Many thanks - I've been struggling with linux to unpack the blob.LNX file. Blob tools isn't working for me for some reason as ./blobunpack blob.LNX says unrecognised blob format.
What is the secret to get it working?
I firmly believe that this should not be a secret, so I'll explain in detail how to do it and not only give you steps but also a lot of background info. Many steps in the following tutorial are optional and useful for learning, for checking things and for troubleshooting - I don't want you to blindly follow a recipe without understanding what you are doing.
Unpacking the blob
We start with the full firmware, unpacked once - in this case, WW_epad-user-10.4.4.25.zip. First extract the blob from the zip:
Code:
[email protected] ~/android/tf700/kernel $ unzip WW_epad-user-10.4.4.25.zip blob
Archive: WW_epad-user-10.4.4.25.zip
signed by SignApk
inflating: blob
This is a signed blob. It begins with "-SIGNED-BY-SIGNBLOB-". You can check this with hexdump (this step is completely optional, just for illustration):
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 48 blob
00000000 2d 53 49 47 4e 45 44 2d 42 59 2d 53 49 47 4e 42 |-SIGNED-BY-SIGNB|
00000010 4c 4f 42 2d e9 1a d7 30 00 01 00 00 4d 53 4d 2d |LOB-...0....MSM-|
00000020 52 41 44 49 4f 2d 55 50 44 41 54 45 00 00 01 00 |RADIO-UPDATE....|
00000030
Also note the MSM-RADIO-UPDATE after the signature header - it is the start of an unsigned blob.
But what exactly is a blob? In database terminology, it stands for "binary large object" - a bunch of data that is just stored with no further interpretation by the database.The blobs for flashing NVIDIA Tegra devices have a specific structure - they contain one or more raw images to be flashed to one or more partitions on the device.
The full firmware blob for the TF700 contains images for 5 partitions, which we will now extract from the blob. If you don't have blobunpack, get the sources from
https://github.com/AndroidRoot/BlobTools - how to do that and how to compile them may be part of another tutorial. I also assume you have blobunpack somewhere in your $PATH.
Code:
[email protected] ~/android/tf700/kernel $ blobunpack blob
Header size: 60
5 partitions starting at offset 0x3C
Blob version: 65536
Partition 0
Name: PT
Offset: 140 (0x8C)
Size: 524288 (0x80000)
Writing file blob.PT (524288 bytes)
Partition 1
Name: EBT
Offset: 524428 (0x8008C)
Size: 1566813 (0x17E85D)
Writing file blob.EBT (1566813 bytes)
Partition 2
Name: SOS
Offset: 2091241 (0x1FE8E9)
Size: 6410496 (0x61D100)
Writing file blob.SOS (6410496 bytes)
Partition 3
Name: LNX
Offset: 8501737 (0x81B9E9)
Size: 5595392 (0x556100)
Writing file blob.LNX (5595392 bytes)
Partition 4
Name: APP
Offset: 14097129 (0xD71AE9)
Size: 805306368 (0x30000000)
Writing file blob.APP (805306368 bytes)
OK, now we have separate files for each partition. The next step is again optional, just to illustrate what happened:
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 2059512
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 819403781 22. Mär 2011 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5595392 19. Jän 19:39 blob.LNX
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
What's inside a firmware blob
blob.APP is a raw image of a Linux ext4 partition. If you are interested in the contents, you can loop mount this file and access all files inside directly on your Linux computer. This is the filesystem that is mounted as /system - the ROM as you all know it.
blob.EBT is the bootloader.
blob.LNX is the boot image - the Linux kernel and the ramdisk, which is later seen as "/". This is the one we are after.
blob.PT is the Tegra partition table. This is a proprietary format, not a PC-compatible partition table.
blob.SOS is the recovery boot image - another Linux kernel and another ramdisk, but this ramdisk contains the stock recovery (incl. the dreaded "dead android").
The boot image
Now, what's actually a boot image, and how do we recognize one if we see one? As usual, the best answer is: look inside! (again - optional, just to learn something)
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 16 blob.LNX
00000000 41 4e 44 52 4f 49 44 21 58 0c 4e 00 00 80 00 10 |ANDROID!X.N.....|
00000010
An Android boot image begins with the text "ANDROID!". Easy.
(maybe sbdags now starts to smack his head and realizes why blobunpack does not work here
- and if anyone is still in doubt - a boot image is not a blob)
Splitting a boot image
Good, but how do we crack this thing open? Fortunately there are two small perl scripts I extracted from dsixda's Android kitchen (they are in tools/extract_boot_files):
OK, what happened?
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 2064524
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 819403781 22. Mär 2011 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5595392 19. Jän 19:39 blob.LNX
drwxr-xr-x 8 that users 4096 19. Jän 19:59 blob.LNX-ramdisk
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
-rw-r--r-- 1 that users 5114968 19. Jän 19:58 zImage
We got a new file (zImage) and a directory (blob.LNX-ramdisk)!
zImage is the kernel image. It is the result of compiling a kernel from sources. Today, we will not compile a new kernel, just repack this image with our modified ramdisk.
What blob.LNX-ramdisk contains should be obvious.
Modifying the ramdisk
Now we can start modifying the ramdisk. A CleanROM ramdisk differs from a stock ramdisk in only 3 files:
- sbin/adbd
- default.prop
- init.rc
sbin/adbd is the adb daemon - the program that handles the tablet's side of the adb protocol. The stock version runs under the "shell" user account, which has limited permissions and cannot access all files on the tablet. The CleanROM version of adbd runs as "root", so "adb push" and "adb pull" can write or read any file on the device, and "adb shell" goes directly to a root prompt.
I don't know where scrosler got this binary from, I just copied it from his CleanROM kernel's ramdisk. You can do the same - now you already know how to extract a ramdisk, so just do the same with the CleanROM kernel in another directory.
default.prop contains only a few lines. Here is the already modified CleanROM version:
Code:
#
# ADDITIONAL_DEFAULT_PROPERTIES
#
ro.secure=0
ro.allow.mock.location=0
ro.debuggable=1
persist.sys.usb.config=mtp,adb
In the original file, ro.secure=1 (which would limit adb to the "shell" account again), ro.debuggable=0 (I'd have to look up what exactly this does), and persist.sys.usb.config=mtp without adb (I think this modification is what automatically enables "USB debugging" at boot time).
Finally, init.rc has a few additional lines to enable init.d support. This means the boot sequence will run any scripts in /etc/init.d - this a simplified version of a usual convention for desktop Linux systems. CleanROM uses it to set up various tweaks (look into /system/etc/init.d on your device).
To illustrate the changes in init.rc, I use the "diff" command (and you can too, if you want - just use the correct paths for your setup instead of mine):
Code:
[email protected] ~/android/tf700/kernel $ diff -u blob.LNX-ramdisk/init.rc ~/android/tf700/cleanrom/blob.LNX-ramdisk/init.rc
--- blob.LNX-ramdisk/init.rc 2013-01-19 19:59:03.000000000 +0100
+++ /home/that/android/tf700/cleanrom/blob.LNX-ramdisk/init.rc 2013-01-19 01:24:43.000000000 +0100
@@ -309,6 +309,8 @@
# Set this property so surfaceflinger is not started by system_init
setprop system_init.startsurfaceflinger 0
+ start sysinit
+
class_start core
class_start main
@@ -413,6 +415,10 @@
user drm
group drm system inet drmrpc sdcard_r
+service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
+ disabled
+ oneshot
+
service media /system/bin/mediaserver
class main
user media
If you have never read a diff before (also called a patch), here a quick guide:
- Lines beginning with "---" and "+++" identify the two files that were compared.
- Lines beginning with "@@" specify line numbers in the two files. -309,6 +309,8 means: The next block contains 6 lines starting with line 309 from file 1 (---), and 8 lines starting with line 309 in file 2 (+++). In other words, this means that two lines were inserted there.
- The following lines start with a blank - they are context lines. These lines are common in both files and only used to see some context around the change.
- Lines beginning with "+" exist only in file 2 (+++). You can easily remember all this if you know that a diff describes how to edit file 1 to be identical to file 2 - you delete lines marked with "-" and insert lines marked with "+". With only these two operations, you can transform any text file to any other - in the worst case you will delete all lines of the original file and insert all lines of the new file.
The second block works like the first one - here we insert 4 lines near line 413 (which has become line 415 after our first insertion).
You have the following options to apply this change:
1. Simply copy over the changed version from the previous CleanROM kernel. Do a diff before to verify that Asus didn't change anything, otherwise you will undo Asus' changes.
2. Use the "patch" tool to apply the diff file from above. This will automatically perform the edit steps in the diff file and it also has some tolerance if something changed in other sections of the file.
3. Use a text editor and perform the changes manually. If you copy lines from the diff (and not the original init.rc), make sure you don't accidentally copy the leading "+" - this is only for the diff and should not part of the file.
Update 2013-01-21 Start
Unlocking DPI
As usual I don't just tell you how to do it, but some background info first:
There is a read-only system property in Android named "lcd_density" that specifies the display resolution. This is done to ensure that the approximate size of display items is kept consistent across the wide range of devices Android runs on.
For normal users this is great, because they don't have to buy a magnifying glass and a pointed stylus just to use their super-hires-devices, but users here on xda-developers of course see a setting that can be modified and therefore it *must* be modified.
Smaller values produce smaller icons and text (and thus more visible content at once); bigger values produce bigger icons and text.
The ASUS stock firmware is not only used on the TF700, but also for several other Transformers. The file /system/build.prop sets ro.sf.lcd_density=160, which means 160 dpi. This is good for the majority of tablets that have 1280*800 displays, but the TF700's display has a resolution of 224 pixels per inch, and with a setting of 160 dpi everything would be very tiny.
The bootloader passes an important parameter to the kernel: "androidboot.productid=0x04", This value depends on the Transformer model, and it is 4 for the TF700. If you look into the ramdisk, there are several files named init.00.rc to init.0d.rc. These are device-specific, and you guessed right if you think that init.04.rc is for the TF700. This file sets lcd_density to 240 dpi, and since it is a read-only property, it keeps its value even if the system's build.prop (which is read later) tries to set lcd_density to 160.
So how do we unlock dpi? Delete the line that says "setprop ro.sf.lcd_density 240" in init.04.rc! Of course this requires you to set the desired dpi value in /system/build.prop. If you install CleanROM Inheritance, you can choose this value in the installer. If you run this kernel with the stock ROM, everything will become very tiny, and now you know why.
Update 2013-01-21 End
Re-packing the boot image
OK, phew, we finally have a modified ramdisk - now we need to repack everything, basically the reverse of the first few steps. For this I use a self-written shell script. First let's just run it:
Code:
[email protected] ~/android/tf700/kernel $ repack-kernel.sh
Found 1 partitions as commandline arguments
Partname: LNX Filename: blob.LNX
Size: 60
1 partitions starting at offset 0x3C
Offset: 76
8+0 records in
8+0 records out
8 bytes (8 B) copied, 0,000136275 s, 58,7 kB/
OK, what happened here? Since I wrote this script myself, I can explain it line by line:
Code:
[email protected] ~/android/tf700/kernel $ cat ~/bin/repack-kernel.sh
#!/bin/bash
~/android/Android-Kitchen/tools/mkboot/mkbootfs blob.LNX-ramdisk | gzip > ramdisk.gz
~/android/Android-Kitchen/tools/mkboot/mkbootimg --kernel zImage --ramdisk ramdisk.gz -o blob.LNX
blobpack blobtmp LNX blob.LNX
echo -n "-SIGNED-BY-SIGNBLOB-" > blob
dd if=/dev/zero bs=1 count=8 >> blob
cat blobtmp >> blob
The ramdisk is repacked and zipped again. This creates ramdisk.gz.
The kernel is merged with the ramdisk to a new boot image, to create blob.LNX. If you paid attention before, yes, this will overwrite the old blob.LNX. If you want, you can check again with hexdump that it is really a boot image (it begins with "ANDROID!").
Re-packing a blob
Now we pack the boot image into a blob. However, the "original" version of blobtools does not create a signed blob. See:
Code:
[email protected] ~/android/tf700/kernel $ hexdump -C -n 16 blobtmp
00000000 4d 53 4d 2d 52 41 44 49 4f 2d 55 50 44 41 54 45 |MSM-RADIO-UPDATE|
00000010
That was the blob format for the TF101. For later Transformers, we need to prepend a signature header. As we don't have Asus' cryptographic keys, we cannot create a valid signature - but fortunately, since we have an unlocked bootloader, we don't actually need to add a proper signature - the header is enough. There are two ways to create this header - there is a modified version of blobtools in circulation that has a command line option for it, but I do it "manually" with echo and dd.
The final output is again simply called "blob" - this is the new blob that now contains only the kernel. (It overwrites the original blob, which is not really intended, but I just didn't care).
For comparison, now we have this:
Code:
[email protected] ~/android/tf700/kernel $ ls -l
total 1274980
-rw-rw-r-- 1 that users 468028852 28. Dez 12:33 WW_epad-user-10.4.4.25.zip
-rw-r--r-- 1 that users 5599336 19. Jän 20:35 blob
-rw-r--r-- 1 that users 805306368 19. Jän 19:39 blob.APP
-rw-r--r-- 1 that users 1566813 19. Jän 19:39 blob.EBT
-rw-r--r-- 1 that users 5599232 19. Jän 20:35 blob.LNX
drwxr-xr-x 8 that users 4096 19. Jän 19:59 blob.LNX-ramdisk
-rw-r--r-- 1 that users 524288 19. Jän 19:39 blob.PT
-rw-r--r-- 1 that users 6410496 19. Jän 19:39 blob.SOS
-rw-r--r-- 1 that users 5599308 19. Jän 20:35 blobtmp
-rw-r--r-- 1 that users 479752 19. Jän 20:35 ramdisk.gz
-rw-r--r-- 1 that users 5114968 19. Jän 19:58 zImage
Congratulations, you have a flashable blob with your new kernel. But how do you get it onto your device?
Installation
Again, there are two ways:
Installing the blob directly with dd
The first option is to copy the blob to your tablet's staging partition manually and reboot:
Code:
[email protected] ~/android/tf700/kernel $ adb push blob /sdcard/kernel-10.4.4.25.blob
1324 KB/s (5599336 bytes in 4.128s)
[email protected] ~/android/tf700/kernel $ adb shell
sh-3.2# dd if=/sdcard/kernel-10.4.4.25.blob of=/dev/block/mmcblk0p4
10936+1 records in
10936+1 records out
5599336 bytes transferred in 0.129 secs (43405705 bytes/sec)
Be very careful with the dd command and double-check that you didn't swap "if" and "of", and that you specified the correct target partition *before* you hit Enter.
Now reboot your tablet, and the bootloader will flash your new kernel from the blob to the actual kernel partition (notice the progress bar on the boot screen with the slightly broken graphics, as usual). The tablet will reboot again automatically, and if you did everything right, your shiny new kernel will be running!
Creating a zip for flashing in the recovery
OK, now to option 2 - pack the blob into a flashable zip that you can flash in the recovery. The easiest way to do this is to reuse the original firmware zip. Just replace the blob inside the big firmware zip, and that's it - you have a recovery-flashable zip!
Code:
[email protected] ~/android/tf700/kernel $ mv WW_epad-user-10.4.4.25.zip kernel-10.4.4.25.zip
[email protected] ~/android/tf700/kernel $ zip -d kernel-10.4.4.25.zip blob
deleting: blob
[email protected] ~/android/tf700/kernel $ zip kernel-10.4.4.25.zip blob
adding: blob (deflated 1%)
This would not be a good tutorial if I didn't explain how and why this actually works. Let's look inside!
Code:
[email protected] ~/android/tf700/kernel $ unzip kernel-10.4.4.25.zip META-INF/*
Archive: kernel-10.4.4.25.zip
signed by SignApk
inflating: META-INF/com/google/android/resource
inflating: META-INF/com/google/android/rule
inflating: META-INF/com/google/android/update-binary
inflating: META-INF/com/google/android/updater-script
inflating: META-INF/com/android/otacert
inflating: META-INF/MANIFEST.MF
inflating: META-INF/CERT.SF
inflating: META-INF/CERT.RSA
The core of a flashable zip is the updater-script. For a full firmware, it is very short:
Code:
[email protected] ~/android/tf700/kernel $ cat META-INF/com/google/android/updater-script
assert(package_extract_file("blob", "/tmp/blob"), write_raw_image("/tmp/blob", "staging"),delete("/tmp/blob"));
This line instructs the recovery to extract the blob from the zip file to /tmp/blob and then copy it to the staging partition - essentially it is exactly the same as we did manually in option #1 above!
OK, now the secret is fully disclosed - I hope you learned something. If you have further questions, or if you need help compiling some of the mentioned binaries, don't hesitate to ask (if it's not something like "how do I use the Linux command line"
).
