Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,812,209 Members 37,551 Now Online
XDA Developers Android and Mobile Development Forum

HOWTO: Unpack, Edit, and Repack Boot Images

Tip us?
 
alansj
Old
(Last edited by alansj; 8th December 2008 at 07:23 PM.) Reason: added mkbootimg binary for OS X 10.5
#1  
Member - OP
Thanks Meter 30
Posts: 44
Join Date: Nov 2008
Default HOWTO: Unpack, Edit, and Repack Boot Images

Several people have already figured out the details on their own, but I have gotten requests to do a more comprehensive tutorial on how the boot and recovery images are structured, and how you can edit them.

Background
Your phone has several devices which hold different parts of the filesystem:

Code:
#cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00280000 00020000 "boot"
mtd3: 04380000 00020000 "system"
mtd4: 04380000 00020000 "cache"
mtd5: 04ac0000 00020000 "userdata"
In this tutorial, we will deal with "recovery" and "boot". The "boot" device holds the files that are automatically loaded onto the root of your filesystem every time you boot (details below).

"system" holds everything that gets mounted in your system/ directory, and userdata/ is everything that shows up in data/ (this is all the apps you've installed, your preferences, etc).

The recovery and boot partitions are at /dev/mtd/mtd1 and /dev/mtd/mtd2, and before you do anything else you should back these up (note: this may not be the best way of doing this because it may not deal properly with bad blocks etc, but it's all we've got until somebody comes up with a better method, and besides you will probably be restoring from update.zip anyway):

Code:
# cat /dev/mtd/mtd1 > /sdcard/mtd1.img
# cat /dev/mtd/mtd2 > /sdcard/mtd2.img
The other thing you should do is put your favorite update.zip file into the root directory of your sd card so that if you screw up your boot partition you can boot into recovery mode and re-apply the update. You probably want one of the pre-rooted recovery images found elsewhere on the forums.

There is also another important file you should know about. In /system/recovery.img there is a full copy of everything that is loaded on mtd1. This file is automatically flashed onto mtd1 every time you shut down. That means two things: 1. Any changes you make directly to /dev/mtd/mtd1 get blown away on reboot and 2. If you want to change /dev/mtd/mtd1 you're probably better off just sticking the image in /system/recovery.img and rebooting. When creating your own custom update.zip files (especially when adapting the stock images), you can get tripped up if you forget to replace /system/recovery.img and it ends up overwriting /dev/mtd/mtd1 unbeknownst to you. Watch out.


Structure of boot and recovery images

The boot and recovery images are not proper filesystems. Instead, they are a custom android format consisting of a 2k header, followed by a gzipped kernel, followed by a ramdisk, followed by a second stage loader (optional, we have not seen these in the wild yet). This structure is outlined in mkbootimg.h:

Code:
+-----------------+ 
| boot header     | 1 page
+-----------------+
| kernel          | n pages  
+-----------------+
| ramdisk         | m pages  
+-----------------+
| second stage    | o pages
+-----------------+

n = (kernel_size + page_size - 1) / page_size
m = (ramdisk_size + page_size - 1) / page_size
o = (second_size + page_size - 1) / page_size

0. all entities are page_size aligned in flash
1. kernel and ramdisk are required (size != 0)
2. second is optional (second_size == 0 -> no second)
A ramdisk is basically a small filesystem containing the core files needed to initialize the system. It includes the critical init process, as well as init.rc, which is where you can set many system-wide properties. If you really want to know more about it, here is the documentation. Here's a list of files on a typical ramdisk:

Code:
./init.trout.rc
./default.prop
./proc
./dev
./init.rc
./init
./sys
./init.goldfish.rc
./sbin
./sbin/adbd
./system
./data
The recovery image typically has a few extra files, which constitute the recovery binary and supporting files (the application that gets run if you hold down home+power when rebooting). These files are:

Code:
./res
./res/images
./res/images/progress_bar_empty_left_round.bmp
./res/images/icon_firmware_install.bmp
./res/images/indeterminate3.bmp
./res/images/progress_bar_fill.bmp
./res/images/progress_bar_left_round.bmp
./res/images/icon_error.bmp
./res/images/indeterminate1.bmp
./res/images/progress_bar_empty_right_round.bmp
./res/images/icon_firmware_error.bmp
./res/images/progress_bar_right_round.bmp
./res/images/indeterminate4.bmp
./res/images/indeterminate5.bmp
./res/images/indeterminate6.bmp
./res/images/progress_bar_empty.bmp
./res/images/indeterminate2.bmp
./res/images/icon_unpacking.bmp
./res/images/icon_installing.bmp
./sbin/recovery
Unpacking, Editing, and Re-Packing the images

Note: below I give you the details for unpacking and repacking manually, but I have attached two perl scripts that do most of this for you

If you are good with a hex editor, you can open up any of these images and strip off the first 2k of data. Then, look for a bunch of zeroes followed by the hex 1F 8B (which is the magic number of a gzip file). Copy everything from the first line of the file, through the zeroes, and stopping at the 1F 8B. That is the kernel. Everything from the 1F 8B through the end is the ramdisk. You could save each of these files separately. In order to see the contents of the ramdisk, you need to un-gzip it and then un-cpio it. You could use a command like this (ideally after creating a new directory and cd'ing into it):

Code:
gunzip -c ../your-ramdisk-file | cpio -i
That will place all of the files from the ramdisk in your working directory. You can now edit them.

In order to re-create the ramdisk, you need to re-cpio them and re-gzip those files, with a command like the following (remember, cpio will include everything in the current working directory, so you probably want to remove any other cruft you might have in there):

Code:
find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz
The final step is to combine the kernel and your new ramdisk into the full image, using the mkbootimg program (which you should download and compile from the git repository):

Code:
mkbootimg --cmdline 'no_console_suspend=1 console=null' --kernel your-kernel-file --ramdisk newramdisk.cpio.gz -o mynewimage.img
Now, there's a lot of hassle in pulling apart files in hex editors and remembering all of these commands, so I wrote unpack and repack perl scripts for you (attached). Hooray.

Flashing your new image back onto the phone

You will probably only ever be flashing boot images directly to the phone, given the fact that /system/recovery.img automatically flashes the recovery device for you (as noted above). If you have created a new recovery image, just stick it in /system/recovery.img and reboot. If you are flashing a boot image, stick it on your phone via adb (a tool included in the Android SDK):

Code:
adb push ./mynewimage.img /sdcard
Then, open a shell to your phone via 'adb shell', get root, and do the following two commands to flash your new boot image:

Code:
# cat /dev/zero >> /dev/mtd/mtd2
   write: No space left on device [this is ok, you can ignore]
# flash_image boot /sdcard/mynewimage.img
Reboot.

If your phone starts all the way up, congratulations. If not, you did something wrong and you'll need to boot into recovery mode and apply your update.zip file (reboot while holding down home+power, when you get the recovery screen press alt+L and then alt+S).


Something fun to do with your new found power

If you place a file titled initlogo.rle in the root directory of your boot image, the phone will display this image upon boot (after the "G1" image and before the Android animation). In order to create this file, you need to create a 320x480 image in Photoshop or Gimp and save it as a "raw image" file. You then need to compress that image with the program to565. More details on that here.


This is not the same thing as applying an update.zip

You will see other places on the forums that describe how to create customized update.zip files, as well as update.zip files that people are sharing. For example, there is a recent update.zip which is a modified version of rc30 (with the anti-root aspects disabled). The update.zip files include new boot images, recovery images, and typically replacements for the entire system/ directory as well as other updates. If you are creating a custom boot or recovery image, it is typically a good idea to start with the image distributed with the most recent update you have applied (flashing an image from an older release could have unintended consequences).


Questions?
Attached Files
File Type: zip unpack-bootimg.pl.zip - [Click for QR Code] (968 Bytes, 14297 views)
File Type: zip repack-bootimg.pl.zip - [Click for QR Code] (491 Bytes, 7596 views)
File Type: zip mkbootimg-OS_X-10_5.zip - [Click for QR Code] (2.9 KB, 4550 views)
The Following 22 Users Say Thank You to alansj For This Useful Post: [ Click to Expand ]
 
jriley60
Old
#2  
Senior Member
Thanks Meter 0
Posts: 194
Join Date: Nov 2008
Location: Atlanta
hooray! you're awesome
 
Dimath
Old
#3  
Dimath's Avatar
Member
Thanks Meter 1
Posts: 77
Join Date: Nov 2008
Location: Los Angeles
Where does boot.img flash? What is the corresponding part of the system?
 
alansj
Old
#4  
Member - OP
Thanks Meter 30
Posts: 44
Join Date: Nov 2008
Quote:
Originally Posted by Dimath View Post
Where does boot.img flash? What is the corresponding part of the system?
I'm not sure what exactly you mean, but when you do flash_image boot imagefile.img it will write imagefile.img to /dev/mtd/mtd2, which is where your phone looks for the boot files. Did that answer your question?
The Following 2 Users Say Thank You to alansj For This Useful Post: [ Click to Expand ]
 
eckzow
Old
#5  
Junior Member
Thanks Meter 1
Posts: 7
Join Date: Nov 2008
For your command...

Code:
cat /dev/zero >> /dev/mtd/mtd2
Do you mean

Code:
cat /dev/zero > /dev/mtd/mtd2
?

The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?
 
andonnguyen
Old
#6  
Senior Member
Thanks Meter 3
Posts: 835
Join Date: Nov 2008
Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.
 
jnasti1
Old
#7  
Junior Member
Thanks Meter 0
Posts: 1
Join Date: Nov 2008
Would this be the only way to rebuild a system app (i.e. Settings.apk) with more debug (Log. to extract via adb logcat over usb), then rebuild the entire system.img, then flash into the G1?
 
alansj
Old
#8  
Member - OP
Thanks Meter 30
Posts: 44
Join Date: Nov 2008
Quote:
Originally Posted by eckzow View Post
For your command...

Code:
cat /dev/zero >> /dev/mtd/mtd2
Do you mean

Code:
cat /dev/zero > /dev/mtd/mtd2
?

The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?
I think you're right. I copied that from somebody else's instructions and it certainly seems to make more sense with one '>'. Anybody know for sure?

In any event, this is unnecessary in most cases because flash_image should overwrite the whole thing. The only exception is when you have an identical header on your image to the one that is already on the device. This shouldn't happen in my instructions (mkbootimg creates a header that includes the build timestamp) but I kept the instruction there just for good measure.
The Following User Says Thank You to alansj For This Useful Post: [ Click to Expand ]
 
alansj
Old
(Last edited by alansj; 12th November 2008 at 08:40 PM.)
#9  
Member - OP
Thanks Meter 30
Posts: 44
Join Date: Nov 2008
Quote:
Originally Posted by andonnguyen View Post
Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.
The adb push command I gave is no different, but you would still have to unpack/repack and flash_image according to my instructions.
The Following User Says Thank You to alansj For This Useful Post: [ Click to Expand ]
 
Koush
Old
#10  
Retired Recognized Developer
Thanks Meter 894
Posts: 917
Join Date: Sep 2007
I've tried these instructions on RC30 1.3. Basically I extracted, unpacked, and repacked, just to see if it would work. The resultant file is too large; it fails when you run flash_image.

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes