HOWTO: Unpack, Edit, and Repack Boot Images

Search This thread

alansj

Member
Nov 6, 2008
44
47
Several people have already figured out the details on their own, but I have gotten requests to do a more comprehensive tutorial on how the boot and recovery images are structured, and how you can edit them.

Background
Your phone has several devices which hold different parts of the filesystem:

Code:
#cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "misc"
mtd1: 00500000 00020000 "recovery"
mtd2: 00280000 00020000 "boot"
mtd3: 04380000 00020000 "system"
mtd4: 04380000 00020000 "cache"
mtd5: 04ac0000 00020000 "userdata"

In this tutorial, we will deal with "recovery" and "boot". The "boot" device holds the files that are automatically loaded onto the root of your filesystem every time you boot (details below).

"system" holds everything that gets mounted in your system/ directory, and userdata/ is everything that shows up in data/ (this is all the apps you've installed, your preferences, etc).

The recovery and boot partitions are at /dev/mtd/mtd1 and /dev/mtd/mtd2, and before you do anything else you should back these up (note: this may not be the best way of doing this because it may not deal properly with bad blocks etc, but it's all we've got until somebody comes up with a better method, and besides you will probably be restoring from update.zip anyway):

Code:
# cat /dev/mtd/mtd1 > /sdcard/mtd1.img
# cat /dev/mtd/mtd2 > /sdcard/mtd2.img

The other thing you should do is put your favorite update.zip file into the root directory of your sd card so that if you screw up your boot partition you can boot into recovery mode and re-apply the update. You probably want one of the pre-rooted recovery images found elsewhere on the forums.

There is also another important file you should know about. In /system/recovery.img there is a full copy of everything that is loaded on mtd1. This file is automatically flashed onto mtd1 every time you shut down. That means two things: 1. Any changes you make directly to /dev/mtd/mtd1 get blown away on reboot and 2. If you want to change /dev/mtd/mtd1 you're probably better off just sticking the image in /system/recovery.img and rebooting. When creating your own custom update.zip files (especially when adapting the stock images), you can get tripped up if you forget to replace /system/recovery.img and it ends up overwriting /dev/mtd/mtd1 unbeknownst to you. Watch out.


Structure of boot and recovery images

The boot and recovery images are not proper filesystems. Instead, they are a custom android format consisting of a 2k header, followed by a gzipped kernel, followed by a ramdisk, followed by a second stage loader (optional, we have not seen these in the wild yet). This structure is outlined in mkbootimg.h:

Code:
+-----------------+ 
| boot header     | 1 page
+-----------------+
| kernel          | n pages  
+-----------------+
| ramdisk         | m pages  
+-----------------+
| second stage    | o pages
+-----------------+

n = (kernel_size + page_size - 1) / page_size
m = (ramdisk_size + page_size - 1) / page_size
o = (second_size + page_size - 1) / page_size

0. all entities are page_size aligned in flash
1. kernel and ramdisk are required (size != 0)
2. second is optional (second_size == 0 -> no second)

A ramdisk is basically a small filesystem containing the core files needed to initialize the system. It includes the critical init process, as well as init.rc, which is where you can set many system-wide properties. If you really want to know more about it, here is the documentation. Here's a list of files on a typical ramdisk:

Code:
./init.trout.rc
./default.prop
./proc
./dev
./init.rc
./init
./sys
./init.goldfish.rc
./sbin
./sbin/adbd
./system
./data

The recovery image typically has a few extra files, which constitute the recovery binary and supporting files (the application that gets run if you hold down home+power when rebooting). These files are:

Code:
./res
./res/images
./res/images/progress_bar_empty_left_round.bmp
./res/images/icon_firmware_install.bmp
./res/images/indeterminate3.bmp
./res/images/progress_bar_fill.bmp
./res/images/progress_bar_left_round.bmp
./res/images/icon_error.bmp
./res/images/indeterminate1.bmp
./res/images/progress_bar_empty_right_round.bmp
./res/images/icon_firmware_error.bmp
./res/images/progress_bar_right_round.bmp
./res/images/indeterminate4.bmp
./res/images/indeterminate5.bmp
./res/images/indeterminate6.bmp
./res/images/progress_bar_empty.bmp
./res/images/indeterminate2.bmp
./res/images/icon_unpacking.bmp
./res/images/icon_installing.bmp
./sbin/recovery

Unpacking, Editing, and Re-Packing the images

Note: below I give you the details for unpacking and repacking manually, but I have attached two perl scripts that do most of this for you

If you are good with a hex editor, you can open up any of these images and strip off the first 2k of data. Then, look for a bunch of zeroes followed by the hex 1F 8B (which is the magic number of a gzip file). Copy everything from the first line of the file, through the zeroes, and stopping at the 1F 8B. That is the kernel. Everything from the 1F 8B through the end is the ramdisk. You could save each of these files separately. In order to see the contents of the ramdisk, you need to un-gzip it and then un-cpio it. You could use a command like this (ideally after creating a new directory and cd'ing into it):

Code:
gunzip -c ../your-ramdisk-file | cpio -i

That will place all of the files from the ramdisk in your working directory. You can now edit them.

In order to re-create the ramdisk, you need to re-cpio them and re-gzip those files, with a command like the following (remember, cpio will include everything in the current working directory, so you probably want to remove any other cruft you might have in there):

Code:
find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz

The final step is to combine the kernel and your new ramdisk into the full image, using the mkbootimg program (which you should download and compile from the git repository):

Code:
mkbootimg --cmdline 'no_console_suspend=1 console=null' --kernel your-kernel-file --ramdisk newramdisk.cpio.gz -o mynewimage.img

Now, there's a lot of hassle in pulling apart files in hex editors and remembering all of these commands, so I wrote unpack and repack perl scripts for you (attached). Hooray.

Flashing your new image back onto the phone

You will probably only ever be flashing boot images directly to the phone, given the fact that /system/recovery.img automatically flashes the recovery device for you (as noted above). If you have created a new recovery image, just stick it in /system/recovery.img and reboot. If you are flashing a boot image, stick it on your phone via adb (a tool included in the Android SDK):

Code:
adb push ./mynewimage.img /sdcard

Then, open a shell to your phone via 'adb shell', get root, and do the following two commands to flash your new boot image:

Code:
# cat /dev/zero >> /dev/mtd/mtd2
   write: No space left on device [this is ok, you can ignore]
# flash_image boot /sdcard/mynewimage.img

Reboot.

If your phone starts all the way up, congratulations. If not, you did something wrong and you'll need to boot into recovery mode and apply your update.zip file (reboot while holding down home+power, when you get the recovery screen press alt+L and then alt+S).


Something fun to do with your new found power

If you place a file titled initlogo.rle in the root directory of your boot image, the phone will display this image upon boot (after the "G1" image and before the Android animation). In order to create this file, you need to create a 320x480 image in Photoshop or Gimp and save it as a "raw image" file. You then need to compress that image with the program to565. More details on that here.


This is not the same thing as applying an update.zip

You will see other places on the forums that describe how to create customized update.zip files, as well as update.zip files that people are sharing. For example, there is a recent update.zip which is a modified version of rc30 (with the anti-root aspects disabled). The update.zip files include new boot images, recovery images, and typically replacements for the entire system/ directory as well as other updates. If you are creating a custom boot or recovery image, it is typically a good idea to start with the image distributed with the most recent update you have applied (flashing an image from an older release could have unintended consequences).


Questions?
 

Attachments

  • unpack-bootimg.pl.zip
    968 bytes · Views: 33,780
  • repack-bootimg.pl.zip
    491 bytes · Views: 17,607
  • mkbootimg-OS_X-10_5.zip
    2.9 KB · Views: 10,534
Last edited:

eckzow

Member
Nov 11, 2008
7
1
For your command...

Code:
cat /dev/zero >> /dev/mtd/mtd2

Do you mean

Code:
cat /dev/zero > /dev/mtd/mtd2

?

The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?
 

andonnguyen

Senior Member
Nov 12, 2008
842
3
Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.
 

jnasti1

New member
Nov 12, 2008
1
0
Would this be the only way to rebuild a system app (i.e. Settings.apk) with more debug (Log. to extract via adb logcat over usb), then rebuild the entire system.img, then flash into the G1?
 

alansj

Member
Nov 6, 2008
44
47
For your command...

Code:
cat /dev/zero >> /dev/mtd/mtd2

Do you mean

Code:
cat /dev/zero > /dev/mtd/mtd2

?

The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?

I think you're right. I copied that from somebody else's instructions and it certainly seems to make more sense with one '>'. Anybody know for sure?

In any event, this is unnecessary in most cases because flash_image should overwrite the whole thing. The only exception is when you have an identical header on your image to the one that is already on the device. This shouldn't happen in my instructions (mkbootimg creates a header that includes the build timestamp) but I kept the instruction there just for good measure.
 
  • Like
Reactions: ak

alansj

Member
Nov 6, 2008
44
47
Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.

The adb push command I gave is no different, but you would still have to unpack/repack and flash_image according to my instructions.
 
Last edited:
  • Like
Reactions: ak

Koush

Retired Recognized Developer
Sep 21, 2007
917
872
I've tried these instructions on RC30 1.3. Basically I extracted, unpacked, and repacked, just to see if it would work. The resultant file is too large; it fails when you run flash_image.
 

[RiS]

Senior Member
Dec 13, 2008
135
8
I'm trying to modify my boot image of my ADP1 using the perl scripts, but I receive the following warning while decompressing the ramdisk:

Code:
$ unpack-bootimg.pl mtd2.img 

kernel written to mtd2.img-kernel.gz
ramdisk written to mtd2.img-ramdisk.cpio.gz

removed old directory mtd2.img-ramdisk

[B]gzip: ../mtd2.img-ramdisk.cpio.gz: decompression OK, trailing garbage ignored
462 blocks[/B]

extracted ramdisk contents to directory mtd2.img-ramdisk/

Is this warning expected? Is safe to continue?

Also, I've found that the size of the modified packed image is far smaller than the original one :confused:

Code:
$ ll mtd2.img mtd2-modified.img 
-rwx------ 1 ris ris 2621440 2009-01-07 19:19 mtd2.img
-rw-r--r-- 1 ris ris 1533952 2009-01-07 20:49 mtd2-modified.img

Update: I've tried anyway.
For the record, I've obtained the boot.img using cat, uncompressed with the perl script, modified default.prop, repacked with the perl script.
As you see from the code above, the img file is much smaller (Opening with a hex editor you can see that the end of the original image is full of 0xFF, so I believe it's ok, both the gzip warning and the different file sizes).
Reflashed it from recovery mode, using
fastboot flash boot mt2-modified.img
fastboot reboot

... and worked flawlessly :D

I'm leaving the coment for future references.
Thanks for the tutorial :)
 
Last edited:

alansj

Member
Nov 6, 2008
44
47
gzip: ../mtd2.img-ramdisk.cpio.gz: decompression OK, trailing garbage ignored
462 blocks


Is this warning expected? Is safe to continue?

Yes, you would expect trailing zeroes, which would give you that error. The trailing zeroes exist in order to pad the image size to the nearest page boundary. They are added by mkbootimg.

Also, I've found that the size of the modified packed image is far smaller than the original one

As you see from the code above, the img file is much smaller (Opening with a hex editor you can see that the end of the original image is full of 0xFF, so I believe it's ok, both the gzip warning and the different file sizes).

Yep, that's exactly why. Nothing to worry about.

Thanks for the informative clarification.
 
  • Like
Reactions: ak

[RiS]

Senior Member
Dec 13, 2008
135
8
There is also another important file you should know about. In /system/recovery.img there is a full copy of everything that is loaded on mtd1. This file is automatically flashed onto mtd1 every time you shut down. That means two things: 1. Any changes you make directly to /dev/mtd/mtd1 get blown away on reboot and 2. If you want to change /dev/mtd/mtd1 you're probably better off just sticking the image in /system/recovery.img and rebooting.

I'm using an stock ADP1, and the file /system/recovery.img does not exist. Is this expected?
Also, I've found out in JFv.1.31 that the recovery image is in /data/recovery.img (although there is no /data/recovery.img in my ADP1 neither..)
 

JesusFreke

Inactive Recognized Developer
Oct 23, 2008
736
54
Dallas
I'm using an stock ADP1, and the file /system/recovery.img does not exist. Is this expected?
Also, I've found out in JFv.1.31 that the recovery image is in /data/recovery.img (although there is no /data/recovery.img in my ADP1 neither..)

It deletes it after it flashes on the first bootup after you apply the update.
 

slevin.yu

Member
Dec 2, 2008
12
0
When I use the unpack script on JF1.31 boot.img, it prints out like this:
"Could not find any embedded ramdisk images. Are you sure this is a full boot image?"
Any help?
 

Top Liked Posts

  • There are no posts matching your filters.
  • 37
    Several people have already figured out the details on their own, but I have gotten requests to do a more comprehensive tutorial on how the boot and recovery images are structured, and how you can edit them.

    Background
    Your phone has several devices which hold different parts of the filesystem:

    Code:
    #cat /proc/mtd
    dev:    size   erasesize  name
    mtd0: 00040000 00020000 "misc"
    mtd1: 00500000 00020000 "recovery"
    mtd2: 00280000 00020000 "boot"
    mtd3: 04380000 00020000 "system"
    mtd4: 04380000 00020000 "cache"
    mtd5: 04ac0000 00020000 "userdata"

    In this tutorial, we will deal with "recovery" and "boot". The "boot" device holds the files that are automatically loaded onto the root of your filesystem every time you boot (details below).

    "system" holds everything that gets mounted in your system/ directory, and userdata/ is everything that shows up in data/ (this is all the apps you've installed, your preferences, etc).

    The recovery and boot partitions are at /dev/mtd/mtd1 and /dev/mtd/mtd2, and before you do anything else you should back these up (note: this may not be the best way of doing this because it may not deal properly with bad blocks etc, but it's all we've got until somebody comes up with a better method, and besides you will probably be restoring from update.zip anyway):

    Code:
    # cat /dev/mtd/mtd1 > /sdcard/mtd1.img
    # cat /dev/mtd/mtd2 > /sdcard/mtd2.img

    The other thing you should do is put your favorite update.zip file into the root directory of your sd card so that if you screw up your boot partition you can boot into recovery mode and re-apply the update. You probably want one of the pre-rooted recovery images found elsewhere on the forums.

    There is also another important file you should know about. In /system/recovery.img there is a full copy of everything that is loaded on mtd1. This file is automatically flashed onto mtd1 every time you shut down. That means two things: 1. Any changes you make directly to /dev/mtd/mtd1 get blown away on reboot and 2. If you want to change /dev/mtd/mtd1 you're probably better off just sticking the image in /system/recovery.img and rebooting. When creating your own custom update.zip files (especially when adapting the stock images), you can get tripped up if you forget to replace /system/recovery.img and it ends up overwriting /dev/mtd/mtd1 unbeknownst to you. Watch out.


    Structure of boot and recovery images

    The boot and recovery images are not proper filesystems. Instead, they are a custom android format consisting of a 2k header, followed by a gzipped kernel, followed by a ramdisk, followed by a second stage loader (optional, we have not seen these in the wild yet). This structure is outlined in mkbootimg.h:

    Code:
    +-----------------+ 
    | boot header     | 1 page
    +-----------------+
    | kernel          | n pages  
    +-----------------+
    | ramdisk         | m pages  
    +-----------------+
    | second stage    | o pages
    +-----------------+
    
    n = (kernel_size + page_size - 1) / page_size
    m = (ramdisk_size + page_size - 1) / page_size
    o = (second_size + page_size - 1) / page_size
    
    0. all entities are page_size aligned in flash
    1. kernel and ramdisk are required (size != 0)
    2. second is optional (second_size == 0 -> no second)

    A ramdisk is basically a small filesystem containing the core files needed to initialize the system. It includes the critical init process, as well as init.rc, which is where you can set many system-wide properties. If you really want to know more about it, here is the documentation. Here's a list of files on a typical ramdisk:

    Code:
    ./init.trout.rc
    ./default.prop
    ./proc
    ./dev
    ./init.rc
    ./init
    ./sys
    ./init.goldfish.rc
    ./sbin
    ./sbin/adbd
    ./system
    ./data

    The recovery image typically has a few extra files, which constitute the recovery binary and supporting files (the application that gets run if you hold down home+power when rebooting). These files are:

    Code:
    ./res
    ./res/images
    ./res/images/progress_bar_empty_left_round.bmp
    ./res/images/icon_firmware_install.bmp
    ./res/images/indeterminate3.bmp
    ./res/images/progress_bar_fill.bmp
    ./res/images/progress_bar_left_round.bmp
    ./res/images/icon_error.bmp
    ./res/images/indeterminate1.bmp
    ./res/images/progress_bar_empty_right_round.bmp
    ./res/images/icon_firmware_error.bmp
    ./res/images/progress_bar_right_round.bmp
    ./res/images/indeterminate4.bmp
    ./res/images/indeterminate5.bmp
    ./res/images/indeterminate6.bmp
    ./res/images/progress_bar_empty.bmp
    ./res/images/indeterminate2.bmp
    ./res/images/icon_unpacking.bmp
    ./res/images/icon_installing.bmp
    ./sbin/recovery

    Unpacking, Editing, and Re-Packing the images

    Note: below I give you the details for unpacking and repacking manually, but I have attached two perl scripts that do most of this for you

    If you are good with a hex editor, you can open up any of these images and strip off the first 2k of data. Then, look for a bunch of zeroes followed by the hex 1F 8B (which is the magic number of a gzip file). Copy everything from the first line of the file, through the zeroes, and stopping at the 1F 8B. That is the kernel. Everything from the 1F 8B through the end is the ramdisk. You could save each of these files separately. In order to see the contents of the ramdisk, you need to un-gzip it and then un-cpio it. You could use a command like this (ideally after creating a new directory and cd'ing into it):

    Code:
    gunzip -c ../your-ramdisk-file | cpio -i

    That will place all of the files from the ramdisk in your working directory. You can now edit them.

    In order to re-create the ramdisk, you need to re-cpio them and re-gzip those files, with a command like the following (remember, cpio will include everything in the current working directory, so you probably want to remove any other cruft you might have in there):

    Code:
    find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz

    The final step is to combine the kernel and your new ramdisk into the full image, using the mkbootimg program (which you should download and compile from the git repository):

    Code:
    mkbootimg --cmdline 'no_console_suspend=1 console=null' --kernel your-kernel-file --ramdisk newramdisk.cpio.gz -o mynewimage.img

    Now, there's a lot of hassle in pulling apart files in hex editors and remembering all of these commands, so I wrote unpack and repack perl scripts for you (attached). Hooray.

    Flashing your new image back onto the phone

    You will probably only ever be flashing boot images directly to the phone, given the fact that /system/recovery.img automatically flashes the recovery device for you (as noted above). If you have created a new recovery image, just stick it in /system/recovery.img and reboot. If you are flashing a boot image, stick it on your phone via adb (a tool included in the Android SDK):

    Code:
    adb push ./mynewimage.img /sdcard

    Then, open a shell to your phone via 'adb shell', get root, and do the following two commands to flash your new boot image:

    Code:
    # cat /dev/zero >> /dev/mtd/mtd2
       write: No space left on device [this is ok, you can ignore]
    # flash_image boot /sdcard/mynewimage.img

    Reboot.

    If your phone starts all the way up, congratulations. If not, you did something wrong and you'll need to boot into recovery mode and apply your update.zip file (reboot while holding down home+power, when you get the recovery screen press alt+L and then alt+S).


    Something fun to do with your new found power

    If you place a file titled initlogo.rle in the root directory of your boot image, the phone will display this image upon boot (after the "G1" image and before the Android animation). In order to create this file, you need to create a 320x480 image in Photoshop or Gimp and save it as a "raw image" file. You then need to compress that image with the program to565. More details on that here.


    This is not the same thing as applying an update.zip

    You will see other places on the forums that describe how to create customized update.zip files, as well as update.zip files that people are sharing. For example, there is a recent update.zip which is a modified version of rc30 (with the anti-root aspects disabled). The update.zip files include new boot images, recovery images, and typically replacements for the entire system/ directory as well as other updates. If you are creating a custom boot or recovery image, it is typically a good idea to start with the image distributed with the most recent update you have applied (flashing an image from an older release could have unintended consequences).


    Questions?
    3
    Where does boot.img flash? What is the corresponding part of the system?

    I'm not sure what exactly you mean, but when you do flash_image boot imagefile.img it will write imagefile.img to /dev/mtd/mtd2, which is where your phone looks for the boot files. Did that answer your question?
    1
    hooray! you're awesome:D
    1
    For your command...

    Code:
    cat /dev/zero >> /dev/mtd/mtd2

    Do you mean

    Code:
    cat /dev/zero > /dev/mtd/mtd2

    ?

    The idea being that you erase flash in the version with one '>', whereas you... append to the end of a device in the version with two '>'s? I can see the utility of erasing flash with one '>' but appending seems... odd. Am I missing something?

    I think you're right. I copied that from somebody else's instructions and it certainly seems to make more sense with one '>'. Anybody know for sure?

    In any event, this is unnecessary in most cases because flash_image should overwrite the whole thing. The only exception is when you have an identical header on your image to the one that is already on the device. This shouldn't happen in my instructions (mkbootimg creates a header that includes the build timestamp) but I kept the instruction there just for good measure.
    1
    Is this any different than using the Dalvik Debug Monitor (DDMS) file manager found the the Android SDK? I'm able to push, pull, and delete files on my G1 with no problem.

    The adb push command I gave is no different, but you would still have to unpack/repack and flash_image according to my instructions.