Nandroid v2.0 - Full NAND backup and restore tool (tried and tested!)

[infernix]

* What is Nandroid Backup v2.0?
Nandroid Backup is a set of tools and a script that will enable anyone who has root on their G1 and has the engineering/dev spl bootloader [1] (or has a dev phone) + a recovery image with busybox and adbd running as root [2] to make full system backups. These can then be restored using the fastboot [3] commandline tool and your phone in SPL/bootloader mode (hold camera + power). This allows you to create regular backups of your entire phones NAND flash so that you can restore a backup when something breaks. Its backup method works on a level that allows you to completely break, wipe or corrupt any or all partitions and still restore the phone to a working state within a few minutes. This works great for those who use their G1 for normal activities but also want to test their ideas which might break the phone, or when not having an extra G1 to keep purely as a development phone.

* Requirements for use
- Android G1 phone in recovery mode with busybox installed, including appropriate symlinks to at least tar and md5sum, as well as 'adb shell' support running as root. I recommend using JesusFreke US RC30 or UK RC8 v1.2 recovery image. You can flash the RC30 v1.2 recovery.img on your G1 if you don't want to bother constructing your own image. Of course you are free to use your own recovery image and it will work if you set it up correctly.
- dump_image-arm and mkyaffs2image|mkyaffs2image-$ARCH compiled in the current dir, as well as root on a linux/mac (POSIX) machine for constructing system and data images. root is required because the uid, gid and file permissions need to be preserved while extracting and creating the yaffs2 images. You are free to run the script as a normal user and then run the appropriate commands yourself as root; the commands are printed when you run the script as a regular user.

If you don't trust the provided binaries, you are free to compile them yourself. The source for the tools are included. For dump_image, edit nandtools/android-imagetools/Makefile so that you use the proper cross compiling gcc. For mkyaffs2img, just running make in tartools/yaffs2/utils/ should give you a binary. You are on your own in getting stuff compiled, but it should be easy to figure out from the Makefile.

* How to use this (the easy part)
Just boot your phone in recovery mode with the correct recovery.img (one that has adb shell as root + busybox installed with symlinks to tar, md5sum etc) and connect it to your pc/laptop/server. Then run the nandroid.sh script. If something is wrong or missing, it will tell you.

* What about restore?
If you have the dev phone or have root on the G1 and flashed the engineering/dev bootloader, then you can power on your phone while holding camera to get into the bootloader/SPL. It should then show some androids on skateboards (if not, you don't have the engineering/dev bootloader). Plug in your USB cable and push back until it says FASTBOOT instead of Serial0. Now you can fire up fastboot flash {partition} {file.img}.
Fastboot itself is part of the android SDK. If you download and compile your own mydroid build, it should sit in mydroid/out/host/linux-x86/bin/fastboot Perhaps in a future version I'll include a restore script, but for now I don't see the need.
One tip, if you are in fastboot mode on your phone and 'fastboot devices' doesn't show anything, try as root. You might need to edit your udev setup to allow your user to access the phone's usb device (which is different from the one you're in when in recovery or normal boot).

* Where's the source?
Sources are included. They are copies of the Android sources with various changes mostly made by Brainaid.
Everything in the zip is also also available through subversion:

svn co http://svn.infernix.net/nandroid/

Note that I DO NOT recommend checking out the latest revision unless you know damn well what you are doing. No guarantees if you play with fire. At the very least do diff against the latest released version.

* Credits
Bash script hacked together by infernix.
dump_image adapted from flash_image and extended by Brainaid.
Original mtdutils source and mkyaffs2image source by Google.

[1] http://forum.xda-developers.com/showthread.php?t=455860
[2] http://forum.xda-developers.com/showthread.php?t=443713
[3] http://www.gotontheinter.net/fastboot


DOWNLOAD HERE or:
wget http://dx.infernix.net/nandroid-2.0.zip

 

[infernix]

Originally Posted by trmacdonal

How to Restore your phone on a Windows PC using a Nandroid backup

I am going to assume you already have a Nandroid backup created on your SD card using JF 1.31's recovery Alt-B feature. The backup will create a folder called nandroid on your SD.

What you need:
The Android SDK:
Fastboot Windows Binary in this post:http://forum.xda-developers.com/showpost.php?p=3083753&postcount=1
From your Nandroid backup you need three files:
data.img
system.img
boot.img

Steps to restore your phone
1) Put the files Adb and AdbWinApi.dll from the tools folder in the Android SDK into C:\WINDOWS\System32 folder on your PC. Substitute the correct drive letter if windows is not installed on you C: drive
2) Put the Windows Fastboot.exe into the C:\WINDOWS\System32 folder on your PC
3) Create a folder on the C: drive of your PC called android. The path should be C:\Android.
4) Copy the Nandroid backup files data.img, system.img, and boot.img from your SD card to the folder called Android you created by mounting your SD card as removable disk.
5) Unplug the USB cord and power off your phone
6) Power up your phone by holding CAMERA+POWER, you should see three androids on skateboards. If you don't see this go install the Engineering boot loader
7) Plug the USB cord back into your phone and press back. The screen on your phone should say fastboot.
8) Make sure your pc is using the correct driver. Open the device manager on your pc. It is helpfull to have all other USB storage devices besides your phone unplugged for this part. Look for a USB Mass Storage device in the list of the USB devices. Right click on it and update the driver. Pick the option to browse for a driver on your pc. The driver is located in the Android SDK your downloaded in the folder called usb_driver. If this is done right you will now see a device called HTC Dream
9) Press start, go to run and type cmd (If you are on Vista just type cmd in the search box and hit enter). The command prompt will pop up.
10) Type cd C:\android
then enter the following commands:

fastboot flash system system.img
it will say Sending, then writing and say OKAY if it was successful
then type
fastboot flash userdata data.img
wait for the second OKAY and type
fastboot flash boot boot.img
wait for the second OKAY and type
fastboot reboot

Your phone should now be restored exactly as you had it when it was backed up.

 

[SplasPood]

This sounds great! I just recently used the other script to make a backup, but this sounds like it's been a bit more polished

I attempted to compile the yaffs2 utils under OS X 10.5 and I encountered a problem during linking:

ln -s ../yaffs_ecc.c yaffs_ecc.c
gcc -c -I/usr/include -I.. -O2 -Wall -DCONFIG_YAFFS_UTIL -Wshadow -Wpointer-arith -Wwrite-strings -Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes -Wredundant-decls -Wnested-externs -Winline -static yaffs_ecc.c -o yaffs_ecc.o
gcc -c -I/usr/include -I.. -O2 -Wall -DCONFIG_YAFFS_UTIL -Wshadow -Wpointer-arith -Wwrite-strings -Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes -Wredundant-decls -Wnested-externs -Winline -static mkyaffsimage.c -o mkyaffsimage.o
mkyaffsimage.c:120: warning: ‘yaffs_CalcNameSum’ defined but not used
gcc -o mkyaffsimage yaffs_ecc.o mkyaffsimage.o
ld: absolute addressing (perhaps -mdynamic-no-pic) used in _process_directory from mkyaffsimage.o not allowed in slidable image
collect2: ld returned 1 exit status
make: *** [mkyaffsimage] Error 1


Any thoughts? I'm heading off to google to figure out what a 'slidable image' is.

-jwb

 

[infernix]

No idea, and I don't have access to a mac. You could change the makefile and skip compiling of mkyaffsimage, you only need mkyaffs2image, but you'll probably get similar errors.

 

[SplasPood]

No idea, and I don't have access to a mac. You could change the makefile and skip compiling of mkyaffsimage, you only need mkyaffs2image, but you'll probably get similar errors.

Google seems to imply its a common issue, possibly solved in a later XCode rev.. We'll see how that pans out.

 

[brocktice]

mkyaffs2image on mac

It built just fine for me when I did the whole mydroid build. It's in out/host/darwin-x86/bin.

Is there some way to mount the yaffs2 image, though? Does one need to do the kernel patch etc on a linux box and then use loop?

 

[dcman008]

Wow this a awesome. Thanks for all the hard work everyone.

 

[hbguy]

you are are ridiculous.....ly good!

so you telling me if we have root and jf's modified rc30 v1.2 and be brick our phone, there's another quarter in your pocket after you die (game over) and you can start a whole new game?

hbguy

 

[infernix]

so you telling me if we have root and jf's modified rc30 v1.2 and be brick our phone, there's another quarter in your pocket after you die (game over) and you can start a whole new game?

If you have flashed the engineering bootloader (the androids on skateboards appear when powering on your phone while holding camera button), yes.

 

[hbguy]

haha very nice ::borat voice::

P.S. For the borat fans out there, he is making a new movie, insider info baby

hbguy!

 

[brocktice]

Error messages but still makes backups?

I got the following when I ran nandroid.sh on my mac. Are the error messages safe to disregard -- it seems like it worked:

nandroid v2.0
mounting system and data read-only on device
error: device not found
error: device not found
start adb portforward on port 4531
error: device not found
checking free space on cache
error: device not found
./nandroid.sh: line 152: [: -le: unary operator expected
pushing tools to /cache: dump_image-arm... error: device not found
done
Getting md5sum on device for boot...error: device not found
done ()
Dumping boot from device over tcp to backup_2008-12-19//boot.img...
error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for boot.img
Getting md5sum on device for recovery...error: device not found
done ()
Dumping recovery from device over tcp to backup_2008-12-19//recovery.img...
error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for recovery.img
Getting md5sum on device for misc...error: device not found
done ()
Dumping misc from device over tcp to backup_2008-12-19//misc.img...
error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for misc.img
Getting md5sum on device for tar for system...error: device not found
done ()
Dumping tar file for system backup_2008-12-19//system.tar...error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for system.tar
To convert system.tar to system.img, run the following commands as root:

mkdir /tmp/-system-tmp
tar x -C /tmp/-system-tmp -f backup_2008-12-19//system.tar
/Users/brock/Inbox/nandroid-2.0/mkyaffs2image /tmp/-system-tmp/system backup_2008-12-19//system.img

Make sure that /tmp/-system-tmp doesn't exist befor you extract, or use different paths.
Remember to remove the tmp dirs when you are done.
Getting md5sum on device for tar for data...error: device not found
done ()
Dumping tar file for data backup_2008-12-19//data.tar...error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for data.tar
To convert data.tar to data.img, run the following commands as root:

mkdir /tmp/-data-tmp
tar x -C /tmp/-data-tmp -f backup_2008-12-19//data.tar
/Users/brock/Inbox/nandroid-2.0/mkyaffs2image /tmp/-data-tmp/data backup_2008-12-19//data.img

Make sure that /tmp/-data-tmp doesn't exist befor you extract, or use different paths.
Remember to remove the tmp dirs when you are done.
Getting md5sum on device for tar for cache...error: device not found
done ()
Dumping tar file for cache backup_2008-12-19//cache.tar...error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for cache.tar
To convert cache.tar to cache.img, run the following commands as root:

mkdir /tmp/-cache-tmp
tar x -C /tmp/-cache-tmp -f backup_2008-12-19//cache.tar
/Users/brock/Inbox/nandroid-2.0/mkyaffs2image /tmp/-cache-tmp/cache backup_2008-12-19//cache.img

Make sure that /tmp/-cache-tmp doesn't exist befor you extract, or use different paths.
Remember to remove the tmp dirs when you are done.
removing tools from /cache: dump_image-arm... error: device not found
done
unmounting system and data on device
error: device not found
error: device not found
Backup successful.

 

[infernix]

I got the following when I ran nandroid.sh on my mac. Are the error messages safe to disregard -- it seems like it worked:

nandroid v2.0
mounting system and data read-only on device
error: device not found
[/QUOTE]

This didn't work at all.

What I don't really get is that it continues at all, if the device isn't connected it should not even start.

It's a bit hard to tell because I don't have a mac here. Please run the following commands on your mac and post them:

[code]which adb
adb devices
BLAH=foo
echo "without $BLAH"
echo "with ${BLAH}"

I'll look at this soon, should have access to a mac sometime in the next few weeks

 

[SplasPood]

I got the following when I ran nandroid.sh on my mac. Are the error messages safe to disregard -- it seems like it worked:

nandroid v2.0
mounting system and data read-only on device
error: device not found
error: device not found
start adb portforward on port 4531
error: device not found
checking free space on cache
error: device not found
./nandroid.sh: line 152: [: -le: unary operator expected
pushing tools to /cache: dump_image-arm... error: device not found
done
Getting md5sum on device for boot...error: device not found
done ()
Dumping boot from device over tcp to backup_2008-12-19//boot.img...
error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for boot.img
Getting md5sum on device for recovery...error: device not found
done ()
Dumping recovery from device over tcp to backup_2008-12-19//recovery.img...
error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for recovery.img
Getting md5sum on device for misc...error: device not found
done ()
Dumping misc from device over tcp to backup_2008-12-19//misc.img...
error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for misc.img
Getting md5sum on device for tar for system...error: device not found
done ()
Dumping tar file for system backup_2008-12-19//system.tar...error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for system.tar
To convert system.tar to system.img, run the following commands as root:

mkdir /tmp/-system-tmp
tar x -C /tmp/-system-tmp -f backup_2008-12-19//system.tar
/Users/brock/Inbox/nandroid-2.0/mkyaffs2image /tmp/-system-tmp/system backup_2008-12-19//system.img

Make sure that /tmp/-system-tmp doesn't exist befor you extract, or use different paths.
Remember to remove the tmp dirs when you are done.
Getting md5sum on device for tar for data...error: device not found
done ()
Dumping tar file for data backup_2008-12-19//data.tar...error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for data.tar
To convert data.tar to data.img, run the following commands as root:

mkdir /tmp/-data-tmp
tar x -C /tmp/-data-tmp -f backup_2008-12-19//data.tar
/Users/brock/Inbox/nandroid-2.0/mkyaffs2image /tmp/-data-tmp/data backup_2008-12-19//data.img

Make sure that /tmp/-data-tmp doesn't exist befor you extract, or use different paths.
Remember to remove the tmp dirs when you are done.
Getting md5sum on device for tar for cache...error: device not found
done ()
Dumping tar file for cache backup_2008-12-19//cache.tar...error: device not found
done
Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory
md5sum verified for cache.tar
To convert cache.tar to cache.img, run the following commands as root:

mkdir /tmp/-cache-tmp
tar x -C /tmp/-cache-tmp -f backup_2008-12-19//cache.tar
/Users/brock/Inbox/nandroid-2.0/mkyaffs2image /tmp/-cache-tmp/cache backup_2008-12-19//cache.img

Make sure that /tmp/-cache-tmp doesn't exist befor you extract, or use different paths.
Remember to remove the tmp dirs when you are done.
removing tools from /cache: dump_image-arm... error: device not found
done
unmounting system and data on device
error: device not found
error: device not found
Backup successful.

I had this problem too but disconnecting and reconnecting the usb cable fixed it.

 

[infernix]

I had this problem too but disconnecting and reconnecting the usb cable fixed it.

Well the script is supposed to check for a device and stop when theres a problem.

also if this can happen at all then other things might go wrong on mac too. I think mac requires bash variables to be called like #{VAR} and doesn't handle $VAR for some reason.

 

[brocktice]

update on mac issues

OK, it seems it was all a problem on my end -- didn't have the correct set-up. I now have the appropriate recovery image (thought I did, but I guess I didn't) and it seems to work, except for this:

Comparing md5sum...parseopts.c:76: setup_check: fopen '--status': No such file or directory

The images and such now have non-zero filesize, though. That's an improvement over before.

 

[infernix]

yeah i've gotten some more reports. will make it mac OS X compatible soonish

 

[Phromik]

I can't seem to figure out how to get this to work. If someone who knows linux can IM me, that'd be great. (or is there a way to do this on XP?)
AIM:adventninja
Thanks in advance

 

[jockyw2001]

Does this also back up radio and htc secure nand partitions?

 

[unrafa]

tranks

backed up successfully.

for those who get errors in the script.
i assume you have installed modiffied recovery image + jesusfreke's rc30v2 and
before running the script check this.
this steps involves having linux i dont know how its done in windows maybe its the same i dont know.

1: disconnect usb cable
2: check that you have adb program in your path, eg: /usr/bin/adb
if you dont know what adb is, adb is a program to comunicate with phone via usb (its not the same as transfering files between pc and g1),

adb program is found in android-sdk-linux_x86-1.0_r2, unpack it, and copy adb to your path, just google and find latest android sdk.

once you downloaded sdk unpacked it and copied the executable file adb to /usr/bin go to next step.

3: boot the phone into recovery mode ( power up the phone and holding "power + home" keys and a triangle will appear on its screen.

4: in your pc open konsole, su root, and plug your USB G1

5: type adb devices

if the response to this command is "HTC G1 bla bla.. found.", then you are ready to execute the script, it will no give errors and you will backup its contents.

if you dont see HTC G1 words, then its likely your phone was detected as "USB MASS STORAGE", and we dont want that. we need that its detected as a "HTC DIRECT ACCESS DEVICE".

a: control +c
b: killall adb (this is important)
c: unplug USB, plug it in another port,
d: go to step 5


hope it helps.

 

[korndub]

* What is Nandroid Backup v2.0?
Nandroid Backup is a set of tools and a script that will enable anyone who has root on their G1 and has the engineering/dev spl bootloader [1] (or has a dev phone) + a recovery image with busybox and adbd running as root [2] to make full system backups. These can then be restored using the fastboot [3] commandline tool and your phone in SPL/bootloader mode (hold camera + power). This allows you to create regular backups of your entire phones NAND flash so that you can restore a backup when something breaks. Its backup method works on a level that allows you to completely break, wipe or corrupt any or all partitions and still restore the phone to a working state within a few minutes. This works great for those who use their G1 for normal activities but also want to test their ideas which might break the phone, or when not having an extra G1 to keep purely as a development phone.

* Requirements for use
- Android G1 phone in recovery mode with busybox installed, including appropriate symlinks to at least tar and md5sum, as well as 'adb shell' support running as root. I recommend using JesusFreke US RC30 or UK RC8 v1.2 recovery image. You can flash the RC30 v1.2 recovery.img on your G1 if you don't want to bother constructing your own image. Of course you are free to use your own recovery image and it will work if you set it up correctly.
- dump_image-arm and mkyaffs2image|mkyaffs2image-$ARCH compiled in the current dir, as well as root on a linux/mac (POSIX) machine for constructing system and data images. root is required because the uid, gid and file permissions need to be preserved while extracting and creating the yaffs2 images. You are free to run the script as a normal user and then run the appropriate commands yourself as root; the commands are printed when you run the script as a regular user.

If you don't trust the provided binaries, you are free to compile them yourself. The source for the tools are included. For dump_image, edit nandtools/android-imagetools/Makefile so that you use the proper cross compiling gcc. For mkyaffs2img, just running make in tartools/yaffs2/utils/ should give you a binary. You are on your own in getting stuff compiled, but it should be easy to figure out from the Makefile.

* How to use this (the easy part)
Just boot your phone in recovery mode with the correct recovery.img (one that has adb shell as root + busybox installed with symlinks to tar, md5sum etc) and connect it to your pc/laptop/server. Then run the nandroid.sh script. If something is wrong or missing, it will tell you.

* What about restore?
If you have the dev phone or have root on the G1 and flashed the engineering/dev bootloader, then you can power on your phone while holding camera to get into the bootloader/SPL. It should then show some androids on skateboards (if not, you don't have the engineering/dev bootloader). Plug in your USB cable and push back until it says FASTBOOT instead of Serial0. Now you can fire up fastboot flash {partition} {file.img}.
Fastboot itself is part of the android SDK. If you download and compile your own mydroid build, it should sit in mydroid/out/host/linux-x86/bin/fastboot Perhaps in a future version I'll include a restore script, but for now I don't see the need.
One tip, if you are in fastboot mode on your phone and 'fastboot devices' doesn't show anything, try as root. You might need to edit your udev setup to allow your user to access the phone's usb device (which is different from the one you're in when in recovery or normal boot).

* Where's the source?
Sources are included. They are copies of the Android sources with various changes mostly made by Brainaid.
Everything in the zip is also also available through subversion:

svn co http://svn.infernix.net/nandroid/

Note that I DO NOT recommend checking out the latest revision unless you know damn well what you are doing. No guarantees if you play with fire. At the very least do diff against the latest released version.

* Credits
Bash script hacked together by infernix.
dump_image adapted from flash_image and extended by Brainaid.
Original mtdutils source and mkyaffs2image source by Google.

[1] http://forum.xda-developers.com/showthread.php?t=455860
[2] http://forum.xda-developers.com/showthread.php?t=443713
[3] http://www.gotontheinter.net/fastboot


DOWNLOAD HERE or:
wget http://dx.infernix.net/nandroid-2.0.zip

hey infernix, did you happen to get a mac version of this together? thanks!