[CWM Scripts] Relatively painless backup/mod/restore scripts

---

Ok, these are specifically for MIUI, but they can easily be adapted to any other rom by editing the shell scripts. I'll explain how they work and it should be fairly apparent how one would need to modify for their particular rom/needs. Don't know if anyone will actually use these, but I thought I'd share. I also think it's a really good example of how CWM can be used for more than just flashing stuff.

So I've been running MIUI for a while now and I'm really happy. One little issue that popped up was every Friday when the rom was updated, the OTA update app icon gained a little "1" badge, indicating an update was available. While it's nice to be notified of the update, I occasionally didn't have the time or interest in updating so the little badge remained. I put that app in a folder, but then the folder had the little 1 badge. After a while, I started deleting the update app after install since I could always download and install updates through ROM Manager just as easily. Then I started deleting the update app from the downloaded ROM zip, along with several other apps I didn't want (Facebook, Twitter, the MIUI FileExplorer, etc) before flashing it. But this entailed downloading the Friday update on my phone, transferring it to my desktop, opening the zip in 7 Zip, deleting the unwanted files, uploading it back to my phone and then flashing. Quite a PITA.

Another issue I ran into was that the built-in MIUI backup app failed to backup my SMS/MMS messages. Actually, I don't know if it was failing to back them up, or failing to properly restore them, but at the end of the day, it wouldn't keep my text messages. I know I could avoid this by just installing each update over the current ROM, but I've always believed cleanly installing an OS helps avoid issues. I prefer to wipe everything, install clean and then restore my data.

So, the solution(s) I came up with was to mod the downloaded ROM zip file right on the phone, and manually backup and restore my data. While it took a little while to get set up, I find it actually works considerably faster and smoother than using MIUI's backup/restore app. I now have 3 simple shell scripts that are run through Clockwork Recovery just like installing any zip - one to mod the ROM after downloading, and two to backup and restore my data. Here they are:

MIUI-Mod.zip

This contains a shell script and a zip binary to handle the adding and deleting of files to/from the downloaded rom zip. When run through CWM, it copies zip and the script to /tmp and runs the script. Because the ROM zip is named differently every Friday, I figured the easiest solution was just to rename it manually. So, this script assumes the ROM zip is named "miui.zip" and is located in the root of the SD card. It's a simply matter to download the ROM on Friday, move it to /sdcard and rename it miui.zip. Here's what the script does in 2 commands:

/tmp/zip -d /sdcard/miui.zip /data/app/Twitter.apk /data/app/Facebook.apk /system/app/Email.apk /system/app/FileExplorer.apk /system/app/Gmail.apk /system/app/MIUI.us_Forums_1.1.2_AdSupported.apk /system/app/Superuser.apk /system/app/Updater.apk /system/app/MIUIStats.apk /system/media/lockscreen/* /system/media/wallpaper/*

/tmp/zip /sdcard/miui.zip /system/app/Gmail.apk

The first line simply opens the zip and deletes the listed files: Twitter, Facebook, Email, etc. It also removes all the wallpapers for the lock and main screens (I don't use them, so I didn't want them, but it's a good example of the use of wildcards like *).

The second line adds in the Gmail.apk from my current ROM. The latest versions of the Gmail app included in MIUI lock the mail database, which conflicts with a widget I use, so I just replace it with an older version. Every time I run the script, it removes the included Gmail app and adds in my current one. If you didn't want to add anything to the ROM, you could just delete this line entirely.

Data-Backup.zip

This just runs a shell script that creates a tar file in the root of your sdcard containing the listed files and directories. Tar files retain file permissions and owner information, so everything can be restored seamlessly. Here's how it works, it's pretty simple:

tar -c -f sdcard/backup.tar data/app/com.adobe.flashplayer-1.apk data/data/com.adobe.flashplayer data/app/com.speedsoftware.rootexplorer-1.apk data/data/com.speedsoftware.rootexplorer data/app/com.touchtype.swiftkey-1.apk data/data/com.touchtype.swiftkey data/data/com.android.browser data/data/com.android.deskclock data/data/com.android.launcher data/data/com.android.mms data/data/com.android.phone data/data/com.android.providers.contacts data/data/com.android.providers.settings data/data/com.android.providers.telephony data/data/com.android.providers.userdictionary data/data/com.android.thememanager data/data/com.android.vending data/data/com.google.android.gsf data/data/com.miui.camera data/data/com.miui.notes data/data/com.noshufou.android.su data/system data/misc/wifi

This creates the tar file and includes Adobe's Flash player and it's data, Root Explorer and it's data, and SwiftKey and it's data (I paired it down to just these three for this example). After that, it backs up a bunch of other data directories which include the current Browser settings, alarms, launcher settings and home screen layouts including widgets, phone logs, the current theme settings, any MIUI notes, current Wifi connections and, yes, SMS and MMS messages.

Obviously, you'd want to add in your specific apps and data directories. Sure, it takes a little work to set this up, but as I said, backing up is very quick and easy afterwards.

Data-Restore.zip

This just runs a shell script that restores all the files from the backup.tar on your SD card:

tar -xf /sdcard/backup.tar

Doesn't need much explanation.

So, the process is:

1-Download the Friday update through ROM Manager, move it to the root of my SD card and rename it to miui.zip
2-Reboot into CWM and run Backup-Data.zip to back up my apps and data
3-Run MIUI-Mod.zip to modify the downloaded MIUI rom
4-Wipe everything (cache/data/system)
5-Install the modded MIUI rom
6-Run Data-Restore to restore apps/data
7-Reboot

It's literally about 5 minutes after downloading the ROM and I'm updated and rebooted with a CLEAN os install and all my apps, data and settings intact, my home screen layouts, keyboard, wallpapers, wifi settings and everything retained.

Some notes:

1-Before running these in CWM, you have to make sure you have system, data and sdcard mounted since the scripts will be reading and/or writing to all of these. When you first reboot to CWM, just go to mounts and manually mount each one. The scripts could certainly be set up to mount them automatically, but mount points vary from phone to phone, and it's not difficult to mount through CWM. Also, after flashing a ROM, the ROM install script may unmount some of these, so before you restore your data, you might have to manually remount.
2-If you're adding anything from your current ROM to the zip (like how I add in my current Gmail.apk), make sure you wipe everything *after* running the mod script, otherwise you're deleting everything and then trying to copy file(s) that were just deleted.
3-I would highly recommend using Notepad++ to edit scripts, since it properly saves Unix line breaks.
4-as always, use at your own risk and a full CWM backup is highly recommended before messing with these.

That's pretty much it. Again, it does take a little while to get set up, but once it is, it's really simple. If you install a new app between updates, you just have to edit the backup script, or you can just keep the scripts maintaining a base setup and reinstall various apps through the market each time. This specific process is probably most useful for ROMs that are updated regularly - nightly or weekly, although I'm sure people can find other uses for running scripts through CWM.

Hope someone might find these useful.

This is really cool. I run Miui and would really like to be able to do a fresh install each week, but hate the hassle of resetting up my phone, even using Titanium backup, it's still a time consuming process. So needless to say, I flash the new rom over top of the old version without wiping just to save time.
So, basically I can open up the data restore zip in 7 zip, add in a few of my apps/data such as Tasker and so forth, close it and flash? If so, that will be great.
Thanks for sharing this!!!

You don't need to touch the Data-Restore.zip, all that does is extract the backup.tar file created with the backup script. What you'd do is open the Data-Backup.zip and edit the backup file (it's just a text file). Add in any of your apps and their corresponding data directories and when you flash the Data-Backup.zip in CWM, it'll back up those files/folders.

To find the files you want to backup, use something like Root Explorer and look in /data/app. There you'll find all the 3rd party apps you've installed. Add each one to the tar command in the backup file just like the ones I included. Then go into /data/data and find each app you want to back up's corresponding data folder. Add them to the tar command too.

Save the file, re-zip it and when you flash it through CWM, it'll back up the new files.

Just what i wanted

---

I also always install everything from clean system so setting up everything is a pain.
Would this work for Internal storage? I have a Motorola RAZR XT910 and running JBX roms.

Google does install all the apps when i sign in into the new rom but then i always had to move each folder to the correct destination manually using some file explorer. was thinking of making something like this but didnt know how to start. Thanks, I will now try and mod these to suit my phone