[GUIDE] How to extract, create or edit android adb backups

What is an android adb backup?
An adb backup is a file with ab extension, generated by android's backup manager when we request it via adb shell. This allows you to backup some data of the phone, but is not a replacement of a clockworkmod backup:

- Java 7 or higher is required because of SYNC_FLUSH mode for the Deflater.
- If the backup contains apk+data for an app, restore will work. If contains only the data, you must install the app first on the device and then restore. Installing the app later won't work.
- Some apps include a policy where the apk is never backed up even if specified.
- Inside an ab file is a tar file, which contains files and folders in a certain order. You have you respect that order, which is not necessarily alphabetical like tar does by default. Such order is listed in "Full local backup infrastructure".
- Inside the tar file, directories must not have trailing slash, for that reason pax, star or equivalent has to be used.
- There are some bugs present in the android source code.
- 'adb backup' or 'adb restore' are pretty slow, between 1 and 2.6 MBps.
- The adb backups (usually with .ab extension) can be password protected or not. If the device is encrypted this is a must and has to be the same used for the device.
- Star is recommended instead of pax, which supports path length only up to 100 characters, so will fail in some cases. This usually happens with browsers history.
- ADB backups are not a replacement for a Nandroid backup. The whole /data partition is not backed up, only a part of it. Also, other partitions like /system, /preload, /cache, modem, RIL, efs, kernel or recovery are never backed up because are not user data. This is to prevent issue when restoring on a different device. It has also the biggest
- If you have the device encrypted with a password, you must use that particular one for backup creation and restore. You can't create a backup without password or a different one in that particular case, or if you try to restore will fail.
- star for cygwin for windows is attached (move it to C:\cygwin\bin\star.exe) since there is no package available. It can be compiled from the schily (ftp://ftp.berlios.de/pub/schily/).
- The best way to test if an adb backup has errors, is to convert it to tar and then check.advantage that doesn't require root to operate, so is totally compatible with stock roms, locked bootloaders and device encryption.
- There's also a bash script called adb-split.sh that creates individual adb backups for each recognized app from the full one, so you can restore apps individually. Same encryption is preserved to them, if any.

You can use java or perl, although with perl can be more complicated because requires downloading some modules from cpan and some ssl headers.

To know more information about types of android backups:
[Guide] Types of Android backups

Software needed
The entire process can be done from Windows, but is better to use a unix-like operating system, like Linux, OS X or BSD, since we should extract the files on a filesystem that preservers file permissions and owners and repack the tar archive. Furthermore some tools like star are easier to get for linux. In such cases, using Virtualbox or VMWare Workstation is highly recommended. Using Windows should work in any case whatsoever.

Cygwin
Preferably 32 bits: has more packages and is less prone to errors. Furthermore, the star for cygwin attached is 32 bits.

Java 7 or higher

• Java 7 or OpenJDK 7 (more builds here)

• Oracle Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 7 if you are going to work with password encrypted backups.
  You need to install the files local_policy.jar and US_export_policy.jar under jre's lib/security folder, for example:
  - For Windows:
  • C:\Program Files\Java\jdk1.7.0_09\jre\lib\security\
  • C:\Program Files\Java\jre7\lib\security\
  • C:\Program Files (x86)\Java\jdk1.7.0_07\jre\lib\security\
  • C:\Program Files (x86)\Java\jre7\lib\security\
  - For Linux or BSD:
  /usr/local/jdk1.7/jre/lib/security/
  /usr/lib/jvm/java-7-openjdk-*/jre/lib/security/
  /usr/local/openjdk7/jre/lib/security/
  - For OS X:
  • /Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home/jre/lib/security/

Perl

• Perl is available for several operating systems
• libssl or openssl headers. If you are using Linux or Cygwin is much more easier
• cpan modules required by use functions

pax or star
pax is an archiving utility that compresses in ustar by default. In this case is extremely useful because stores directories without trailing slash. Is available for all operating systems, and in windows can be installed via cygwin's setup.exe or download a native version like gnuwin32 (sometimes available via bsdtar with pax option). pax doesn't work when paths are more than 100 characters lenght, so I recommend star instead.

star allows storing directories without trailing slash. You can get the ubuntu version 1.5 here. For Windows you can use Cygwin's version included.

Android Backup Extractor
Android Backup Extractor is the java application that does all the job. It includes the perl scripts.


How it works, better with an example
I will use an example to demonstrate how it works, with java version. If you like to use perl, just grab the perl scripts and is nearly the same.
This is extracted from the readme file, and will extract a whole adb backup and repack only the data for the game Grand Theft Auto III for android.

1) Convert the original adb backup to tar format:

2) Extract the contents of the tar archive. This should be done on a filesystem where the permissions of the files inside the tar are preserved, for example using linux, mac or bsd. Up to two folders may appear, apps and shared:

3) Make a list of all the contents of the original archive in the order they are archived:

4) Create a new list only with the files and folders you want, in proper order. For example for the GTA 3 (you can try savegames instead of all data):

5) Create the new tar archive. The directories stored on tar shouldn't contain trailing slashes, so I use pax or star instead of tar. Pax works also if paths are up to 100 of lenght:
OR

6) Create the adb backup from the tar archive. Password is optional:

Note: if the backup is not encrypted zlib can be used instead for both unpack and pack the ab archive:
- Quick unpacking:
- Quick packing:

Split an Android backup
Use Android Backup Splitter (is a shell script). Works on Cygwin too:
sh adb-split.sh backup.ab [password if needed]
Resulting files go in app-ab folder

# ADB Backup Splitter:
#
# This bash script utility generates an adb backup for each item
# on apps/ folder. That equals to generating an adb backup for
# each app.
#
# shared/0 and shared/1 (sdcards) are ignored, so that means that
# external data and OBB files, although may be backed up, may not
# be restored. For doing so backup the folder Android of the sdcards.
# Creating a backup with -shared flag has know issues (corruption)
# so is not recommended. Future releases of this script may add
# the option to automatically detect things from sdcards inside
# the adb backup file.
#
# This script works on Cygwin also, and in general where star does.
#
# There may be apps in the backup that don't include the apk. In that
# particular case, the data won't be restored, so the individual backup
# is generated for testing purposes only. Installing the apk afterwards won't work.
# There is generated an html file called apk-missing.html so you can open it and
# install all applications from Play Store on any of your devices.

Patch apps to allow adb backup
Some apps specify in the AndroidManifest.xml android:allowBackup="false" so it won't be backed up by adb backup. If you have root and use Titanium Backup or CWM or TWRP they will do the job, but if you don't want root the only solution is to modify the apk to edit that string. Read here how to do that and the pros and cons:
[GUIDE] How to enable adb backup for any app changing android:allowBackup

Links
Android Backup Splitter (@ Sourceforge)
Android Backup Extractor (@ Sourceforge)
Android Backup Extractor (@ GitHub)
Documentation at Blogspot
Full local backup infrastructure
[Guide] Types of Android backups
Backing Up Android Games
Titanium Backup Decrypter
Perl scripts to encrypt/decrypt adb backup files
[GUIDE] Full Phone Backup without Unlock or Root
[GUIDE] Internal Memory Data Recovery - Yes We Can!
[Q] [adb] backup not working for Temple Run

Related adb backup and restore bugs:
Issue 55860: adb backup skips the apk for certain apps and doesn't prompt for download when restoring
Issue 48126: device adb connecting to localhost adbd cannot execute commands anymore since 4.2.2 (error device offline)
Issue 3254: adb shell doesn't return exit code of program
Issue 53065: Debugging via ADB stopping the onClick action on ImageView widget
Issue 53668: ADB : An existing connection was forcibly closed by the remote host
Issue 54781: adb install: single quotes in filenames cause quoting issues
Issue 55173: (Game) Offroad Legends does not detect obb when restored via adb restore
Issue 55178: (Game) Carmageddon Promo does not detect obb when restored via adb restore
Issue 28303: adb backup doesn't respect -noshared flag
Issue 32830: adb restore errors not displayed on device
Issue 34311: Galaxy Nexus gets stuck when restoring adb backup
Issue 25780: BackupManager causes reboot when BackupAgent missing
Issue 40691: ab backup and restore Ski Safari does not restore saved data
Issue 16286: Restoration of phone not working properly
Issue 39014: Nexus 7 adb restore freezes when restoring udk.android.reader

Warning about Helium (Carbon): helium backups are just android adb backups that are not compressed with Java Deflater. It means they were created with the option static final boolean COMPRESS_FULL_BACKUPS = false.
A normal adb restore should work.

ball3t said:

I have tried similar attempt to restore my save of another game but that doesn't work.
How did you replace the files in the .tar file? I am using 7z to replace the .tar file but nothing happen after i restored.
Thanks.

Click to expand...

Click to collapse

Read the guide. 7z is not compatible, you have to use star or equivalent and meet the requirements.

Great guide, and can confirm that it does work!

I managed to use this method to restore my angry birds data from a Froyo phone (HTC Desire), to a Jelly Bean phone (HTC One X+). This is no easy task since you cannot use adb backup/restore on Froyo, but you can use adb pull, and you can't use adb push on Jelly Bean, but you can use adb backup/restore. Just in case anyone is looking to do the same, here is the process I used (requires having adb up and running but does not require root)...

As the adb backup function doesn't work on Froyo, I had to use adb pull to grab the angry birds data files (highscores.lua and settings.lua). Then, making sure that you have angry birds installed on the jelly bean phone and have played the first level, use the adb backup command to backup just the angry birds app. You can then use scandiuns guide to extract a tar file from the backup file, and then extract the "app" folder from the tar file.

Once this is done, you will find the highscores.lua and settings.lua files in the apps/com.rovio.angrybirds/f directory. Replace these two files with your original ones that you pulled from the old device. Then package up the app folder using the guide. Then just use adb restore to restore the modified backup file to your new device, and you should have all your old scores restored!

Hope that helps and thanks to scandiun for the great guide! :good:

AAhrens said:

Maybe I'm missing something.

I've installed Java, ran cygwins setup.exe and I think it installed everything. Also downloaded the Android Backup Extractor. I try running a windows command prompt with:
java -jar abe.jar unpack nexus7.ab nexus7.tar

All I get is "java is not an internal or external command". I'm running the command prompt in the android backup extractor folder. Does everything have to be in the same folder? Where do I dump everything?

Click to expand...

Click to collapse

Probably java is not in your path, try


It should result something like

Add Java to the Windows Path

Then restart cygwin

ball3t said:

Because i am using windows, so does it mean that i couldn't modify the .tar file in any way?
thanks.

Click to expand...

Click to collapse

Yes, from windows you can use pax. You can download it from cygwin's setup.exe. I've updated the guide, see the step 5.