FORUMS
Remove All Ads from XDA

[GUIDE] How to extract, create or edit android adb backups

1,701 posts
Thanks Meter: 914
 
By scandiun, XDA Ad-Free Senior Member on 24th November 2012, 10:51 PM
Post Reply Email Thread
11th February 2013, 10:55 PM |#21  
Junior Member
Thanks Meter: 0
 
More
Quote:
Originally Posted by scandiun

On Cygwin you have cat and the other utilities you have to install them from the setup.

I thought I did, as the tar command works fine. I think cygwin installed them all, but I will check.
 
 
18th February 2013, 05:20 PM |#22  
Junior Member
Thanks Meter: 0
 
More
Quote:
Originally Posted by scandiun

What is an android adb backup?


5) Create the new tar archive. The directories stored on tar shouldn't contain trailing slashes, so I use pax or star instead of tar:

Code:
cat gta.list | pax -wd > gta3.tar
OR
Code:
star -c -v -f gta3.tar -no-dirslash list=gta3.list

Well,

I have made progress up to step 5 trying to transfer Angry Birds scores to my Jelly Bean phone. Following the tutorial I decided to use the pax option in Cygwin with this command:
cat ab.list | pax -wd > ab.tar
having created the .tar archive and .list file before as described in the tutorial.

But when I use the command pax gives me the following error on every file on the list:
pax: ustar header field is too small for apps/com.rovio.angrybirds/f/settings.lua
(this is just one instance of the error; the archive consists of many more files which all create the same error)

I have Googled the error but can't seem to find the solution.
Any idea?

Maybe I could use the star option, but did not find a version of star that works on my Mint installation yet.

Thanks
18th February 2013, 09:25 PM |#23  
Junior Member
Thanks Meter: 0
 
More
Quote:
Originally Posted by ttv

Well,

But when I use the command pax gives me the following error on every file on the list:
pax: ustar header field is too small for apps/com.rovio.angrybirds/f/settings.lua
(this is just one instance of the error; the archive consists of many more files which all create the same error)

I have Googled the error but can't seem to find the solution.
Any idea?

Maybe I could use the star option, but did not find a version of star that works on my Mint installation yet.

Thanks

Ok, so I managed to solve this one myself. The problem was with the pax I installed in Cygwin. It gave me the error. I used the Mint VM for the file system that keeps the permissions in tact, but I couldn't find a star-package for it. But then I thought: "Could there be a pax-package for Mint?". Surprise: there is! I followed the guide to the end and guess what: it works! My beloved Angry Birds and Angry Birds Seasons scores have been restored.

Thanks for the help everyone.

Now Rovio: solve this please; I guess there are a lot of sad players with new phones out there.
18th February 2013, 11:25 PM |#24  
scandiun's Avatar
OP Senior Member
Thanks Meter: 914
 
More
Quote:
Originally Posted by ttv

Well,

I have made progress up to step 5 trying to transfer Angry Birds scores to my Jelly Bean phone. Following the tutorial I decided to use the pax option in Cygwin with this command:
cat ab.list | pax -wd > ab.tar
having created the .tar archive and .list file before as described in the tutorial.

But when I use the command pax gives me the following error on every file on the list:
pax: ustar header field is too small for apps/com.rovio.angrybirds/f/settings.lua
(this is just one instance of the error; the archive consists of many more files which all create the same error)

I have Googled the error but can't seem to find the solution.
Any idea?

Maybe I could use the star option, but did not find a version of star that works on my Mint installation yet.

Thanks

The problem is that the GID / UID is just too big to be compatibly recorded, so pax blows chunks. Example of error:
https://trac.macports.org/ticket/28965#comment:7

I'm glad that you finally made it work. star can be compiled under cygwin but takes a lot of time, getting the schily bundle.
20th February 2013, 11:28 AM |#25  
Junior Member
Thanks Meter: 0
 
More
I see. But the version of pax that I installed with Mint 13 works ok. So it seems to be related to the version that gets installed with Cygwin. Is the link you are referring to about the Cygwin version? Not sure what the subject of Macports-website is. Still a noob in Linux-land, although I've learned a lot by following this guide
20th February 2013, 11:25 PM |#26  
scandiun's Avatar
OP Senior Member
Thanks Meter: 914
 
More
Quote:
Originally Posted by ttv

I see. But the version of pax that I installed with Mint 13 works ok. So it seems to be related to the version that gets installed with Cygwin. Is the link you are referring to about the Cygwin version? Not sure what the subject of Macports-website is. Still a noob in Linux-land, although I've learned a lot by following this guide

I'm not sure but is possible that owner is coded different when you extract the tar files under linux or under windows. You can see all the information with this

Code:
ls -l apps/com.rovio.angrybirds/f/settings.lua
4th April 2013, 10:06 PM |#27  
Member
Thanks Meter: 7
 
More
Thumbs up
Quote:
Originally Posted by scandiun

What is an android adb backup?

An adb backup is a file with ab extension, generated by android's backup manager when we request it via adb shell. This allows you to backup some data of the phone, but is not a replacement of a clockworkmod backup:

- Each application files may be backed up or not depending on its policies. In some cases, the apk itself is never backed up even if requested.
- Usually for restoring an application's data it requires you to have it installed first.
- Inside an ab file is a tar file, which contains files and folders in a certain order. You have you respect that order in order to create a valid tar file.
- Inside the tar file, directories must not have trailing slash, for that reason pax, star or equivalent has to be used.
- There are some bugs present in the android source code.
- 'adb backup' or 'adb restore' are pretty slow, no matter what computer or phone you have, because the android debugging bridge has a maximum internal speed of about 1 MBps. So, each gigabyte can take about 15 minutes to backup or restore.
- If you have the device encrypted with a password, you must use that as well when your device asks to confirm backup creation or restoration. You can't create a backup without password in this particular case.

You can use java or perl, although with perl can be more complicated because requires downloading some modules from cpan and some ssl headers.

To know more information about types of android backups:
[Guide] Types of Android backups

Software needed
The entire process can be done from Windows, but is better to use a unix-like operating system, like Linux, OS X or BSD, since we should extract the files on a filesystem that preservers file permissions and owners and repack the tar archive. Furthermore some tools like star are easier to get for linux. In such cases, using Virtualbox or VMWare Workstation is highly recommended. Using Windows should work in any case whatsoever.

Java 7 or higher
  • Oracle Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files 7 if you are going to work with password encrypted backups.
    You need to install the files local_policy.jar and US_export_policy.jar under jre's lib/security folder, for example:
    - For Windows:
    • C:\Program Files\Java\jdk1.7.0_09\jre\lib\security\
    • C:\Program Files\Java\jre7\lib\security\
    • C:\Program Files (x86)\Java\jdk1.7.0_07\jre\lib\security\
    • C:\Program Files (x86)\Java\jre7\lib\security\
    - For Linux or BSD:
    /usr/local/jdk1.7/jre/lib/security/
    /usr/lib/jvm/java-7-openjdk-*/jre/lib/security/
    /usr/local/openjdk7/jre/lib/security/
    - For OS X:
    • /Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home/jre/lib/security/

Perl
  • Perl is available for several operating systems
  • libssl or openssl headers. If you are using Linux or Cygwin is much more easier
  • cpan modules required by use functions

pax or star
pax is an archiving utility that compresses in ustar by default. In this case is extremely useful because stores directories without trailing slash. Is available for all operating systems, and in windows can be installed via cygwin's setup.exe or download a native version like gnuwin32 (sometimes available via bsdtar with pax option).

star allows storing directories without trailing slash. You can get the ubuntu version 1.5 here. There is no version of star for Windows or Cygwin, and other versions like tar, gtar or bsdtar don't implement the option. So, the only option is to user another operating system, installing it or running it from VMWare Workstation or Virtualbox, and share a folder. See here images.

Android Backup Extractor
Android Backup Extractor is the java application that does all the job. It includes the perl scripts.


How it works, better with an example
I will use an example to demonstrate how it works, with java version. If you like to use perl, just grab the perl scripts and is nearly the same.
This is extracted from the readme file, and will extract a whole adb backup and repack only the data for the game Grand Theft Auto III for android.

1) Convert the original adb backup to tar format:
Code:
java -jar abe.jar unpack nexus7.ab nexus7.tar <password>
2) Extract the contents of the tar archive. This should be done on a filesystem where the permissions of the files inside the tar are preserved, for example using linux, mac or bsd. Up to two folders may appear, apps and shared:
Code:
tar -xvf nexus7.tar
3) Make a list of all the contents of the original archive in the order they are archived:
Code:
tar -tf nexus7.tar > nexus7.list
4) Create a new list only with the files and folders you want, in proper order. For example for the GTA 3 (you can try savegames instead of all data):
Code:
cat nexus7.list | grep com.rockstar.gta3 > gta3.list
5) Create the new tar archive. The directories stored on tar shouldn't contain trailing slashes, so I use pax or star instead of tar:
Code:
cat gta.list | pax -wd > gta3.tar
OR
Code:
star -c -v -f gta3.tar -no-dirslash list=gta3.list
6) Create the adb backup from the tar archive. Password is optional:
Code:
java -jar abe.jar pack gta3.tar gta3.ab <password>
Note: if the backup is not encrypted zlib can be used instead for both unpack and pack the ab archive:
- Quick unpacking:
Code:
dd if=nexus7.ab bs=24 skip=1 | openssl zlib -d > nexus7.tar
- Quick packing:
Code:
dd if=nexus7.ab bs=24 count=1 of=gta3.ab ; openssl zlib -in gta3.tar >> gta3.ab
Links
Android Backup Extractor (@ Sourceforge)
Android Backup Extractor (@ GitHub)
Documentation at Blogspot
[Guide] Types of Android backups
Perl scripts to encrypt/decrypt adb backup files
[GUIDE] Full Phone Backup without Unlock or Root
[GUIDE] Internal Memory Data Recovery - Yes We Can!

Related adb backup and restore bugs:
Issue 28303: adb backup doesn't respect -noshared flag
Issue 32830: adb restore errors not displayed on device
Issue 34311: Galaxy Nexus gets stuck when restoring adb backup
Issue 25780: BackupManager causes reboot when BackupAgent missing
Issue 40691: ab backup and restore Ski Safari does not restore saved data
Issue 16286: Restoration of phone not working properly

Well firstly, one massive thumbs up, I now can migrate my Angry Birds scores

I do however have 2 questions
  1. Is PAX installed as a default option in Cygwin? - only I am writing a tutorial for another site and I want to keep the instructions as simple as possible - and I cannot remember if I left it as "default" or "installed" when I searched for it on the Cygwin Installer.
  2. Is it possible to run all the command from 1 terminal? - again this is for simplicites sake, at the moment undr windows I am running the Java command under CMD and everything else under Cygwin.

Thanks again.
5th April 2013, 09:05 AM |#28  
scandiun's Avatar
OP Senior Member
Thanks Meter: 914
 
More
Pax is not included in Cygwin base install. You can run all commands from Mintty in a single terminal.
The Following User Says Thank You to scandiun For This Useful Post: [ View ] Gift scandiun Ad-Free
5th April 2013, 10:48 PM |#29  
Member
Thanks Meter: 7
 
More
Quote:
Originally Posted by scandiun

Pax is not included in Cygwin base install. You can run all commands from Mintty in a single terminal.

Thank you - so if I am telling people about Cygwin, I need to tell them to search PAX and select "installed". OK.

RE: Mintty - I did a search on it and this page says that

Quote:

Mintty is a terminal emulator for Cygwin and MSYS. In Cygwin, it is installed as the default terminal by Cygwin's setup.exe.

Does this mean that when I double click the Cygwin shortcut, Mintty is the terminal window that opens?

I ask this because when I navigate to the folder where the adb.exe is located, and run the adb backup command I get the response

-bash: adb: command not found
5th April 2013, 11:54 PM |#30  
scandiun's Avatar
OP Senior Member
Thanks Meter: 914
 
More
Quote:
Originally Posted by escribblings

Thank you - so if I am telling people about Cygwin, I need to tell them to search PAX and select "installed". OK.

RE: Mintty - I did a search on it and this page says that



Does this mean that when I double click the Cygwin shortcut, Mintty is the terminal window that opens?

I ask this because when I navigate to the folder where the adb.exe is located, and run the adb backup command I get the response

-bash: adb: command not found

Yes, you have to intentionally install pax from setup.exe and Mintty in the default shortcut to "Cygwin".
The Following User Says Thank You to scandiun For This Useful Post: [ View ] Gift scandiun Ad-Free
6th April 2013, 12:30 AM |#31  
Member
Thanks Meter: 7
 
More
Quote:
Originally Posted by scandiun

Yes, you have to intentionally install pax from setup.exe and Mintty in the default shortcut to "Cygwin".

OK - so how do I get Mintty to run adb?
Post Reply Subscribe to Thread
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes