Welcome to XDA

Search to go directly to your device's forum

Register an account

Unlock full posting privileges

Ask a question

No registration required
Post Reply

Perl scripts to encrypt/decrypt adb backup files

OP puterboy

25th June 2012, 12:23 AM   |  #1  
OP Junior Member
Thanks Meter: 15
 
10 posts
Join Date:Joined: Jun 2012
I wrote the following attached PERL routines for reading/decrypting/decompressing and writing/encrypting/compressing adb backup format backup files.

The routines are:
backupdecrypt.pl: Decrypt (and decompress) android backup file
backupencrypt.pl: Encrypt (and decompress) android backup file
tarfix.pl: Fix broken tar files produced by android backup when using -shared flag

The first two routines allow for reading and writing to the standard ".ab" adb backup format.

Backupdecrypt.pl takes an '.ab' file as input and outputs a standard format tar file (which may be optionally gzip'd).

Backupencrypt.pl takes an arbitrary file (though typically it should be tar file) as input and outputs a standard ".ab" format backup file. Options include the ability to encrypt (or not) and deflate (or not) the backup. Also, one can automatically decompress most standard input formats before encrypting.

For encryption, passwords can be queried for or passed on the command line or read from a file.

NOTE: unfortunately the standard 'adb backup' routine seem to have a SEVERE *BUG* in it when using the '--shared' option in combination with certain other options.

First, the backup is not compressed even though the header claims it is. To get around this, backupdecrypt.pl has a --nocompress option to override the header.

Second, the encapsulated tar file is corrupted by the insertion of 4 extra bytes before every file header and before every group of 64 512-byte blocks of data.

The third routine tarfix.pl fixes this corruption and outputs a normal readable tar file. So, if you are not able to recover a valid tar backup file using backupdecrypt.pl, try doing the following:

backupdecrypt.pl --nocompress <backup.ab> <backupdata>
tarfix.pl backupdata | tar xv


Enjoy!

NOTE: I am incredibly grateful to Nikolay Elenkov for providing sample java routines and for help in understanding the encryption formats
Attached Files
File Type: tar abbackuproutines.tar - [Click for QR Code] (40.0 KB, 2469 views)
The Following 15 Users Say Thank You to puterboy For This Useful Post: [ View ]
30th June 2012, 01:43 AM   |  #2  
Junior Member
Thanks Meter: 0
 
2 posts
Join Date:Joined: Jun 2012
Thanks for this. I was looking for away to peek into the backup file.
trogdan
5th July 2012, 09:56 PM   |  #3  
Guest
Thanks Meter: 0
 
n/a posts
Problem decrypting
just what i was looking for, unfortunately, the tarfix.pl doesn't seem to like my backup.

Code:
user@machine:~/Sandbox/transformerprime$ ~/bin/adbbackup/backupdecrypt.pl --nocompress backup.ab decrypted
the following is where things get funky. not recognized as a tar archive
Code:
user@machine:~/Sandbox/transformerprime$ ~/bin/adbbackup/tarfix.pl decrypted | tar xv
Illegal binary digit ']' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Wide character in oct at /home/user/bin/adbbackup/tarfix.pl line 107.
Illegal binary digit '�������������' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
tar: This does not look like a tar archive
tar: Skipping to next header
Illegal octal digit '8' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Wide character in oct at /home/user/bin/adbbackup/tarfix.pl line 107.
Illegal binary digit '�������������' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Illegal hexadecimal digit 'X' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Illegal hexadecimal digit '' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Illegal octal digit '9' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Wide character in oct at /home/user/bin/adbbackup/tarfix.pl line 107.
Illegal binary digit '�������������' ignored at /home/user/bin/adbbackup/tarfix.pl line 107.
Wide character in oct at /home/user/bin/adbbackup/tarfix.pl line 107.
20th July 2012, 08:05 PM   |  #4  
Junior Member
Thanks Meter: 2
 
2 posts
Join Date:Joined: Jul 2012
Fix for tarfix.pl issue
I had ran into issues with ADB backups performed under Android 4.0.4 before the JB upgrade (on a Samsung Galaxy Nexus). I did include the shared storage (accidentally or intentionally I don't remember) and I ran into this bug (Android issue 28303; sorry, I am new here and not allowed to post outside links). Some investigation revealed that while the backup was supposedly AES encrypted and "Deflate" compressed, this was only true for the first "part" of it. At around 150 MB into the file, a simple tar archive of the SD card content was appended. ADB was unable to restore any SD card content per the bug linked to above.

What I did to resolve:
  • Use a hex editor (HxD) to get to the start of the appended tar archive
  • Copy this part to a new ".tar" file
  • Experiment with tarfix.pl and run into the same issues as the previous poster
  • Look at the code and TAR file content and find out that 00 00 02 00 needs to be prepended to the tar file for tarfix.pl to do its job

Key learning was that the ADB backup tool will create plaintext, corrupted tar format backups that it cannot restore. It is problematic that while the user will believe they have an encrypted backup they can restore, they actually have a plaintext backup that they cannot restore...
The Following 2 Users Say Thank You to binaryhero For This Useful Post: [ View ]
21st July 2012, 10:50 AM   |  #5  
Junior Member
Thanks Meter: 0
 
8 posts
Join Date:Joined: Jun 2010
Hi,

where did you get PBKDF2.pm from?

I couldnt find it in Fedora repos and the only one I got from the internat has an issue:


Undefined subroutine &Crypt:penSSL::PBKDF2::derive called at ./backupdecrypt.pl line 266, <STDIN> line 1.

Thanks,
Klement
21st July 2012, 01:59 PM   |  #6  
Junior Member
Thanks Meter: 2
 
2 posts
Join Date:Joined: Jul 2012
CPAN
All modules were pulled from CPAN directly (I had to use Cygwin as I was on the road), e.g. "perl -M CPAN -e shell" and then issuing "install Crypt:penSSL::PBKDF2".
3rd August 2012, 10:26 PM   |  #7  
Junior Member
Thanks Meter: 1
 
1 posts
Join Date:Joined: Dec 2009
Quote:
Originally Posted by binaryhero

I had ran into issues with ADB backups performed under Android 4.0.4 before the JB upgrade (on a Samsung Galaxy Nexus). I did include the shared storage (accidentally or intentionally I don't remember) and I ran into this bug (Android issue 28303; sorry, I am new here and not allowed to post outside links). Some investigation revealed that while the backup was supposedly AES encrypted and "Deflate" compressed, this was only true for the first "part" of it. At around 150 MB into the file, a simple tar archive of the SD card content was appended. ADB was unable to restore any SD card content per the bug linked to above.

What I did to resolve:

  • Use a hex editor (HxD) to get to the start of the appended tar archive
  • Copy this part to a new ".tar" file
  • Experiment with tarfix.pl and run into the same issues as the previous poster
  • Look at the code and TAR file content and find out that 00 00 02 00 needs to be prepended to the tar file for tarfix.pl to do its job

Key learning was that the ADB backup tool will create plaintext, corrupted tar format backups that it cannot restore. It is problematic that while the user will believe they have an encrypted backup they can restore, they actually have a plaintext backup that they cannot restore...

I ran afoul of this (foolishly didn't finish the 30 page forum post on it before diving in ). I'm trying your perl solution now, but I'm afraid I'm unfamiliar with tar headers in a hex viewer. Could I trouble you for some pointers on how best to determine where the TAR starts? I understand there is some sort of header, but I can't figure out what to look for.

Thanks, though, the perl runs well and I'm learning alot (far more than I ever wanted, tbh) about tars,encrypted backups, adb, etc.
The Following User Says Thank You to KentigernEnnis For This Useful Post: [ View ]
28th October 2012, 11:19 PM   |  #8  
munjeni's Avatar
Recognized Contributor / Recognized Developer
Thanks Meter: 16,766
 
5,360 posts
Join Date:Joined: Jun 2011
Thanks for your great tools! Here is some better way for unpacking?
dd if=mybackup.ab bs=1 skip=24 | openssl zlib -d > mybackup.tar

For packing (just need to add 24 butes to header (41 4E 44 52 4F 49 44 20 42 41 43 4B 55 50 0A 31 0A 31 0A 6E 6F 6E 65 0A)?
openssl zlib -in mybackup.tar -out gg.ab

Hope it helps!
Last edited by munjeni; 28th October 2012 at 11:32 PM.
The Following 2 Users Say Thank You to munjeni For This Useful Post: [ View ]
29th October 2012, 11:25 AM   |  #9  
scandiun's Avatar
Senior Member
Thanks Meter: 779
 
1,338 posts
Join Date:Joined: Jul 2010
Quote:
Originally Posted by munjeni

Thanks for your great tools! Here is some better way for unpacking?
dd if=mybackup.ab bs=1 skip=24 | openssl zlib -d > mybackup.tar

For packing (just need to add 24 butes to header (41 4E 44 52 4F 49 44 20 42 41 43 4B 55 50 0A 31 0A 31 0A 6E 6F 6E 65 0A)?
openssl zlib -in mybackup.tar -out gg.ab

Hope it helps! :good:

Of course. You have to concatenate the created ab backup to the first 24 bytes.

Get the first 24 bytes of an unencrypted backup
Code:
dd if=mybackup.ab bs=24 count=1 of=first24
Concatenate
Code:
cp first24 backup.ab
openssl zlib -in mybackup.tar >> backup.ab


---------- Post added at 10:25 AM ---------- Previous post was at 10:21 AM ----------

Quote:
Originally Posted by binaryhero

All modules were pulled from CPAN directly (I had to use Cygwin as I was on the road), e.g. "perl -M CPAN -e shell" and then issuing "install Crypt::eek:openSSL::PBKDF2".

Says "Missing argument to -M."
The Following User Says Thank You to scandiun For This Useful Post: [ View ]
29th October 2012, 02:38 PM   |  #10  
munjeni's Avatar
Recognized Contributor / Recognized Developer
Thanks Meter: 16,766
 
5,360 posts
Join Date:Joined: Jun 2011
You concatenation is wrong! Your command "cp" will overwrite backup.ab! You can concetate 2 files using "cat" for example "cat first24 backup.ab > new.backup.ab"

Do you have idea how I can generate timestamp of these file in "13 number" format?

Quote:

<?xml version="1.0" encoding="UTF-8"?>
<recordset version="1" timestamp="1344764788434" size="61667">
<record name="back.ab" type="1" size="12662" order="1" catagory="1" id="back.ab"><packagelist><package>com.android.set tings</package></packagelist></record>
</recordset>

Edit:
found a way for generating timestamt with 13 numbers
stat -c '%Y000' backup.ab
Last edited by munjeni; 29th October 2012 at 10:43 PM.

Post Reply Subscribe to Thread
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes