FORUMS
Remove All Ads from XDA

[TOOL] Whatsapp Xtract: Backup Messages Extractor / Database Analyzer / Chat-Backup

207 posts
Thanks Meter: 290
 
By ztedd, Senior Member on 5th April 2012, 07:13 AM
Post Reply Email Thread
1st May 2014, 03:00 PM |#1071  
Member
Thanks Meter: 12
 
More
Quote:
Originally Posted by Lucabest98

Hi, i've got this problem

[email protected]:~/Scrivania/Whatsapp_Xtract$ python whatsapp_xtract.py msgstore.db -w wa.db
Python Version 2.x
Android mode!

Traceback (most recent call last):
File "whatsapp_xtract.py", line 2453, in <module>
main(sys.argv[1:])
File "whatsapp_xtract.py", line 1486, in main
curr_chat = Chatsession(chats["id"],chats["display_name"],chats["jid"],None,chats["unseen_msg_count"],chats["status"],lastmessagedate)
IndexError: No item with that key
[email protected]:~/Scrivania/Whatsapp_Xtract$

How can i solve it?
thank you

Even though this problem may be too old, I have searched for the solution in the thread and I found nothing. This happens when you provide both wa.db and msgstore.db.

The way to solve it is just to replace chats["id"] with chats["_id"] in whatsapp_xtract.py, as that's the right name for the column in the database.
 
 
1st May 2014, 07:44 PM |#1072  
Senior Member
Thanks Meter: 151
 
More
Quote:
Originally Posted by Frantch

ps: last night I put my table (N7 running kitkat wand whatsapp 2.11.230) in flight mode and when whatsapp did the 4am backup this morning i found a - msgstore.db.crypt7 (created at 4am) and a msgstore.db.crypt (created at 4.15am)

I'm assuming that you have Tri-Crypt installed? The Tri-Crypt service will check for a new backup file every 15 minutes. If it finds one then it will re-crypt it to legacy crypt. I.E. WhatsApp created msgstore.db.crypt7 at 04:00. Tri-Crypt created msgstore.db.crypt at 04:15 (15 minutes later).

If the automated backup at 04:00 works in flight-mode then that's positive news as it means they're reading the key file. Hopefully they won't change that. I know you need to be online to do a manual backup, but did not test the automated one. Thanks for the info.
1st May 2014, 08:48 PM |#1073  
Junior Member
Thanks Meter: 0
 
More
Quote:
Originally Posted by TripCode

I'm assuming that you have Tri-Crypt installed? The Tri-Crypt service will check for a new backup file every 15 minutes. If it finds one then it will re-crypt it to legacy crypt. I.E. WhatsApp created msgstore.db.crypt7 at 04:00. Tri-Crypt created msgstore.db.crypt at 04:15 (15 minutes later).

If the automated backup at 04:00 works in flight-mode then that's positive news as it means they're reading the key file. Hopefully they won't change that. I know you need to be online to do a manual backup, but did not test the automated one. Thanks for the info.

Yes i had tri-crypt thats why.

"know you need to be online to do a manual backup, but did not test the automated one"
I tried just now, put my N7 in flight mode, tap the manual backup. and the msgstore.de.crypt7 has been created (didn't try to decrypt but I guess it will work) ? Interesting
2nd May 2014, 12:15 AM |#1074  
Senior Member
Thanks Meter: 151
 
More
Quote:
Originally Posted by Frantch

I tried just now, put my N7 in flight mode, tap the manual backup. and the msgstore.de.crypt7 has been created (didn't try to decrypt but I guess it will work) ? Interesting

That is interesting. That's behaviour that wasn't observed in previous betas. Testing this further, I deleted my main database so that WhatsApp would allow me to restore from a backup. Oddly, I still have to be online to restore, but not to backup (unless I delete the key file). The good news is that this latest beta actually uses the key file so hopefully they will be keeping it. Earlier betas builds did not need it at all.
2nd May 2014, 12:23 AM |#1075  
Junior Member
Thanks Meter: 0
 
More
Quote:
Originally Posted by TripCode

That is interesting. That's behaviour that wasn't observed in previous betas. Testing this further, I deleted my main database so that WhatsApp would allow me to restore from a backup. Oddly, I still have to be online to restore, but not to backup (unless I delete the key file). The good news is that this latest beta actually uses the key file so hopefully they will be keeping it. Earlier betas builds did not need it at all.

Well yes that's a good news . I will have some free time tho weekend. I will try to go through Whatsapp code and play around with the key to understand how it get generate.
2nd May 2014, 01:36 AM |#1076  
Senior Member
Thanks Meter: 151
 
More
Quote:
Originally Posted by Frantch

Well yes that's a good news . I will have some free time tho weekend. I will try to go through Whatsapp code and play around with the key to understand how it get generate.

The actual cipher key (for encryption and decryption) is not generated locally. It's generated by WhatsApp's server. The generation process is clearly not random and is being hashed using the information contained within the first 67 bytes of the database. Bytes 0-3 is padding. Bytes 3-35 contain a challenge key which is randomly generated by the app. Bytes 35-51 contain a sha generation key that is seeded against your gmail to create a sha (comparison) challenge. Bytes 51-67 contain the IV (again this is randomly generated, but one half of the crypto challenge at least).

As far as I can tell the sha challenge is for internal use only and is what prevents you from importing a crypt file belonging to another account (similar to crypt5, but instead of md5 they use sha-256).

Example:

Code:
	  private static byte[] getRandom(int i) throws NoSuchAlgorithmException
	  {
	      byte[] B = new byte[i];
	      SecureRandom.getInstance("SHA1PRNG").nextBytes(B);
	      return B;
	  }
Calling this and asking for 16 bytes will give you a randomly generated seed which is one half of the sha challenge (this is what is stored in bytes 51-67). The other half is your gmail.

So...

Code:
	  private static byte[] getShaChallenge(String gMail, byte[] challengeSha) throws NoSuchAlgorithmException, UnsupportedEncodingException
	  {
	      MessageDigest locMessageDigest = null;
	      MessageDigest shaMessageDigest = MessageDigest.getInstance("SHA-256");
	      locMessageDigest = shaMessageDigest;
	      byte[] challengeBytes = gMail.getBytes("UTF-8");
	      int i = challengeBytes.length;
	      byte[] challengeFinalBytes = new byte[i + challengeSha.length];
	      System.arraycopy(challengeBytes, 0, challengeFinalBytes, 0, challengeBytes.length);
	      System.arraycopy(challengeSha, 0, challengeFinalBytes, i, challengeSha.length);
	      locMessageDigest.reset();
	      locMessageDigest.update(challengeFinalBytes);
	      return locMessageDigest.digest();
	  }
Call the above and feed in your gmail and sha seed. This will produce the sha challenge. If you move a database from one device to another the gmail will be different so the resulting hash will not match up. As I said, the above challenge is for internal use only, but the seed itself may perhaps being used to keygen the cipher key along with the challenge key and / or iv. Unfortunately, we have no access to WhatsApp's server so we have no idea what their hashing routines are, or if they're seeding or not (and if they are, what that seed is). What I can tell you is that your telephone number and/or jabber id (more or less the same thing) is not being used to seed. This is because I manipulated two installs of WhatsApp to return the same cipher, based on the information in the backup file only. This was in crypt6 so I may have to re-test with crypt7 though I think crypt7 is just the final version of crypt6 ready for the next stable release. In a nutshell, if you want to figure out how the cipher keys are being generated you will a) have to get very lucky (several billion hashes later my luck is still out) or b) hack WhatsApp's server for the hashing routines (not recommended). The first 67 bytes of the crypt file are all WhatsApp's server need to generate the valid cipher. Unless you, I or anybody else can reproduce what they're doing online then crypt6 and crypt7 will remain secure for non-rooted users of WhatsApp.

Just thought I would save you some time getting lost in smali this weekend.
2nd May 2014, 02:13 AM |#1077  
Senior Member
Thanks Meter: 23
 
More
Quote:
Originally Posted by TripCode

The actual cipher key (for encryption and decryption) is not generated locally. It's generated by WhatsApp's server. The generation process is clearly not random and is being hashed using the information contained within the first 67 bytes of the database. Bytes 0-3 is padding. Bytes 3-35 contain a challenge key which is randomly generated by the app. Bytes 35-51 contain a sha generation key that is seeded against your gmail to create a sha (comparison) challenge. Bytes 51-67 contain the IV (again this is randomly generated, but one half of the crypto challenge at least).

As far as I can tell the sha challenge is for internal use only and is what prevents you from importing a crypt file belonging to another account (similar to crypt5, but instead of md5 they use sha-256).

Example:

Code:
	  private static byte[] getRandom(int i) throws NoSuchAlgorithmException
	  {
	      byte[] B = new byte[i];
	      SecureRandom.getInstance("SHA1PRNG").nextBytes(B);
	      return B;
	  }
Calling this and asking for 16 bytes will give you a randomly generated seed which is one half of the sha challenge (this is what is stored in bytes 51-67). The other half is your gmail.

So...

Code:
	  private static byte[] getShaChallenge(String gMail, byte[] challengeSha) throws NoSuchAlgorithmException, UnsupportedEncodingException
	  {
	      MessageDigest locMessageDigest = null;
	      MessageDigest shaMessageDigest = MessageDigest.getInstance("SHA-256");
	      locMessageDigest = shaMessageDigest;
	      byte[] challengeBytes = gMail.getBytes("UTF-8");
	      int i = challengeBytes.length;
	      byte[] challengeFinalBytes = new byte[i + challengeSha.length];
	      System.arraycopy(challengeBytes, 0, challengeFinalBytes, 0, challengeBytes.length);
	      System.arraycopy(challengeSha, 0, challengeFinalBytes, i, challengeSha.length);
	      locMessageDigest.reset();
	      locMessageDigest.update(challengeFinalBytes);
	      return locMessageDigest.digest();
	  }
Call the above and feed in your gmail and sha seed. This will produce the sha challenge. If you move a database from one device to another the gmail will be different so the resulting hash will not match up. As I said, the above challenge is for internal use only, but the seed itself may perhaps being used to keygen the cipher key along with the challenge key and / or iv. Unfortunately, we have no access to WhatsApp's server so we have no idea what their hashing routines are, or if they're seeding or not (and if they are, what that seed is). What I can tell you is that your telephone number and/or jabber id (more or less the same thing) is not being used to seed. This is because I manipulated two installs of WhatsApp to return the same cipher, based on the information in the backup file only. This was in crypt6 so I may have to re-test with crypt7 though I think crypt7 is just the final version of crypt6 ready for the next stable release. In a nutshell, if you want to figure out how the cipher keys are being generated you will a) have to get very lucky (several billion hashes later my luck is still out) or b) hack WhatsApp's server for the hashing routines (not recommended). The first 67 bytes of the crypt file are all WhatsApp's server need to generate the valid cipher. Unless you, I or anybody else can reproduce what they're doing online then crypt6 and crypt7 will remain secure for non-rooted users of WhatsApp.

Just thought I would save you some time getting lost in smali this weekend.

Yes. I agree that for non-rooted seems ok.
Other alternative which i am working is using smaili injection on whatsapp.apk to create a log file of the key to sdcard.
I have decompiled it using JEB and working on it. This is probably the partial solution.
Thats why, to be secure, downlaod whatsapp from google play directly
2nd May 2014, 04:09 AM |#1078  
Senior Member
Thanks Meter: 151
 
More
Quote:
Originally Posted by droopyar

Other alternative which i am working is using smaili injection on whatsapp.apk to create a log file of the key to sdcard.

Already done this. It works in theory and if you take a backup and re-install the original WhatsApp after you have gotten your cipher then WhatsApp will give you the same cipher (most of the time). Unfortunately it rolls so a few days later you're given a completely different cipher. This means that you would have to leave the modified WhatsApp installed permanently to keep getting the latest key. You only have to change a few lines of smali to get WhatsApp to revert back to crypt or crypt5 which would be far less hassle (for the user). If they go stable with crypt7 then I will be releasing a patching app (WhatPatch) that patches the latest apk to revert back to an older crypt method or one that stores the curent key in the database file and increase its header size to accomodate it. Other patches will include the ability to disable latest version notification and the "WhatsApp has been illegally modified" warning that will randomly appear on modified apk's due to the signature verification checks. I decided a patching app would be better than releasing a modified apk because a) patching apps are not illegal, b) it gives the user a choice, c) the user doesn't have to worry about any malicious code injections and d) it will work with future builds, unless of course the code base changes drastically.

The best solution would be to crack their hashing methods, unfortunately I do not see this happening anytime soon.

P.S. My modified version of WhatsApp replaces the crypt7 method with my own, which is called crypt42. It's the same as crypt7, but the decryption key is stored where the challenge key should be. The database is also compressed before encryption commences so the backup files take upto 70%-80% less space on external storage compared to WhatsApp which does not compress. It's likely that if I do go ahead with WhatPatch then I will include the option to use my crypt42 method.
The Following User Says Thank You to TripCode For This Useful Post: [ View ] Gift TripCode Ad-Free
6th May 2014, 12:31 AM |#1079  
Member
Thanks Meter: 27
 
More
Resolved. Fixed my own app
8th May 2014, 06:20 AM |#1080  
jics91's Avatar
Senior Member
Flag David
Thanks Meter: 7
 
More
Hello everyone, it's been a long time since I used Whatsapp Xtract to read some databases. Today I'm struggling to open the mstore... but I get this issue:


C:\Users\Joel_2\Downloads>python "C:\Users\Joel_2\Downloads\wx\wx\whatsapp_xtra c
t.py" C:\Users\Joel_2\Downloads\msgstore-2014-04-29.1.db.crypt5
Python Version 2.x
Android mode!

trying to repair android database...
trying to decrypt android database...
decrypted database written to C:\Users\Joel_2\Downloads\msgstore-2014-04-29.15.p
lain.db
trying to repair decrypted android database...
Could not open database file. Guess it's not a valid Android or Iphone database
file.
Presione una tecla para continuar (Press a button to continue) . . .

I use the ''drag and drop'' method. Any fix for that?
8th May 2014, 02:56 PM |#1081  
Junior Member
Thanks Meter: 0
 
More
invalid database
First I have to say sorry for my bad english. But I hope someone understand me an can help me.
My problem is, what I have chance my phone. Now I want to reinstall whatsapp with my old database.
During the Installation whatsapp is recognise the old database, but than there is no chat in.
So I have install whatsapp_xtract an ActivePython-2.7.6.9-win32-x86.
Python is still running. But the old database of Whatsapp is not valid.
So there is a error: could not open the database file.Gues it´s not a valid Androi or Iphone database.
Can samebody tell me what is wrong?
Thanks.
Post Reply Subscribe to Thread

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes