[TOOL] Whatsapp Xtract: Backup Messages Extractor / Database Analyzer / Chat-Backup

Search This thread

advance512

Member
Feb 26, 2013
31
10
hi advance512

nice enhancement :good:
I have a problem when opening the html file on another machine. The full original path is written in the html file (which dows not exists on the second machine)

Thanks. I actually added other enhancements - a Media file to see all photos quickly, but I am too lazy to release it as I did it in a sort of ugly way (code-wise).

As far as your problem, I would have to see a specific file to understand. The code generates HTML files which refer to relative files names, all based on the input filename, the command line argument. I assume you input a complete file path as a command line argument? If so, try and just give the filename.

Otherwise, I'd have to understand more about your process.
 

logge22

New member
Mar 6, 2013
4
0
How do you expect anyone to help you without all of the information?
What is this "old" file? What version of WhatsApp is it from? Is there any chance it got corrupted? Do you have a backup? Better yet - is the "plain.db" file a decrypted version of this encrypted file? If so, why do you need to decrypt said file?

Please read the actual instruction to see you can open a decrypted file with the script, just as you can open an encrypted file.

The msgstore.plain.db-File was generated during the process using Whatsapp_Extract like you can see in the picture. Also you can see that the file is from December 2012, i don't know exactly the Whatsapp-version I used at this time.
The plain.db-File is decrypted but I only can use it with sqlite-manager but not with Whatsapp_Extract. And no I don't have a backup this is the only file I have for this chat-history i want to get back. And yes it is possible that the file got courrupted because i have had several problems tranferring my data from the sd-card to my computer.

Hope this is enough information and you can help me? Otherwise i can send you the file if you have time to have a short look at it.
 

Attachments

  • Error.jpg
    Error.jpg
    86.2 KB · Views: 649

advance512

Member
Feb 26, 2013
31
10
The msgstore.plain.db-File was generated during the process using Whatsapp_Extract like you can see in the picture. Also you can see that the file is from December 2012, i don't know exactly the Whatsapp-version I used at this time.
The plain.db-File is decrypted but I only can use it with sqlite-manager but not with Whatsapp_Extract. And no I don't have a backup this is the only file I have for this chat-history i want to get back. And yes it is possible that the file got courrupted because i have had several problems tranferring my data from the sd-card to my computer.

Hope this is enough information and you can help me? Otherwise i can send you the file if you have time to have a short look at it.

Well, the error you showed me comes from the SQLite library.

You probably have a bit of corruption in the decrypted file - even 1 bit off in the encrypted file could affect large parts of the decrypted file. Anyways, the fact that you can see your info (or some of it) in SQLite Manager is great. This is your solution to save what remains.

Open the database, and export the data into an SQL file.
Then, create a NEW database, and import the data from that SQL file. Save the new database.

Now, feed the new database into WhatsApp_Xtract using the "decrypted database" option found in the OP.

It should work.

Let me know how it goes. :)
 

logge22

New member
Mar 6, 2013
4
0
Well, the error you showed me comes from the SQLite library.

You probably have a bit of corruption in the decrypted file - even 1 bit off in the encrypted file could affect large parts of the decrypted file. Anyways, the fact that you can see your info (or some of it) in SQLite Manager is great. This is your solution to save what remains.

Open the database, and export the data into an SQL file.
Then, create a NEW database, and import the data from that SQL file. Save the new database.

Now, feed the new database into WhatsApp_Xtract using the "decrypted database" option found in the OP.

It should work.

Let me know how it goes. :)

I created a new database like u told me (the imported sql-file had about 6 mb, the new created file had about 4 mb) and called it msgstore.db and pasted it into the whatsapp_xtract folder like you can see on the image. then i ran whatsapp_xtract_android but it didn't work, it also didn't work when i pasted the file into whatsapp_xtract_drag_and_drop_databse(s), where the following error message occured.
Sorry but I really tried a lot of things (and wasted a lot of time) but it doesn't work... :( Do you think you could manage it when i would send you the file?
 

Attachments

  • error2.jpg
    error2.jpg
    173.9 KB · Views: 440

advance512

Member
Feb 26, 2013
31
10
I created a new database like u told me (the imported sql-file had about 6 mb, the new created file had about 4 mb) and called it msgstore.db and pasted it into the whatsapp_xtract folder like you can see on the image. then i ran whatsapp_xtract_android but it didn't work, it also didn't work when i pasted the file into whatsapp_xtract_drag_and_drop_databse(s), where the following error message occured.
Sorry but I really tried a lot of things (and wasted a lot of time) but it doesn't work... :( Do you think you could manage it when i would send you the file?

Okay, post it somewhere so I could download it, I'll see what I can do.. though I am not promising anything.
 

hape70

New member
May 3, 2012
3
0
Thanks. I actually added other enhancements - a Media file to see all photos quickly, but I am too lazy to release it as I did it in a sort of ugly way (code-wise).

As far as your problem, I would have to see a specific file to understand. The code generates HTML files which refer to relative files names, all based on the input filename, the command line argument. I assume you input a complete file path as a command line argument? If so, try and just give the filename.

Otherwise, I'd have to understand more about your process.

your're right, when entering the command manually all is fine.
When I drag and drop the db-file to "whatsapp_xtract_drag'n'drop_database(s)_here.bat" the full path will be submitted.

Thanks for your help!

Update:
I changed the lines in whatsapp_xtract_drag'n'drop_database(s)_here.bat
from
python "%~dp0whatsapp_xtract.py" %1
to
python "%~dp0whatsapp_xtract.py" %~nx1

now it's ok
 
Last edited:

pippo81

New member
Jan 5, 2009
3
0
Thank You Fabio :D !
It works perfectly for restoring my backups (cambiero' 3-4 rom al mese, ma il fatto che non mi ripristinasse la chat non era mai capitato..)
 

Snapy1991

New member
Mar 10, 2013
4
0
Whatsapp Backup gefunden,aber nicht wiederhergestellt

Hallo ,
Ich habe folgendes Problem, und zwar kann ich meine Backup Datei nicht mehr auf dem Handy wiederherstellen.Ich habe mein Handy formatiert und habe aber vorher den Whatsapp Ordner von der SD Karte auf den PC kopiert.Darin befindet sich im Ordner Databases die Datei msgstore.db.crypt.
Ich habe dann die Anleitung befolgt und es klappte alles bis zum Fehler :
Error sqlite3.Error while reading Chat #: database disk image is malformed
So habe ich dann deinen Rat befolgt :
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
- python whatsapp_xtract.py msgstore_new.db

und konnte am Ende die Datei auf dem PC anzeigen und hatte die Verläufe zumindest auf dem PC.
Nun meine Frage: Kann ich nun irgendwie die msgstore.db.crypt Datei so reparieren und auf dem Handy wieder kopieren, damit Whatsapp auch die Nachrichten tatsächlich wiederherstellt oder kann ich sie nur maximal auf dem PC anzeigen lassen?Kannst du mir bitte sagen,welche Datei ich wohin genau verschieben soll?Denn es heißt ja im Normalfall,wenn es klappt die Datei msgstore.plain.db in den Whatsapp Ordner auf dem Handy kopieren und in msgstore.db umbenennen,jedoch wenn ich
whatsapp_xtract_android_crypted.bat
ausführe, entsteht ja der oben in rot angezeigte Fehler und die msgstore.plain.db Datei ensteht auch mit genau der gleichen Dateigröße wie die msgstore.db.crypt Datei.Nun als ich Whatsapp wieder installiert habe, fragte es mich,ob ich den Verlauf wiederherstellen will,ich bestätigte und es zeigte an,dass es Nachrichten wiederherstellt,jedoch waren keine Verläufe da und alles war weg.
Ich hoffe ihr habt verstanden was ich meine und hoffe, dass ihr mir helfen könnt, das wäre echt lieb.

Noch ein kleiner Nachtrag,wenn ich diese Befehle ausführe
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
ensteht ja die msgstore_new.db Datei und die ist nur minimal kleiner als die msgstore.db oder msgstore.plain.db Datei und die ist ja dann die reparierete Datei oder? Und durch sie kann ich den Inhalt auf dem PC sehen und so erhalte ich ja auch die msgstore.db als HTML Datei. Müsste ich diese Datei umbennen und irgendwohin kopieren oder was muss ich jetzt genau machen,damit Whatsapp die Backup Datei wirklich richtig erkennt und die Nachrichten tatsächlich wiederherstellt?
Zur Zeit ist Whatsapp installiert,müsste ich es deinstallieren und vorher noch irgendwas sichern oder kopieren?
Vielen lieben Dank im Vorraus
 

Labersack

Member
Nov 16, 2006
12
9
Hallo ,
Ich habe folgendes Problem, und zwar kann ich meine Backup Datei nicht mehr auf dem Handy wiederherstellen.Ich habe mein Handy formatiert und habe aber vorher den Whatsapp Ordner von der SD Karte auf den PC kopiert.Darin befindet sich im Ordner Databases die Datei msgstore.db.crypt.
Ich habe dann die Anleitung befolgt und es klappte alles bis zum Fehler :
Error sqlite3.Error while reading Chat #: database disk image is malformed
So habe ich dann deinen Rat befolgt :
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
- python whatsapp_xtract.py msgstore_new.db

und konnte am Ende die Datei auf dem PC anzeigen und hatte die Verläufe zumindest auf dem PC.
Nun meine Frage: Kann ich nun irgendwie die msgstore.db.crypt Datei so reparieren und auf dem Handy wieder kopieren, damit Whatsapp auch die Nachrichten tatsächlich wiederherstellt oder kann ich sie nur maximal auf dem PC anzeigen lassen?Kannst du mir bitte sagen,welche Datei ich wohin genau verschieben soll?Denn es heißt ja im Normalfall,wenn es klappt die Datei msgstore.plain.db in den Whatsapp Ordner auf dem Handy kopieren und in msgstore.db umbenennen,jedoch wenn ich
whatsapp_xtract_android_crypted.bat
ausführe, entsteht ja der oben in rot angezeigte Fehler und die msgstore.plain.db Datei ensteht auch mit genau der gleichen Dateigröße wie die msgstore.db.crypt Datei.Nun als ich Whatsapp wieder installiert habe, fragte es mich,ob ich den Verlauf wiederherstellen will,ich bestätigte und es zeigte an,dass es Nachrichten wiederherstellt,jedoch waren keine Verläufe da und alles war weg.
Ich hoffe ihr habt verstanden was ich meine und hoffe, dass ihr mir helfen könnt, das wäre echt lieb.

Noch ein kleiner Nachtrag,wenn ich diese Befehle ausführe
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
ensteht ja die msgstore_new.db Datei und die ist nur minimal kleiner als die msgstore.db oder msgstore.plain.db Datei und die ist ja dann die reparierete Datei oder? Und durch sie kann ich den Inhalt auf dem PC sehen und so erhalte ich ja auch die msgstore.db als HTML Datei. Müsste ich diese Datei umbennen und irgendwohin kopieren oder was muss ich jetzt genau machen,damit Whatsapp die Backup Datei wirklich richtig erkennt und die Nachrichten tatsächlich wiederherstellt?
Zur Zeit ist Whatsapp installiert,müsste ich es deinstallieren und vorher noch irgendwas sichern oder kopieren?
Vielen lieben Dank im Vorraus

This is an international forum.
I understand german, but me (and maybe some others) will not answer you, if you don't even try to write english.
If everyone posts in his native language, this thread will soon be a mess.
Get help by some friend (good) or try the google translator (worse)

Dieses Forum ist international
Ich verstehe deutsch, aber ich (und evtl. einige Andere) werden nicht antworten, wenn du nichmal versuchst, englisch zu schreiben.
Wenn jeder hier in seiner eigenen Sprache schreibt, wird dieser Thread ein Chaos.
Lass dir von einem Freund helfen (gut) oder versuche den Google Translator (schlechter).
 

abhi.v

New member
Mar 12, 2013
1
0
hi all i face the same problem and here is my screenshot for the same :



This looks like PyCrypto is not correctly installed.
Which versions of
- windows/linux/.. (I assume windows)
- ActivePython
Do you have?

Please download the newest archive, extract and then run install pyCrypto.bat again.

Now the window will stay open after executing the install commands. Make a screenshot of what it shows.

Then start whatsapp_xtract_console.bat
there you type:
python
then hit Enter. Then python shows something, then you type:
from Crypto.Cipher import AES

Make a screenshot of this as well. Send the screenshots here.
 

Attachments

  • python.jpg
    python.jpg
    178.4 KB · Views: 571

LizTheCat

New member
Mar 12, 2013
3
0
Hallo ,
Ich habe folgendes Problem, und zwar kann ich meine Backup Datei nicht mehr auf dem Handy wiederherstellen.Ich habe mein Handy formatiert und habe aber vorher den Whatsapp Ordner von der SD Karte auf den PC kopiert.Darin befindet sich im Ordner Databases die Datei msgstore.db.crypt.
Ich habe dann die Anleitung befolgt und es klappte alles bis zum Fehler :
Error sqlite3.Error while reading Chat #: database disk image is malformed
So habe ich dann deinen Rat befolgt :
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
- python whatsapp_xtract.py msgstore_new.db

und konnte am Ende die Datei auf dem PC anzeigen und hatte die Verläufe zumindest auf dem PC.
Nun meine Frage: Kann ich nun irgendwie die msgstore.db.crypt Datei so reparieren und auf dem Handy wieder kopieren, damit Whatsapp auch die Nachrichten tatsächlich wiederherstellt oder kann ich sie nur maximal auf dem PC anzeigen lassen?Kannst du mir bitte sagen,welche Datei ich wohin genau verschieben soll?Denn es heißt ja im Normalfall,wenn es klappt die Datei msgstore.plain.db in den Whatsapp Ordner auf dem Handy kopieren und in msgstore.db umbenennen,jedoch wenn ich
whatsapp_xtract_android_crypted.bat
ausführe, entsteht ja der oben in rot angezeigte Fehler und die msgstore.plain.db Datei ensteht auch mit genau der gleichen Dateigröße wie die msgstore.db.crypt Datei.Nun als ich Whatsapp wieder installiert habe, fragte es mich,ob ich den Verlauf wiederherstellen will,ich bestätigte und es zeigte an,dass es Nachrichten wiederherstellt,jedoch waren keine Verläufe da und alles war weg.
Ich hoffe ihr habt verstanden was ich meine und hoffe, dass ihr mir helfen könnt, das wäre echt lieb.

Noch ein kleiner Nachtrag,wenn ich diese Befehle ausführe
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
ensteht ja die msgstore_new.db Datei und die ist nur minimal kleiner als die msgstore.db oder msgstore.plain.db Datei und die ist ja dann die reparierete Datei oder? Und durch sie kann ich den Inhalt auf dem PC sehen und so erhalte ich ja auch die msgstore.db als HTML Datei. Müsste ich diese Datei umbennen und irgendwohin kopieren oder was muss ich jetzt genau machen,damit Whatsapp die Backup Datei wirklich richtig erkennt und die Nachrichten tatsächlich wiederherstellt?
Zur Zeit ist Whatsapp installiert,müsste ich es deinstallieren und vorher noch irgendwas sichern oder kopieren?
Vielen lieben Dank im Vorraus

I am a nativ german speaker and have exactly the same problem that Snappy has, so I will explain it:

I have 9 older backup files with dates (msgstore-2013-02-22.1.db.crypt) (Snappy has just msgstore.db.crypt). My phone (HTC Sensation) was in repair for a couple of weeks and I now tried to restore the WhatsApp Backup. It all worked fine till we got the following error: Error sqlite3.Error while reading Chat #13: database disk image is malformed
I tried it with all of my backup files and got the same error with the newest 5 of them - the others where not readable by the program at all. We both got msgstore.plain.db files but they were just about the same size as the crypt ones and although WhatsApp was able to find the databases it was not able to restore the chats. In my case I had all the groups and contacts but not a single message - and the plain file was about 11 MB, so this can't really be. Please help - I spent half a day now trying to solve the problem and I have no clue anymore... :p

Snappy tried all your advices and got the readable HTML file but this still didn't solve the problem - obviously - and he or she is mainly asking how to repair the plain file and where to save it so the backup would be restored and working.

Thank you very, very much for your help and this thread - I really, really hope you can help me/us!

Liz
 

Snapy1991

New member
Mar 10, 2013
4
0
if you don't speak or understand german i will try to text you in englisch so that you can help me because i need my chat.
Liz has translated my text and i want to give you a detailed description of my problem and the way how i tried to solve it so you can understand and follow my problem.
I have tried to restore my chat but Whatsapp didn't restore anything allthough the question appeared during the installation if i want to restore the chat and i accepted.
I have just the Whatsapp folder from sd card where the file msgstore.db.crypt is in the databases. I copied this file and overwrote the file in the xtract whatsapp folder.Than i started the !install pyCrypto file and executed the whatsapp_xtract_android_crypted file but the following error appeared:
Error sqlite3.Error while reading Chat #: database disk image is malformed
and i got the msgstore.plain db file which has exactly the same size like the msgstore.db.crypt file (5,439 KB)
Than i copied the msgstore.plain db file again to the folder sdcard/Whatsapp/databases and renamed it to msgstore.db but Whatsapp didn't restore anything allthough it showed me that it seems to restore something during the installation.
So i did this to repair the database:
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
- python whatsapp_xtract.py msgstore_new.db
Now i got the msgstore_new db file which has just a little bit smaller size than the msgstore.db.crypt and msgstore.plain.db file (5,291 KB) and it seemed to be correct because i got also the HTML file msgstore.db file which showed me all my chat in firefox.
My question now: Which file do i have to copy to my sd card so i get my chat back and so Whatsapp can restore them correctly?
I would be very grateful for your help
ps. Liz thank you for your help and the translation :)
 

LizTheCat

New member
Mar 12, 2013
3
0
Than i copied the msgstore.plain db file again to the folder sdcard/Whatsapp/databases and renamed it to msgstore.db but Whatsapp didn't restore anything allthough it showed me that it seems to restore something during the installation.
So i did this to repair the database:
- start whatsapp_xtract_console.bat and then type:
- echo .dump | sqlite3.exe msgstore.plain.db > Temp.sql
- echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
- python whatsapp_xtract.py msgstore_new.db
Now i got the msgstore_new db file which has just a little bit smaller size than the msgstore.db.crypt and msgstore.plain.db file (5,291 KB) and it seemed to be correct because i got also the HTML file msgstore.db file which showed me all my chat in firefox.
My question now: Which file do i have to copy to my sd card so i get my chat back and so Whatsapp can restore them correctly?
I would be very grateful for your help
ps. Liz thank you for your help and the translation :)

That is where my problem differs from Snappy's: When I work with whatsapp_xtract_console.bat, first everything looks normal (tryubg to repair android database... trying to decrypt android database... decrypted database written to msgstorenew.db.plain.db trying to repair decrypted andorid database...) but then I get an error telling me: "Could not open database file. Guess it's not a valid Android or Iphone database file." The file msgstore_new.db is created but with a size of 0 KB. I really have no clue what to do there...
 

advance512

Member
Feb 26, 2013
31
10
Hi there,

Just wanted to post my "unofficial" (as in - I needed extra features for my own usage of this tool, so I coded them, but I am not a part of the official dev team) version 2.5 of this tool.

This version can now automatically handle malformed databases (that received the "database disk image is malformed" error).

--------------

CHANGELOG:

v2.5 (updated by Alon Diamant - Mar 14, 2013)
- Improved encrypted Android database detection and decryption code
- Can now repair malformed Android databases (depends on availability of sqlite3 executable)

v2.4 (updated by Alon Diamant - Mar 06, 2013)
- Generates media index file - but crappily, we should set this up better..

v2.3 (updated by Alon Diamant - Mar 05, 2013)
- now generates separate file for each contact
- fixed file search to search for image files in days prior to date given (to fix a bug where because of timezone differences the image file exists but is not found)
- fixed message counts for contacts
- does not list contacts with 0 messages
- now writes version number in generated files
- (Android Version) displays WhatsApp name (server based) if no display name is found for a contact
- (Android Version) Supports Python 2.6

--------------

How to use:

After doing step 2 of the original post, and before doing step 3 - extract attached archive (Whatsapp_Xtract_Py_2_5.zip) to the same folder selected in step 2. Continue to step 3 normally.

Good luck :)

Make sure you click the "Thanks" button beneath this post if you like this!

--------------

Original poster: feel free to merge this into your code base, if you feel like it, or contact me to take it down, if you don't like it. :)
 

Attachments

  • Whatsapp_Xtract_Py_2_5.zip
    20.7 KB · Views: 500
Last edited:
  • Like
Reactions: sam@xda29

LizTheCat

New member
Mar 12, 2013
3
0
Hi there,

Just wanted to post my "unofficial" (as in - I needed extra features for my own usage of this tool, so I coded them, but I am not a part of the official dev team) version 2.5 of this tool.

This version can now automatically handle malformed databases (that received the "database disk image is malformed" error).

--------------

CHANGELOG:

v2.5 (updated by Alon Diamant - Mar 14, 2013)
- Improved encrypted Android database detection and decryption code
- Can now repair malformed Android databases (depends on availability of sqlite3 executable)

v2.4 (updated by Alon Diamant - Mar 06, 2013)
- Generates media index file - but crappily, we should set this up better..

v2.3 (updated by Alon Diamant - Mar 05, 2013)
- now generates separate file for each contact
- fixed file search to search for image files in days prior to date given (to fix a bug where because of timezone differences the image file exists but is not found)
- fixed message counts for contacts
- does not list contacts with 0 messages
- now writes version number in generated files
- (Android Version) displays WhatsApp name (server based) if no display name is found for a contact
- (Android Version) Supports Python 2.6

--------------

How to use:

After doing step 2 of the original post, and before doing step 3 - extract attached archive (Whatsapp_Xtract_Py_2_5.zip) to the same folder selected in step 2. Continue to step 3 normally.

Good luck :)

Make sure you click the "Thanks" button beneath this post if you like this!

--------------

Original poster: feel free to merge this into your code base, if you feel like it, or contact me to take it down, if you don't like it. :)


Hello advance512

I tried your tool exactly how you wrote to, hoping it would solve my problem. But unfortunately it didn't work at all and I get the error attached when trying to use whatsapp_xtract_android_crypted.bat after having unpacked your tool.

Could you please help?

Thank you!

Liz
 

Attachments

  • Bild1.jpg
    Bild1.jpg
    22.9 KB · Views: 368
Last edited:

advance512

Member
Feb 26, 2013
31
10
Okay, so a quick fix to 2.5.

I forgot test it on Python 3 (silly me programming late at night), but now I did.

--------------

CHANGELOG:

v2.51 (updated by Alon Diamant - Mar 15, 2013)
- v2.5 was now tested with Python 3 as well. :S

v2.5 (updated by Alon Diamant - Mar 14, 2013)
- Improved encrypted Android database detection and decryption code
- Can now repair malformed Android databases (depends on availability of sqlite3 executable)

v2.4 (updated by Alon Diamant - Mar 06, 2013)
- Generates media index file - but crappily, we should set this up better..

v2.3 (updated by Alon Diamant - Mar 05, 2013)
- now generates separate file for each contact
- fixed file search to search for image files in days prior to date given (to fix a bug where because of timezone differences the image file exists but is not found)
- fixed message counts for contacts
- does not list contacts with 0 messages
- now writes version number in generated files
- (Android Version) displays WhatsApp name (server based) if no display name is found for a contact
- (Android Version) Supports Python 2.6

--------------

How to use:

After doing step 2 of the original post, and before doing step 3 - extract attached archive (Whatsapp_Xtract_Py_2_51.zip) to the same folder selected in step 2. Continue to step 3 normally.

Good luck :)

Make sure you click the "Thanks" button beneath this post if you like this!

--------------

Original poster: feel free to merge this into your code base, if you feel like it, or contact me to take it down, if you don't like it. :)
 

Attachments

  • Whatsapp_Xtract_Py_2_51.zip
    20.9 KB · Views: 787

Blinkin89

Member
Mar 28, 2012
42
7
First I thought that I wouldn't like the unofficial future development because of the separated contacts in html files...
But I've tried it and I must admit: well done. It looks pretty good now. Especially now the loading time in my browser is way lower.
I'm using your unofficial version now and I it's a pretty good addition. Great work!
 

ggclanlord

Senior Member
Feb 11, 2011
1,290
203
Downloading unofficial v2.5 now.

@OP, if at all you check this thread now - Are you going to update the tool now? Or there's going to be no more support for it?
 

Top Liked Posts

  • There are no posts matching your filters.
  • 181
    Have you ever wanted to
    - have a useful backup of your whatsapp chats?
    - be able to read your whatsapp chats again using a backup file?
    - delete the grown-too-big message history of whatsapp without losing the possibility to read the old conversations?
    - read older messages without the need of endlessly pressing the annoying "load older messages" button?
    - be able to search old messages?
    - ...?

    Here is a little Tool to display whatsapp chats on your computer, using the database files from your android installation of whatsapp. :)

    It's called WhatsApp Xtract and was created by Fabio Sangiacomo and me.

    Preview of the result:
    attachment.php


    So here is WhatsApp Xtract v2.3
    Released on April 25th, 2018

    Tested with Whatsapp (Android) 2.18.46
    Tested with Whatsapp (iPhone) 2.5.1

    CHANGELOG:

    v2.3 (updated by ztedd - Apr 25, 2018)
    - fixed image preview to work with new Whatsapp database format
    - some minor fixes

    v2.2 (updated by ztedd - Nov 17, 2012)
    - now supports new emoji smileys
    - (Android Version) hotfix for TypeError in b64encode
    - (Android Version) decoded file won't be deleted even if it can't be opened

    v2.1 (updated by Fabio Sangiacomo and ztedd - May 7th, 2012)
    - improved install pyCrypto.bat
    - added easy drag and drop possibility with whatsapp_xtract_drag'n'drop_database(s)_here.bat
    - (Android Version) added support to fix corrupted android whatsapp database (needs sqlite3, for windows sqlite3.exe is contained in the archive)
    - (Android Version) removed wrong extraction of owner in android version
    - (Iphone Version) information from Z_METADATA table will be printed to shell
    - (Iphone Version) fixed bug in support of older Iphone whatsapp databases

    V2.0 (updated by Fabio Sangiacomo and ztedd - Apr 28, 2012)
    - supports WhatsApp DBs coming from both Android and iPhone platforms
    - (Android Version) wa.db is optional
    - (Android Version) now also crypted msgstore.db.crypt from the SD card can be imported
    - chat list is sorted by the last sent message
    - fixed some bugs (e.g. that the script didn't work with python 3)


    ------------------------
    Wish list:
    Feel free to take the code and improve it. So maybe we could include some code from this different version which is able to export separate htmls for each chat:
    https://github.com/BurakDev/hotoloti/issues/10

    ------------------------

    How to use: :cool:
    (instrucciones en español / deutsche Anleitung - old)

    1. You need a copy of the whatsapp database and (if no root) get your key.

      a) if your Android phone is rooted:

      get these files:
      /data/data/com.whatsapp/databases/msgstore.db and wa.db
      (for this you need root access. detailed instructions in the bottom of this file.
      The advantage of having wa.db is that the corresponding contact names of phone numbers will be displayed.)

      b) if you have a normal Android (not rooted):

      you need this to get the key:
      [Tool] WhatsApp Key/DB Extractor | CRYPT6-12 | NON-ROOT
      https://xdaforums.com/showthread.php?t=2770982

      then you need the .crypt12 database from /sdcard/WhatsApp/Databases/

      then you need a tool like Whatsapp Viewer to decrypt the database file:
      Whatsapp Viewer
      http://andreas-mausch.de/whatsapp-viewer/
      https://xdaforums.com/showthread.php?t=2719741

      The decrypted database can already be viewed in Whatsapp Viewer. If you want the output style from Whatsapp xtract then you can continue with the decrypted file.

      c) On IPhone, get this file (don't know if it still works in 2018):

      net.whatsapp.WhatsApp/Documents/ChatStorage.sqlite
      (You can use an Iphone Backup Tool to get the file, e.g. I-Twin or Iphone Backup Extractor. Make sure to create an unencrypted backup with Itunes, as these tools can't handle encrypted backups. Another possibility are forensic tools like UFED Physical Analyzer.)

    2. Extract this archive (Whatsapp_Xtract....zip) to a certain folder on your computer, e.g. C:\WhatsApp.

    3. Copy the database(s) to e.g. C:\WhatsApp (on Android, you simply copy the whole folder WhatsApp on SD card to your computer e.g. to C:\WhatsApp and then copy the database file from C:\WhatsApp\Databases to C:\WhatsApp)

    4. You need Python

      https://www.python.org/downloads/

      It's available for Windows, Linux and Mac. However in my HowTo, I focus on Windows.

    5. Now run whatsapp_xtract_android.bat or whatsapp_xtract_iphone.bat

      OR simply drag and drop the database file(s) to whatsapp_xtract_drag'n'drop_database(s)_here.bat

      OR whatsapp_xtract_console.bat and then manually specify the input file with one of these commands:

      COMMAND LINE OPTIONS:

      For Android DB:
      python whatsapp_xtract.py msgstore.db -w wa.db
      OR (if wa.db is unavailable)
      python whatsapp_xtract.py msgstore.db

      For iPhone DB: (-w option is ignored)
      python whatsapp_xtract.py ChatStorage.sqlite

      Once finished, your browser will open and show the chats.
      The resulting file size of the .html file will be slightly bigger than the size of the original .db.crypt12 database.

    Attachment: :)

    Whatsapp Xtract 2.3 2018-04-25 (~ 1,8 MB)
    contains:
    !README.txt
    bplist.py
    convert_smileys_python_2.py
    SetEnv.exe
    sqlite3.exe
    whatsapp_xtract.py
    whatsapp_xtract_android.bat
    whatsapp_xtract_iphone.bat
    whatsapp_xtract_console.bat
    whatsapp_xtract_drag'n'drop_database(s)_here.bat
    folder: data
    samples of needed input files and Media folder


    also available here:

    https://www.xup.in/dl,18324514/Whatsapp_Xtract_2.3_2018-04-25.zip/

    IF SOMETHING DOESN'T WORK:

    1. First check if you already have the newest download (check full name of the .zip file: is it the same as in mentioned in this thread?). Maybe your error already has been fixed.
    2. Read the post Some additional information and read the last pages of this thread to see if your problem is already being discussed.
    Use the Search inside this thread function!
    3. If not or if it doesn't help, post your error with a detailed description (and error message) in the thread.
    29
    Merge multiple sqlite database files

    Do you know if it's possible to merge the db-files into one with all messages?
    e.g. I have 7 backup files, i deleted some messages every day, and now I want to have an overview of all messages over these last 7 days.
    I saw that every message have an ID defined as primary key in the DB-file.
    Merging of multiple whatsapp databases is still on the feature request list.

    But you can use this tool to merge two sqlite databases and afterwards convert the merged database with Whatsapp Xtract:
    SQLite Compare
    http://www.codeproject.com/Articles/220018/SQLite-Compare-Utility

    HOW TO MERGE SQLITE DATABASE FILES WITH SQLite Compare:
    • 1. Open the old database file in SQLite Database Browser ( http://sqlitebrowser.sourceforge.net/ ). Open the table messages in Browse Data. Press the Button " > " as often as necessary to get to the last page. Scroll down to the last message. Now write down the number (_id) of that last message, e.g. 65422
    • 2. Now open the new database file in SQLite Database Browser. Open the table messages in Browse Data. Click on the first entry (which has most fields with value "-1"). Click on Delete Record. Have a look at the now first entry and write down the number (_id) of that first message, e.g. 12
      Now Click on Execute SQL and execute the following SQL statement:
      UPDATE messages set _id = (65422 + 1 + _id - 12)
      (replace 65422 by your last message _id in the old database and 12 by your first message _id in the new database)
    • 3. Now all messages in the new database have the correct _id's.
      Click on Save. Close all windows of SQLite Database Browser.
    • 4. Download & Install SQLite Compare ( http://www.codeproject.com/Articles/220018/SQLite-Compare-Utility ). Open SQLite Compare.
    • 5. File - Compare
      For example:
      Left file: msgstore-2012-06-01.1.plain.db
      Right file: msgstore-2012-06-07.1.plain.db
      (Make sure to have a backup of both files!)
      x Compare schema and data
      x Compare BLOB fields
      OK
    • 6. Double Click on table "messages" in the result window OR left click on table "messages" and hit button "Edit selected difference..."
    • 7. A new window opens, "Table messages"
      Click on blue button "R" (it's located after "Refresh Comparison", "L" and before "≠", "=")
    • 8. Now on the right side all messages are shown that aren't included in the left file.
      Now you can select multiple messages (first left click on the first message to select, then scroll down to the last message to select and press SHIFT + left click on the last message to select).
      I noticed that you can't handle too many messages at once as it produces an error. Just try. I succeeded with about 1000-1500 messages.

      Now click on the button with the arrow from right to left: ←

      Now the selected messages are copied from the right database to the left database.
    • 9. Repeat the steps in (8.) until all the messages are copied from the right file to the left file.
    • 10. Maybe you also need to to this for the table chat_list if there are some new chat contacts in the newer backup file.
    • 11. Now the left file (e.g. msgstore-2012-06-01.1.plain.db) contains all the messages from the prior file msgstore-2012-06-01.1.plain.db and the newer file msgstore-2012-06-07.1.plain.db. (No saving necessary as the tool automatically applies the changes to the file after pressing the ← button).
    • 12. Now you have 3 options how to go on:

      a) Do you want to merge another database file, appending it to the merged file? Then repeat the steps with an even newer database file on the right, e.g. msgstore-2012-06-12.1.plain.db, if you have multiple database files to merge.

      b) Do you want to import the merged database file to Whatsapp? Then rename the merged database file (the left one) to "msgstore.db", copy it to the sdcard to location /sdcard/WhatsApp/Databases/msgstore.db and make sure that it's the only file in that folder. Now install Whatsapp and confirm when asked to import the Backup during installation.

      c) Or do you want to convert the merged database file to browser-friendly html using Whatsapp Xtract? Then give the left file msgstore-2012-06-01.1.plain.db to Whatsapp Xtract, e.g. using drag and drop onto the whatsapp_xtract_drag'n'drop_database(s)_here.bat.
    21
    Changelog, Recent Bugfixes, Feature Requests

    CHANGELOG:

    v2.3 (updated by ztedd - Apr 25, 2018)
    - fixed image preview to work with new Whatsapp database format
    - updated Readme, refering to Whatsapp Viewer (which is able to decrypt the .crypt12 files)
    - some minor fixes

    v2.2 (updated by ztedd - Nov 17, 2012)
    - now supports new emoji smileys
    - (Android Version) hotfix for TypeError in b64encode
    - (Android Version) decoded file won't be deleted even if it can't be opened

    v2.1-bugsfixed-4 (2012-05-16)
    - fixed .bat files again to support script execution from each external directory
    - fixed .bat files if error "python" not found occurs (if you get that error, run !install pyCrypto.bat first; rightclick on it and choose "run as administrator")
    - updated compatibility for newer android whatsapp versions to show media files with thumbnails correctly

    v2.1 (updated by Fabio Sangiacomo and ztedd - May 7th, 2012)
    - improved install pyCrypto.bat
    - added easy drag and drop possibility with whatsapp_xtract_drag'n'drop_database(s)_here.bat
    - (Android Version) added support to fix corrupted android whatsapp database (needs sqlite3, for windows sqlite3.exe is contained in the archive)
    - (Android Version) removed wrong extraction of owner in android version
    - (Iphone Version) information from Z_METADATA table will be printed to shell
    - (Iphone Version) fixed bug in support of older Iphone whatsapp databases

    v2.0-bugsfixed-10
    file: Whatsapp_Xtract_V2.0_2012-05-02-3.zip
    - On errors, now the window will not close but show you an error message
    - Now installation of PyCrypto is indeed optional. You only need it if you want to import a crypted db.
    - added some advanced instructions in the Readme on how to use this script with the command line (also added a .bat that will open the command prompt and show the help message of the script)
    - changed error management to let the script continue on errors like these:
    * 'NoneType' object is not subscriptable in line 1028
    * 'NoneType' object is not subscriptable in line 1126
    The errors will be displayed in the message table and in the DOS Box, but the script will go on
    - fixed syntax error in line 216
    - fixed error that it tries to name output file "N/A" which is not possible because of the "/"
    - fixed error of handling timestamps in Iphone database
    - fixed " AttributeError: 'int' object has no attribute 'split' "
    - changed parsing of sqlite databases, columns are now accessed by name and not by number (e.g., msgs["ZTEXT"] instead of msgs[10])
    - included Iphone support for images, videos, vcard and (beta) gps. Fixed bug of opening a wrong output file in browser.
    - changed install pyCrypto.bat

    V2.0 (updated by Fabio Sangiacomo and ztedd - Apr 28, 2012)
    - supports WhatsApp DBs coming from both Android and iPhone platforms
    - (Android Version) wa.db is optional
    - (Android Version) now also crypted msgstore.db.crypt from the SD card can be imported
    - chat list is sorted by the last sent message
    - fixed some bugs (e.g. that the script didn't work with python 3)

    V1.3 (updated by ztedd - Apr 17, 2012)
    - corrected linking of offline files (now linking according to media file size)

    V1.2 (updated by ztedd - Apr 5, 2012)
    - media files also linked to offline files
    - corrected hyperlinks

    V1.1 (updated by ztedd - Apr 5, 2012)
    - changed database structure, Android only
    - show contact names
    - show smileys
    - show images
    - link / popup for images, video, audio, gps
    - clickable links

    V1.0 (created by Fabio Sangiacomo - Dec 10, 2011)
    - first release, iPhone only:
    it takes in input the file "ChatStorage.sqlite",
    extracts chat sessions and the bare text
    - sortable js allows table sorting to make chat sessions easily readable


    FEATURE REQUESTS:

    These features below are quite outdated. For example, conversion/migration from Iphone / Windows Phone to Android was made possible by paid apps (Wazzapmigrator / WinWazzapmigrator).

    Anyway, if you want to contribute, please take my source file, put a repo on github and go on!

    - Blackberry Support: Step 1 - How is it possible to get the unencrypted messagestore.db file from Blackberry Whatsapp? (valek07); Step 2 - get a sample db file (see how to create sample db) and include Blackberry support
    - Nokia Support: Step 1 - How to get the database file from Nokia? (oscargpes) Step 2 - Get a sample db file (see how to create sample db) and include Nokia support
    - Windows Phone Support (beastie_carrion): Step 1 - How to get the database file from Windows Phone? Step 2 - Get a sample db file (see how to create sample db) Step 3 - include Windows Phone support
    - create an android app for this, ideally offering "export to googlemail" function as in SMS Backup + (nomad4ever)
    / conversion of whatsapp to sms xml (ofmb)
    - conversion between Android and Iphone database:
    -- Android -> Iphone: ark0n3 is working on a Java tool called WhatsAppMigrator, here you can download WhatsAppMigrator
    -- Iphone -> Android: no solution yet
    - merge multiple backup files (darKpoiSonend) (you can use the tool SQLite Compare as a workaround, see howto in this post by ztedd)
    - extract deleted messages (sometimes deleted messages remain in the database file but can't be accessed with normal Sqlite commands anymore) (Rauken)
    - include Iphone support for audio, gps, gps of a place (like a restaurant), group messages and change of group name -> we would need an Iphone sample database containing these message types!
    14
    Some Additional Information

    General advice on how to backup Whatsapp

    Some general advice on how to backup Whatsapp and get the database file:

    Android:
    - In Whatsapp go to settings - more - Backup Chats
    - Copy the folder "Whatsapp" on the SD card to your backup location (e.g., PC)
    - (ideally also) use the app Titanium Backup to backup the full whatsapp application together with its data, copy the backup from the folder "TitaniumBackup" on the SD card to your backup location (e.g., PC)
    - Use this tool Whatsapp Backup Extractor (download in this thread) to show the chats in a friendly readable format. The necessary files "wa.db" and "msgstore.db" you will find inside the Titanium Backup archive "com.whatsapp-[Date]-[some digits].tar.gz", alternatively (without Titanium Backup) you can use the msgstore.db.crypt file from the folder Whatsapp/Databases on the SD card.

    Iphone:
    - use Itunes to create an unencrypted Backup
    - use an Iphone Backup Tool to get the file net.whatsapp.WhatsApp/Documents/ChatStorage.sqlite, e.g. I-Twin or Iphone Backup Extractor. Make sure to create an unencrypted backup with Itunes, as these tools can't handle encrypted backups. Another possibility are forensic tools like UFED Physical Analyzer.)

    Blackberry:
    - sync your blackberry with desktop manager and then copy the messagestore.db file from SD
    - however, it seems that this file is encrypted? Currently we don't know how to get the unencrypted messagestore.db file
    - Blackberry not supported yet!

    Nokia:
    - not known yet
    - Nokia not supported yet!

    CHECK IF DATABASE FILE CAN BE OPENED / EDIT CHAT MESSAGES INSIDE DATABASE

    You may want to open the decrypted whatsapp database (ChatStorage.sqlite, msgstore.db or msgstore.plain.db) without this tool, e.g. if something goes wrong to check if it can be opened at all or to edit chat messages.
    For this use SQLite Database Browser ( http://sqlitebrowser.sourceforge.net/ ).
    However it is much more confusing and the messages are ordered by date, not by conversations. Also you won't see the smileys and media files...

    IF DATABASE CAN'T BE OPENED: REPAIR DATABASE

    Update: This function is now included in Whatsap Xtract 2.1.
    If you get an error like this: Error: file is encrypted or is not a database and you tried to open the unencrypted database with SQLite Database Browser but it can't be opened with it, then try the following:
    Open the database with Editor, can you see it starts with "SQLite format 3"? Then you may try to repair the database.
    Thanks to hape70 for the instructions on how to repair (I translated it from German to English):
    - download sqlite3.exe from http://www.sqlite.org/download.html
    - start whatsapp_xtract_console.bat and then type:
    - echo .dump | sqlite3.exe msgstore.db > Temp.sql
    - echo .quit | sqlite3.exe -init Temp.sql msgstore_new.db
    - python whatsapp_xtract.py msgstore_new.db
    (make sure to replace the underlined msgstore.db with the name of the database that is unencrypted but can't be opened, sometimes it will be msgstore.plain.db)

    MEDIA FILES

    If you want to watch the videos, audios and images, you can click on the thumbnails and media links. Popups should open displaying the media.
    However, online media files are available only for the last ~ 3 weeks.
    But you still can open the offline media files, they are linked as well.
    For this it is necessary to at first copy the folder "Media" from /sdcard/Whatsapp (Android) or net.whatsapp.WhatsApp (Iphone) to the certain folder of your computer where this tool is installed. Then run the script. Now the media files should be correctly linked.

    HOW TO MERGE MULTIPLE DATABASE FILES WITH SQLite Compare

    Merging of multiple backup files is still on the feature request list, but you can use the tool SQLite Compare as a workaround, see howto in this post by ztedd.
    5
    I'd like to me able to import a backup into my current conversations in whatsapp or merge one or more db.crypt files together so that I can import them all into whatsapp.

    I had a lot of conversations in my backup, but instead of importing them, my whatsapp received 613 new messages so I don't have any of my old conversations. I can import them, but then I lose my current conversations and vice versa.

    I'm willing to play around with code if it helps. My speciality is SQL, ASP, Java (not as good at it anymore, but can pick it up fairly quickly)
    You can try the following:

    1. Create a new backup (menu Button - Settings - Chat Settings - Backup Chat history)
    2. Force-Close Whatsapp so that it won't receive any new messages.
    3. now you should have two .db.crypt files, the older one with all your backupped chats and the newer one with ~ 613+ messages. Use Whatsapp Xtract to convert the .db.crypt files to .htm and unencrypted .db files
    4. Now use the tool SQLite Compare (see howto) to merge the two .db files into one
    5. Now copy the merged .db file as "msgstore.db" into the folder /sdcard/WhatsApp/Databases and make sure that there isn't any other file there (copy the other files in another folder)
    6. uninstall whatsapp, install whatsapp and choose yes when asked to import the backup

    If you want to include the merging function into the code of Whatsapp Xtract, you should have a look at the SQLite commands of Python. It's quite similar to SQL. Until someone has done this, you can use the mentioned tool SQLite Compare, though it's a bit difficult to use.