[TOOL] yaffs extractor, mmssms.db & contacts2.db converter

[abbot2]

Some time ago I sold my old android phone and forgot to make a backup of SMS messages and call logs, but kept a complete image backup made by clockworkmod recovery (system.img, data.img, etc.). I wanted to import my SMS messages and call logs to a new phone, but without a reasonable backup this seemed to be impossible. Some quick search over the internet showed that a question about extracting/importing SMS messages from mmssms.db is not so uncommon, but there are no tools to do this. So I wrote one, and decided to share

Sources are available at github.com abbot/android-restore-tools.
It requires Python 2.6 and above to run.

I've also made compiled binaries for windows, attached to this post.

[Binaries last updated on 12.04.2011]

 

[jamestts]

Thanks for the tools. Nice work.

Sent from my GT-I8150 using XDA

 

[dcarr622]

Hello,

I'm trying to use your tool to extract SMS from a nandroid backup.

I'm using the data.yaffs2.img file from the backup.

When I choose #2 for mmssms.db and then "s" to extract SMS I get the following error:

Failed to extract messages: file is encrypted or is not a database
DatabaseError('file is encrypted or is not a database',)
Warning: failed to remove temporary file...

What does this mean, and is there a solution?

 

[abbot2]

Hi,

This may be caused by two things: either my tool can't properly read/extract the image file, or it can't read the database.

Please try to extract the image (extract -x data.yaffs2.img). If this does not produce any errors, find the mmssms.db file in the extracted data, it will probably be in data/com.android.providers.telephony/databases/mmssms.db. Then try to run extract -s mmssms.db. Please post if you get any errors doing these steps.

 

[dcarr622]

There was an error while extracting the image...it got through partway, but then failed at some bluetooth directory with colons in the path.

So I used a different tool to extract it, which went successfully.

Then I ran extract -s mmssms.db which came with the same error.

I'm pretty sure there isn't some weird encryption since I opened it up with Notepad ++ and I could read bits and pieces of conversations.

Anything else to try?

 

[abbot2]

This might be caused by an older sqlite3 version bundled with binaries. I have updated the binaries in the first message to a newer version, could you download it and try again, extract -s mmssms.db?

 

[dcarr622]

Different error this time:

Failed to extract messages: no such column: failure_cause
OperationalError('no such column: failure_cause',)
Warning: failed to remove temporary file...

FYI the first thing at the top of the file when I open in Notepad++ is SQLite format 3 and this is from an HTC device running Android 4.x does that help at all?

 

[abbot2]

This is much better and now makes sense: android 4.0 usually has sqlite 3.7.x, previous binary build of this tool had sqlite 3.6.21, and that was the reason for the 'file is encrypted or is not a database' error.

Now it looks like mmssms.db format in Android 4.0 has changed a little bit. I will have a look on these changes and update the app accordingly. Hope it will not take too much time

 

[dcarr622]

This is much better and now makes sense: android 4.0 usually has sqlite 3.7.x, previous binary build of this tool had sqlite 3.6.21, and that was the reason for the 'file is encrypted or is not a database' error.

Now it looks like mmssms.db format in Android 4.0 has changed a little bit. I will have a look on these changes and update the app accordingly. Hope it will not take too much time

That would be awesome, perhaps keep the old version available for other folks too though

If you can get this working I'll be sure to send a couple bucks your way. Thanks!

 

[abbot2]

I have updated the extractor again, did some limited testing with android 4.0.3 on the emulator - seems to work. This database has a number of fields removed on android 4 compared to android 2, however everything required for xml dump is still there - just had to remove some unused stuff. Download the new version and try again.

Regarding the older versions, no reason to keep them - new one works fine with old database formats.

 

[dcarr622]

Awesome works perfectly! Send me a PM where to donate.

Sent from my GT-I9000 using xda premium

 

[Bonehead Fraggle]

Thanks! It really works perfectly. Any plans to include MMS?

 

[niuhap]

Sorry for the noobish question but I'm not familiar with Python.
Installed newest version of Python and also got the extracted mmssms.db and contacts2.db (used Nandriod Browser to extract them), which command lines do I need/what do I have to do? :s

€dit: Downloaded the zip from github and got as far as opening mmssms2xml.py which gives me an error when opening the mmssms.db

Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Python27\lib\lib-tk\Tkinter.py", line 1410, in __call__
return self.func(*args)
File "D:\Downloads\abbot-android-restore-tools-bc8584d\abbot-android-restore-t
ools-bc8584d\mmssms2xml.py", line 99, in open_file
self.messages = read_messages(filename)
File "D:\Downloads\abbot-android-restore-tools-bc8584d\abbot-android-restore-t
ools-bc8584d\mmssms2xml.py", line 33, in read_messages
c.execute("SELECT _id, thread_id, address, person, date, protocol, read, pri
ority, status, type, callback_number, reply_path_present, subject, body, service
_center, failure_cause, locked, error_code, stack_type, seen, sort_index FROM sm
s ORDER BY date DESC")
OperationalError: no such column: priority

€dit2: No problems with contacts2.db?!

€dit3: Well..this should be the last, found this explanation of rani2001 over here (http://forum.xda-developers.com/showpost.php?p=25173166&postcount=7). That did it for me!

 

[KRiODOXiS]

Dude you just made my day

 

[klauer7]

Can you please compile a new windows version with the latest files from github?
Or give us a manual how to use the .py files.

I think this will help many of us.



Thank you very much.

BTW: Your tool worked perfectly with my mmssms.db but failed with contacts2.db. Maybe the new files requested above will help me.

 

[arielc]

Got all my sms back thanks to you.

Amazing script, Thanks for sharing and explaining and updating.

 

[RubiksHnK]

60% There

I was just wondering if there was anything special I had to do to merge my old SMS list and the ones from the new list. I didn't see an option there for it and I don't want to just paste it in there and load it. Many thanks ahead of time

 

[kienerii]

Hi I want to extract the data.yaffs2.img from my HTC Desire nandroid backup but get the error message:

> extract.exe -s data.yaffs2.img
Failed to extract messages: file is encrypted or is not a database

Any ideas?

 

[freakymonkey]

I tried this tool yesterday evening and I'm truly grateful for your work!
I was ready to spend the whole night to work with these damn tables when I found your work. Many thanks! :highfive:

 

[damastah]

Where is the file stored after i run it? This is what I got.

C:\Users\Damastah>C:\Users\Damastah\Downloads\yaffs-mmssmsdb-calls-extractor\ext
ract.exe -s D:\S3_recovery\mmssms.db
Read 1782 messages
Save as (empty=sms-20130330111807.xml): y