5,594,282 Members 43,072 Now Online
XDA Developers Android and Mobile Development Forum

eMMC sudden death research

Tip us?
 
odoto
Old
#11  
Member
Thanks Meter 21
Posts: 56
Join Date: Dec 2010
Quote:
Originally Posted by Rob2222 View Post
@ivan:
Sorry, can't do that. Cause of high air humidity my humidity indicator is already a little soaked. Cause of the warranty-repair reports in out local forums I am not sure if I would get warranty. I think theres a fair chance, that they would deny my warranty. Cause of that I don't want to take any extra risk. I am on unrooted stock at the moment, cause of that.

@Oranav:
In our local forum we get some reports about a rising count of locks and restarts on S3's in the last time. Some like my freeze.
It also seems that after a while this problems gets better and even disappear completely.
Cause of that I am thinking, if it could be, that the fix maybe locks the eMMC if it finds a bad data structure, then this locks maybe could bring a phone-freeze (already stated that), and in the same time it repairs the data structure in this block with the bad data structure.
At least this would explain some rising count of freezes with the fix and the point, that the freezes become less and less over time.

I have no idea if it's that way, I just wanted to post it as theory to think about.

BTW, do you think when the watchdog restarts the eMMC that it goes that fast that the phone isn't affected?

BR
Robert
if those freezes are really caused by the firmware fix, I don't see how the would disappear over time...

I mean if it really is the case that the fix trades data corruption for eMMC survival, it would make sense to see freezes... but depending on what data is affected, they should only be treatable by reinstalling the affected app or deleting its data/cache.

updated theory:
for all we know the error condition where the eMMC dies is quite rare, since most devices have been used for month before they passed away. So under the assumption that the error condition appears randomly and that there is a chance of data corruption every time the condition appears with fixed kernel, we could expect to see freezes and other problems some time after the fix was applied. So that would explain the raising number of freezes reported. Furthermore I'd assume that people getting freezes would try to do something about it, like reinstalling/deleting apps wiping caches and/or data... or even reflashing, thus repairing the corrupted data. So freezes would disappear.

Wait, doesn't most evidence point to the fact that the error condition does NOT appear in a random fashion, since there were no cases in the beginning, and then a lot all of a sudden? Well, it might be that this is just the way we perceive the issue. Maybe there were cases before, but they weren't reported... phones died, people sent them in, got new ones and went on with their lives. But after some time the issue got known... bloggers wrote about it... and so on... people realized their phones died because of a wider problem... voila, steep raise in reported cases. Also the number of dying S3s would simply rise by a rising number of overall S3s, I mean Samsung kept selling phones, right?

But even under the assumption the bug is related to wear-levelling and not random, here is another idea: I have no clue how the algorithms work, but maybe it uses some sort of pseudo-random data to do whatever, with the same seed on all eMMCs... and thus all of them go through the same series of numbers. And now imagine the error condition is only triggered by a specific number or number set (say someone screwed up a boundary condition). Under this theory the error condition wouldn't appear randomly, but after a certain amount of write ops (or something).

Another question I asked myself is: shouldn't there be cases were data corruption does damage beyond all repair except for reflashing?
Well, it might be, but it seems reasonable to assume that it is a lot less likely than user-data corruption, since most critical files on the phone shouldn't be opened writeable (or are on a read-only mounted partition in the first place), hence shouldn't be affected by ****-ups during writes.

Like the previous poster I want to add that this is most likely all bull****... but it is what came to my mind looking for a theory that supports the data we got.
 
Oranav
Old
(Last edited by Oranav; 18th January 2013 at 02:19 AM.) Reason: Added even more results
#12  
Member - OP
Thanks Meter 256
Posts: 50
Join Date: Oct 2010
Okay, got a RAM dump
I won't post it here (or anywhere else for that matter) because I don't want to get sued by Samsung.

I might release a kernel which allows you to dump the RAM yourself if there's enough demand, but I don't want to right now, because:
1. The code is ugly as hell, not implemented as a kernel module, not thread-safe etc.
2. It is highly dangerous (messing with the eMMC chip - I really don't know how much stable this thing is), so if you want to do it on your device, you should be an expert. In that case, you can write the code yourself (with little effort)


Anyway, I hope the FTL is Whimory, since I'm familiar with it. Would be easier.
I'll let you know if I find anything interesting.


PS I've attached a little teaser. (Yes, this is the patched function. 0x40300 is red because I've opened a partial RAM dump.)



EDIT - Some initial results:
0. The CPU is a Cortex-M3.
1. No strings at all Just some uninteresting release asserts ("REL_ASSERT")
2. Found the Smart Report generator function -> found the MMC command handlers.
3. Most MMC commands handlers are stored in a function table. There are 3 special commands: MMC60, MMC62, MMC64. Depends on the arguments these special commands are provided, they modify the function table (this is the so called "vendor mode").
4. There are a lot of possible arguments for MMC62, not the only ones we know.
5. If you trace back the function they patch all the way up the call stack, you get to MMC24 and MMC25 handler. These commands are MMC_WRITE_BLOCK and MMC_WRITE_MULTIPLE_BLOCK. Since the function they patch is deep down the call stack, it's very likely that it is the wear level.

Anyway, because of the lack of strings I guess it would be very hard to truly understand the SDS bug we're facing
Attached Thumbnails
Click image for larger version

Name:	emmc.png
Views:	1728
Size:	8.2 KB
ID:	1654102  
The Following 28 Users Say Thank You to Oranav For This Useful Post: [ Click to Expand ]
 
resore
Old
#13  
Member
Thanks Meter 23
Posts: 96
Join Date: Apr 2011
Default Odp: eMMC sudden death research

i cant say i have an idea whats going on inside emmc but usually in this case of mistakes/failures debug or diagnostics code is used for release.
maybe some debug info repeatedly written triggers wear levelling failure
so fix has to simply disable it
 
Rebellos
Old
#14  
Senior Recognized Developer
Thanks Meter 3384
Posts: 1,334
Join Date: May 2009
Location: Gdańsk

 
DONATE TO ME
Awesome research.
So we're dealing with bug in exactly the same eMMC subsystem as in faulty SGS2 eMMC chips, but in device that was released after proving SGS2 eMMCs to be faulty.


Oranav, for some reason I cannot send you PMs. Could you send me your dump? Does your eMMC come from faulty serie?
Feedback on my development is highly appreciated, but first you should read this GUIDE and watch this MOVIE.

If you like my work - you can help me getting various cool stuff by clicking donation link in my profile. It's not required while pressing is, just appreciated.

Pretty owsom Android/Kernel dev tips&tricks: http://omappedia.org/wiki/Android_How-tos

Git HOW-TO by eagleeyetom: http://forum.xda-developers.com/show...php?p=31304826
15-minutes GIT introduction: http://try.github.com
If you want to submit patches to my git projects - use the guides above and make a pull request.
The Following 6 Users Say Thank You to Rebellos For This Useful Post: [ Click to Expand ]
 
owl74
Old
#15  
Senior Member
Thanks Meter 558
Posts: 327
Join Date: Nov 2012

 
DONATE TO ME
Hi all, after reading this thread, I am now scared....

I have a Note 2 N7100 which is running ARHD V8.0 with Perseus Kernel V31.2 and TWRP recovery 3.2.2.3

The above all include the fix for SDS and Exynos hole.

I have been running the device for nearly 1 week I think. Last night I fully charged my phone, used it for 3 minutes surfing the forum (chrome) via wifi connection. After 3 minutes, I left the phone on the table for 3 hours. The only running app is Viber... when I tried to wake the phone up, it did wake up but it froze... no button worked, I tried for about 2 minutes... nothing worked except the power button which booted the device.

This is weird, never experienced this before... I am now scared the phone will die unexpectedly.
 
thealgorithm
Old
#16  
Junior Member
Thanks Meter 7
Posts: 25
Join Date: Jan 2007
Quote:
Originally Posted by owl74 View Post
Hi all, after reading this thread, I am now scared....

I have a Note 2 N7100 which is running ARHD V8.0 with Perseus Kernel V31.2 and TWRP recovery 3.2.2.3

The above all include the fix for SDS and Exynos hole.

I have been running the device for nearly 1 week I think. Last night I fully charged my phone, used it for 3 minutes surfing the forum (chrome) via wifi connection. After 3 minutes, I left the phone on the table for 3 hours. The only running app is Viber... when I tried to wake the phone up, it did wake up but it froze... no button worked, I tried for about 2 minutes... nothing worked except the power button which booted the device.

This is weird, never experienced this before... I am now scared the phone will die unexpectedly.
How long were you using the system before you had updated to the 'fix'? None the less, it does not necessarily mean that the phone is getting near to the SDS. I have had a few android phones which would sometimes reboot or hang for other reasons.
 
Oranav
Old
#17  
Member - OP
Thanks Meter 256
Posts: 50
Join Date: Oct 2010
Default Re: eMMC sudden death research

I tried to simulate an eMMC freeze (by forcing it to go into an infinite loop). It behaves exactly as you describe - the phone works for a second, then becomes totally unresponsive. Seems like there is no watchdog.

Rebellos, I enabled the private messaging system for me. I do have the faulty chip.

Sent from my GT-I9300 using xda app-developers app
The Following 8 Users Say Thank You to Oranav For This Useful Post: [ Click to Expand ]
 
Rob2222
Old
#18  
Senior Member
Thanks Meter 295
Posts: 405
Join Date: Feb 2008
Quote:
Originally Posted by Oranav View Post
I tried to simulate an eMMC freeze (by forcing it to go into an infinite loop). It behaves exactly as you describe - the phone works for a second, then becomes totally unresponsive. Seems like there is no watchdog.
Damn Oranav, nice work!
So does it mean you get a total screen freeze? Every time?

BR
Robert
 
owl74
Old
#19  
Senior Member
Thanks Meter 558
Posts: 327
Join Date: Nov 2012

 
DONATE TO ME
Quote:
Originally Posted by thealgorithm View Post
How long were you using the system before you had updated to the 'fix'? None the less, it does not necessarily mean that the phone is getting near to the SDS. I have had a few android phones which would sometimes reboot or hang for other reasons.
About 2 weeks. Forgot to mention i reebooted the phone before I charged it. I will return this phone before it dies on me.... I think I will get an S3 but I will check if it has a new chip... otherwise I will return again and stick to my desire hd which is already running S3 rom....

---------- Post added at 02:24 PM ---------- Previous post was at 02:22 PM ----------

Quote:
Originally Posted by Oranav View Post
I tried to simulate an eMMC freeze (by forcing it to go into an infinite loop). It behaves exactly as you describe - the phone works for a second, then becomes totally unresponsive. Seems like there is no watchdog.

Rebellos, I enabled the private messaging system for me. I do have the faulty chip.

Sent from my GT-I9300 using xda app-developers app
So everyone's speculation is right about thw fix causing the freeze...
 
SamsungPisser
Old
#20  
Member
Thanks Meter 5
Posts: 43
Join Date: Jun 2011
Default AW: eMMC sudden death research

Suppose this fix addresses wear leveling. If firmwares without this fix wear out the eMMC would then not the device still boot and then crash? As far as I know flash is still readable but not writable any more when worn out. Could it be that the wear leveling algorithm has a problem so that after some time it replaces cells from the bootloader and that causes the death?

In short: I want to know if it had a negative effect using the old firmware for some time because that old software caused extreme aging for the eMMC.

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes