Welcome to XDA

Search to go directly to your device's forum

Register an account

Unlock full posting privileges

Ask a question

No registration required
Post Reply

eMMC sudden death research

OP Oranav

17th January 2013, 09:48 PM   |  #11  
Member
Thanks Meter: 21
 
56 posts
Join Date:Joined: Dec 2010
More
Quote:
Originally Posted by Rob2222

@ivan:
Sorry, can't do that. Cause of high air humidity my humidity indicator is already a little soaked. Cause of the warranty-repair reports in out local forums I am not sure if I would get warranty. I think theres a fair chance, that they would deny my warranty. Cause of that I don't want to take any extra risk. I am on unrooted stock at the moment, cause of that.

@Oranav:
In our local forum we get some reports about a rising count of locks and restarts on S3's in the last time. Some like my freeze.
It also seems that after a while this problems gets better and even disappear completely.
Cause of that I am thinking, if it could be, that the fix maybe locks the eMMC if it finds a bad data structure, then this locks maybe could bring a phone-freeze (already stated that), and in the same time it repairs the data structure in this block with the bad data structure.
At least this would explain some rising count of freezes with the fix and the point, that the freezes become less and less over time.

I have no idea if it's that way, I just wanted to post it as theory to think about.

BTW, do you think when the watchdog restarts the eMMC that it goes that fast that the phone isn't affected?

BR
Robert

if those freezes are really caused by the firmware fix, I don't see how the would disappear over time...

I mean if it really is the case that the fix trades data corruption for eMMC survival, it would make sense to see freezes... but depending on what data is affected, they should only be treatable by reinstalling the affected app or deleting its data/cache.

updated theory:
for all we know the error condition where the eMMC dies is quite rare, since most devices have been used for month before they passed away. So under the assumption that the error condition appears randomly and that there is a chance of data corruption every time the condition appears with fixed kernel, we could expect to see freezes and other problems some time after the fix was applied. So that would explain the raising number of freezes reported. Furthermore I'd assume that people getting freezes would try to do something about it, like reinstalling/deleting apps wiping caches and/or data... or even reflashing, thus repairing the corrupted data. So freezes would disappear.

Wait, doesn't most evidence point to the fact that the error condition does NOT appear in a random fashion, since there were no cases in the beginning, and then a lot all of a sudden? Well, it might be that this is just the way we perceive the issue. Maybe there were cases before, but they weren't reported... phones died, people sent them in, got new ones and went on with their lives. But after some time the issue got known... bloggers wrote about it... and so on... people realized their phones died because of a wider problem... voila, steep raise in reported cases. Also the number of dying S3s would simply rise by a rising number of overall S3s, I mean Samsung kept selling phones, right?

But even under the assumption the bug is related to wear-levelling and not random, here is another idea: I have no clue how the algorithms work, but maybe it uses some sort of pseudo-random data to do whatever, with the same seed on all eMMCs... and thus all of them go through the same series of numbers. And now imagine the error condition is only triggered by a specific number or number set (say someone screwed up a boundary condition). Under this theory the error condition wouldn't appear randomly, but after a certain amount of write ops (or something).

Another question I asked myself is: shouldn't there be cases were data corruption does damage beyond all repair except for reflashing?
Well, it might be, but it seems reasonable to assume that it is a lot less likely than user-data corruption, since most critical files on the phone shouldn't be opened writeable (or are on a read-only mounted partition in the first place), hence shouldn't be affected by ****-ups during writes.

Like the previous poster I want to add that this is most likely all bull****... but it is what came to my mind looking for a theory that supports the data we got.
18th January 2013, 12:41 AM   |  #12  
OP Member
Thanks Meter: 259
 
52 posts
Join Date:Joined: Oct 2010
More
Okay, got a RAM dump
I won't post it here (or anywhere else for that matter) because I don't want to get sued by Samsung.

I might release a kernel which allows you to dump the RAM yourself if there's enough demand, but I don't want to right now, because:
1. The code is ugly as hell, not implemented as a kernel module, not thread-safe etc.
2. It is highly dangerous (messing with the eMMC chip - I really don't know how much stable this thing is), so if you want to do it on your device, you should be an expert. In that case, you can write the code yourself (with little effort)


Anyway, I hope the FTL is Whimory, since I'm familiar with it. Would be easier.
I'll let you know if I find anything interesting.


PS I've attached a little teaser. (Yes, this is the patched function. 0x40300 is red because I've opened a partial RAM dump.)



EDIT - Some initial results:
0. The CPU is a Cortex-M3.
1. No strings at all Just some uninteresting release asserts ("REL_ASSERT")
2. Found the Smart Report generator function -> found the MMC command handlers.
3. Most MMC commands handlers are stored in a function table. There are 3 special commands: MMC60, MMC62, MMC64. Depends on the arguments these special commands are provided, they modify the function table (this is the so called "vendor mode").
4. There are a lot of possible arguments for MMC62, not the only ones we know.
5. If you trace back the function they patch all the way up the call stack, you get to MMC24 and MMC25 handler. These commands are MMC_WRITE_BLOCK and MMC_WRITE_MULTIPLE_BLOCK. Since the function they patch is deep down the call stack, it's very likely that it is the wear level.

Anyway, because of the lack of strings I guess it would be very hard to truly understand the SDS bug we're facing
Attached Thumbnails
Click image for larger version

Name:	emmc.png
Views:	1853
Size:	8.2 KB
ID:	1654102  
Last edited by Oranav; 18th January 2013 at 03:19 AM. Reason: Added even more results
The Following 28 Users Say Thank You to Oranav For This Useful Post: [ View ]
18th January 2013, 09:13 PM   |  #13  
Member
Thanks Meter: 23
 
98 posts
Join Date:Joined: Apr 2011
Odp: eMMC sudden death research
i cant say i have an idea whats going on inside emmc but usually in this case of mistakes/failures debug or diagnostics code is used for release.
maybe some debug info repeatedly written triggers wear levelling failure
so fix has to simply disable it
19th January 2013, 09:26 AM   |  #14  
Senior Recognized Developer
Flag Gdańsk
Thanks Meter: 3,437
 
1,340 posts
Join Date:Joined: May 2009
Donate to Me
Awesome research.
So we're dealing with bug in exactly the same eMMC subsystem as in faulty SGS2 eMMC chips, but in device that was released after proving SGS2 eMMCs to be faulty.
[IMG]http://***********************/instances/400x/33621297.jpg[/IMG]

Oranav, for some reason I cannot send you PMs. Could you send me your dump? Does your eMMC come from faulty serie?
The Following 6 Users Say Thank You to Rebellos For This Useful Post: [ View ]
20th January 2013, 12:36 PM   |  #15  
Senior Member
Thanks Meter: 570
 
328 posts
Join Date:Joined: Nov 2012
Donate to Me
Hi all, after reading this thread, I am now scared....

I have a Note 2 N7100 which is running ARHD V8.0 with Perseus Kernel V31.2 and TWRP recovery 3.2.2.3

The above all include the fix for SDS and Exynos hole.

I have been running the device for nearly 1 week I think. Last night I fully charged my phone, used it for 3 minutes surfing the forum (chrome) via wifi connection. After 3 minutes, I left the phone on the table for 3 hours. The only running app is Viber... when I tried to wake the phone up, it did wake up but it froze... no button worked, I tried for about 2 minutes... nothing worked except the power button which booted the device.

This is weird, never experienced this before... I am now scared the phone will die unexpectedly.
20th January 2013, 02:20 PM   |  #16  
Junior Member
Thanks Meter: 7
 
25 posts
Join Date:Joined: Jan 2007
Quote:
Originally Posted by owl74

Hi all, after reading this thread, I am now scared....

I have a Note 2 N7100 which is running ARHD V8.0 with Perseus Kernel V31.2 and TWRP recovery 3.2.2.3

The above all include the fix for SDS and Exynos hole.

I have been running the device for nearly 1 week I think. Last night I fully charged my phone, used it for 3 minutes surfing the forum (chrome) via wifi connection. After 3 minutes, I left the phone on the table for 3 hours. The only running app is Viber... when I tried to wake the phone up, it did wake up but it froze... no button worked, I tried for about 2 minutes... nothing worked except the power button which booted the device.

This is weird, never experienced this before... I am now scared the phone will die unexpectedly.

How long were you using the system before you had updated to the 'fix'? None the less, it does not necessarily mean that the phone is getting near to the SDS. I have had a few android phones which would sometimes reboot or hang for other reasons.
20th January 2013, 02:24 PM   |  #17  
OP Member
Thanks Meter: 259
 
52 posts
Join Date:Joined: Oct 2010
More
Re: eMMC sudden death research
I tried to simulate an eMMC freeze (by forcing it to go into an infinite loop). It behaves exactly as you describe - the phone works for a second, then becomes totally unresponsive. Seems like there is no watchdog.

Rebellos, I enabled the private messaging system for me. I do have the faulty chip.

Sent from my GT-I9300 using xda app-developers app
The Following 8 Users Say Thank You to Oranav For This Useful Post: [ View ]
20th January 2013, 03:21 PM   |  #18  
Senior Member
Thanks Meter: 300
 
412 posts
Join Date:Joined: Feb 2008
Quote:
Originally Posted by Oranav

I tried to simulate an eMMC freeze (by forcing it to go into an infinite loop). It behaves exactly as you describe - the phone works for a second, then becomes totally unresponsive. Seems like there is no watchdog.

Damn Oranav, nice work!
So does it mean you get a total screen freeze? Every time?

BR
Robert
20th January 2013, 03:24 PM   |  #19  
Senior Member
Thanks Meter: 570
 
328 posts
Join Date:Joined: Nov 2012
Donate to Me
Thumbs up
Quote:
Originally Posted by thealgorithm

How long were you using the system before you had updated to the 'fix'? None the less, it does not necessarily mean that the phone is getting near to the SDS. I have had a few android phones which would sometimes reboot or hang for other reasons.

About 2 weeks. Forgot to mention i reebooted the phone before I charged it. I will return this phone before it dies on me.... I think I will get an S3 but I will check if it has a new chip... otherwise I will return again and stick to my desire hd which is already running S3 rom....

---------- Post added at 02:24 PM ---------- Previous post was at 02:22 PM ----------

Quote:
Originally Posted by Oranav

I tried to simulate an eMMC freeze (by forcing it to go into an infinite loop). It behaves exactly as you describe - the phone works for a second, then becomes totally unresponsive. Seems like there is no watchdog.

Rebellos, I enabled the private messaging system for me. I do have the faulty chip.

Sent from my GT-I9300 using xda app-developers app

So everyone's speculation is right about thw fix causing the freeze...
22nd January 2013, 06:48 AM   |  #20  
Member
Thanks Meter: 6
 
44 posts
Join Date:Joined: Jun 2011
AW: eMMC sudden death research
Suppose this fix addresses wear leveling. If firmwares without this fix wear out the eMMC would then not the device still boot and then crash? As far as I know flash is still readable but not writable any more when worn out. Could it be that the wear leveling algorithm has a problem so that after some time it replaces cells from the bootloader and that causes the death?

In short: I want to know if it had a negative effect using the old firmware for some time because that old software caused extreme aging for the eMMC.

Post Reply Subscribe to Thread
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes