Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,810,941 Members 38,537 Now Online
XDA Developers Android and Mobile Development Forum

eMMC sudden death research

Tip us?
 
Oranav
Old
(Last edited by Oranav; 17th February 2013 at 05:50 PM.)
#1  
Member - OP
Thanks Meter 259
Posts: 52
Join Date: Oct 2010
Default eMMC sudden death research

Update from Feb 17th:
Samsung has started to upgrade eMMC firmwares on the field - only for GT-I9100 for now.
See post #79 for additional details.

Update from Feb 13th:
If you want to dump the eMMC's RAM yourself, go ahead to post #72.
I'm looking for a dump of firmware revision 0xf7 if you've got one.
-----------------------


Since it's very likely that the recent eMMC firmware patch by Samsung is their patch for the "sudden death" issue, it would be very nice to understand what is really going on there.

According to a leaked moviNAND datasheet, it seems that MMC CMD62 is vendor-specific command that moviNAND implements.
If you issue CMD62(0xEFAC62EC), then CMD62(0xCCEE) - you can read a "Smart report". To exit this mode, issue CMD62(0xEFAC62EC), then CMD62(0xDECCEE).


So what are they doing in their patch?

1. Whenever an MMC is attached:
a. If it is "VTU00M", revision 0xf1, they read a Smart report.
b. The DWORD at Smart[324:328] represents a date (little-endian); if it is not 0x20120413, they don't patch the firmware. (Maybe only chips from 2012/04/13 are buggy?)
2. If the chip is buggy, whenever an MMC is attached or the device is resumed:
a. Issue CMD62(0xEFAC62EC) CMD62(0x10210000) to enter RAM write mode. Now you can write to RAM by issuing MMC_ERASE_GROUP_START(Address to write) MMC_ERASE_GROUP_END(Value to be written) MMC_ERASE(0).
b. *(0x40300) = 10 B5 03 4A 90 47 00 28 00 D1 FE E7 10 BD 00 00 73 9D 05 00
c. *(0x5C7EA) = E3 F7 89 FD
d. Exit RAM write mode by issuing CMD62(0xEFAC62EC) CMD62(0xDECCEE).
10 B5 looks like a common Thumb push (in ARM architecture). Disassembling the bytes that they write to 0x40300 yields the following code:
Code:
ROM:00040300                 PUSH    {R4,LR}
ROM:00040302                 LDR     R2, =0x59D73
ROM:00040304                 BLX     R2
ROM:00040306                 CMP     R0, #0
ROM:00040308                 BNE     locret_4030C
ROM:0004030A
ROM:0004030A loc_4030A                               ; CODE XREF: ROM:loc_4030Aj
ROM:0004030A                 B       loc_4030A
ROM:0004030C ; ---------------------------------------------------------------------------
ROM:0004030C
ROM:0004030C locret_4030C                            ; CODE XREF: ROM:00040308j
ROM:0004030C                 POP     {R4,PC}
ROM:0004030C ; ---------------------------------------------------------------------
Disassembling what they write to 0x5C7EA yields this:
Code:
ROM:0005C7EA                 BL      0x40300
Looks like it is indeed Thumb code.
If we could dump the eMMC RAM, we would understand what has been changed.


By inspecting some code, it seems that we know how to dump the eMMC RAM:
Look at the function mmc_set_wearlevel_page in line 206. It patches the RAM (using the method mentioned before), then it validates what it has written (in lines 255-290). Seems that the procedure to read the RAM is as following:
1. CMD62(0xEFAC62EC) CMD62(0x10210002) to enter RAM reading mode
2. MMC_ERASE_GROUP_START(Address to read) MMC_ERASE_GROUP_END(Length to read) MMC_ERASE(0)
3. MMC_READ_SINGLE_BLOCK to read the data
4. CMD62(0xEFAC62EC) CMD62(0xDECCEE) to exit RAM reading mode


I don't want to run this on my device, because I'm afraid - messing with the eMMC doesn't sound like a very good idea on my device (I don't have a spare one).
Does someone have a development device which he doesn't mind to risk, and want to dump the eMMC firmware from it?
The Following 51 Users Say Thank You to Oranav For This Useful Post: [ Click to Expand ]
 
sponix2ipfw
Old
#2  
sponix2ipfw's Avatar
Member
Thanks Meter 22
Posts: 39
Join Date: Jul 2012
Quote:
Originally Posted by Oranav View Post
Since it's very likely that the recent eMMC firmware patch by Samsung is their patch for the "sudden death" issue, it would be very nice to understand what is really going on there.

According to a leaked moviNAND datasheet, it seems that MMC CMD62 is vendor-specific command that moviNAND implements.
If you issue CMD62(0xEFAC62EC), then CMD62(0xCCEE) - you can read a "Smart report". To exit this mode, issue CMD62(0xEFAC62EC), then CMD62(0xDECCEE).


So what are they doing in their patch?

1. Whenever an MMC is attached:
a. If it is "VTU00M", revision 0xf1, they read a Smart report.
b. The DWORD at Smart[324:328] represents a date (little-endian); if it is not 0x20120413, they don't patch the firmware. (Maybe only chips from 2012/04/13 are buggy?)
2. If the chip is buggy, whenever an MMC is attached or the device is resumed:
a. Issue CMD62(0xEFAC62EC) CMD62(0x10210000) to enter RAM write mode. Now you can write to RAM by issuing MMC_ERASE_GROUP_START(Address to write) MMC_ERASE_GROUP_END(Value to be written) MMC_ERASE(0).
b. *(0x40300) = 10 B5 03 4A 90 47 00 28 00 D1 FE E7 10 BD 00 00 73 9D 05 00
c. *(0x5C7EA) = E3 F7 89 FD
d. Exit RAM write mode by issuing CMD62(0xEFAC62EC) CMD62(0xDECCEE).
10 B5 looks like a common Thumb push (in ARM architecture). Disassembling the bytes that they write to 0x40300 yields the following code:
Code:
ROM:00040300                 PUSH    {R4,LR}
ROM:00040302                 LDR     R2, =0x59D73
ROM:00040304                 BLX     R2
ROM:00040306                 CMP     R0, #0
ROM:00040308                 BNE     locret_4030C
ROM:0004030A
ROM:0004030A loc_4030A                               ; CODE XREF: ROM:loc_4030Aj
ROM:0004030A                 B       loc_4030A
ROM:0004030C ; ---------------------------------------------------------------------------
ROM:0004030C
ROM:0004030C locret_4030C                            ; CODE XREF: ROM:00040308j
ROM:0004030C                 POP     {R4,PC}
ROM:0004030C ; ---------------------------------------------------------------------
Disassembling what they write to 0x5C7EA yields this:
Code:
ROM:0005C7EA                 BL      0x40300
Looks like it is indeed Thumb code.
If we could dump the eMMC RAM, we would understand what has been changed.


By inspecting some code, it seems that we know how to dump the eMMC RAM:
Look at the function mmc_set_wearlevel_page in line 206. It patches the RAM (using the method mentioned before), then it validates what it has written (in lines 255-290). Seems that the procedure to read the RAM is as following:
1. CMD62(0xEFAC62EC) CMD62(0x10210002) to enter RAM reading mode
2. MMC_ERASE_GROUP_START(Address to read) MMC_ERASE_GROUP_END(Length to read) MMC_ERASE(0)
3. MMC_READ_SINGLE_BLOCK to read the data
4. CMD62(0xEFAC62EC) CMD62(0xDECCEE) to exit RAM reading mode


I don't want to run this on my device, because I'm afraid - messing with the eMMC doesn't sound like a very good idea on my device (I don't have a spare one).
Does someone have a development device which he doesn't mind to risk, and want to dump the eMMC firmware from it?
--> **Ultimate GS3 sudden death thread**
Just wanted to link to a prior thread with some information/testing that as been done. Completely understand if you nuke it because it doesn't meet the proper criteria or is way to noobish to be posted here. Anyway, just though it _might_ help, so giving it a shot..

.
Verizon Samsung Galaxy S4 SCH-I545 (Rooted Stock NC5)
Retired: Sprint Samsung Galaxy S III (Rooted Stock LJ7) SPH-L710
Retired: Sprint HTC Evo 4G
.
<--- Did I help you? If So, Click Thanks!
Fun times,
sponix2ipfw
The Following 3 Users Say Thank You to sponix2ipfw For This Useful Post: [ Click to Expand ]
 
Oranav
Old
#3  
Member - OP
Thanks Meter 259
Posts: 52
Join Date: Oct 2010
So I decided to do a small RAM dump after all.

Before the patch, 0x5C7EA reads FD F7 C2 FA, which is "BL 0x59D72".
As I thought, they replace a function call to the new one.

I will dump function 0x59D72 later this week.
The Following 16 Users Say Thank You to Oranav For This Useful Post: [ Click to Expand ]
 
odewdney
Old
#4  
Junior Member
Thanks Meter 2
Posts: 1
Join Date: Feb 2008
Quote:
Originally Posted by Oranav View Post
So I decided to do a small RAM dump after all.

Before the patch, 0x5C7EA reads FD F7 C2 FA, which is "BL 0x59D72".
As I thought, they replace a function call to the new one.

I will dump function 0x59D72 later this week.
So it looks like the new function calls the old function, and then if it returns ZERO then in goes into an INFINITE loop?!?

Seems like an odd fix, maybe self presevation?

Oli
The Following 2 Users Say Thank You to odewdney For This Useful Post: [ Click to Expand ]
 
Rob2222
Old
(Last edited by Rob2222; 15th January 2013 at 05:07 PM.)
#5  
Senior Member
Thanks Meter 298
Posts: 412
Join Date: Feb 2008
Quote:
Originally Posted by odewdney View Post
So it looks like the new function calls the old function, and then if it returns ZERO then in goes into an INFINITE loop?!?
Seems like an odd fix, maybe self presevation?
Oli
WELL... after I changed to XXELLA stock firmware and stock kernel in 01/13 my 06/12 SGS3 had the _first freeze ever_ on XXELLA. Maybe its completely unrelated and was only a random thing.

But, could it be, that this fix temporary (until reboot) locks the eMMC in a bad situation to avoid damaging internal data structures?
But then in this cases you get a phone freeze, cause the eMMC is temporary unaviable so the phone crashes until you reboot it. But it avoided eMMC data structure damage.

Sounds not very logical, but when you have to fix a problem and only have a few bytes to patch it (cause it must be run on every emmc-start), and the problem only occurs on a hand full devices (out of millions) then it is maybe acceptable to have a freeze instead of a dead eMMC in that rare cases that it occurs.

But this is only a idea... don't now if it is like that.

BR
Rob

PS: Oranav, thank you very much for your effort.
The Following 4 Users Say Thank You to Rob2222 For This Useful Post: [ Click to Expand ]
 
Oranav
Old
(Last edited by Oranav; 15th January 2013 at 09:26 PM.)
#6  
Member - OP
Thanks Meter 259
Posts: 52
Join Date: Oct 2010
Quote:
Originally Posted by odewdney View Post
So it looks like the new function calls the old function, and then if it returns ZERO then in goes into an INFINITE loop?!?

Seems like an odd fix, maybe self presevation?

Oli
Right, haven't spotted this. Thanks for the observation.
Self preservation sounds possible.

Quote:
Originally Posted by Rob2222 View Post
WELL... after I changed to XXELLA stock firmware and stock kernel in 01/13 my 06/12 SGS3 had the _first freeze ever_ on XXELLA. Maybe its completely unrelated and was only a random thing.

But, could it be, that this fix temporary (until reboot) locks the eMMC in a bad situation to avoid damaging internal data structures?
But then in this cases you get a phone freeze, cause the eMMC is temporary unaviable so the phone crashes until you reboot it. But it avoided eMMC data structure damage.

Sounds not very logical, but when you have to fix a problem and only have a few bytes to patch it (cause it must be run on every emmc-start), and the problem only occurs on a hand full devices (out of millions) then it is maybe acceptable to have a freeze instead of a dead eMMC in that rare cases that it occurs.

But this is only a idea... don't now if it is like that.

BR
Rob

PS: Oranav, thank you very much for your effort.
This could be possible - this patch looks like a quick and dirty fix, so maybe they didn't have the time to properly fix this. Instead, they just avoid the bug absolutely (with the cost of data corruption).
But I don't think this would cause lockups - I believe the chip has a watchdog...


All in all, I think the best thing we can do right now is to dump the whole firmware out of it. I will do it soon.
The Following 10 Users Say Thank You to Oranav For This Useful Post: [ Click to Expand ]
 
Bäcker
Old
#7  
Bäcker's Avatar
Senior Member
Thanks Meter 92
Posts: 225
Join Date: Jan 2007
there is also a chance that this is just a temporary workaround to prevent further bricking - until there is a final fix.

As of now we asume that this is the fix as it directly adresses the eMMC in concern, but all this is just based on asumptions.
 
ivan4o5
Old
#8  
Member
Thanks Meter 6
Posts: 77
Join Date: Nov 2010
Quote:
Originally Posted by Rob2222 View Post
WELL... after I changed to XXELLA stock firmware and stock kernel in 01/13 my 06/12 SGS3 had the _first freeze ever_ on XXELLA. Maybe its completely unrelated and was only a random thing.

But, could it be, that this fix temporary (until reboot) locks the eMMC in a bad situation to avoid damaging internal data structures?
But then in this cases you get a phone freeze, cause the eMMC is temporary unaviable so the phone crashes until you reboot it. But it avoided eMMC data structure damage.

Sounds not very logical, but when you have to fix a problem and only have a few bytes to patch it (cause it must be run on every emmc-start), and the problem only occurs on a hand full devices (out of millions) then it is maybe acceptable to have a freeze instead of a dead eMMC in that rare cases that it occurs.

But this is only a idea... don't now if it is like that.

BR
Rob

PS: Oranav, thank you very much for your effort.
I think we can prove that the fix is actualy locking into loop ,but you must risk your phone :/ . If you are in ->
1) Flash back to older version without the fix
2) Wait and pray
a) If your phone dies --> the guys were right about the fix and the loop
b) If it stays alive ,then ...

We wont know for sure ,but your phone is maybe in the "perfect" condition for the test :/ .
(Sorry if this makes no sence)
 
Rob2222
Old
(Last edited by Rob2222; 17th January 2013 at 02:48 PM.)
#9  
Senior Member
Thanks Meter 298
Posts: 412
Join Date: Feb 2008
@ivan:
Sorry, can't do that. Cause of high air humidity my humidity indicator is already a little soaked. Cause of the warranty-repair reports in out local forums I am not sure if I would get warranty. I think theres a fair chance, that they would deny my warranty. Cause of that I don't want to take any extra risk. I am on unrooted stock at the moment, cause of that.

@Oranav:
In our local forum we get some reports about a rising count of locks and restarts on S3's in the last time. Some like my freeze.
It also seems that after a while this problems gets better and even disappear completely.
Cause of that I am thinking, if it could be, that the fix maybe locks the eMMC if it finds a bad data structure, then this locks maybe could bring a phone-freeze (already stated that), and in the same time it repairs the data structure in this block with the bad data structure.
At least this would explain some rising count of freezes with the fix and the point, that the freezes become less and less over time.

I have no idea if it's that way, I just wanted to post it as theory to think about.

BTW, do you think when the watchdog restarts the eMMC that it goes that fast that the phone isn't affected?

BR
Robert
 
Oranav
Old
#10  
Member - OP
Thanks Meter 259
Posts: 52
Join Date: Oct 2010
Quote:
Originally Posted by Rob2222 View Post
I have no idea if it's that way, I just wanted to post it as theory to think about.
The problem is that there are too many theories imaginable, but I can't think of no way to prove them but to reverse engineer the MoviNAND firmware.

Quote:
Originally Posted by Rob2222 View Post
BTW, do you think when the watchdog restarts the eMMC that it goes that fast that the phone isn't affected?
Certainly not. Watchdogs are slow, drivers running on a Cortex-A9 are blazing fast.
But I do think Linux's MMC driver can handle device restarts during an MMC operation.

The Following 2 Users Say Thank You to Oranav For This Useful Post: [ Click to Expand ]
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes