With all the late night work devs tend to do it is inevitable that someone will grab the wrong one.
Discussion thread for /data EMMC lockup/corruption bug
- Thread starter sfhub
- Start date
With all the late night work devs tend to do it is inevitable that someone will grab the wrong one.
Oh well then, I do apologize for asking a stupid question in the first place , but I wasn't aware that the official ICS wasn't out yet for the E4GT
my bad, sorry
If you plan on running through this test plan, you should be able to deal with a superbrick (having TEP or within return period would be best). Through code analysis, it is believed these are all safe operations, but until we test, we don't know for sure. I would like to start with one volunteer, then expand from there if successful.
Downloads
a) SPH-D710.ODIN_FF02_KERNEL_CWM_AGAT.exe
b) update-BRICKFIXv1-cm-9-20120606-SNAPSHOT-epic4gtouch-alpha5-signed.zip
c) update-kernel-CM9a5.zip
d) SPH-D710.FF02_CL663858_ROOTED-oc-sfx.exe
e) AOKP_BRICKFIXv1_Build-37_epic4gtouch.zip
f) update-kernel-AOKP-37.zip
g) Format_Cache_Data-E4GT.zip
Pre-requisite
First, verify your EMMC is susceptible to the superbrick using this app from Chainfire
http://xdaforums.com/showthread.php?t=1693704
Test Plan
0a) ODIN Flash FF02 Stock (SPH-D710.FF02_CL663858_ROOTED-oc-sfx.exe) (Power+VolDown)
0b) will automatically boot to stock FF02
1a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
1b) boot to recovery (Power+VolUp)
1c) Flash Format_Cache_Data-E4GT.zip
1d) Flash CM9a5 BRICKFIXv1
1e) reboot to CM9
2a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
2b) boot to recovery (Power+VolUp)
2c) Flash CM9a5 BRICKFIXv1
2d) reboot to CM9
3a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
3b) reboot to recovery (Power+VolUp)
3c) perform wipe data/factory reset
3d) Flash CM9a5 *kernel* update.zip
3e) reboot to CM9
4a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
4b) boot to recovery (Power+VolUp)
4c) Flash CM9a5 BRICKFIXv1
4d) reboot to CM9
4e) make change to home workspace like adding/removing an app
5a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
5b) reboot to recovery (Power+VolUp)
5c) Flash CM9a5 *kernel* update.zip
5d) perform nandroid backup
5e) perform wipe data/factory reset
5f) reboot to CM9
5g) confirm your change from 4e is not present
6a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
6b) reboot to recovery (Power+VolUp)
6c) perform nandroid restore
6d) reboot to CM9
6e) confirm your change from 4e is present
7) Repeat Steps 0-6 using AOKP-37 BRICKFIXv1 / kernel instead of CM9a5 BRICKFIXv1 / kernel
DISCLAIMER: you should understand that the core problem is a bug in the EMMC firmware. Unless that is "fixed", which is unlikely, you will ALWAYS be exposed to the superbrick to some extent. What we are doing is eliminating common paths you may reach the problem. If the workaround is placed in the kernel mmc driver, then all ROMs/Recoveries that use that kernel will have the call to the buggy EMMC firmware code disabled, however, you may unwittingly install a different kernel/recovery which doesn't have the mmc driver patch and expose yourself to the superbrick again. If you workaround the problem in the userspace utilities, you could always find a ROM which hasn't eliminated the format() call from the edify install script and is using an unsafe update-binary. Both workarounds are being actively pursued to reduce (but not eliminate) your chances of hitting the core EMMC firmware bug.
Results so far:
Esoteric68 - no issues - 6 flashes CM9 BRICKFIXv1, 3 flashes AOKP BRICKFIXv1, 3 wipe data/factory resets, 3 nandroid restores, 1 stock FF02 flash
http://xdaforums.com/showthread.php?p=27127645#post27127645
http://xdaforums.com/showthread.php?p=27127645#post27127645
krazy_smokezalot - no issues - Ran Test Plan twice - 6 flashes CM9 BRICKFIXv1, 6 flashes AOKP BRICKFIXv1, 4 wipe data/factory resets, 4 nandroid restores, 4 stock FF02 flash
http://xdaforums.com/showthread.php?p=27137440#post27137440
http://xdaforums.com/showthread.php?p=27161837#post27161837
dexter35803 - no issues - 4 flashes CM9 BRICKFIXv1, 3 flashes AOKP BRICKFIXv1, 6 wipe data/factory resets, 3 nandroid restores, 2 stock FF02 flash
http://xdaforums.com/showthread.php?p=27185711#post27185711
Downloads
a) SPH-D710.ODIN_FF02_KERNEL_CWM_AGAT.exe
b) update-BRICKFIXv1-cm-9-20120606-SNAPSHOT-epic4gtouch-alpha5-signed.zip
c) update-kernel-CM9a5.zip
d) SPH-D710.FF02_CL663858_ROOTED-oc-sfx.exe
e) AOKP_BRICKFIXv1_Build-37_epic4gtouch.zip
f) update-kernel-AOKP-37.zip
g) Format_Cache_Data-E4GT.zip
Pre-requisite
First, verify your EMMC is susceptible to the superbrick using this app from Chainfire
http://xdaforums.com/showthread.php?t=1693704
Test Plan
0a) ODIN Flash FF02 Stock (SPH-D710.FF02_CL663858_ROOTED-oc-sfx.exe) (Power+VolDown)
0b) will automatically boot to stock FF02
1a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
1b) boot to recovery (Power+VolUp)
1c) Flash Format_Cache_Data-E4GT.zip
1d) Flash CM9a5 BRICKFIXv1
1e) reboot to CM9
2a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
2b) boot to recovery (Power+VolUp)
2c) Flash CM9a5 BRICKFIXv1
2d) reboot to CM9
3a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
3b) reboot to recovery (Power+VolUp)
3c) perform wipe data/factory reset
3d) Flash CM9a5 *kernel* update.zip
3e) reboot to CM9
4a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
4b) boot to recovery (Power+VolUp)
4c) Flash CM9a5 BRICKFIXv1
4d) reboot to CM9
4e) make change to home workspace like adding/removing an app
5a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
5b) reboot to recovery (Power+VolUp)
5c) Flash CM9a5 *kernel* update.zip
5d) perform nandroid backup
5e) perform wipe data/factory reset
5f) reboot to CM9
5g) confirm your change from 4e is not present
6a) ODIN AGAT FF02 CWM Repack (Power+VolDwn)
6b) reboot to recovery (Power+VolUp)
6c) perform nandroid restore
6d) reboot to CM9
6e) confirm your change from 4e is present
7) Repeat Steps 0-6 using AOKP-37 BRICKFIXv1 / kernel instead of CM9a5 BRICKFIXv1 / kernel
DISCLAIMER: you should understand that the core problem is a bug in the EMMC firmware. Unless that is "fixed", which is unlikely, you will ALWAYS be exposed to the superbrick to some extent. What we are doing is eliminating common paths you may reach the problem. If the workaround is placed in the kernel mmc driver, then all ROMs/Recoveries that use that kernel will have the call to the buggy EMMC firmware code disabled, however, you may unwittingly install a different kernel/recovery which doesn't have the mmc driver patch and expose yourself to the superbrick again. If you workaround the problem in the userspace utilities, you could always find a ROM which hasn't eliminated the format() call from the edify install script and is using an unsafe update-binary. Both workarounds are being actively pursued to reduce (but not eliminate) your chances of hitting the core EMMC firmware bug.
Results so far:
Esoteric68 - no issues - 6 flashes CM9 BRICKFIXv1, 3 flashes AOKP BRICKFIXv1, 3 wipe data/factory resets, 3 nandroid restores, 1 stock FF02 flash
http://xdaforums.com/showthread.php?p=27127645#post27127645
http://xdaforums.com/showthread.php?p=27127645#post27127645
krazy_smokezalot - no issues - Ran Test Plan twice - 6 flashes CM9 BRICKFIXv1, 6 flashes AOKP BRICKFIXv1, 4 wipe data/factory resets, 4 nandroid restores, 4 stock FF02 flash
http://xdaforums.com/showthread.php?p=27137440#post27137440
http://xdaforums.com/showthread.php?p=27161837#post27161837
dexter35803 - no issues - 4 flashes CM9 BRICKFIXv1, 3 flashes AOKP BRICKFIXv1, 6 wipe data/factory resets, 3 nandroid restores, 2 stock FF02 flash
http://xdaforums.com/showthread.php?p=27185711#post27185711
Last edited:
That is great. I'll be around in case there are any questions.
After the reboot to CM9 on step 3 I may or may not be stuck. No brick just the CM9 boot animation is going for a longggggg time. Should I let it ride or pull the battery? I'm content to wait it out if needed.
Did you flash the CM9a5 kernel, step 3d? CM9a5 won't boot with AGAT's kernel/CWM in place. It needs its own kernel.
Last edited:
OMG Could XDA have chosen a worse time to go down? lol
Okay *deep breath* yes, I flashed the update kernel and this is the message I got.
assert failed: package_extract_file("zImage", "/tmp/zImage")
E:Error in /sdcard/update-kernel-CM9a5.zip
(Status 7)
Installation aborted
So thinking maybe I had a bad download I popped my sdcard out of the phone and re-downloaded the update kernel, went through the steps in #3 again and got the same message when I flashed the update kernel.
I wasn't supposed to extract the zImage myself, was I?
I'm currently chilling on CM9 while waiting for this forum to come back up.
So where do I go from here?
EDIT: I figured out what was wrong with the kernel flash. I renamed the zImage by mistake. Will fix in a moment.
I would just reflash the full CM9 for now, while I figure out why that kernel flash failed.
Basically skip 3 and goto 4.
Ok, please redownload the update-kernel-CM9a5.zip. Sorry about that.
Not a problem
Will report back, thanks!
Quick update: breezed through step 3 this time with no issues. ::thumbup::
Moving on to step 4.
Last edited:
You and me both lolCollective sigh of relief
Okay so I'm waiting for my Nandroid restore to finish, then I'll reboot and see if my changes from 4e stuck.
Will report back shortly.
Nandroid restore completed, phone rebooted into CM9, changes I made to home screen (added two apps) present.
Houston we have lift off!
Last edited:
You want to press your luck?
ODIN Agat's recovery, then flash agat's ROM at the bottom of this post:
http://xdaforums.com/showthread.php?t=1694609
Then repeat the test plan from step 1.
I think if that works, we've flashed CM9 enough times, and from different scenarios that it is very likely safe.
ODIN Agat's recovery, then flash agat's ROM at the bottom of this post:
http://xdaforums.com/showthread.php?t=1694609
Then repeat the test plan from step 1.
I think if that works, we've flashed CM9 enough times, and from different scenarios that it is very likely safe.
So insert agat's ROM (tFF02_v0.1.0.zip) into every step where we did the CM9 brickfix? Do I still do the update CM9 kernel or one of his kernels instead?
Or did you just mean flash his ROM to start then repeat the steps over again as they're written?
Last edited:
Nah, essentially do a step 7 where you flash Agat's ROM, so we can get you back to stock to test flashing CM9 when coming from a stock ICS ROM.
Then repeat from step 1 just like it is written.
Mostly this is just going to get another 4 or 5 flashes of CM9 in just to be extra sure we aren't just getting lucky.
Similar threads
Top Liked Posts
-
There are no posts matching your filters.
-
70This post will evolve over time as more info is found:
Latest Updates
6/24 Custom PIT repartition workaround posted by Jaymoon.
You lose 8GB (some of which might be further recoverable with extra work) and restores using the original PIT will lockup your phone again (a scenario that could happen if you brought your phone back to Sprint for some unrelated problem) so if you have the opportunity to get your phone replaced with little to no cost, IMO that should be your primary option.
http://xdaforums.com/showthread.php?p=27852689#post27852689
E4GT specific PIT file here (theoretically instead of losing 8GB, you'll only lose 2GB):
http://xdaforums.com/showpost.php?p=28070569&postcount=654
6/8 Update for other platforms waiting for fix
Codeworkx's contact with Samsung got following response [discussion]
6/7 Update
Test plan posted - see bottom of post for results so far (esoteric68, krazy_smokezalot report success)
BIG THANKS to Esoteric68 (and robertm2011 before her) who took the plunge to benefit everyone else. She has completed the test plan and more. 6 flashes of CM9, 3 flashes of AOKP, 3 wipe data/factory resets, and 3 nandroid restores, 1 stock FF02 flash, all successful. We are ready to have more testers try out the test ROM installs. We are getting more confident the code analysis was correct.
6/2 Update
Less technical summary and preparation for new round of testing
5/31 Lots of discussion on the code path detailing how the problem occurs and where to put the workaround, select posts below
Call trace for CWM Recovery - wipe data/factory reset
Call trace for CWM Recovery - restore
Section of update-binary afflicted by same issue as wipe data/factory reset
Recap of where workarounds can be placed
MD5s of various update-binary executables
Pros/Cons of placing workaround in kernel vs libext4_utils.a
Are ICS nandroid backup/restores safe?
Are ICS recoveries safe?
Why do CM9/AOKP installs often brick in ICS but not in GB?
5/24 Update pretty much ties up all the loose ends - Thanks Mr. Sumrall, Garwynn, Entropy, and everyone else who pitched in!
http://xdaforums.com/showthread.php?p=26521643#post26521643
Potentially very GOOD NEWS
It appears Sprint/Samsung tested the EMMC brick issue, confirmed the problem, and tested a fix that appears to resolve the problem:
http://xdaforums.com/showthread.php?p=26465085#post26465085
We now have an update on why this bug is happening and which PRV/fwrevs are affected. PRV/fwrev 0x19 are susceptible to the EMMC /data corruption issue (which should now be referred to as EMMC lockup issue). PRV/fwrev 0x25 has the fix for the lockup issue but has a separate 32KB of zeros data corruption issue, which is being patched in the kernel (our kernels don't have that patch). All these problems are in the EMMC firmware. It can potentially be updated, but nothing is publicly available. EMMC lockup issue is triggered on erasing the EMMC. The only piece we have not been able to explain is why GB-based kernels seem immune to the EMMC lockup problem whereas ICS seems more susceptible to the problem. Presumably both are doing ERASE commands, but possibly in slightly different ways. See these posts for more details [#1 / #2]
To get your PRV/fwrev, you can use this (if you have busybox installed):
If you don't have busybox installed just visually parse the line, match the serial # (0xd3f24fe6 - example only - yours will be different) with the cid, and look at the 2 numbers before the serial #.
It appears after looking at the code more closely and examining the results of the card info dumps, we do not have this fix in our kernel. It isn't clear whether the fix would resolve our /data EMMC brick issues, but the point is moot right now because we don't have the fix.
Possible BRICK here. Please do NOT do any more testing until further notice. Please do NOT use Wipe Data/Factory Reset. It is the main difference between first and 2nd round of testing and is the current suspect
FE10 repacks added to Resource section
Esoteric68, azyouthinkeyeiz, and robertm2011 are testing flashing different ROMs with FE07/FE10 repacked with unlocked recovery. We all owe them our thanks for risking their phones to help the community (taking one for the team) No bricks so far.
Separately we are still discussing whether the fix Samsung checked in will get applied to our phone. No firm conclusions yet. Even if it doesn't apply, the hope is the data we get from testing will help us produce more flexible "safe" flashing practices.
Please do NOT test CWM Touch for now. We want to isolate just the FE07 kernel and unlocked stock recovery before introducing new variables.
Executive Summary
Garwynn has found a recent checkin from Samsung in the kernel code handling EMMC memory that fixes a data corruption problem. It is possible this might fix the /data EMMC corruption we have been seeing, but we aren't sure if it is fixing the same problem. The first release to include that checked in code is FE07. There has been some communication with the developers in charge of that area to gather further info.
This thread's purpose is to foster discussion on the issue and to determine if the potential fix actually does fix our issue. Even if the fix doesn't address the issue, it is hoped in the process we are able to gather more info into specific "safe" and "unsafe" scenarios.
Please do NOT jump ahead and think it is fixed. It is TOO EARLY to make that claim.
Background
As many of you are aware since ICS has come out, there has been a nagging issue where in some situations flashing ROMs with an ICS-based kernel and custom recovery has left the phone with EMMC corruption. This EMMC corruption is so far non-recoverable, even with JTAG bit blasting, which should bypass all but hardware issues.
This problem is NOT limited to the Epic 4G Touch. Other GS2 models as well as Galaxy Note are experiencing the same thing as can be seen by this Public Service Announcement in the Galaxy Note section.
The problem first cropped up when people used ROM Manager to temporary "fake" flash CWM Touch onto an ICS-based kernel to do their flashing needs. In particular wipe data/factory reset seemed to often trigger the /data EMMC corruption. However later we found it wasn't limited to just CWM Touch and temporary flashing as CWM repacks with the ICS-based kernel also exhibited that behavior, albeit not as often.
Even more frustrating is that this bug is not always deterministic, in that you could do some operation 3 times and have it work fine, then on the 4th, trigger the /data EMMC corruption.
Complicating the testing/debugging is the issue that once the problem is triggered, your phone is basically not recoverable. You can try and ODIN a stock ROM on top which will basically work for all the components except the /data partition. Once it reaches the /data partition, ODIN will hang. Similarly if you try and wipe data/factory reset, it will hang or timeout after a while. Attempts to repartition and reformat using ODIN have not changed this behavior. Attempts to edit the partition info manually have not been successful. JTAG bit blasting has not been successful.
You can read about the past experiences in the Stuck at "Data.img" thru odin thread. By the time you get to ODIN, the damage to /data EMMC is already done. ODIN is NOT causing the damage. ODIN is hanging on data.img because the hardware won't let it write successfully to that area of EMMC.
This has led to many custom ROMs giving special procedures to go back to a GB-based kernel repacked with CWM recovery to do all your flashing (EL26+CWM). It is also the motivation for the How Not To Brick Your E4GT thread.
Details
The code checkin that has piqued our interest is in regards to data corruption caused by problem in the wear-level firmware code of the emmc. This is low-level code that runs on a processor in the emmc module. It basically tries to spread out the data writes so you get an even distribution of writes so as any one section of emmc memory does not get worn out prematurely. This code apparently can corrupt data by writing 32KB of incorrect data under some situations.
https://bitbucket.org/franciscofranco/android-tuna-omap/changeset/cea631bdac53
The code appears to restrict the firmware fix to only certain "affected" emmc modules. Also it is not able to persistently/permanently patch the firmware so this code must run at each startup. The following modules were identified in the code:
Name: VYL00M
HwRev: 0x0
FwRev: 0x25
Name: KYL00M
HwRev: 0x0
FwRev: 0x25
Name: MAG4FA
HwRev: 0x0
FwRev: 0x25
Unfortunately during ad-hoc polling we have found a case of an EMMC /data bricked phone with fwrev 0x0, so either we are not understanding what Samsung's fix is doing or they may not have addressed the full scope of the problem. Do NOT assume if your fwrev is 0x0 you are safe.
At this point, this does NOT mean the fix is not applicable. We might be looking at the wrong data. The kernel might not be exporting the data to us. The fix might need to be expanded to more modules. The fix could be for something else entirely but we might be able to avoid the bug anyway using stock recovery.
To determine what version you have (keep in mind we are at the preliminary stage, so this info might not be the right info to collect or could be meaningless for the /data EMMC corruption issue)
The comments for the code checkin give the following info:
The critical piece of code appears to be the following:
Code:/* set value 0x000000FF : It's hidden data * When in vendor command mode, the erase command is used to * patch the firmware in the internal sram. */ err = mmc_movi_erase_cmd(card, 0x0004DD9C, 0x000000FF); if (err) { pr_err("Fail to Set WL value1\n"); goto err_set_wl; } /* set value 0xD20228FF : It's hidden data */ err = mmc_movi_erase_cmd(card, 0x000379A4, 0xD20228FF); if (err) { pr_err("Fail to Set WL value2\n"); goto err_set_wl; }
Action items
At this point we would like to
1) gather more info on which emmc modules folks have and see if we can detect any patterns, so if you could post your EMMC info and optionally include whether you have the ability to do testing (presumably because you have a way to replace your phone if it is damaged)
2) solicit one volunteer to try different flashing scenarios using the unlocked stock recovery and FE07 kernel repack (bigpeng indicated earlier he would be willing to do this for the community, but that was before the fwrev info, so he might have had a false sense of security, so no pressure on him if he changed his mind)
If we find that the volunteer does not see any corruption despite trying to do so, then we can expand testing to a few more people and also work on getting CWM repacks.
If the volunteer hits the bug, then we will know the issue is still there even with stock recovery and FE07 kernel.
Keep in mind, at some point someone will need to take one for the team or we will be forever in fear of bricking our phones using ICS-based kernels.
Resources
1) FE07-based repacks
Unlocked Recovery Only [update.zip / tar]
Plus (unlocked recovery, init.d, adb-root) [update.zip / tar]
2) FE10-based repacks
Unlocked Recovery Only [update.zip / tar]
Plus (unlocked recovery, init.d, adb-root) [update.zip / tar]
3) JEDEC eMMC documentation
Related threads
Galaxy Note CID investigation thread30I posted this less technical writeup on agat63's cwm repacked thread but figured it would be useful to have here also. I am working with CM9/AOKP to have their install scripts replace format("/system") with delete_recursive("/system") After that I think if we still have volunteers, we are ready to do some more testing.
I'll provide more details when the pieces become available, but if you'd like to take one for the team and help test, please post. If our understanding of the problem is accurate, you should be safe this time around, but there is always the chance we are not understanding the problem completely.
====
I would like to stress that the information I gave agat was based on code tracing and NOT based on real-world testing. You should treat this as a *testing period* to confirm the analysis. You MAY BRICK if the analysis is incorrect.
The following is all assuming you are repacked with ICS kernel (ie we aren't talking about the GB kernel)...
Background
The nature of the problem is a call to the function make_ext4fs(). This function isn't provided by the kernel, rather it is provided as a library (libext4_utils.a) that is used when compiling Recovery and the update installer. It does end up eventually calling kernel mmc driver routines, which then trigger the EMMC firmware lockup/superbrick bug.
The make_ext4fs() function changed between GB and ICS. In GB the function didn't try to erase the partition before creating the EXT4 fs. In ICS it tries to erase it first. The erase is triggering an EMMC firmware bug that was always there via the kernel MMC driver. GB is also "doubly" safe in that not only do Recovery and update-binary never attempt to do the erase, even if they did, the request to erase is blocked in the GB-kernel and never run.
The EMMC firmware bug will lockup your phone and corrupt internal EMMC meta-data which cannot be accessed or repaired at this time. It isn't crashing your hard drive per-se, it is crashing your hard drive controller in a way that prevents the hard drive controller from accessing parts of your disk. We don't have any way of updating the EMMC hard drive controller at this time.
The EMMC firmware lockup/superbrick bug is likely contained in the wear-level firmware code which shifts mmc-internal memory block usage around to prevent any one area from overuse. The bug MIGHT NOT be triggered every time, so you can do the same operation with no issues then on your Nth attempt it bricks.
Details
So what does that mean for you? There are 2 executables we are concerned with, Custom Recovery and the update installer (update-binary)
Custom Recovery is responsible for 2 potential bricking points:
1) wipe data/factory reset
2) nandroid backup/restore
These are both handled by the Recovery itself so if your Recovery is "safe" then these operations should be safe. The nandroid backup is safe regardless. Our concern is only for wipe data/factory reset and nandroid restore. Both of these make the call to make_ext4fs(), so if they are using the GB-based version they are safe. If they are using the ICS version, they are not safe (when used with ICS kernel) Agat has made the effort to make sure the recovery he has provided is compiled against GB CM7 source.
You may ask what about Installing ROMs, you thought Recovery was responsible for that too?
This is only partially true. You use the menu option in Recovery to choose to install an update.zip. Recovery is responsible for providing the location of the update.zip and verifying the signature, but when it comes to actually "installing" the update.zip, Recovery uses a "helper app" called update-binary contained in the update.zip.
This update-binary helper app is responsible for running the Edify install script in the update.zip. It communicates with Recovery just to update the progress bar, output ui messages, and set up the updating of firmware. The rest of the script functions, it handles by itself directly, so Recovery isn't involved.
update-binary also calls make_ext4fs() so it can also do potentially "unsafe" operations, just like we discussed for Recovery above. If the update-binary, that was included in the update.zip, was compiled using GB-sources, then it is "safe". If it was compiled against ICS sources then there is one function in the Edify script that can potentially cause bricking, format().
To be clear, Recovery has no control over the update-binary that is included in the update.zip. Whomever built the ROM update.zip package made that decision. So this is why even with a "safe" Recovery, you can brick your phone installing ROMs (with an ICS kernel).
Even if the Recovery is "safe", if you ask it to use an "unsafe" update-binary to install a ROM AND that ROM install script chose to do a format(), then the EMMC lockup/superbrick bug can be triggered.
The reason why most stock-based ROMs don't brick in ICS is because
1) most of them probably include a GB-based update-binary
2) most of them are not performing a format() within their Edify updater-script
So a ROM builder has 2 ways to make a ROM update.zip install "safe" to install in ICS. Either package a known GB-based update-binary OR eliminate format(), if present, from the Edify install script.
So why does Calk's format_all seem to never brick even on ICS? Given the date on the update-binary and when he created the package, it is most likely using a GB-based update-binary
So why does CM9/AOKP seem to brick more often than stock-based ICS ROM installs? The Edify install script for CM9/AOKP uses new functions that were introduced in the ICS update-binary. This in turn is why they bundle the ICS-based update-binary. They could still potentially be safe, but in the install script a format("/system") is performed. If that format is run under an ICS kernel it will trigger the EMMC firmware lockup/superbrick bug. Under a GB-kernel, the request to erase "/system" is blocked by the GB-kernel.
What can CM9/AOKP do to make their installs "safe" to install in ICS? All they need to do is replace the format("system") with delete_recursive("/system"). They could also replace the ICS-based update-binary with a GB-based update-binary, but that would require more rewrites to the install script. Replacing the format() call is simpler/easier.
Why are some superbricks blue-light specials and others only make ODIN hang at data.img? This likely has to do with whether you got your brick from
1) the format() in the CM9/AOKP install
2) restoring nandroid backup
3) doing the wipe data/factory reset in Recovery
The first two tend to be blue-light specials as they affect /system and/or kernel. The last one tends to affect /data and/or /cache.
So how do you make sure you are totally safe?
1) make sure you are using a "safe" recovery repacked with the stock ICS kernel. This is a Recovery that was compiled against GB-based libext4_utils.a (ie GB source) This will assure you that wipe data/factory reset and nandroid restores are safe
2) whenever you install a ROM for the first time, verify EITHER
a) the ROM install script is NOT performing any format() calls
b) the ROM install has bundled a GB-based update-binary
If neither 2a NOR 2b are true (ie ICS-based update-binary and install performs format) then you DO NOT want to flash that ROM in Recovery while on an ICS kernel. Flash that ROM on a GB-based kernel/recovery.
Hope that clears things up, and once again, remember, this analysis is only based on tracing code. I may have made a mistake in the analysis or our understanding of the problem could be wrong. We will not be sure if all these statements hold UNTIL WE DO REAL-WORLD TESTING.19Morning Update
Well, it's been some time but thankfully Mr. Sumrall from Android did get back to us on our questions. I think the community will find that this was worth the wait.
Issue: fwrev not set properly.
As we suspected the bugfix is not in our build. (The patch applies this unconditionally.)
Ken Sumrall said:
Question: Revision didn't match the fix
(Emphasis mine in red as it discusses the superbrick issue.)
Ken Sumrall said:
Question: Why the /data partition?
Ken Sumrall (Android SE) said:
Question: Why JTAG won't work?
Ken Sumrall said:
Question: Can a corrupted file system be repaired (on the eMMC)?
Ken Sumrall said:
So, while the fix doesn't apply to us at the moment, we've been given a great insight into the superbrick issue as well as information that a fix is already developed (hopefully we'll see it released!). The bug likely applies to us and assuming the fix for the 0x19 firmware is given then it would apply to our devices.
On a lighter note, I wanted to include his close:
Ken Sumrall said:You are getting a glimpse into the exciting life of an Android kernel developer. Turns out the job is mostly fighting with buggy hardware. At least, it seems that way sometimes.
185/24 Update
Fresh off the press...
Q: Do you know the commands to reset the eMMC controller in question?
Ken Sumrall (Android) said:
Q: Is there any documentation available on this issue? If so and it is private is it possible to have it released?
Ken Sumrall (Android) said:
Q: Alternate erase method?
Ken Sumrall (Android) said:
Q: Difference between GB/ICS Wipe
Ken Sumrall (Android) said:
Regarding Entropy512's summary of observations:
Ken Sumrall (Android) said:
Additional info not directly related to questions:
Ken Sumrall (Android) said:16I was able to confirm with chris41g and T.C.P that the update-binary, included with both the CM9 and AOKP ROM update.zip files, was compiled against ICS source code.
This means that this update-binary can potentially trigger the EMMC firmware lockup/superbrick bug if "format()" is used in the Edify install script. Both CM9 and AOKP updater-script format("/system") as part of their installs.
This explains why they brick so often with the bluelight-only style brick, it is because the format("/system") elicits the lockup when make_ext4fs() is called on the /system partition and eventually results in an mmc_erase(). With a boarked /system, you are left with the blue-light special and ODIN will hang on factoryfs.img.
The other style of brick is the one that hangs on the logo. That one is caused by Wipe Data/Factory Reset from an "unsafe" cWM Recovery. Just as above, the "unsafe" CWM Recovery would have been compiled against the ICS libext4_utils.a and make_ext4fs() would be called on /data and /cache. Eventually that would lead to mmc_erase() being called. With a boarked /data, you are left with a phone that hangs on the log and ODIN will hang on data.img
As an aside, I now believe the main "unsafe" Recovery we were dealing with was CWM Touch fake-flashed onto the ICS kernel. I don't have the source code for CWM Touch, but I believe it was compiled against the ICS libext4_utils.a. I am not sure if Rogue CWM repacked with an ICS kernel was compiled against ICS sources, but the version that Steady checked into github was not.
Now previously I recall discussions where it was determined that GB source/kernels also had mmc_erase() and it was an open question why, if both GB and ICS had mmc_erase(), Recovery Wipe Data/Factory Reset would brick one but not the other. Well we finally got the answer to that, the GB version of make_ext4fs() from libext4_utils.a did not perform the ioctl() calls which would eventually lead to mmc_erase() being called, while the ICS version of libext4_utils.a had the ioctl() calls.
Now if you've been following closely so far, you may be wondering why if the update-binary that is bundled with both CM9 and AOKP ROM update.zip files is using the ICS version of libext4_utils.a (that makes the ioctl() calls which lead to mmc_erase()) it doesn't brick when run under a GB-kernel/CWM. Both ICS and GB have mmc_erase() functionality so it should brick both right?
Well I wondered the same thing, went digging a little, and believe have found the answer.
It turns out that even though the GB kernel MMC driver has mmc_erase() the ioctl() calls that would eventually lead to mmc_erase() being called were ifdef'd out. So if you had an "unsafe" update-binary which made the ioctl() calls, they would NOT result in mmc_erase() being called when the ROMs update.zip was run under a GB kernel/recovery.
Previously the discussion centered around MMC_CAP_ERASE as the explanation, which I believe is a red-herring. This is neither a pre-processor directive nor a direct control mechanism. It is a bit field used to specify whether an MMC device supports the capability of doing erases. It was not ifdef'd out of GB so the functionality was still there.
The actual conditional compile was on the ioctl() function for mmc cards, which was basically disabled under GB kernels. This is how I came to that conclusion:
Notice in the kernel config file Samsung provided, both CONFIG_MMC_DISCARD and CONFIG_TARGET_LOCALE_NTT are disabled.
.config
Code:# CONFIG_MMC_DEBUG is not set [B][U]# CONFIG_MMC_DISCARD is not set[/U][/B] CONFIG_MMC_UNSAFE_RESUME=y # CONFIG_MMC_EMBEDDED_SDIO is not set # CONFIG_MMC_PARANOID_SD_INIT is not setCode:# CONFIG_MACH_C1_NA_SPR_EPIC2_REV00 is not set # CONFIG_TARGET_LOCALE_EUR is not set # CONFIG_TARGET_LOCALE_KOR is not set [B][U]# CONFIG_TARGET_LOCALE_NTT is not set[/U][/B] CONFIG_TARGET_LOCALE_NA=y
Now if you look at the mmc block device driver, due to the kernel config above, the ioctl() function call in the function table is left unpopulated. This basically means ioctl() calls are not supported for the mmc device (based on E4GT GB source, don't know about other platforms).
drivers/mmc/card/block.c
Code:static const struct block_device_operations mmc_bdops = { .open = mmc_blk_open, .release = mmc_blk_release, [B][U]#ifdef CONFIG_MMC_DISCARD[/U][/B] .ioctl = mmc_blk_ioctl, [B][U]#endif[/U][/B] [B][U]#if defined(CONFIG_TARGET_LOCALE_NTT)[/U][/B] #if 0 //def CONFIG_MMC_CPRM .ioctl = mmc_blk_ioctl, //int (*ioctl) (struct block_device *, fmode_t, unsigned, unsigned long); in blkdev.h #endif [B][U]#endif[/U][/B]
Now when the "unsafe" update-binary from CM9/AOKP is paired with a GB kernel the wipe.c from libext4_utils.a will try and make these calls:
system/extras/ext4_utils/wipe.c
Code:int wipe_block_device(int fd, s64 len) { u64 range[2]; int ret; range[0] = 0; range[1] = len; ret = [B][U]ioctl(fd, BLKSECDISCARD, &range);[/U][/B] if (ret < 0) { range[0] = 0; range[1] = len; ret = [B][U]ioctl(fd, BLKDISCARD, &range);[/U][/B] } return 0; }
I believe that both those calls have basically been disabled because ioctl() calls are not supported in the mmc block device driver under the GB kernel (for E4GT EL29 GB kernel source).
So what does that all mean?
I believe GB systems are doubly "safe". The kernel will not execute mmc_erase() even though the functionality is there because the ioctl() call entry point from user space to kernel space has been disabled and also GB recovery and GB update-binary never even attempt to make the ioctl() calls.
That explains why, if you pair an "unsafe" update-binary from the CM9/AOKP ROM update.zip with a GB kernel, it is still safe.
