[PSA][MUST SEE] (Updated) Things You Should Know Before Unlocking Bootloader

[topjohnwu]

System Verity Checks​

Thanks to the investigation of @nkk71

first off, sorry @jollywhitefoot that word just doesnt work well with me, i know what it means, how it's used (harmless), it just hits a nerve


My Final Conclusion

As I have suspected it's the

[ro.boot.veritymode]: [enforcing] --> good

vs

[ro.boot.veritymode]: [logging] --> BAD, and will result with the
"There was an unexpected error and the file system may be corrupted. Please contact HTC support for further assistance."

encryption has nothing to do with it by the way.


So here's a breakdown of what happens during a normal boot.
1) aboot starts, "does stuff"
2) then hands over to boot.img for further processing
3) which should eventually end up with a booting the ROM

This is what a normal (verity=enforcing) would look like:

aboot part:
...
[510] read_device_info_mmc:read device info from mmc
[520] htc_read_device_info: verity_mode = enforcing
....

kernel/boot.img part:
...
...

How to break it (DO NOT DO THIS): have your boot.img use verified boot (ie verity checking on a modified system partition, by having the "verify" flag in fstab), since system won't pass the check, this will happen:

aboot is still OK:
...
[500] read_device_info_mmc:read device info from mmc
[510] htc_read_device_info: verity_mode = enforcing
...

BUT once it hits the kernel/boot.img
....
[    2.737990] c0    466 fs_mgr: Enabling dm-verity for system (mode 1)
[    2.801811] c1     53 device-mapper: verity: verity_verify_io:io_real_digest=a48df664 io_want_digest=a43baac3
[    2.801826] c1     53 device-mapper: verity: 259:30: data block 0 is corrupted
[    2.802645] c1     53 device-mapper: verity: verity_verify_io:io_real_digest=5c67cc54 io_want_digest=5864f2b1
[    2.802659] c1     53 device-mapper: verity: 259:30: data block 1 is corrupted
[    2.808372] c0      6 device-mapper: verity: verity_verify_io:io_real_digest=2a70e9c0 io_want_digest=7bafd223
[    2.808386] c0      6 device-mapper: verity: 259:30: data block 243 is corrupted
[    2.826504] c1     53 device-mapper: verity: verity_verify_io:io_real_digest=6dff1798 io_want_digest=b40f453c
[    2.826520] c1     53 device-mapper: verity: 259:30: data block 950780 is corrupted

unfortunately no matter what you restore in TWRP, any following bootup will already be set to "logging" from aboot (because one or several partitions have been "flagged",
so the kernel/boot.img verification won't matter any more

...
[530] read_device_info_mmc:read device info from mmc
[540] htc_read_device_info: verity_mode = logging
....

this will be passed on to kernel/boot.img and then to the ROM, which the Updater will now consider a corrupt emmc


How to fix: (you tell me)
i've had limited success, in restoring back to "enforcing" instead of "logging", on occasion (no RUU yet), and nothing "conclusive"

and given the amount of [low level] partitions that are now messed up, i have no further info


so yay.... i'm about as close to bricking as possible for the moment

Data Encryption and Data Signal​

Short Story:
Decrypting the DATA partition will break signal

Full Story:
I was testing out data decryption one day and did some tests. Soon I found out that my phone cannot receive any signal.
At first, I thought it was caused by Xposed, so I restore system and boot, flash SuperSU to patch the boot image. The result is still the same, no signal.
Then I thought it might be SuperSU, so I manually modified the fstab of the ramdisk, the signal is still broken.
Before doubting the hardware might be broken, I restored everything back to stock (include forceencrypt boot), and it everything works!!
I've done the tests above several times, just in case I missed anything. The results are always the same: once the data is decrypted, the signal won't work.

Conclusion:
Don't try to decrypt your data until people found out the reasons.

 

[Captain_Throwback]

I look forward to exploring this further once I have my 10. Looks like it'll be loads of fun

 

[jcase]

I wouldn't worry about s-off, soon enough. SunShine will support m10 in the near future (dont ask for ETAs or anything of the sort). There are bound to be some issues we need to fix up once we have our devices, but I am confident we will find a solution shortly.

I'd also stick to flashing full RUUs (official or repacks, makes little difference) when updating a modified device, I've never been a fan of OTAing a modified device, and there is a lot of good reasons not to. Especially if the new boot.img is fully expecting an encrypted userdata partition and you don't have one. Rom developers, recovery developers, kernel developers, and yes exploit developers need to get used to working with an encrypted userdata partition.

If you can get me a logcat from boot on wards on a unencrypted device with radio not working, I will gladly look at it.

To Moderators: I'm not sure if this thread belongs to the developments section, what I'm thinking about is that most people will look at development threads before unlocking bootloaders. Feel free to move this thread to the appropriate sections, but I hope no matter where it goes, this thread could be stickied, since it is quite important.

Everything below is confirmed by various Taiwanese users. Being one of the early adopters, I'm here to step on the mines and do experiments for all of you.

About OTA​

Short Story:
ALL DEVICES BOOTLOADER UNLOCKED CANNOT RECEIVE OTAs
It might be caused by recovery mismatch, we still cannot know

Full Story:
You got your new phone, and you can't wait to unlock your bootloader, flash TWRP and all the goodies, right?
This is what I have done with my 10, but...
Wait a second!!
The first OTA has just be pushed here in Taiwan, and a weird thing is happening...
Everyone with their phone unlocked and flashed TWRP CANNOT receive the OTA.
It doesn't mean that the OTA cannot be applied, what I mean is your device will be greeted with this screen if an update is available:

Viewing the logcat, it spits out this message:

1557  8835 W htcCheckinService: The device is emmc error case, can't let user get fota update.

This doesn't mean my system partition is tampered, I've backup the untouched system images.
I am now updated with some special tricks using my system image backup, this shows that my image is legit and untampered.
So the problem isn't that simple.

After a few hours of investigation, I successfully fixed the issue... WITH QUITE SOME EFFORT.
It will take some time to write a complete guide about the procedure. Stay tuned for more details.

Conclusion:
Follow the tiresome guide (I'm going to release soon), or wait for S-OFF to happen.

About Data Encryption​

Short Story:
ALL DEVICES WITH DECRYPTED DATA CAN NOT RECEIVE DATA SIGNAL

Full Story:
I was testing out data decryption one day and did some tests. Soon I found out that my phone cannot detect any signal.
First I thought it was caused by Xposed, so I restore system and boot, flash SuperSU to patch the boot image. The result is still the same, no signal.
Then I thought it might be SuperSU, so I manually modified the fstab of the ramdisk, the signal is still broken.
Before doubting the hardware might be broken, I restored everything back to stock (include forceencrypt boot), and it everything works!!
I've done the tests above several times, just in case I missed anything. The result is always the same: once the data is decrypted, the signal won't work.

Conclusion:
Don't try to decrypt your data until people found out the reasons.

 

[topjohnwu]

I wouldn't worry about s-off, soon enough. SunShine will support m10 in the near future (dont ask for ETAs or anything of the sort). There are bound to be some issues we need to fix up once we have our devices, but I am confident we will find a solution shortly.

I'd also stick to flashing full RUUs (official or repacks, makes little difference) when updating a modified device, I've never been a fan of OTAing a modified device, and there is a lot of good reasons not to. Especially if the new boot.img is fully expecting an encrypted userdata partition and you don't have one. Rom developers, recovery developers, kernel developers, and yes exploit developers need to get used to working with an encrypted userdata partition.

If you can get me a logcat from boot on wards on a unencrypted device with radio not working, I will gladly look at it.

Here you go
RUUs are not easy to get, especially the newer devices that are not US versions.
Really hope HTC can provide official RUUs to restore our devices.

 

[jcase]

Here you go
RUUs are not easy to get, especially the newer devices that are not US versions.
Really hope HTC can provide official RUUs to restore our devices.

We will see RUUs for it

Will look when i get back home

 

[Captain_Throwback]

@topjohnwu

Can you upload a copy of the OTA zip when you get a chance? I'm curious to see how the contents look. Thanks!

 

[topjohnwu]

@topjohnwu

Can you upload a copy of the OTA zip when you get a chance? I'm curious to see how the contents look. Thanks!

I'm uploading everything now

 

[Superguy]

Just curious - if this ends up being a huge PITA, anyone know what HTC's return policy is? For me it was between the 10 and the Exynos S7, and I'm giving this a try first.

Does Sammy encrypt the data portion on the S7s as well?

Glad to see Sunshine's coming our way!

 

[PhoenixPath]

He mentioned TWRP. Did he try stock recovery? (or did I miss that in the post?)

Don't get me wrong; I know the guy is a wizard with HTC devices, but I want to know that was tried and failed...

Sent from my HTC One_M8 using Tapatalk

 

[topjohnwu]

I had tried restoring the system back to 1.21.709.2, then flash 1.30.709.1 stock recovery, still the OTA won't show up on the device.
So it might detect that the recovery and system doesn't match and refuse to push update. There is possibility that HTC is not actually detecting the boot loader unlock, but instead detecting the stock recovery.
However, I have no chance to test it since there is no way to get 1.20.709.2 stock recovery, only if someone unlocks the boot loader now and test if he can detect the OTA, or HTC releases a new OTA so I can test from 1.30.709.1

 

[Captain_Throwback]

I had tried restoring the system back to 1.21.709.2, then flash 1.30.709.1 stock recovery, still the OTA won't show up on the device.
So it might detect that the recovery and system doesn't match and refuse to push update. There is possibility that HTC is not actually detecting the boot loader unlock, but instead detecting the stock recovery.
However, I have no chance to test it since there is no way to get 1.20.709.2 stock recovery, only if someone unlocks the boot loader now and test if he can detect the OTA, or HTC releases a new OTA so I can test from 1.30.709.1

Did you revert your main version? That's the only way you'd be able to take the OTA again.

 

[topjohnwu]

Did you revert your main version? That's the only way you'd be able to take the OTA again.

Yes of course
It is the same issue, same logcat error.
So I simply guess that HTC blocked all bootloader unlocked phones to receive OTA. But it might be detecting the matched recovery with the system though.
Have to wait for someone to unlock their bootloader, or HTC has to release the next OTA for me to test (right now I only got the new recovery)
However, the new recovery still works on the old version.
The trick I used to upgrade my device is about grabbing/guessing the OTA link from HTC server. Then I can manually flash the OTA zip.

 

[mahdibassam]

So considering these issues, there is no stable root for the 10 as it will cause the signal to drop off right?

Sent from my Galaxy S7 Edge

 

[Captain_Throwback]

So considering these issues, there is no stable root for the 10 as it will cause the signal to drop off right?

Incorrect. You can still root via the usual/normal means. You just can't be decrypted. That's apparently what is breaking signal.

 

[topjohnwu]

(Apr 26 Update)
By tests from other HTC 10 owners, we now confirmed that unlocked bootloader without any modifications can receive the OTA.
So it means that HTC didn't detect the bootloader status when pushing OTAs, it is depending on other factors.

What comes up my mind is the following two possibilities (both personal guesses, no confirmations):

1. The 1.21 stock recovery is different from the 1.30 one, although both functions the same and can accept flashing the OTA.
   HTC checks if the stock recovery is valid before allowing the OTA to show up on the device.
2. HTC introduces some protection method like the Samsung knox: un-official boot images cause the phone to write some flags to the device.
   If this is the case, @jcase might be interested in it

 

[mahdibassam]

Incorrect. You can still root via the usual/normal means. You just can't be decrypted. That's apparently what is breaking signal.

Oh ok. Marshmallow is encrypted by default if I'm not wrong, so if you don't manually decrypt you should be ok then? Also, since there is no root here on XDA for the 10, we would just have to flash twrp and download and flash super su and that's it? Thanks for your reply.

Sent from my Galaxy S7 Edge

 

[Captain_Throwback]

Oh ok. Marshmallow is encrypted by default if I'm not wrong, so if you don't manually decrypt you should be ok then? Also, since there is no root here on XDA for the 10, we would just have to flash twrp and download and flash super su and that's it? Thanks for your reply.

Correct. You can choose any root method (SuperSU isn't the only option), but that's been the standard way to root HTC devices for the past several years.

 

[Superguy]

Maybe I misunderstood, but didn't someone say that we needed to be decrypted in order to put a Rom on, it us that just so we don't have to wipe the /data and internal SD each time?

Sent from my HTC One_M8 using XDA Free mobile app

 

[topjohnwu]

Maybe I misunderstood, but didn't someone say that we needed to be decrypted in order to put a Rom on, it us that just so we don't have to wipe the /data and internal SD each time?

Sent from my HTC One_M8 using XDA Free mobile app

Yes, that's the usual case. What I was pointing out here is that, you can no longer decrypt the data partition anymore (at least for now).
If you want to flash roms, you have to use external sdcard or use adb sideload, and yes if you want to clear data, you have to also clear all internal storage.

 

[Captain_Throwback]

(Apr 26 Update)
By tests from other HTC 10 owners, we now confirmed that unlocked bootloader without any modifications can receive the OTA.
So it means that HTC didn't detect the bootloader status when pushing OTAs, it is depending on other factors.

What comes up my mind is the following two possibilities (both personal guesses, no confirmations):

1. The 1.21 stock recovery is different from the 1.30 one, although both functions the same and can accept flashing the OTA.
   HTC checks if the stock recovery is valid before allowing the OTA to show up on the device.
2. HTC introduces some protection method like the Samsung knox: un-official boot images cause the phone to write some flags to the device.
   If this is the case, @jcase might be interested in it

When you manually modified the boot.img while decrypted, what exactly did you change? It is possible that the system is performing a checksum validation of the boot block device to determine whether it's been modified, prior to allowing an OTA to install. But it's also possible that certain modifications to the ramdisk cause it to error out as well.

It might just perform the same signature check that causes the "red text" to appear when booting the device, in which case yes, the boot.img would need to be fully stock for the device to accept an OTA. As long as the recovery is signed and is from the current OTA of the previous version, that shouldn't be an issue for taking the OTA, as the only check done there should be for the device name and build fingerprint.