Android 10 encryption issue after rom downgrade

[JackSlaterIV]

Hi guys, I am asking you some help due to an emergency.

I had to downgrade an Android 10 rom where I had encryption turnen on (rom).
All I did was flashing a previous (minor) version of the rom via TWRP with just a "wipe cache/dalvik".
After rebooting my pin was not recognized anymore by both Android and TWRP.

I did many tentatives and at some point I typed "default_password" as pin, when asked by Android during the boot, and there was a important change:
1. After rebooting I typed my old pin, and now Android always tells me: "The password you entered is correct, but unfortunately your data is corrupt".
2. Now when TWRP asks for the password, it accepts the old pin too. But it is "unable to mount storage".
3. The system partition's contents are now visible: before they were not showing at all. The data partition is not accessible (error decrypting…).

I have done a lot of studying and tentatives to get the phone working without formatting and losing the data, but I could not solve the issue. I don't think the data is actually corrupted, because the rom downgrade was a minor version and it did not modify anything about encryption.

Could you please point me to the right direction? I am trying to understand what could have gone wrong, and find some possible solution.

EDIT: more details and list of the attempted solutions in this post: https://forum.xda-developers.com/t/...sue-after-rom-downgrade.4168821/post-85210619

 

[Deleted member 1890170]

After rebooting I typed my old pin, and now Android always tells me: "The password you entered is correct, but unfortunately your data is corrupt".

Look inside here.

 

[JackSlaterIV]

Look inside here.

Both methods cause /data to be erased, which is what I don't want. Thanks anyway.

 

[alecxs]

guess if something has changed since your dirty flash, it must be something in last 16384 bytes where the crypto footer is

there are some bytes which are most likely one or eight flag(s)
Flags : 0x00000000


you can locate and copy the crypto footer like this

- check fstab for location if it says encryptable=footer (or see recovery.log)
- get partition size and calculate the offset -16384
- extract the footer to /sdcard with dd (any file name)

on PC open that file with Hex Editor

- the crypto footer will start with magic 0xD0B5B1C* (little endian). in my case it's C5 B1 B5 D0 as it's a samsung device.
- you should also see a string aes-cbc-essiv:sha256 (in my case aes-xts)

inspect the crypto footer with python script. you can't decrypt since android uses scrypt+keymaster but it will give you a nice layout

- install python 2.7
- run that script bruteforce_stdcrypto.py

Android FDE crypto footer
-------------------------
Magic              : 0xD0B5B1C4
Major Version      : 1
Minor Version      : 3
Footer Size        : 2352 bytes
Flags              : 0x00001008
Key Size           : 128 bits
Failed Decrypts    : 36
Crypto Type        : aes-xts
Encrypted Key      : 0xCCE7D93B501B400D3D81726806F92936
Salt               : 0x51B68B017C2A181E3ABD0B041FBFAA14
KDF                : scrypt+keymaster
N_factor           : 15    (N=32768)
r_factor           : 3    (r=8)
p_factor           : 1    (p=2)
crypt type         : PIN
FS size            : 52453304
encrypted upto     : 52453304
-------------------------

as you can see in your case the flags are 0x00001008 so you can easier locate that in your Hex Editor

- convert the string little endian 0x00 00 10 08 -> 08 10 00 00
- you will find that four bytes at offset 13 in the first line
- reset the flags to 00 00 00 00 and save the file

if you prefer linux you can also use that shell script for doing that. fde_crypto.sh


Before messing up your data partition do a full dump for backup purposes (because we don't know what we are doing here, encryption is complicated stuff). In case you broke something you can just adb push it later

adb pull /dev/block/bootdevice/by-name/userdata

Now, write the new crypto footer back to end of userdata partition

- copy the file back to the device (another file name)
- get partition size and calculate the offset -16384
- write the footer to offset with dd (seek)

adb push data-footer.bin /sdcard
adb shell
cd /sdcard
blockdev --getsize /dev/block/bootdevice/by-name/userdata
dd bs=512 seek=$((52453336-32)) count=32 if=data-footer.bin of=/dev/block/bootdevice/by-name/userdata

Note: i don't know if that works. indeed, that's all guesswork based on your input in pm. good luck!

 

[JackSlaterIV]

Hi and thanks again. As you wrote we spoke a lot via PM before your post.
I reset the footer flags to 00 00 00 00. Then used dd as you suggested to overwrite the userdata footer.

During the first Android boot, it asked me to enter the pin, but then it failed to decrypt, and now is always showing the old message "The password you entered is correct, but unfortunately your data is corrupt" .
So looks like the flag at least reset the default mode.
And TWRP still can't decrypt the partition.
It's no surprise because, as you showed me, the userdata partition may be corrupted.


I wanted to get the updated footer back from the phone to my PC. I used this:
dd bs=512 seek=$((52453336-32)) count=32 if=/dev/block/bootdevice/by-name/userdata of=/tmp/data-footer-new.bin
32+0 records in
32+0 records out
16384 bytes (16.0KB) copied, 0.009945 seconds, 1.6MB/s

Then Adb pull tmp/data-footer-new.bin
But it started downloading a few GB of data. I checked the size via ls -l:
-rw-rw-rw- 1 root root 26856108032 Dec 20 14:04 data-footer-new.bin
What I did wrong? Is it a bug?

 

[alecxs]

usage problem - this is expected behavior for dd seek. when the output file is too small or doesn't exist, a zero padding is filled to create a big file before the offset starts, where it finally starts to write the real data (32 x 512 bytes)

you have mixed up parameters skip/seek, in your case copied first 16384 bytes from userdata into the end of a big file data.footer.bin

btw the userdata partition is not corrupt per se (or at least there is no proof i could show ever) you will never find ext4 file system magic 0xEF53 on encrypted userdata, only on dm-0 (if decrypted successfully). but true, mounting is a different case, indeed mount may fail even for successfully decrypted file system (like for Redmi 5). so the safest way to know if decrypted successfully is looking for zero paddings, first 1024 bytes will have enough of it...

you can try lot of other values for this flag (0x00001000 like for LG?) or try other (undiscovered) flags. you need a lot patient and time as you are the first one trying this. also reset the failed decrypts counter as this may important for gatekeeper timeout

i recommend to decrypt straight from twrp command line, should "work" without reboot

edit: i could even imagine automatizing that with script (10 sec/attempt - min timeout)

edit 2: interesting too would be binary (or checksum) compare of userdata before/after failed attempt (without footer) to figure out if changes happen elsewhere (other than footer)

 

[alecxs]

even more interesting, you could factory reset and reproduce the mistake, make a snapshot before/after and bitwise compare where the changes happen

if the key itself has changed, there is no possible way to revert as the old key is lost

but decryption should still be possible on the newer android version, all you need is working twrp that fits

edit: factory reset is maybe not the best idea! turns out for FBE file-based-encryption the KEK is stored in TEE and depending on rollback resistance (not related to version binding) master key may deleted on factory reset. FBE is introduced with Android 7.1 - your device is still running good old FDE full-disk-encryption - but who knows what additional protections Android 10 enforces? can't guarantee that KEK is encrypted by hardware-backed RSA-2048 private key and screenlock credentials only and everything is stored in crypto footer only, although the documentation doesn't indicate contradictory

 

[JackSlaterIV]

usage problem - this is expected behavior for dd seek. when the output file is too small or doesn't exist, a zero padding is filled to create a big file before the offset starts, where it finally starts to write the real data (32 x 512 bytes)

you have mixed up parameters skip/seek, in your case copied first 16384 bytes from userdata into the end of a big file data.footer.bin

Can you confirm this is the correct command to get the new footer?
dd bs=512 skip=$((52453336-32)) count=32 if=/dev/block/bootdevice/by-name/userdata of=/tmp/data-footer-new.bin

I think that this new big file may have caused some corruption.
I want to restore the userdata partition backup, but I read it's not easy as a simple adb push: https://android.stackexchange.com/q...n-image-of-android-partition-from-my-linux-pc
Can you tell me any reliable way to do this, apart using busybox as in the above replies?

btw the userdata partition is not corrupt per se (or at least there is no proof i could show ever) you will never find ext4 file system magic 0xEF53 on encrypted userdata, only on dm-0 (if decrypted successfully). but true, mounting is a different case, indeed mount may fail even for successfully decrypted file system (like for Redmi 5). so the safest way to know if decrypted successfully is looking for zero paddings, first 1024 bytes will have enough of it...

Thanks for clarifying this.

you can try lot of other values for this flag (0x00001000 like for LG?) or try other (undiscovered) flags. you need a lot patient and time as you are the first one trying this. also reset the failed decrypts counter as this may important for gatekeeper timeout

i recommend to decrypt straight from twrp command line, should "work" without reboot

edit: i could even imagine automatizing that with script (10 sec/attempt - min timeout)

edit 2: interesting too would be binary (or checksum) compare of userdata before/after failed attempt (without footer) to figure out if changes happen elsewhere (other than footer)

Indeed I had already tried 0x00001000 and resetting the counter, before the mess up with my dd command.
Do you know any other combination I could try?

Something I could try is see what happens to /userdata if I type default_password at the first boot.

 

[alecxs]

yes, that is the right command

no, you didn't mess up with big file because of= is the only thing written (and /tmp is only in RAM)

yes, simple adb push is fine and works quite well for single partition. the link is talking about something different (whole eMMC including gpt and bootloader)

no, i have no clue about the flags. the source code might help but it's above my knowledge (yet)

 

[alecxs]

found some explanation for flags
https://www.0xf8.org/2019/01/analyz...axy-s7-data-partition-with-samsung-encryption

 

[alecxs]

have implemented the above link, not sure if i am doing it right but have a look into script fde_crypto.sh

 

[JackSlaterIV]

Hello alecxs, thanks for your last messages. Sorry for this long delay.
I did not write any update because I couldn't find anything useful in the footer and the full data images. The phone is still not in use, in a drawer.

I had tried different flags, but after each tentative I had the same result. The "system" tells that data may be corrupted and updates the flag accordingly.

I had compared before vs after data images and did not find any difference. There is only one field in the footer that is modified after each tentative: the sha256 of the footer (offset 90c).

Without further information I cannot tell what causes this issue, if the data is corrupt or not. It would be useful having a more verbose mode in the mount command, so that it shows the reason of the failed mount. I guess it's not possible.

 

[alecxs]

i think it is caused by rollback resistance and you should try higher android version (that one that messed up everything) with compatible TWRP. besides recovery.log you can check dmesg and logcat for additional information

 

[JackSlaterIV]

Hi again,

I am attaching dmesg and recovery log, taken from TWRP after a failed mount of the data partition, using my pin, with the crypto footer flags reset to zero.
I could not find anything, so I hope someone reading this could give me a hint.

From what I can see, anti rollback and verified boot are disabled in Mi5 and in LineageOS based roms (see here).
Regarding TWRP I always used the same version recommended by the rom developer.

EDIT: file attachment not working for me...
See them here:

dmesg.log

Shared with Dropbox

www.dropbox.com

recovery.log

Shared with Dropbox

www.dropbox.com

 

[alecxs]

looks like double encryption bug. try to dump content of dm-0 and restore it to userdata, that should at least eliminate the FDE encryption. second encryption is FBE? let binwalk analyze usually there is unencrypted area

 

[steadfasterX]

... you should try higher android version [...]

just as a reference: for this you would find errors like

E vold : upgrade_key failed, code -38 E Cryptfs : Failed to upgrade key

which is not the case here.

(note: yes it says "upgrade" but in my example the installed key is from a higher version so actually a downgrade would be needed - which is not possible at all.)
(see a full example and details here and google details here)

Hi again,

I am attaching dmesg and recovery log, taken from TWRP after a failed mount of the data partition, using my pin, with the crypto footer flags reset to zero.
I could not find anything, so I hope someone reading this could give me a hint.

From what I can see, anti rollback and verified boot are disabled in Mi5 and in LineageOS based roms (see here).
Regarding TWRP I always used the same version recommended by the rom developer.

EDIT: file attachment not working for me...
See them here:

dmesg.log

Shared with Dropbox

www.dropbox.com

recovery.log

Shared with Dropbox

www.dropbox.com

the interesting part is here:

<3>[    5.880909] QSEECOM: __qseecom_process_incomplete_cmd: fail:resp res= -65,app_id = 0,lstr = 12288
<3>[    6.007678] QSEECOM: __qseecom_process_incomplete_cmd: fail:resp res= -71,app_id = 0,lstr = 12288
<3>[    6.007697] QSEECOM: __qseecom_set_clear_ce_key: process_incomplete_cmd FAILED, resp.result -71
<3>[    6.007716] QSEECOM: qseecom_create_key: Failed to create key: pipe 2, ce 0: -22
<3>[    6.007726] QSEECOM: qseecom_ioctl: failed to create encryption key: -22
<3>[    6.098357] scm_call failed: func id 0x72000501, ret: -2, syscall returns: 0xffffffffffffffbf, 0x0, 0x0
<3>[    6.225071] QSEECOM: __qseecom_process_incomplete_cmd: fail:resp res= -71,app_id = 0,lstr = 12288
<3>[    6.225082] QSEECOM: __qseecom_set_clear_ce_key: process_incomplete_cmd FAILED, resp.result -71
<3>[    6.225096] QSEECOM: qseecom_create_key: Failed to create key: pipe 2, ce 0: -22
<3>[    6.225104] QSEECOM: qseecom_ioctl: failed to create encryption key: -22

the main error is likely:

<3>[    5.880909] QSEECOM: __qseecom_process_incomplete_cmd: fail:resp res= -65,app_id = 0,lstr = 12288
[..]
<3>[    6.007716] QSEECOM: qseecom_create_key: Failed to create key: pipe 2, ce 0: -22
<3>[    6.007726] QSEECOM: qseecom_ioctl: failed to create encryption key: -22

-65 means: ATTESTATION_APPLICATION_ID_MISSING whatever that means actually.

looks like double encryption bug. try to dump content of dm-0 and restore it to userdata, that should at least eliminate the FDE encryption. second encryption is FBE? let binwalk analyze usually there is unencrypted area

interesting idea especially as it actually can decrypt /dev/dm0 according to the recovery.log but then failing to mount it.
I would +1 here and try if you can dump the content of /dev/dm0 after trying the decryption ( e.g. when you have an ext sdcard: `dd if=/dev/dm0 of=/external_sd/dump.img bs=4096` )

Other then that it might be an issue with your blobs - either in TWRP, or the device

 

[alecxs]

i think your issue is bit different and the links provided are about FBE. afaik FDE does not hold keys in TEE (except for hardware-backed RSA-2048 private key which is not flushable) so i am not sure if upgradeKey affects crypto-footer but deleteKey is clearly some keystore thing

to eliminate issues with TWRP i would do decryption test on working block encryption (and maybe try OrangeFox) only then you can determine issues with faulty crypto-footer

 

[JackSlaterIV]

Hello guys, thanks for your help.

I dumped both sda14 and dm-0 partitions (using adb dump).
The dm-0 ("decrypted" partition) is a smaller binary file (26.856.091.648 bytes) vs sda14 (26.856.108.032 bytes).
I compared these binary files using HxD and they look different. dm-0 does not contain the crypto footer section (the 16384 bytes difference).

I just installed binwalk for the suggested purpose, and started analyzing dm-0 (binwalk dm-0). It is outputting something and I don't have any idea of how much time it would take to complete the task.
Let's see if I can attach a screenshot..

 

[alecxs]

okay not sure binwalk may just false detect random data or it may real files. anyway you can concatenate dm-0 with crypto-footer from userdata and check what TWRP says about this garbage then

 

[JackSlaterIV]

okay not sure binwalk may just false detect random data or it may real files. anyway you can concatenate dm-0 with crypto-footer from userdata and check what TWRP says about this garbage then

Yes indeed.
I did not find any text in the dm-0 binary.

Can you suggest me how I concatenate these files? I have dm-0 and crypto-footer in separate files. EDIT: just by using HxD.

To overwrite the partition can I use "adb push dm-0-new /dev/block/sda14"?