Flashing ROM after turning on Encryption

efrant · Apr 26, 2012

bunklung said:
Great news. So what is the size of this partition? I know the Gnex has 1G ram, but I am sure it doesn't use all of this. Is there a way to do a full nandroid backup and restore too? I don't think Nandroids are larger than 1G.

orcruin also devised a solution to flashing custom roms by formatting /cache but it seems near identical to shaaXo. orcruin makes no mention of the error message on boot:
http://xdaforums.com/showpost.php?p=24923240&postcount=16

Are we talking about the same solutions here? Expert chime in

Exactly the same...

bunklung · Apr 26, 2012

Do you think there would be a way to shrink the encrypted sdcard partition (32G) used by the vz gnex and leave it as some sort of temp unencrypted file system so that you can push/pull a nandroid backup file from them?

I just checked my nandroid backups and they are all over 1G. That rules out the device memory.

efrant said:
Exactly the same...

efrant · Apr 26, 2012

bunklung said:
Do you think there would be a way to shrink the encrypted sdcard partition (32G) used by the vz gnex and leave it as some sort of temp unencrypted file system so that you can push/pull a nandroid backup file from them?

I just checked my nandroid backups and they are all over 1G. That rules out the device memory.

Not sure what you are trying to do? Why not just flash a ROM and, then, restore your backup. It's obviously very easy.

As for changing partitions, I'm not sure that is easily done. If Samsung devices work the same way as HTC devices, then in order to change the partition sizes, we would have to have a bootloader that will allow it. Feel like disassembling the bootloader?

shaaXo · Apr 26, 2012

bunklung said:
Is there a way to do a full nandroid backup and restore too?

Making a backup isn't that hard, you could create the tar-files while android is running or get the encrypted images via recovery.

But I don't know a comfortable way to actually restore the backups, especially when they are larger. Splitting up the backups into RAM-sized pieces and puzzling everything back together on the phone is doable, but tedious and error prone. I'll look into getting CWM to expose *all* block devices via USB Mass Storage, so you could just backup/restore using standard tools on the connected PC.

---------- Post added at 10:07 PM ---------- Previous post was at 09:20 PM ----------

I think I got it already. When booted into CWM, use the following command to get access to the entire internal memory (mmc card).

Code:

adb shell "echo /dev/block/mmcblk0 > /sys/devices/platform/usb_mass_storage/lun0/file"

The storage should be detected immediately on the host and can be used to make/restore direct disk images with appropriate tools (I just use linux and dd).

On the Nexus S, this is pretty nice, as it has a separate 1GB /data partition and makes independant backup/restore possible. The partition layout of the Galaxy Nexus looks like there would be just a single huge 'userdata' partition, that has to be read/written as a whole. Obviously this makes the whole process extremely slow and potentially reduces the lifetime of the flash storage (when restoring a lot). But it allows a complete (as in every-single-bit) backup and restore and is relatively safe.

bunklung · Apr 26, 2012

I am trying to do what most people use nandroid restore and backup for, but my hands are tied because of encryption. It's great that we have a sorta kinda work around with flashing roms. When you flash a rom and restore your backup, what do you mean? I can't make a nandroid backup.

efrant said:
Not sure what you are trying to do? Why not just flash a ROM and, then, restore your backup. It's obviously very easy.

As for changing partitions, I'm not sure that is easily done. If Samsung devices work the same way as HTC devices, then in order to change the partition sizes, we would have to have a bootloader that will allow it. Feel like disassembling the bootloader?

---------- Post added at 05:48 PM ---------- Previous post was at 05:38 PM ----------

I recall making a manual backup within Adb/windows when I was trying to figure out how to make a Odin backup. My thinking was that I could restore within Odin. I never got that far, but was able to use fast boot to restore to a stock kernel.

Could I use adb while in recovery and restore all those partitions effectively doing a manual nandroid via a windows host?

shaaXo said:
Making a backup isn't that hard, you could create the tar-files while android is running or get the encrypted images via recovery.

But I don't know a comfortable way to actually restore the backups, especially when they are larger. Splitting up the backups into RAM-sized pieces and puzzling everything back together on the phone is doable, but tedious and error prone. I'll look into getting CWM to expose *all* block devices via USB Mass Storage, so you could just backup/restore using standard tools on the connected PC.

---------- Post added at 10:07 PM ---------- Previous post was at 09:20 PM ----------

I think I got it already. When booted into CWM, use the following command to get access to the entire internal memory (mmc card).

Code:

adb shell "echo /dev/block/mmcblk0 > /sys/devices/platform_mass_storage/lun0/file"

The storage should be detected immediately on the host and can be used to make/restore direct disk images with appropriate tools (I just use linux and dd).

On the Nexus S, this is pretty nice, as it has a separate 1GB /data partition and makes independant backup/restore possible. The partition layout of the Galaxy Nexus looks like there would be just a single huge 'userdata' partition, that has to be read/written as a whole. Obviously this makes the whole process extremely slow and potentially reduces the lifetime of the flash storage (when restoring a lot). But it allows a complete (as in every-single-bit) backup and restore and is relatively safe.

rocketnz · Apr 27, 2012

Hi All, my SD card was encrypted and i have just performed a full wipe on internal to reset to defaults and now the phone doesnt know the SD card is encrypted and wants to format it, how do you access your SD card after a full phone wipe? (internal)

dont tell me all my data is lost?

bunklung · Apr 27, 2012

99% certain it's lost, but maybe someone else can chime in.

rocketnz said:
Hi All, my SD card was encrypted and i have just performed a full wipe on internal to reset to defaults and now the phone doesnt know the SD card is encrypted and wants to format it, how do you access your SD card after a full phone wipe? (internal)

dont tell me all my data is lost?

shaaXo · Apr 28, 2012

rocketnz said:
Hi All, my SD card was encrypted and i have just performed a full wipe on internal to reset to defaults and now the phone doesnt know the SD card is encrypted and wants to format it, how do you access your SD card after a full phone wipe? (internal)

I would have expected the factory reset process to wipe the internal storage.

bunklung said:
99% certain it's lost, but maybe someone else can chime in.

Maybe the encryption master key is still around somewhere. Google says (source.android.com/tech/encryption/android_crypto_implementation.html) that they keep it in the last 16KiB of the partition, but on my Nexus S, it's saved in the /efs/userdata_footer file. You could look in the efs folder (adb shell ls -l /efs) and see if it's still there. If not, the crypto footer may be there, but checking for that is more complicated.

Also a quick status update: I am now able to manually setup dmcrypt and can access the encrypted /data and /sdcard from my linux box. I'll probably write a small script for that.

bunklung · Apr 29, 2012

Sweet. Keep us posted.

Do you think FreeOTFE would do the trick on Windows?

shaaXo said:
Also a quick status update: I am now able to manually setup dmcrypt and can access the encrypted /data and /sdcard from my linux box. I'll probably write a small script for that.

shaaXo · Apr 29, 2012

bunklung said:
Do you think FreeOTFE would do the trick on Windows?

Technically, yes, but it's probaby easier to get CWM Recovery to mount the encrypted partitions.

bunklung · Apr 30, 2012

Slightly confused here. In your below comment, you are still referring to using a Linux host machine connected via a USB cable? And when you say mount the encrypted partitions, you are refering to using ADB when in CWM Recovery and using this command, "adb shell "echo /dev/block/mmcblk0 > /sys/devices/platform_mass_storage/lun0/file""?

It's too bad CWR didn't have native dmcrypt support.

shaaXo said:
Technically, yes, but it's probaby easier to get CWM Recovery to mount the encrypted partitions.

rocketnz · Apr 30, 2012

rocketnz said:
Hi All, my SD card was encrypted and i have just performed a full wipe on internal to reset to defaults and now the phone doesnt know the SD card is encrypted and wants to format it, how do you access your SD card after a full phone wipe? (internal)

dont tell me all my data is lost?

Ok WEIRD. I just did a full wipe on my phone again. then re rooted it, went to CWM and it could see my SD card, so i load up file explorer and boom, there is my SD card and all my files. even though its encrypted, ok now i'm confused.

thank christ. full backup now before anything else goes wrong. praise the lord! hahaha

shaaXo · May 1, 2012

bunklung said:
Slightly confused here. In your below comment, you are still referring to using a Linux host machine connected via a USB cable?

I'm writing it in python right now, so the script itself should run fine on Windows/Mac OS.

bunklung said:
It's too bad CWR didn't have native dmcrypt support.

Actually, it does. dm-crypt is just the kernel component and the CyanogenMod CWM-based recovery does have the necessary modules built in (it uses the same kernel for recovery and normal booting). But besides kernel support, there is just no userspace tool to configure it (on a typical Linux host, there would be dmsetup and cryptsetup). So I compiled a small C program, based on the code that android/vold itself uses to set up dm-crypt and copied it over into the recovery system. Now I can access my encrypted partitions inside CWM and already succeeded in making a backup.

Restoration is a bit tricky, since CWM probably tries to format the wrong device before restoring, but I'm happy with my progress for today.

shaaXo · May 3, 2012

I've finished a beta version of androidcrypt.py, which is able to get the recovery system to decrypt and mount the encrypted partitions. It should detect the connected android device, do some checks, look for the crypto footer, ask for a password and finally decrypt and mount /userdata and /sdcard, so you can backup to or update from /sdcard. It includes a slightly modified binary of dmsetup, I put the source in a git repository.

I tested it on my Nexus S on Linux and on Windows, on both you need to have Python installed (Windows Installer) and have adb on your PATH environment variable (probably python, too, for convenience). It will probably fail with the current version of CWM recovery, as it seems only the CyanogenMod CWM-based recovery has all necessary flags set correctly.

If you have made backups and want to try it nevertheless, here's quick step-by-step guide.

Extract attached androidcrypt.py.zip file, open a terminal window and navigate into folder where you extracted it.
Make sure you can execute 'adb' and 'python' without error (you can exit python by entering 'exit()'). If it doesn't work, check you Android SDK install, especially the PATH, CyanogenMod Wiki has a detailed howto.
Run 'python androidcrypt.py', see what happens and report your output.
Do not try to restore backups or format partitions inside CWM, I have no idea what will happen.

ghost shell · May 7, 2012

Amazing work shaaXo!

But I have some questions since I'm with SG S2 with WanamLite:
- is dmcrypt available on all CWM versions or just the CM9 version?
- adb doesn't see any connected device when I'm in CWM-recovery mode. Any idea why? (maybe a usb or mtp issue?)

It would be great if your work could start to be integrated in the ROMs out there.

shaaXo · May 7, 2012

ghost shell said:
Amazing work shaaXo!

Thanks.

ghost shell said:
- is dmcrypt available on all CWM versions or just the CM9 version?

What I can tell is that the upstream version from koush does not support the required encryption modes, fixing this would be a very minor change, but I haven't contacted him for the moment.

There are some different, CWM-based recovery images floating around, at least the CyanogenMod version for the Nexus S does have the required kernel settings. (It should also be possible to replace the kernel of a recovery image with a different kernel, using the kernel from the system image would guarantee compatibility.)

ghost shell said:
- adb doesn't see any connected device when I'm in CWM-recovery mode. Any idea why? (maybe a usb or mtp issue?)

On Windows, it's probably a driver issue, took me almost an hour setting it up correctly on a VM. Once I had the driver correctly installed and no more question marks in the device manager, it worked fine.

ghost shell said:
It would be great if your work could start to be integrated in the ROMs out there.

I'm trying to get my hands on a few different phones to try it out. It may also be possible to get the dmcrypt/cryptsetup source into the CyanogenMod source tree, to have it preinstalled on every system or recovery image.

ghost shell · May 8, 2012

shaaXo said:
On Windows, it's probably a driver issue, took me almost an hour setting it up correctly on a VM. Once I had the driver correctly installed and no more question marks in the device manager, it worked fine.

I'm using linux.
It's strange because the adb interface works fine when the phone is in "normal user mode".
It's only when in CWM-recovery mode that it doesn't work. Makes me think that it could be something broken with my CWM version or kernel.

shaaXo · May 8, 2012

Are you using a udev rule to avoid restarting running the adb server as root? At least on my Nexus S, the USB product ID changes from 4e21 (normal boot) to d001 (recovery), which is problematic if the udev rule only allows access to the "normal" product ID.

bunklung · May 8, 2012

Do you have any insight into replacing the kernel in a recovery image?

shaaXo said:
Thanks.

What I can tell is that the upstream version from koush does not support the required encryption modes, fixing this would be a very minor change, but I haven't contacted him for the moment.

There are some different, CWM-based recovery images floating around, at least the CyanogenMod version for the Nexus S does have the required kernel settings. (It should also be possible to replace the kernel of a recovery image with a different kernel, using the kernel from the system image would guarantee compatibility.)

shaaXo · May 9, 2012

bunklung said:
Do you have any insight into replacing the kernel in a recovery image?

I successfully replaced the kernel of the current CWM-touch-recovery image by using split_bootimg.pl and put the image back together with mkbootimg. Unfortunately, the whole process is rather complex because it requires compiling the mkbootimg binary (which I already had from building CM from source) and the correct base kernel offset for the device.

Flashing ROM after turning on Encryption

Retired Senior Moderator & Developers Relations

Senior Member

Retired Senior Moderator & Developers Relations

Senior Member

Senior Member

Member

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Member

Senior Member

Senior Member

Attachments

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Similar threads

Top Liked Posts