• Introducing XDA Computing: Discussion zones for Hardware, Software, and more!    Check it out!
  • Fill out your device list and let everyone know which phones you have!    Edit Your Device Inventory

Best way to create and re-flash userdata image on latest N (ryu-nrd91n)

Search This thread

jason_hanley

New member
Nov 29, 2016
4
0
First of all, thanks so much to everyone for all the hard work creating and updating utilities for the Pixel C.

I have a rooted (but otherwise stock) Pixel C running the November update of Android 7/N (ryu-nrd91n) and am trying to:
1. Create an image of the userdata partition
2. Read/modify files
3. Re-flash the modified partition back to the Pixel C

So far I've been able to create a 24.8 GB image file using 'dd' but the file is unreadable, probably due to encryption.

I don't see any Developer Settings option to remove encryption or otherwise unencrypt the data partition.

What is the best approach?
 

followmsi

Senior Member
Oct 10, 2013
4,092
12,268
First of all, thanks so much to everyone for all the hard work creating and updating utilities for the Pixel C.

I have a rooted (but otherwise stock) Pixel C running the November update of Android 7/N (ryu-nrd91n) and am trying to:
1. Create an image of the userdata partition
2. Read/modify files
3. Re-flash the modified partition back to the Pixel C

So far I've been able to create a 24.8 GB image file using 'dd' but the file is unreadable, probably due to encryption.

I don't see any Developer Settings option to remove encryption or otherwise unencrypt the data partition.

What is the best approach?

Do you want to get rid of your encrypted data partition ?
- Format data (inside TWRP - and do not boot into system in between)
- Flash stock boot.img
- Install SuperSU.zip / or flash antother boot.img - with "encryptable" flag.

If you flash stock boot.img and boot into system your are encrypted again - automatically.

yes, "dd" will dump the complete partition including encryption ..
you may can "tar" the complete file system while decrypted .. similar to TWRP backup function.

Why do you want to "backup - modify - restore" at all ?
Why you don´t modify your files directly on data using root-explorer ?

Cheers
 
  • Like
Reactions: jason_hanley

jason_hanley

New member
Nov 29, 2016
4
0
Do you want to get rid of your encrypted data partition ?
- Format data (inside TWRP - and do not boot into system in between)
- Flash stock boot.img
- Install SuperSU.zip / or flash antother boot.img - with "encryptable" flag.

If you flash stock boot.img and boot into system your are encrypted again - automatically.
...
Why do you want to "backup - modify - restore" at all ?
Why you don´t modify your files directly on data using root-explorer ?

Thanks so much for the follow-up. The main goal is actually simply to be able to create images, and re-flash them reliably on a different Pixel C (replicating the exact system state). The modification step is simply to verify that it's working.

I was able to unencrypt /data using the method above -- Thanks, that's extremely helpful!

However, I'm still unable to mount/read the file produced by "dd if=/dev/block/mmcblk0p7"

Is there a better way you can recommend to make an image of the userdata partition?
 

followmsi

Senior Member
Oct 10, 2013
4,092
12,268
How do you mount the unencrypted dd dump of data ?
mount -o loop dumped.data.img /mnt/data

As well you can untar all twrp backup files into one new data.img file.
Or tar the complete /data partition ... etc.
It's an ext4 partition ..

Of course, you can use the twrp backup files directly to restore on another pixel c device.
There is no modification needed at all.

Good luck!



Sent from my Nexus 10 using Tapatalk
 
  • Like
Reactions: jason_hanley

followmsi

Senior Member
Oct 10, 2013
4,092
12,268
That whole boot.img step.. is that why I always get stuck in a weird misplaced bootloop whenever I modify anything in TWRP? I just flash original boot.img in fastboot?
The boot.img step to decrypt data ?
It's a one time task.

Once decrypted you always need to root after new boot.img flash.
If you don't root or don't use encryptable flag boot.img you will get encrypted.

Not really related to bootloops.

What are you changing when you are using TWRP ?
 

gnome9er

Member
Oct 16, 2010
37
1
40
langley
twitter.com
If I use backup then restore it loops. If I flash twrp it just boots to recovery. Now that i think about it i never did root this thing properly. Well i did with a toolkit but that doesnt count. I went beta then hated it lost root on restore then when i tried to do it again i kept running into bootloops. It says in the forums here that its common ans to just fastboot reboot but that dont work. I cant wait for the new nexus 7
 

followmsi

Senior Member
Oct 10, 2013
4,092
12,268
If I use backup then restore it loops. If I flash twrp it just boots to recovery. Now that i think about it i never did root this thing properly. Well i did with a toolkit but that doesnt count. I went beta then hated it lost root on restore then when i tried to do it again i kept running into bootloops. It says in the forums here that its common ans to just fastboot reboot but that dont work. I cant wait for the new nexus 7
Depends what are you restoring ;)
Do you backup / restore other partitions except the data partition ?
Pls don't :)

In general .. pls backup data partition only !

All other .img files should be taken freshly from image.

In systemless rooting world the su.img is placed on data.
You may have an older version inside your backup.
If you restore boot.img from backup, supersu ramdisk modifications will fail.

Flash what ever system.img, vendor.img and install twrp as recovery.
If you flash a boot.img, it should have stock ramdisk, to be able to let supersu.zip do the work for proper rooting.
At the same time the su.img on data will be refreshed too.

To summarize .. if you change something always install a fresh copy of your boot.img and run SuperSu.zip inside TWRP afterwards!

Hope this helps ..
 
  • Like
Reactions: jason_hanley

jason_hanley

New member
Nov 29, 2016
4
0
Depends what are you restoring ;)
Do you backup / restore other partitions except the data partition ?
Pls don't :)

In general .. pls backup data partition only !

In this case, we're only looking to backup and restore the data partition.

Once we have the raw image created by dd, what would you say is the most reliable way to convert it to something that can be flashed by fastboot?

I've tried several different utilities to convert it to a "sparse image" but flashing it just erases data and doesn't copy anything from the image.
 

followmsi

Senior Member
Oct 10, 2013
4,092
12,268
In this case, we're only looking to backup and restore the data partition.

Once we have the raw image created by dd, what would you say is the most reliable way to convert it to something that can be flashed by fastboot?

I've tried several different utilities to convert it to a "sparse image" but flashing it just erases data and doesn't copy anything from the image.
Did you test img2simg as well ?

I use simg2img to convert the sparse to raw image, the other way around.

Or something like this below .. "s" defines sparse image.

make_ext4fs -s -l xxxxxxxx -a data data.img

Sorry, can't help you better now ..
 
  • Like
Reactions: jason_hanley

jason_hanley

New member
Nov 29, 2016
4
0
Did you test img2simg as well ?

Yes, unfortunately we've tried everything.

First we created the file data.dd using
Code:
dd if=/dev/block/mmcblk0p7
, which can be read as an ext4 image.

Then we tried using SparseConverter, ext2simg and img2simg to convert the raw image to a sparse image.

They all produce files of different lengths, and none of those files seem to flash successfully :(

They go through the flash process and report being successful, but the data isn't there.

Is there any way to debug this and figure out what's going wrong?
 

followmsi

Senior Member
Oct 10, 2013
4,092
12,268
Yes, unfortunately we've tried everything.

First we created the file data.dd using
Code:
dd if=/dev/block/mmcblk0p7
, which can be read as an ext4 image.

Then we tried using SparseConverter, ext2simg and img2simg to convert the raw image to a sparse image.

They all produce files of different lengths, and none of those files seem to flash successfully :(

They go through the flash process and report being successful, but the data isn't there.

Is there any way to debug this and figure out what's going wrong?

If you don´t get an error message from fastboot the sparse image seems to be valid.

How do you verify data partition after flashing ?
Are you able to mount and check ?
Also from recovery .. e.g. before you boot into system ?

For example on Nexus 7 flo there is still a userdata.img inside factory rom (6.0.1), but not for Pixel C anymore.
Maybe some permissions are required (file_contexts) - like on Nexus 7 flo too.

Sorry, no idea whats going wrong for you ..

---------- Post added at 12:49 PM ---------- Previous post was at 12:35 PM ----------

Why should other partitions not be backed up? Is it not possible to flash the /data partition if you flash other partitions at the same time?

You can backup all partitions you want .. but you should know the possible impact if you restore them. :)

And Yes - you can flash all partitions at the same time .. but normally you don´t flash data partition via fastboot - not on Pixel C.

In times of systemless rooting and monthly security releases there is no need for system.img backup anymore.
Unless you have special modifications inside your system.img.

If you were rooted before I recommend to use fresh boot.img and reinstall supersu.zip after change.
Just to prevent issues ...
 

Top Liked Posts

  • There are no posts matching your filters.
  • 2
    Yes, unfortunately we've tried everything.

    First we created the file data.dd using
    Code:
    dd if=/dev/block/mmcblk0p7
    , which can be read as an ext4 image.

    Then we tried using SparseConverter, ext2simg and img2simg to convert the raw image to a sparse image.

    They all produce files of different lengths, and none of those files seem to flash successfully :(

    They go through the flash process and report being successful, but the data isn't there.

    Is there any way to debug this and figure out what's going wrong?

    If you don´t get an error message from fastboot the sparse image seems to be valid.

    How do you verify data partition after flashing ?
    Are you able to mount and check ?
    Also from recovery .. e.g. before you boot into system ?

    For example on Nexus 7 flo there is still a userdata.img inside factory rom (6.0.1), but not for Pixel C anymore.
    Maybe some permissions are required (file_contexts) - like on Nexus 7 flo too.

    Sorry, no idea whats going wrong for you ..

    ---------- Post added at 12:49 PM ---------- Previous post was at 12:35 PM ----------

    Why should other partitions not be backed up? Is it not possible to flash the /data partition if you flash other partitions at the same time?

    You can backup all partitions you want .. but you should know the possible impact if you restore them. :)

    And Yes - you can flash all partitions at the same time .. but normally you don´t flash data partition via fastboot - not on Pixel C.

    In times of systemless rooting and monthly security releases there is no need for system.img backup anymore.
    Unless you have special modifications inside your system.img.

    If you were rooted before I recommend to use fresh boot.img and reinstall supersu.zip after change.
    Just to prevent issues ...
    1
    First of all, thanks so much to everyone for all the hard work creating and updating utilities for the Pixel C.

    I have a rooted (but otherwise stock) Pixel C running the November update of Android 7/N (ryu-nrd91n) and am trying to:
    1. Create an image of the userdata partition
    2. Read/modify files
    3. Re-flash the modified partition back to the Pixel C

    So far I've been able to create a 24.8 GB image file using 'dd' but the file is unreadable, probably due to encryption.

    I don't see any Developer Settings option to remove encryption or otherwise unencrypt the data partition.

    What is the best approach?

    Do you want to get rid of your encrypted data partition ?
    - Format data (inside TWRP - and do not boot into system in between)
    - Flash stock boot.img
    - Install SuperSU.zip / or flash antother boot.img - with "encryptable" flag.

    If you flash stock boot.img and boot into system your are encrypted again - automatically.

    yes, "dd" will dump the complete partition including encryption ..
    you may can "tar" the complete file system while decrypted .. similar to TWRP backup function.

    Why do you want to "backup - modify - restore" at all ?
    Why you don´t modify your files directly on data using root-explorer ?

    Cheers
    1
    How do you mount the unencrypted dd dump of data ?
    mount -o loop dumped.data.img /mnt/data

    As well you can untar all twrp backup files into one new data.img file.
    Or tar the complete /data partition ... etc.
    It's an ext4 partition ..

    Of course, you can use the twrp backup files directly to restore on another pixel c device.
    There is no modification needed at all.

    Good luck!



    Sent from my Nexus 10 using Tapatalk
    1
    If I use backup then restore it loops. If I flash twrp it just boots to recovery. Now that i think about it i never did root this thing properly. Well i did with a toolkit but that doesnt count. I went beta then hated it lost root on restore then when i tried to do it again i kept running into bootloops. It says in the forums here that its common ans to just fastboot reboot but that dont work. I cant wait for the new nexus 7
    Depends what are you restoring ;)
    Do you backup / restore other partitions except the data partition ?
    Pls don't :)

    In general .. pls backup data partition only !

    All other .img files should be taken freshly from image.

    In systemless rooting world the su.img is placed on data.
    You may have an older version inside your backup.
    If you restore boot.img from backup, supersu ramdisk modifications will fail.

    Flash what ever system.img, vendor.img and install twrp as recovery.
    If you flash a boot.img, it should have stock ramdisk, to be able to let supersu.zip do the work for proper rooting.
    At the same time the su.img on data will be refreshed too.

    To summarize .. if you change something always install a fresh copy of your boot.img and run SuperSu.zip inside TWRP afterwards!

    Hope this helps ..
    1
    In this case, we're only looking to backup and restore the data partition.

    Once we have the raw image created by dd, what would you say is the most reliable way to convert it to something that can be flashed by fastboot?

    I've tried several different utilities to convert it to a "sparse image" but flashing it just erases data and doesn't copy anything from the image.
    Did you test img2simg as well ?

    I use simg2img to convert the sparse to raw image, the other way around.

    Or something like this below .. "s" defines sparse image.

    make_ext4fs -s -l xxxxxxxx -a data data.img

    Sorry, can't help you better now ..