[DEV][COMPLETE] Project Treble conversion

Status
Not open for further replies.
Search This thread

CosmicDan

Senior Member
Jun 19, 2009
5,862
7,697
34
Sydney
Xiaomi Poco F1
UPDATE4: New Treble guide is posted here. Ask all your questions here that aren't specific to other parts of Treble (e.g. the ROM or the TWRP). Thread lock requested. Thanks all!

UPDATE3: The new TWRP Installer with Treble Manager (for guided repartitioning) is now released. See this thread.

UDPATE2: Experimental RR builds for Treble is now released. This contains the required Vendor Pack for GSI use. See this thread.


UPDATE: We have succeed in booting a Phh-Treble GSI! See screenshot at Post #57. Stay tuned for user releases!


This thread will be kept for posterity, but I will request locking once we release stuff. Thanks you all for the support, and thanks to @ghpranav for lots of help and @phhusson for pointing me in the right direction on the last stretch.

Original post below

----------------------------------------------

Alright, here is my WIP guide for converting the Mi A1 to a Treble device and ROM.

NOT FOR NEWBIES. This is all manual stuff right now and if you do something wrong you CAN firm-brick your device! The project is incomplete and for developers/researchers only.

Please refrain from "good luck" and "can't wait for this" and "OMG you are the King of Kings ComsicDan, plz have my babies" and such posts.

Also note that you will NEED to get into EDL mode to restore stock partition table, if you need to, by flashing stock factory image with MiFlash or my Low-Level Backup Tool. MiFlash will WIPE your Userdata completely and your persist so you MUST make a low-level backup to restore original persist.img, and manually backup internal storage. But really it's perfectly fine to keep using a non-treble ROM even with partition table changes (there's no need to revert for any functional reason that I can think of, unless you somehow get a ROM that's bigger than 2.4GB). Note that the final update ZIP that will automatically repartition can also restore stock partitions, without the need for EDL stuff.


Prerequisite - Low-level backup, TWRP backup and Internal Storage backup


  1. First make a full backup of all partitions, via EDL (low-level which gets all partitions). You can use my new Low-Level tool here. I recommend using the skip-systems-and-userdata list since it takes like an hour, and you can backup system/userdata faster in TWRP and PC manually.
  2. Make a full TWRP backup of our current ROM. System needs to be erased during this procedure. After re-partitioning however, you can simply restore this backup (unless your system is > 2.4GB for some reason, which is very unlikely).
  3. Make a full manual backup of your internal storage files to PC. Userdata partition needs to be completely erased if/when you restore the original stock partitions and firmware. You can postpone this process to later however, for if/when you want to actually restore to stock partition layour.


Phase 1 - Repartition device for vendor_a and vendor_b


This is relatively simple. Our device has a standard GPT table, so we can use sgdisk which is already included in our TWRP build.

  1. Boot into recovery and fire up an adb shell
  2. View current partition table...
    Code:
    sgdisk /dev/block/mmcblk0 --print
    ...and confirm these values...
    Code:
    ...
      25         1185792         7477247   3.0 GiB     FFFF  system_a
      26         7477248        13768703   3.0 GiB     FFFF  system_b
    ...
      49        14550032        61071326   22.2 GiB    FFFF  userdata
    The important values are the partition numbers (25, 26 and 49), and the next two numbers for both system partitions (start/end sectors). The start/end sector of userdata isn't important, only the partition number of 49 is. If your sector start/ends are different, you will need to adjust some of the values in these steps - that's up to you to figure out. But do please let me know what your differences are, for science.
  3. Remove system_a, and create a new system_a partition of size ~2.4GB. Don't worry, this is plenty - my current RR build with a heavy AROMA Gapps is only ~1.9GB. We're taking off ~600MB, which is 1228800 sectors (600*1024*1024/512) so the new end sector will be 6248447 (7477247 - 1228800):
    Code:
    sgdisk /dev/block/mmcblk0 --delete 25
    sgdisk /dev/block/mmcblk0 --new=25:1185792:6248447
    sgdisk /dev/block/mmcblk0 --change-name=25:system_a
  4. Create vendor_a with the reclaimed ~600MB:
    Code:
    sgdisk /dev/block/mmcblk0 --new=50:6248448:7477247
    sgdisk /dev/block/mmcblk0 --change-name=50:vendor_a
    ...notice the partition number of 50 - one after the userdata (49).
  5. Do similarly to above, only for system_b and vendor_b:
    Code:
    sgdisk /dev/block/mmcblk0 --delete 26
    sgdisk /dev/block/mmcblk0 --new=26:7477248:12539903
    sgdisk /dev/block/mmcblk0 --change-name=26:system_b
    sgdisk /dev/block/mmcblk0 --new=51:12539904:13768703
    sgdisk /dev/block/mmcblk0 --change-name=51:vendor_b
  6. Quit shell and reboot recovery...
    Code:
    adb reboot recovery
    ...and verify the new partitions exist...
    Code:
    adb shell ls -la /dev/block/bootdevice/by-name/
    ...should show this at the bottom...
    Code:
    ...
    lrwxrwxrwx    1 root     root            21 Jan  5  1970 userdata -> /dev/block/mmcblk0p49
    lrwxrwxrwx    1 root     root            21 Jan  5  1970 vendor_a -> /dev/block/mmcblk0p50
    lrwxrwxrwx    1 root     root            21 Jan  5  1970 vendor_b -> /dev/block/mmcblk0p51


Repartition is done! You can now restore your TWRP backup (only system) and system works perfectly as before.


Roadblocks of Phase 1

  • How best to deliver this partition table change to users? The obvious answer is an update ZIP that just scripts it all (that's why I used sgdisk in this write-up). But then this requires the user to manually backup, reboot recovery, then restore backup. Is that ideal?
  • Another solution would be to export the new GPT, and make a MiFlash-flashable package. This could be another option for future, but would require new system and vendor images too - may as well be a whole flash package. That's big. We'll look into that once we actually get a Treble ROM working (e.g. stock).


Phase 1b revert - Restore stock partition map

If you want to go back to original partition table for some reason, you will need to flash an official fastboot ROM via EDL mode in MiFlash. After researching, I have discovered that there is no way around this. The way MiFlash works is it actually adjusts the GPT info dynamically based on *your* partition map (64GB vs 32GB have different userdata sizes) and for this to be done it needs to flash a full package. So, to restore stock partition map, you must flash a stock ROM in MiFlash via EDL mode which wipes everything. I hope you made a low-level and internal-storage backup like I warned - because after that you'll want to restore your backup to get persist and stuff back. Flashing in fastboot mode is not enough because it does not restore stock partition table.

Full steps for restoring to stock:

  1. Boot into EDL mode
  2. Load MiFlash, select any Oreo fastboot image (I use 8.1.10 since it's what I have) > Refresh. Should show COM30 (or whatever your QDLoader COM port is).
  3. Reboot EDL by holding Power + VolDown and then 'fastboot oem edl' again
  4. Run the restore in my Low-level flash tool of your backup
  5. Reboot back into recovery by holding power + volume-up for ~10 seconds.
  6. "Skip" on password screen
  7. Wipe > Format Data > type "yes" to remove stock ROM encryption
  8. Restore your TWRP backup
  9. Done!


Phase 2 - Patch kernel DTB and TWRP

This requires Linux. I use ChaletOS in VirtualBox and have my home folder Samba shared, mapped to a network drive on my Windows host.

Probably when all this is finalized and working, we will use a custom kernel built from source. But for now lets just patch our existing kernel. I assume you already have TWRP installed.

Part 1 - Kernel DTB

  1. Download the DTB split tool from:
    https://github.com/dianlujitao/split-appended-dtb
    ...to your Linux machine. Just get the binary, not the .c source file.
  2. Dump existing boot.img while in TWRP and pull from device:
    Code:
    adb shell dd if=/dev/block/bootdevice/by-name/boot of=/tmp/boot.img
    adb pull /tmp/boot.img
  3. Unpack this boot.img. I use AIK by @osm0sis from:
    https://forum.xda-developers.com/showthread.php?t=2073775
    ... In this case, we need boot.img-zImage. Copy it to the Linux machine folder with DTB split.
  4. Split DTB from zImage on Linux machine:
    Code:
    ./split-appended-dtb boot.img-zImage
    ...should produce 1 DTB. Total output will be two files - "kernel" and "dtbdump_1.dtb".
  5. Decompile DTB on Linux machine:
    Code:
    # if you need to install it
    sudo apt install device-tree-compiler
    dtc -O dts -I dtb -o dtbdump_1.dts dtbdump_1.dtb
  6. Edit dtbdump_1.dts in whatever text editor. Search for fstab and edit the vendor entry, adding "slotselect" to fsmgr_flags (so it's the same as system):
    Code:
    fsmgr_flags = "wait,slotselect";
    NOTE: This is on my RR kernel. I need to check any difference with stock or other kernels.
  7. Recompile the dts to dtb and build a new zImage:
    Code:
    dtc -O dtb -I dts -o dtbdump_1_new.dtb dtbdump_1.dts
    cat kernel dtbdump_1_new.dtb > boot.img-zImage.new
  8. Copy the boot.img-zImage.new back to your kernel kitchen tool, replacing the old one.
That's the device tree patched to mount seamless/slot vendor. But don't repack the boot.img yet, we need to edit RAMDisk.



Part 2 - TWRP patch

This is just so we can see Vendor (and Persist, because why not) in TWRP. This is so:

  • Vendor can be formatted (required before use), backed up/restored and flashed to
  • [Bonus] Persist can be backed up/restored and flashed to (for e.g. Persist repair ZIP's that should come soon).

Only one step is needed - edit /etc/recovery.fstab to add vendor mount (and also persist because why not) with flags:
Code:
/vendor          ext4             /dev/block/bootdevice/by-name/vendor     flags=slotselect;backup=1;wipeingui;display="Vendor"
/persist         ext4             /dev/block/bootdevice/by-name/persist    flags=backup=1;display="Persist"
That's it.

For your convenience, I have attached a Treble-modded TWRP installer to this thread with these changes. Don't forget to flash Magisk after re-installing TWRP. Big thanks to @ghpranav


Phase 3 - Compile a Treble ROM (for Vendor files)

Already succeeded. We just need to cleanup and push.

Phase 4 - Treble-ize an installed ROM

  • Is this possible? Not sure... I'll check it out later sometime


Did I miss anything? Probably. Let me know!
 

Attachments

  • Twrp-recoveryInstaller-3.2.1-2-oreo-TrebleMod.zip
    8.5 MB · Views: 590
Last edited:

joeyhuab

Recognized Developer
  • Nov 5, 2012
    3,118
    12,954
    26
    Antipolo
    evolution-x.org
    Xiaomi Mi A1
    Redmi K20 Pro
    Just finished reading everything. You did a wonderful job explaining this and I'm eager to test this out even if my device is my daily driver. With that being said, I really hope inexperienced users would stay away from this thread and refrain asking silly questions. I especially want them to stay away from complaining that they bricked their device, etc etc. By the looks of this, it should be easy considering you got the partition sectors noted down and most if not all users have the 64GB variant. It should be fairly easy making the ROM boot as long as it caters to A/B partitioning (already available in XDA) and that the Vendor HALs wouldn't cause further trouble.

    With all of that, I'm looking forward to testing Android P as soon as we got treble tested up and running.

    Thanks for taking your time to explain this thoroughly and I appreciate all the efforts you've put into this, especially automating the process by making a backup tool and hopefully, a working TWRP for Treble.

    Looking at the Treble threads, I see some devs have started working on Android P. Link here.

    Sent from my Xiaomi Mi A1 using XDA Labs
     
    Last edited:

    CosmicDan

    Senior Member
    Jun 19, 2009
    5,862
    7,697
    34
    Sydney
    Xiaomi Poco F1
    Just finished reading everything. You did a wonderful job explaining this and I'm eager to test this out even if my device is my daily driver. With that being said, I really hope inexperienced users would stay away from this thread and refrain asking silly questions. I especially want them to stay away from complaining that they bricked their device, etc etc. By the looks of this, it should be easy considering you got the partition sectors noted down and most if not all users have the 64GB variant. It should be fairly easy making the ROM boot as long as it caters to A/B partitioning (already available in XDA) and that the Vendor HALs wouldn't cause further trouble.

    With all of that, I'm looking forward to testing Android P as soon as we got treble tested up and running.

    Thanks for taking your time to explain this thoroughly and I appreciate all the efforts you've put into this, especially automating the process by making a backup tool and hopefully, a working TWRP for Treble.

    Looking at the Treble threads, I see some devs have started working on Android P. Link here.

    Sent from my Xiaomi Mi A1 using XDA Labs

    Thanks mate! Yep I've seen them :) but based on Phh's work, at least Oreo Treble GSI depends on VTS which isn't released yet. So I'm not sure how long until we get it.

    I just learned that in order to get stock GPT back you basically have to fully erase the device via flash_factory in MiFlash. Technical details added to the OP.

    So, apart from low-level flash, you will also want to backup internal storage before reverting - since factory flash wipes absolutely everything and TWRP won't backup storage. Then of course you can use my low-level tool to restore partitions via EDL to get your (e.g.) persist and stuff back, then restore the associated TWRP backup.
     

    JoKeRLeE

    Senior Member
  • Aug 15, 2016
    282
    171
    Behind You
    Xiaomi Mi 9
    Just a reminder ,

    EDL mode uses rawprogram0 , prog_emmc_firehose_8953_ddr.mbn , patch0 or partition isn't it ? Flash_factory will erase all the partition and causes device lost of IMEI , ESN, PSN and others sensors settings . and only flash-able during Fastboot mode .

    fastboot %* getvar product 2>&1 | findstr /r /c:"^product: *tissot" || @ECHO "error : Missmatching image and device" & exit /B 1
    fastboot %* oem unlock || @ECHO "unlock error" & exit /B 1
    fastboot %* flash partition %~dp0images\gpt_both0.bin || @ECHO "Flash gpt_both0 error" & exit /B 1
    fastboot %* flash modem_a %~dp0images\NON-HLOS.bin || @ECHO "Flash modem_a error" & exit /B 1
    fastboot %* flash modem_b %~dp0images\NON-HLOS.bin || @ECHO "Flash modem_b error" & exit /B 1
    fastboot %* flash sbl1 %~dp0images\sbl1.mbn || @ECHO "Flash sbl1 error" & exit /B 1
    fastboot %* flash sbl1bak %~dp0images\sbl1.mbn || @ECHO "Flash sbl1bak error" & exit /B 1
    fastboot %* flash rpm %~dp0images\rpm.mbn || @ECHO "Flash rpm error" & exit /B 1
    fastboot %* flash rpmbak %~dp0images\rpm.mbn || @ECHO "Flash rpmbak error" & exit /B 1
    fastboot %* flash tz %~dp0images\tz.mbn || @ECHO "Flash tz error" & exit /B 1
    fastboot %* flash tzbak %~dp0images\tz.mbn || @ECHO "Flash tzbak error" & exit /B 1
    fastboot %* flash devcfg %~dp0images\devcfg.mbn || @ECHO "Flash devcfg error" & exit /B 1
    fastboot %* flash devcfgbak %~dp0images\devcfg.mbn || @ECHO "Flash devcfgbak error" & exit /B 1
    fastboot %* flash dsp %~dp0images\adspso.bin || @ECHO "Flash dsp error" & exit /B 1
    fastboot %* flash fsg %~dp0images\fs_image.tar.gz.mbn.img || @ECHO "Flash fs_image error" & exit /B 1
    fastboot %* flash sec %~dp0images\sec.dat || @ECHO "Flash sec error" & exit /B 1
    fastboot %* flash splash %~dp0images\splash.img || @ECHO "Flash splash error" & exit /B 1
    fastboot %* flash aboot %~dp0images\emmc_appsboot.mbn || @ECHO "Flash aboot error" & exit /B 1
    fastboot %* flash abootbak %~dp0images\emmc_appsboot.mbn || @ECHO "Flash abootbak error" & exit /B 1
    fastboot %* flash boot_a %~dp0images\boot.img || @ECHO "Flash boot_a error" & exit /B 1
    fastboot %* flash boot_b %~dp0images\boot.img || @ECHO "Flash boot_b error" & exit /B 1
    fastboot %* flash system_a %~dp0images\system.img || @ECHO "Flash system_a error" & exit /B 1
    fastboot %* flash system_b %~dp0images\system.img || @ECHO "Flash system_a error" & exit /B 1
    fastboot %* flash lksecapp %~dp0images\lksecapp.mbn || @ECHO "Flash lksecapp error" & exit /B 1
    fastboot %* flash lksecappbak %~dp0images\lksecapp.mbn || @ECHO "Flash lksecappbak error" & exit /B 1
    fastboot %* flash cmnlib %~dp0images\cmnlib.mbn || @ECHO "Flash cmnlib error" & exit /B 1
    fastboot %* flash cmnlibbak %~dp0images\cmnlib.mbn || @ECHO "Flash cmnlibbak error" & exit /B 1
    fastboot %* flash cmnlib64 %~dp0images\cmnlib64.mbn || @ECHO "Flash cmnlib64 error" & exit /B 1
    fastboot %* flash cmnlib64bak %~dp0images\cmnlib64.mbn || @ECHO "Flash cmnlib64bak error" & exit /B 1
    fastboot %* flash keymaster %~dp0images\keymaster.mbn || @ECHO "Flash keymaster error" & exit /B 1
    fastboot %* flash keymasterbak %~dp0images\keymaster.mbn || @ECHO "Flash keymasterbak error" & exit /B 1
    fastboot %* flash userdata %~dp0images\userdata.img || @ECHO "Flash userdata error" & exit /B 1
    fastboot %* flash persist %~dp0images\persist.img || @ECHO "Flash persist error" & exit /B 1
    fastboot %* erase modemst1
    fastboot %* erase modemst2
    fastboot %* erase config || @ECHO "Erase config error" & exit /B 1
    fastboot %* reboot bootloader
    ping 127.0.0.1 -n 5 -w 1000 > nul
    fastboot %* set_active a
    fastboot %* reboot

    also , those code are NOT compatible for Mi A1 as modem uses modem.img not NON-HLOS.bin, sbl.img not sbl.mbn etc . for more information you may compare with flash_all with flash_factory .
     

    murtaza1326

    Senior Member
    Jun 21, 2014
    1,094
    1,227
    24
    godhra
    murtazapiplod.ml
    Redmi Note 10 Pro
    well just curious i read whole thread but i didn't get that is
    repartitioning affect A/B partition system or
    we are lucky and have A/B and Treble both at same time?

    also except developer if someone tries and flash this successfully then do post SS ???

    p.s don't tell me to try it neither i am developer or researcher ?
    Regards
     

    CosmicDan

    Senior Member
    Jun 19, 2009
    5,862
    7,697
    34
    Sydney
    Xiaomi Poco F1

    Yes you are right, flash_factory is only for fastboot mode. It doesn't matter what is chose when in EDL mode as it will flash everything.

    But I think EDL mode is still needed, since MiFlash recalculates the partition table. Maybe it does in fastboot mode too but I don't know, it just flashes to "partition" via fastboot.

    YMMV. I do it in EDL and it works fine.

    Don't know or care if any of the firmware files are incompatible (are you sure? That seems strange if so, why would they be in the A1 package if they're not) since we restore a backup from my tool anyway which does all partitions exactly how they were on your device. The only purpose of MiFlash in this case is to rewrite partition table.


    well just curious i read whole thread but i didn't get that is
    repartitioning affect A/B partition system or
    we are lucky and have A/B and Treble both at same time?

    also except developer if someone tries and flash this successfully then do post SS ??? regards

    p.s don't tell me to try it neither i am developer or researcher ?
    Regards

    Yes, as you can see it's "vendor_a" and "vendor_b", one for each slot.

    There are no screenshots lol, unless you want to see a screenshot of TWRP with a Vendor mount listing ? ?
     

    murtaza1326

    Senior Member
    Jun 21, 2014
    1,094
    1,227
    24
    godhra
    murtazapiplod.ml
    Redmi Note 10 Pro
    Yes, as you can see it's "vendor_a" and "vendor_b", one for each slot.

    There are no screenshots lol, unless you want to see a screenshot of TWRP with a Vendor mount listing ? ?

    well yeah i missed that

    about screen shot that treble app on playstore or any other thing which state both
    also i wonder if GSI images are flashable or moded treble roms should be there ?
     

    CosmicDan

    Senior Member
    Jun 19, 2009
    5,862
    7,697
    34
    Sydney
    Xiaomi Poco F1
    well yeah i missed that

    about screen shot that treble app on playstore or any other thing which state both
    also i wonder if GSI images are flashable or moded treble roms should be there ?

    Not there yet mate, patience ?

    Interesting.

    As i know, we also need to work with .dts to make it possible with treble inside kernel source with A/B partition layout.

    Really? This might be what I'm stuck on. No idea what it is though.

    I just get recovery loop if I try to Treble-ize a ROM, and GSI's kick me to fastboot. I was suspecting that it could be kernel, but I am seeking help from Phh about it.

    Do you have any links that might point me in the right direction? Meanwhile I'll see what I find in Google.
     
    • Like
    Reactions: VanoSM

    khusika

    Recognized Contributor
  • Feb 14, 2013
    1,591
    5,245
    Malang
    khusika.com
    Xiaomi Mi A1
    Xiaomi Mi 8
    Really? This might be what I'm stuck on. No idea what it is though.

    I just get recovery loop if I try to Treble-ize a ROM, and GSI's kick me to fastboot. I was suspecting that it could be kernel, but I am seeking help from Phh about it.

    Do you have any links that might point me in the right direction? Meanwhile I'll see what I find in Google.

    I'll try to provide treble for A/B with the kernel, in the next day
     

    CosmicDan

    Senior Member
    Jun 19, 2009
    5,862
    7,697
    34
    Sydney
    Xiaomi Poco F1
    Great job. Just a question, if you restore by doing flash_factory.bat - it deletes your IMEI, or not?

    Maybe. I don't know. It's kind of a moot point because if it does, my low-level backup tool will backup and restore it anyway.

    I'll try to provide treble for A/B with the kernel, in the next day

    Cool beans!

    I was reading on Android source docs that it might be related to early mount (vboot or whatever our device uses) missing vendor. So there's still leads to work on. But no rush, whenever you can :)

    Oh BTW, if it's compatible with stock blobs would be good (i.e. stock kernel/ROM). Using stock vendor seems like the ideal way to go.
     
    Last edited:

    xemisxu

    Senior Member
    Nov 14, 2015
    217
    49
    We can use /b partition as vendor making phone /a system only without use of edl (this makes more easy recover this partition). Also if we have /cust we can use it as vendor without repartition only with recovery/kernel support.
     

    SomratMJX

    Senior Member
    Sep 12, 2014
    94
    44
    Dinajpur
    www.prohelika.org
    I think it's not ideal

    We can use /b partition as vendor making phone /a system only without use of edl (this makes more easy recover this partition). Also if we have /cust we can use it as vendor without repartition only with recovery/kernel support.

    As no one wants to get rid of A/B in exchange of Treble! Both are important! And Google is pushing hard OEMs to implement A/B in near future! So, both are actually important! Moreover getting rid of it seems loosing integrity to me!:p

    As far I know, A1 doesn't have a /cust partition!

    BTW, as @CosmicDan has found a way to re-partiton, then there's no point of getting rid of anything!
    And to me using /b as vendor is awful waste of space! 3 GB for vendor... Seriously?!?:cyclops:
    He is using (or we need) 600*2=1200MB though!:(

    I think he could shrink some spaces from /data instead of system! But it seems he got the 32 GB variant! :p
    Just kidding LoL!:laugh::laugh::laugh:
     

    JoKeRLeE

    Senior Member
  • Aug 15, 2016
    282
    171
    Behind You
    Xiaomi Mi 9
    As no one wants to get rid of A/B in exchange of Treble! Both are important! And Google is pushing hard OEMs to implement A/B in near future! So, both are actually important! Moreover getting rid of it seems loosing integrity to me!:p

    As far I know, A1 doesn't have a /cust partition!

    BTW, as @CosmicDan has found a way to re-partiton, then there's no point of getting rid of anything!
    And to me using /b as vendor is awful waste of space! 3 GB for vendor... Seriously?!?:cyclops:
    He is using (or we need) 600*2=1200MB though!:(

    I think he could shrink some spaces from /data instead of system! But it seems he got the 32 GB variant! :p
    Just kidding LoL!:laugh::laugh::laugh:


    LoL :) , Repartiton system is the best choice . i don't think user want to waste another 1.2GB for Vendor partition ... just my thought ;)
     
    • Like
    Reactions: CosmicDan

    xemisxu

    Senior Member
    Nov 14, 2015
    217
    49
    As no one wants to get rid of A/B in exchange of Treble! Both are important! And Google is pushing hard OEMs to implement A/B in near future! So, both are actually important! Moreover getting rid of it seems loosing integrity to me!:p

    As far I know, A1 doesn't have a /cust partition!

    BTW, as @CosmicDan has found a way to re-partiton, then there's no point of getting rid of anything!
    And to me using /b as vendor is awful waste of space! 3 GB for vendor... Seriously?!?:cyclops:
    He is using (or we need) 600*2=1200MB though!:(

    I think he could shrink some spaces from /data instead of system! But it seems he got the 32 GB variant! :p
    Just kidding LoL!:laugh::laugh::laugh:

    Sure,re-partition is the best option cause we dont have /cust partition.

    I know that 5x have it but i didn know if A1 have it.

    Ty for answer!
     

    CosmicDan

    Senior Member
    Jun 19, 2009
    5,862
    7,697
    34
    Sydney
    Xiaomi Poco F1
    Yeah guys, ROM's are not big anymore since Google themselves don't bother doing odex'ing (or ART'ing, whatever you want to call it now; you know like odex/deodex) since they know that most system apps just get updated from Play Store anyway and their odex goes on userdata. Also in modern Android apps can run fast before full odex/ART optimisation (that's why we have such fast booting) :)

    Yeah I'm 32GB, it's fine I have plenty of space (we have microSD remember lol). But vendor_a and vendor_b is the way it should be for compatibility. It also keeps the doors open for dual boot in future ;)

    I don't want to mess around with resizing data. 2.4GB for System and 600MB for vendor is a comfortable figure.
     
    Last edited:
    Status
    Not open for further replies.

    Top Liked Posts

    • There are no posts matching your filters.
    • 105
      UPDATE4: New Treble guide is posted here. Ask all your questions here that aren't specific to other parts of Treble (e.g. the ROM or the TWRP). Thread lock requested. Thanks all!

      UPDATE3: The new TWRP Installer with Treble Manager (for guided repartitioning) is now released. See this thread.

      UDPATE2: Experimental RR builds for Treble is now released. This contains the required Vendor Pack for GSI use. See this thread.


      UPDATE: We have succeed in booting a Phh-Treble GSI! See screenshot at Post #57. Stay tuned for user releases!


      This thread will be kept for posterity, but I will request locking once we release stuff. Thanks you all for the support, and thanks to @ghpranav for lots of help and @phhusson for pointing me in the right direction on the last stretch.

      Original post below

      ----------------------------------------------

      Alright, here is my WIP guide for converting the Mi A1 to a Treble device and ROM.

      NOT FOR NEWBIES. This is all manual stuff right now and if you do something wrong you CAN firm-brick your device! The project is incomplete and for developers/researchers only.

      Please refrain from "good luck" and "can't wait for this" and "OMG you are the King of Kings ComsicDan, plz have my babies" and such posts.

      Also note that you will NEED to get into EDL mode to restore stock partition table, if you need to, by flashing stock factory image with MiFlash or my Low-Level Backup Tool. MiFlash will WIPE your Userdata completely and your persist so you MUST make a low-level backup to restore original persist.img, and manually backup internal storage. But really it's perfectly fine to keep using a non-treble ROM even with partition table changes (there's no need to revert for any functional reason that I can think of, unless you somehow get a ROM that's bigger than 2.4GB). Note that the final update ZIP that will automatically repartition can also restore stock partitions, without the need for EDL stuff.


      Prerequisite - Low-level backup, TWRP backup and Internal Storage backup


      1. First make a full backup of all partitions, via EDL (low-level which gets all partitions). You can use my new Low-Level tool here. I recommend using the skip-systems-and-userdata list since it takes like an hour, and you can backup system/userdata faster in TWRP and PC manually.
      2. Make a full TWRP backup of our current ROM. System needs to be erased during this procedure. After re-partitioning however, you can simply restore this backup (unless your system is > 2.4GB for some reason, which is very unlikely).
      3. Make a full manual backup of your internal storage files to PC. Userdata partition needs to be completely erased if/when you restore the original stock partitions and firmware. You can postpone this process to later however, for if/when you want to actually restore to stock partition layour.


      Phase 1 - Repartition device for vendor_a and vendor_b


      This is relatively simple. Our device has a standard GPT table, so we can use sgdisk which is already included in our TWRP build.

      1. Boot into recovery and fire up an adb shell
      2. View current partition table...
        Code:
        sgdisk /dev/block/mmcblk0 --print
        ...and confirm these values...
        Code:
        ...
          25         1185792         7477247   3.0 GiB     FFFF  system_a
          26         7477248        13768703   3.0 GiB     FFFF  system_b
        ...
          49        14550032        61071326   22.2 GiB    FFFF  userdata
        The important values are the partition numbers (25, 26 and 49), and the next two numbers for both system partitions (start/end sectors). The start/end sector of userdata isn't important, only the partition number of 49 is. If your sector start/ends are different, you will need to adjust some of the values in these steps - that's up to you to figure out. But do please let me know what your differences are, for science.
      3. Remove system_a, and create a new system_a partition of size ~2.4GB. Don't worry, this is plenty - my current RR build with a heavy AROMA Gapps is only ~1.9GB. We're taking off ~600MB, which is 1228800 sectors (600*1024*1024/512) so the new end sector will be 6248447 (7477247 - 1228800):
        Code:
        sgdisk /dev/block/mmcblk0 --delete 25
        sgdisk /dev/block/mmcblk0 --new=25:1185792:6248447
        sgdisk /dev/block/mmcblk0 --change-name=25:system_a
      4. Create vendor_a with the reclaimed ~600MB:
        Code:
        sgdisk /dev/block/mmcblk0 --new=50:6248448:7477247
        sgdisk /dev/block/mmcblk0 --change-name=50:vendor_a
        ...notice the partition number of 50 - one after the userdata (49).
      5. Do similarly to above, only for system_b and vendor_b:
        Code:
        sgdisk /dev/block/mmcblk0 --delete 26
        sgdisk /dev/block/mmcblk0 --new=26:7477248:12539903
        sgdisk /dev/block/mmcblk0 --change-name=26:system_b
        sgdisk /dev/block/mmcblk0 --new=51:12539904:13768703
        sgdisk /dev/block/mmcblk0 --change-name=51:vendor_b
      6. Quit shell and reboot recovery...
        Code:
        adb reboot recovery
        ...and verify the new partitions exist...
        Code:
        adb shell ls -la /dev/block/bootdevice/by-name/
        ...should show this at the bottom...
        Code:
        ...
        lrwxrwxrwx    1 root     root            21 Jan  5  1970 userdata -> /dev/block/mmcblk0p49
        lrwxrwxrwx    1 root     root            21 Jan  5  1970 vendor_a -> /dev/block/mmcblk0p50
        lrwxrwxrwx    1 root     root            21 Jan  5  1970 vendor_b -> /dev/block/mmcblk0p51


      Repartition is done! You can now restore your TWRP backup (only system) and system works perfectly as before.


      Roadblocks of Phase 1

      • How best to deliver this partition table change to users? The obvious answer is an update ZIP that just scripts it all (that's why I used sgdisk in this write-up). But then this requires the user to manually backup, reboot recovery, then restore backup. Is that ideal?
      • Another solution would be to export the new GPT, and make a MiFlash-flashable package. This could be another option for future, but would require new system and vendor images too - may as well be a whole flash package. That's big. We'll look into that once we actually get a Treble ROM working (e.g. stock).


      Phase 1b revert - Restore stock partition map

      If you want to go back to original partition table for some reason, you will need to flash an official fastboot ROM via EDL mode in MiFlash. After researching, I have discovered that there is no way around this. The way MiFlash works is it actually adjusts the GPT info dynamically based on *your* partition map (64GB vs 32GB have different userdata sizes) and for this to be done it needs to flash a full package. So, to restore stock partition map, you must flash a stock ROM in MiFlash via EDL mode which wipes everything. I hope you made a low-level and internal-storage backup like I warned - because after that you'll want to restore your backup to get persist and stuff back. Flashing in fastboot mode is not enough because it does not restore stock partition table.

      Full steps for restoring to stock:

      1. Boot into EDL mode
      2. Load MiFlash, select any Oreo fastboot image (I use 8.1.10 since it's what I have) > Refresh. Should show COM30 (or whatever your QDLoader COM port is).
      3. Reboot EDL by holding Power + VolDown and then 'fastboot oem edl' again
      4. Run the restore in my Low-level flash tool of your backup
      5. Reboot back into recovery by holding power + volume-up for ~10 seconds.
      6. "Skip" on password screen
      7. Wipe > Format Data > type "yes" to remove stock ROM encryption
      8. Restore your TWRP backup
      9. Done!


      Phase 2 - Patch kernel DTB and TWRP

      This requires Linux. I use ChaletOS in VirtualBox and have my home folder Samba shared, mapped to a network drive on my Windows host.

      Probably when all this is finalized and working, we will use a custom kernel built from source. But for now lets just patch our existing kernel. I assume you already have TWRP installed.

      Part 1 - Kernel DTB

      1. Download the DTB split tool from:
        https://github.com/dianlujitao/split-appended-dtb
        ...to your Linux machine. Just get the binary, not the .c source file.
      2. Dump existing boot.img while in TWRP and pull from device:
        Code:
        adb shell dd if=/dev/block/bootdevice/by-name/boot of=/tmp/boot.img
        adb pull /tmp/boot.img
      3. Unpack this boot.img. I use AIK by @osm0sis from:
        https://forum.xda-developers.com/showthread.php?t=2073775
        ... In this case, we need boot.img-zImage. Copy it to the Linux machine folder with DTB split.
      4. Split DTB from zImage on Linux machine:
        Code:
        ./split-appended-dtb boot.img-zImage
        ...should produce 1 DTB. Total output will be two files - "kernel" and "dtbdump_1.dtb".
      5. Decompile DTB on Linux machine:
        Code:
        # if you need to install it
        sudo apt install device-tree-compiler
        dtc -O dts -I dtb -o dtbdump_1.dts dtbdump_1.dtb
      6. Edit dtbdump_1.dts in whatever text editor. Search for fstab and edit the vendor entry, adding "slotselect" to fsmgr_flags (so it's the same as system):
        Code:
        fsmgr_flags = "wait,slotselect";
        NOTE: This is on my RR kernel. I need to check any difference with stock or other kernels.
      7. Recompile the dts to dtb and build a new zImage:
        Code:
        dtc -O dtb -I dts -o dtbdump_1_new.dtb dtbdump_1.dts
        cat kernel dtbdump_1_new.dtb > boot.img-zImage.new
      8. Copy the boot.img-zImage.new back to your kernel kitchen tool, replacing the old one.
      That's the device tree patched to mount seamless/slot vendor. But don't repack the boot.img yet, we need to edit RAMDisk.



      Part 2 - TWRP patch

      This is just so we can see Vendor (and Persist, because why not) in TWRP. This is so:

      • Vendor can be formatted (required before use), backed up/restored and flashed to
      • [Bonus] Persist can be backed up/restored and flashed to (for e.g. Persist repair ZIP's that should come soon).

      Only one step is needed - edit /etc/recovery.fstab to add vendor mount (and also persist because why not) with flags:
      Code:
      /vendor          ext4             /dev/block/bootdevice/by-name/vendor     flags=slotselect;backup=1;wipeingui;display="Vendor"
      /persist         ext4             /dev/block/bootdevice/by-name/persist    flags=backup=1;display="Persist"
      That's it.

      For your convenience, I have attached a Treble-modded TWRP installer to this thread with these changes. Don't forget to flash Magisk after re-installing TWRP. Big thanks to @ghpranav


      Phase 3 - Compile a Treble ROM (for Vendor files)

      Already succeeded. We just need to cleanup and push.

      Phase 4 - Treble-ize an installed ROM

      • Is this possible? Not sure... I'll check it out later sometime


      Did I miss anything? Probably. Let me know!
      96
      It's aliiiiiiiive!


      AqS4O781_o.png


      I humbly request all follows of this project to thank @ghpranav two posts above (orange display pic) for this fix that I was stuck on :D

      Now to work on GSI...
      60
      And we have GSI booting! This is it people, we've done it!

      https%3A%2F%2Fimages2.imgbox.com%2F3d%2F39%2FcDbBQZi5_o.png


      Congratulations! :) Which ROM are you using during that screenshot? Stock ROM?

      That one was an experimental RR build.

      Partitions wouldn't be changed anymore when flashing GSI (generic system images). The space allotted is already enough for all GSIs available in XDA.

      Sent from my Xiaomi Mi A1 using XDA Labs

      Correct.

      All other questions will be answered soon.

      This GSI boot is a little hacky, I was swapping files and editing crap all over the place on my live device and taking notes. Me and the guys are working on deploying it all to our device tree(s) next and will release more user-friendly stuff soon.

      This will include:
      1. A flashable ZIP that repartitions the device *safely*, as well as another ZIP that can restore STOCK partitions safely
      2. A flashable zip for vendor files only (maybe; or maybe just a full ROM), but either way will allow you to flash Phh GSI treble ROM's via fastboot
      3. ....? ;)
      55
      TWRP installer

      Here's a TWRP installer with vendor mount point added, if anyone wants it. Make sure to flash Magisk after installing TWRP, otherwise you'll get a recovery bootloop.

      https://www.androidfilehost.com/?fid=962339331458992573
      35
      Let's call it Illu-Mi A1.

      ?

      A stupid question... We can flash any gsi after this complete right?... Is all the feature functional?.. or should we stabilize the gsi.. or the vendor one to stabilize?

      Yeah, this is a problem we'll have to work out. All GSI's should work to a limited degree. But even right now, the GSI has poor compatibility on some devices. Sometimes Phh adds fixes in GSI but that seems only for *popular* devices with *official* Treble.

      It really should be up to us, the A1 Treble maintainers, to get GSI working best on our side. There is a lot we can do on Vendor, it gets it's own share of boot initialisation. We can run scripts and anything, we can even inject files onto the GSI /system automatically on bootnif necessary. The only problem with that is it could break GSI OTA, but that doesn't even exist yet - the GSI update system is only in conceptual stages right now, it always has to be updated manually by Phh (that means there are NO official updated directly from Google, just incase anybody didn't know).

      The good news is that because the Mi A1 is an Android One device, it should be far far easier to get 100% daily working GSI support compared to other devices.

      To be completely honest, I didn't really find the Treble porting too difficult. I've been out of the loop of Android dev for a long time and did get some help from Phh and Team Oreo, but it was all mostly trivial and I just had to learn about general Android building stuff again. The A1 really is a pure Android device, completely in-line with Google requirements for Treble.

      What I mean to say is, I think this was the easy part. We still have to fix fingerprint and get SELinix Enforcing on GSI Treble (actually I don't know if that's even possible), and there are probably more bugs - this might be beyond me. I think this will be up to Team Oreo, they are fully behind this Treble project and eagerly waiting for me to share final changes :) speaking of, I better get back to it!