[DEV][COMPLETE] Project Treble conversion

Status
Not open for further replies.
Search This thread

SmallTarzan

Senior Member
Nov 28, 2014
244
127
Bratislava
Try doing
Code:
fastboot --set-active=a
(Or b).

There is a boot slot bug currently that exists in TWRP with Treble.

That's the only way I could imagine it causing it.

You can tell if you are effected by this bug by trying to change slot in TWRP, seeing it's changed, then rebooting recovery and noticing it's still stuck on the first slot.

The fastboot command will solve that.

If that's not the problem, then it's not the partitions to blame. OpenGapps sounds buggy and could be reporting wrong version when a ROM isn't even detected at all (because of slot bug).

But you said you even flashed stock ROM in MiFlash, and it still happens.

If you made a backup of your ROM with my Low-level Backup Tool, like I strongly suggested, you will get back a 100% clean stock style partition when you restore it. And if it still happens then, then it's obviously not a problem with the partitions.

The other way to get back a 100% clean stock system with partitions is with MiFlash in EDL mode, but this will wipe your MAC's and IMEI. So you should of made a Low-level backup either way.

Thanks for help, but I get this error when trying to use the command.
 

Attachments

  • Screenshot_2.png
    Screenshot_2.png
    2.5 KB · Views: 1,397

dback31

Senior Member
Sep 10, 2016
233
81

Attachments

  • Minimal_ADB_and_Fastboot.zip
    1.1 MB · Views: 14
Last edited:

SmallTarzan

Senior Member
Nov 28, 2014
244
127
Bratislava
Try the fastboot version in the attachments.
I always flash stock ROM via EDL mode and set the A partition as default and it works fine for me with the command

Virustotal link in case you have doubts
https://www.virustotal.com/#/file/3...00c2c88cfba49ac835d5dd57557a1be31df/detection

Edit: It should look like this
6bcc175c5d7b.PNG

Used the command, everything went fine. Booted into TWRP, changed to partition B, restarted the device into recovery, and I'm back at partition A...
EDIT: Switched to partition B through fastboot and it works now, thanks!
 
Last edited:

dback31

Senior Member
Sep 10, 2016
233
81
Last edited:

SomratMJX

Senior Member
Sep 12, 2014
94
44
Dinajpur
www.prohelika.org
Yes, the new TWRP menu for repartition has option to shrink data. See my post on progress update back a page or so.

Hmm! Seen that! But I was talking about checking if shrinking and repartitioning data instead of system solves the OTA problem or not! Have you tried or want someone to try?
All they need to have is prev stock builds (feb/march) backup and willingness of taking risks off course!
I have April fastboot stock! (I actually bought the device on April 4th) ;)

I think the process of checking it pretty straightforward:
1) Backup the earlier stock on non treblize partition
2) Treblize (data shrinking)
3) Restore backup
4) Flash stock recovery (don't know if it actually exists)
5) Reboot to system and check for OTA
6) If OTA is available (off course it is) then update

If the update completes without issue, then everyone should prefer shrinking data and should agree on loosing some space from their storage!

I hope everyone has sdcards like you! :D
 

speedunderx

Senior Member
Feb 12, 2013
70
17
Hmm! Seen that! But I was talking about checking if shrinking and repartitioning data instead of system solves the OTA problem or not! Have you tried or want someone to try?
All they need to have is prev stock builds (feb/march) backup and willingness of taking risks off course!
I have April fastboot stock! (I actually bought the device on April 4th) ;)

I think the process of checking it pretty straightforward:
1) Backup the earlier stock on non treblize partition
2) Treblize (data shrinking)
3) Restore backup
4) Flash stock recovery (don't know if it actually exists)
5) Reboot to system and check for OTA
6) If OTA is available (off course it is) then update

If the update completes without issue, then everyone should prefer shrinking data and should agree on loosing some space from their storage!

I hope everyone has sdcards like you! :D

your idea seems more favorable, by reducing the data partition and from there creating vendor a and b so you would not be touching the partitions of the system.
 
  • Like
Reactions: SomratMJX

SomratMJX

Senior Member
Sep 12, 2014
94
44
Dinajpur
www.prohelika.org
Was saying that for sooooo long!

your idea seems more favorable, by reducing the data partition and from there creating vendor a and b so you would not be touching the partitions of the system.

I was saying that (actually arguing ;)) from the beginning!
But the OP chose system because it's same for both 64 and 32 GB variants! And may be also to avoid potential user data loss or at least the hassle to back up and restore data! It was simpler for him to touch system rather than data! Which is different from his device anyways unless he get both variants!
Therefore he tended to go for the initial easiest way! That is for system!

But you know not everything seemed easier initially may not actually be that simple/easy all the time!

BTW, kudos to him anyways! He is such an amazing person and talented guy that I have full faith on him that he somehow will find a way neat and clean!
 

CosmicDan

Senior Member
Jun 19, 2009
5,906
7,746
37
Sydney
Xiaomi Poco X3 Pro
Hmm! Seen that! But I was talking about checking if shrinking and repartitioning data instead of system solves the OTA problem or not! Have you tried or want someone to try?
All they need to have is prev stock builds (feb/march) backup and willingness of taking risks off course!
I have April fastboot stock! (I actually bought the device on April 4th) ;)

I think the process of checking it pretty straightforward:
1) Backup the earlier stock on non treblize partition
2) Treblize (data shrinking)
3) Restore backup
4) Flash stock recovery (don't know if it actually exists)
5) Reboot to system and check for OTA
6) If OTA is available (off course it is) then update

If the update completes without issue, then everyone should prefer shrinking data and should agree on loosing some space from their storage!

I hope everyone has sdcards like you! :D


your idea seems more favorable, by reducing the data partition and from there creating vendor a and b so you would not be touching the partitions of the system.

The new TWRP Installer with Treble Manager I just released has this option, so try it yourself.

But if OTA doesn't work with shrunk Userdata, then there's nothing I can do about it.
 
Status
Not open for further replies.

Top Liked Posts

  • There are no posts matching your filters.
  • 105
    UPDATE4: New Treble guide is posted here. Ask all your questions here that aren't specific to other parts of Treble (e.g. the ROM or the TWRP). Thread lock requested. Thanks all!

    UPDATE3: The new TWRP Installer with Treble Manager (for guided repartitioning) is now released. See this thread.

    UDPATE2: Experimental RR builds for Treble is now released. This contains the required Vendor Pack for GSI use. See this thread.


    UPDATE: We have succeed in booting a Phh-Treble GSI! See screenshot at Post #57. Stay tuned for user releases!


    This thread will be kept for posterity, but I will request locking once we release stuff. Thanks you all for the support, and thanks to @ghpranav for lots of help and @phhusson for pointing me in the right direction on the last stretch.

    Original post below

    ----------------------------------------------

    Alright, here is my WIP guide for converting the Mi A1 to a Treble device and ROM.

    NOT FOR NEWBIES. This is all manual stuff right now and if you do something wrong you CAN firm-brick your device! The project is incomplete and for developers/researchers only.

    Please refrain from "good luck" and "can't wait for this" and "OMG you are the King of Kings ComsicDan, plz have my babies" and such posts.

    Also note that you will NEED to get into EDL mode to restore stock partition table, if you need to, by flashing stock factory image with MiFlash or my Low-Level Backup Tool. MiFlash will WIPE your Userdata completely and your persist so you MUST make a low-level backup to restore original persist.img, and manually backup internal storage. But really it's perfectly fine to keep using a non-treble ROM even with partition table changes (there's no need to revert for any functional reason that I can think of, unless you somehow get a ROM that's bigger than 2.4GB). Note that the final update ZIP that will automatically repartition can also restore stock partitions, without the need for EDL stuff.


    Prerequisite - Low-level backup, TWRP backup and Internal Storage backup


    1. First make a full backup of all partitions, via EDL (low-level which gets all partitions). You can use my new Low-Level tool here. I recommend using the skip-systems-and-userdata list since it takes like an hour, and you can backup system/userdata faster in TWRP and PC manually.
    2. Make a full TWRP backup of our current ROM. System needs to be erased during this procedure. After re-partitioning however, you can simply restore this backup (unless your system is > 2.4GB for some reason, which is very unlikely).
    3. Make a full manual backup of your internal storage files to PC. Userdata partition needs to be completely erased if/when you restore the original stock partitions and firmware. You can postpone this process to later however, for if/when you want to actually restore to stock partition layour.


    Phase 1 - Repartition device for vendor_a and vendor_b


    This is relatively simple. Our device has a standard GPT table, so we can use sgdisk which is already included in our TWRP build.

    1. Boot into recovery and fire up an adb shell
    2. View current partition table...
      Code:
      sgdisk /dev/block/mmcblk0 --print
      ...and confirm these values...
      Code:
      ...
        25         1185792         7477247   3.0 GiB     FFFF  system_a
        26         7477248        13768703   3.0 GiB     FFFF  system_b
      ...
        49        14550032        61071326   22.2 GiB    FFFF  userdata
      The important values are the partition numbers (25, 26 and 49), and the next two numbers for both system partitions (start/end sectors). The start/end sector of userdata isn't important, only the partition number of 49 is. If your sector start/ends are different, you will need to adjust some of the values in these steps - that's up to you to figure out. But do please let me know what your differences are, for science.
    3. Remove system_a, and create a new system_a partition of size ~2.4GB. Don't worry, this is plenty - my current RR build with a heavy AROMA Gapps is only ~1.9GB. We're taking off ~600MB, which is 1228800 sectors (600*1024*1024/512) so the new end sector will be 6248447 (7477247 - 1228800):
      Code:
      sgdisk /dev/block/mmcblk0 --delete 25
      sgdisk /dev/block/mmcblk0 --new=25:1185792:6248447
      sgdisk /dev/block/mmcblk0 --change-name=25:system_a
    4. Create vendor_a with the reclaimed ~600MB:
      Code:
      sgdisk /dev/block/mmcblk0 --new=50:6248448:7477247
      sgdisk /dev/block/mmcblk0 --change-name=50:vendor_a
      ...notice the partition number of 50 - one after the userdata (49).
    5. Do similarly to above, only for system_b and vendor_b:
      Code:
      sgdisk /dev/block/mmcblk0 --delete 26
      sgdisk /dev/block/mmcblk0 --new=26:7477248:12539903
      sgdisk /dev/block/mmcblk0 --change-name=26:system_b
      sgdisk /dev/block/mmcblk0 --new=51:12539904:13768703
      sgdisk /dev/block/mmcblk0 --change-name=51:vendor_b
    6. Quit shell and reboot recovery...
      Code:
      adb reboot recovery
      ...and verify the new partitions exist...
      Code:
      adb shell ls -la /dev/block/bootdevice/by-name/
      ...should show this at the bottom...
      Code:
      ...
      lrwxrwxrwx    1 root     root            21 Jan  5  1970 userdata -> /dev/block/mmcblk0p49
      lrwxrwxrwx    1 root     root            21 Jan  5  1970 vendor_a -> /dev/block/mmcblk0p50
      lrwxrwxrwx    1 root     root            21 Jan  5  1970 vendor_b -> /dev/block/mmcblk0p51


    Repartition is done! You can now restore your TWRP backup (only system) and system works perfectly as before.


    Roadblocks of Phase 1

    • How best to deliver this partition table change to users? The obvious answer is an update ZIP that just scripts it all (that's why I used sgdisk in this write-up). But then this requires the user to manually backup, reboot recovery, then restore backup. Is that ideal?
    • Another solution would be to export the new GPT, and make a MiFlash-flashable package. This could be another option for future, but would require new system and vendor images too - may as well be a whole flash package. That's big. We'll look into that once we actually get a Treble ROM working (e.g. stock).


    Phase 1b revert - Restore stock partition map

    If you want to go back to original partition table for some reason, you will need to flash an official fastboot ROM via EDL mode in MiFlash. After researching, I have discovered that there is no way around this. The way MiFlash works is it actually adjusts the GPT info dynamically based on *your* partition map (64GB vs 32GB have different userdata sizes) and for this to be done it needs to flash a full package. So, to restore stock partition map, you must flash a stock ROM in MiFlash via EDL mode which wipes everything. I hope you made a low-level and internal-storage backup like I warned - because after that you'll want to restore your backup to get persist and stuff back. Flashing in fastboot mode is not enough because it does not restore stock partition table.

    Full steps for restoring to stock:

    1. Boot into EDL mode
    2. Load MiFlash, select any Oreo fastboot image (I use 8.1.10 since it's what I have) > Refresh. Should show COM30 (or whatever your QDLoader COM port is).
    3. Reboot EDL by holding Power + VolDown and then 'fastboot oem edl' again
    4. Run the restore in my Low-level flash tool of your backup
    5. Reboot back into recovery by holding power + volume-up for ~10 seconds.
    6. "Skip" on password screen
    7. Wipe > Format Data > type "yes" to remove stock ROM encryption
    8. Restore your TWRP backup
    9. Done!


    Phase 2 - Patch kernel DTB and TWRP

    This requires Linux. I use ChaletOS in VirtualBox and have my home folder Samba shared, mapped to a network drive on my Windows host.

    Probably when all this is finalized and working, we will use a custom kernel built from source. But for now lets just patch our existing kernel. I assume you already have TWRP installed.

    Part 1 - Kernel DTB

    1. Download the DTB split tool from:
      https://github.com/dianlujitao/split-appended-dtb
      ...to your Linux machine. Just get the binary, not the .c source file.
    2. Dump existing boot.img while in TWRP and pull from device:
      Code:
      adb shell dd if=/dev/block/bootdevice/by-name/boot of=/tmp/boot.img
      adb pull /tmp/boot.img
    3. Unpack this boot.img. I use AIK by @osm0sis from:
      https://xdaforums.com/showthread.php?t=2073775
      ... In this case, we need boot.img-zImage. Copy it to the Linux machine folder with DTB split.
    4. Split DTB from zImage on Linux machine:
      Code:
      ./split-appended-dtb boot.img-zImage
      ...should produce 1 DTB. Total output will be two files - "kernel" and "dtbdump_1.dtb".
    5. Decompile DTB on Linux machine:
      Code:
      # if you need to install it
      sudo apt install device-tree-compiler
      dtc -O dts -I dtb -o dtbdump_1.dts dtbdump_1.dtb
    6. Edit dtbdump_1.dts in whatever text editor. Search for fstab and edit the vendor entry, adding "slotselect" to fsmgr_flags (so it's the same as system):
      Code:
      fsmgr_flags = "wait,slotselect";
      NOTE: This is on my RR kernel. I need to check any difference with stock or other kernels.
    7. Recompile the dts to dtb and build a new zImage:
      Code:
      dtc -O dtb -I dts -o dtbdump_1_new.dtb dtbdump_1.dts
      cat kernel dtbdump_1_new.dtb > boot.img-zImage.new
    8. Copy the boot.img-zImage.new back to your kernel kitchen tool, replacing the old one.
    That's the device tree patched to mount seamless/slot vendor. But don't repack the boot.img yet, we need to edit RAMDisk.



    Part 2 - TWRP patch

    This is just so we can see Vendor (and Persist, because why not) in TWRP. This is so:

    • Vendor can be formatted (required before use), backed up/restored and flashed to
    • [Bonus] Persist can be backed up/restored and flashed to (for e.g. Persist repair ZIP's that should come soon).

    Only one step is needed - edit /etc/recovery.fstab to add vendor mount (and also persist because why not) with flags:
    Code:
    /vendor          ext4             /dev/block/bootdevice/by-name/vendor     flags=slotselect;backup=1;wipeingui;display="Vendor"
    /persist         ext4             /dev/block/bootdevice/by-name/persist    flags=backup=1;display="Persist"
    That's it.

    For your convenience, I have attached a Treble-modded TWRP installer to this thread with these changes. Don't forget to flash Magisk after re-installing TWRP. Big thanks to @ghpranav


    Phase 3 - Compile a Treble ROM (for Vendor files)

    Already succeeded. We just need to cleanup and push.

    Phase 4 - Treble-ize an installed ROM

    • Is this possible? Not sure... I'll check it out later sometime


    Did I miss anything? Probably. Let me know!
    96
    It's aliiiiiiiive!


    AqS4O781_o.png


    I humbly request all follows of this project to thank @ghpranav two posts above (orange display pic) for this fix that I was stuck on :D

    Now to work on GSI...
    60
    And we have GSI booting! This is it people, we've done it!

    https%3A%2F%2Fimages2.imgbox.com%2F3d%2F39%2FcDbBQZi5_o.png


    Congratulations! :) Which ROM are you using during that screenshot? Stock ROM?

    That one was an experimental RR build.

    Partitions wouldn't be changed anymore when flashing GSI (generic system images). The space allotted is already enough for all GSIs available in XDA.

    Sent from my Xiaomi Mi A1 using XDA Labs

    Correct.

    All other questions will be answered soon.

    This GSI boot is a little hacky, I was swapping files and editing crap all over the place on my live device and taking notes. Me and the guys are working on deploying it all to our device tree(s) next and will release more user-friendly stuff soon.

    This will include:
    1. A flashable ZIP that repartitions the device *safely*, as well as another ZIP that can restore STOCK partitions safely
    2. A flashable zip for vendor files only (maybe; or maybe just a full ROM), but either way will allow you to flash Phh GSI treble ROM's via fastboot
    3. ....? ;)
    55
    TWRP installer

    Here's a TWRP installer with vendor mount point added, if anyone wants it. Make sure to flash Magisk after installing TWRP, otherwise you'll get a recovery bootloop.

    https://www.androidfilehost.com/?fid=962339331458992573
    35
    Let's call it Illu-Mi A1.

    ?

    A stupid question... We can flash any gsi after this complete right?... Is all the feature functional?.. or should we stabilize the gsi.. or the vendor one to stabilize?

    Yeah, this is a problem we'll have to work out. All GSI's should work to a limited degree. But even right now, the GSI has poor compatibility on some devices. Sometimes Phh adds fixes in GSI but that seems only for *popular* devices with *official* Treble.

    It really should be up to us, the A1 Treble maintainers, to get GSI working best on our side. There is a lot we can do on Vendor, it gets it's own share of boot initialisation. We can run scripts and anything, we can even inject files onto the GSI /system automatically on bootnif necessary. The only problem with that is it could break GSI OTA, but that doesn't even exist yet - the GSI update system is only in conceptual stages right now, it always has to be updated manually by Phh (that means there are NO official updated directly from Google, just incase anybody didn't know).

    The good news is that because the Mi A1 is an Android One device, it should be far far easier to get 100% daily working GSI support compared to other devices.

    To be completely honest, I didn't really find the Treble porting too difficult. I've been out of the loop of Android dev for a long time and did get some help from Phh and Team Oreo, but it was all mostly trivial and I just had to learn about general Android building stuff again. The A1 really is a pure Android device, completely in-line with Google requirements for Treble.

    What I mean to say is, I think this was the easy part. We still have to fix fingerprint and get SELinix Enforcing on GSI Treble (actually I don't know if that's even possible), and there are probably more bugs - this might be beyond me. I think this will be up to Team Oreo, they are fully behind this Treble project and eagerly waiting for me to share final changes :) speaking of, I better get back to it!