awesom3: dynamic partition table resizing ~ twrp 2.8.0.0

thewisenerd

Senior Member
Sep 27, 2012
1,385
2,995
0
This started out as a little side project on the OmniROM threads, and it's about time some things are clarified, and what exactly done, is explained.

Currently, only a modified version of TWRP 2.7 recovery is available, and we *might* support other recoveries soon :)
We have moved ahead of the TWRP 2.7.0.0 originally available in orig-development, to 2.7.1.0, and eventually to 2.8.0.0 too. Enjoy!

What is aw3som3?
Developers have long tried to sqeeze in ROMs into legacy devices with low internal memory, and one of the ways to do this was by using custom tailored mtd_parts, which gave the kernel a partition table to use, with resized partitions. This was widely used on many devices like the HTC G1, MT3G, Hero, Evo4G, Desire, etc.

While that was a viable option for the HTC Pico, a small "problem" that popped up was that different phones have different NAND Devices, with different partition tables and pagesizes. Incase the modified custom mtd_parts even accidentally touched the boot partition, or has been offset'ed out of the available space, a partition would be lost.

As a solution, dynamically obtain 32 mB from userdata partition, and 48 mB from the cache partition, and hax the partition table as soon as the information is got from ATAG. This should make this method work on *almost* all Pico's out there, which have partition layouts as misc:recovery:boot:system:cache:userdata:devlog and partitions in the order misc:rec..cache:devlog:userdata.
I've changed the partition table layout, without modifying the actual mtdblock<n>, thus no further fstab/ramdisk editing is required.
table: 0->misc::1->recovery::2->boot::3->system::4->cache::5->userdata::6->devlog
original layout : recovery:boot:system:cache:devlog:userdata:misc
new layout : recovery:boot:misc:devlog:userdata:cache:system

To developers:
In case you want to modify any of the partition size defines, there are three defines, CACHE_SIZE_LEAVE, USERDATA_SIZE_LEAVE, and DEVLOG_SIZE_LEAVE. This, will leave that much mB as defined, and take the rest, put it in system partition. The default defines are: 8, 4 and 1 respectively.
You may clone this repo, and follow the following steps:
  • download and compile kernel from android-4.4-recovery branch from https://github.com/thewisenerd/android_kernel_htc_pico
  • put compiled zImage (arch/arm/boot/zImage) in root directory (i.e. cloned directory)
  • run the following command to repack recovery ./repack.sh
  • output recovery would be in output/recovery.img

Downloads:

  1. TWRP 2.8.0.0: http://forum.xda-developers.com/showpost.php?p=56168368&postcount=125

    [*] TWRP 2.8.0.0: http://forum.xda-developers.com/showpost.php?p=55685051&postcount=108(26 sep 2014)
    [*] Modified TWRP 2.8.0.0: http://forum.xda-developers.com/showpost.php?p=55430289&postcount=106(14 sep 2014)
    [*] Modified TWRP 2.8.0.0: http://forum.xda-developers.com/showpost.php?p=55375649&postcount=104(11 sep 2014)
    [*] Modified TWRP 2.7.1.0: http://forum.xda-developers.com/showpost.php?p=55246774&postcount=97 (05 sep 2014)
    [*] Modified TWRP 2.7.0.0: http://forum.xda-developers.com/showpost.php?p=54709263&postcount=69 (09 Aug 2014)
    [*] Modified TWRP 2.7.0.0: http://forum.xda-developers.com/showpost.php?p=54688800&postcount=65 (08 Aug 2014)
    [*] Modified TWRP 2.7.0.0: http://forum.xda-developers.com/showpost.php?p=54484381&postcount=58 (30 Jul 2014)
    [*] Modified TWRP 2.7.0.0 (resized): http://forum.xda-developers.com/showpost.php?p=53704940&postcount=6
    [*] Modified TWRP 2.7.0.0: http://d-h.st/I4L

Kernel: You may use CodyKernel or the kernel provided by your ROM's developer that supports aw3som3.

Q: How to use?
A:
  • Wipe the system, data and cache partition.
  • fastboot flash the given recovery.img.
  • wipe the system, data and cache partitions, again. preferably use flash_erase on the partitions.
  • flash your preferred ROM
  • flash the given kernel, *with* aw3som3
  • you're good to go!

Sources
you can get the patches here:

PS#1: 0001-aw3som3-hax-the-partition-table.patch

PS#2: 0002-aw3som3-set-msm_nand_data.parts-later.patch

PS#3: 0003-aw3som3-rearrange-and-relay-the-partition-table.patch


I've moved all the partition re-laying, resizing, etc, to a separate file, in drivers/misc, which should help keep clean the nand_partitions.c file, and we would have to worry only about one file, and related code. It also means that awesom3 is reduced to one patch
PS#1 (26Sep'14):
0001-awesom3-hax-the-partition-table.patch
0002-awesom3-allow-a-128-mB-userdata.patch

XDA:DevDB Information
awesom3, Tool/Utility for the HTC Pico (Explorer)

Contributors
thewisenerd

Version Information
Status: Stable

Created 2014-10-21
Last Updated 2014-10-21
 
Last edited:

thewisenerd

Senior Member
Sep 27, 2012
1,385
2,995
0
Reserved

Questions&Answers
Q: Does this work?
A: Yes, it does

Q: Will it work on my phone?
A: no. it will blast into pieces. jk.
Yes, it should. The main reason that partitions are dynamically resized, is to avoid having to create tailor-made hex values for every type of NAND device that different picos have.

Q: why the name awesom3?
A: A couple of reasons, actually. I was using the awesome D.E. in my distro a few days before I wrote this patch. I liked that UI, mainly because it was cluster-free, and the maximum available space was always used.
Another, was a couple of twn_oc_helper scripts which I had written before this. I had decided to get rid of the scripts, once and for all, due to various reasons (you can read them here), and the working dir for those scripts was named awesom3. The name just stuck by, and voila!

misc
- setting timezone
1. settings-> Time Zone -> set timezone (closest) and offset.
 
Last edited:

thewisenerd

Senior Member
Sep 27, 2012
1,385
2,995
0
Reserved

changelog

21Oct2014
- 128 mB userdata partition
- Fix wiping crash when block device is not present (credits to Ethan Yonker)
- mtp: fix log spam: findNode not finding a node is not an error (credits to that)
- Update reference to 24-hour clock (credits to Captain Throwback)
- display cpu temperature in twrp use TW_NO_CPU_TEMP := true to disable display use TW_CPU_CUSTOM_PATH := path to set custom path (credits to Jenkins)
- Add support for TrueType fonts (credits to Vojtech Bocek)
- Fix exfat-fuse handling (credits to Dees Troy)
- performance profile support (credits to Tom Hite)
- mtp: cleanup, fixes and performance improvements (credits to that)
- Update RECOVERY_VARIANT blocking to use project-path-for (credits to Matt Mower)
- Attempt to fix md5 issue with backups (credits to Captain Throwback)
- applypatch: add support for BML partitions (credits to Conn O'Griofa)
- atime is access time, let's just use mtime (credits to Jenkins)
- Remove extra slashes from zip path in openrecovery (credits to Matt Mower)
- Fixes for exfat-fuse (credits to Captain Throwback)
- recovery: Fix adb with linux 3.10 (credits to Benoit Goby)
- ADB sideload command not work (credits to Da Zhou)
- Check sehandle before fixing permissions (credits to Dees Troy)
- Attempt to automatically identify data/media devices (credits to Ethan Yonker)


older logs:

26sep'14
  • Fix handling of MTP fork/child process (upstream)
  • Include no strict aliasing for MTP (upstream)
  • Additional checks to prevent partitioning internal storage (upstream)
  • pico: use twrp.fstab (help unify recovery compilation with device tree)
  • removed UMS
  • updated MTP configs (Sachin Thomas)
  • aw3som3 -> awesom3
14sep'14
  • add mtp config
11sep'14
  • Add MTP support to recovery thanks mostly to bigbiff with a little help from Dees_Troy
  • Add command line capabilities - you can now execute various TWRP features via adb instead of the touchscreen
  • Add support for color in the console and give error, warning, and highlight lines different colors
  • Track backup and restore progress based on file sizes to provide a much more accurate indication of progress
  • Improve handling of /misc thanks to mdmower
  • Improve setting of time on Qualcomm devices thanks to [NUT]
  • Allow using images on slidervalue GUI objects thanks to Tassadar
  • Allow using variables and addition and subtraction in variables for easier theming
  • Add support for 1440x2560, 280x280, and 320x320 resolutions and update 240x240
  • Allow ui.xml file to include additional xml files to help break up the theme and make TWRP easier to maintain
  • Other minor fixes and improvements
05sep'14
  • updated sources to TWRP 2.7.1.0 (till 01 sep 2014)
  • removed TW_EXCLUDE_SUPERSU := true
  • removed TW_NO_EXFAT := true
09aug'14
  • use inverted logo
08aug'14
  • relay partition table
  • add graphic logo && text to differentiate between normal and awesom3 recovery
30jul'14
27jun'14
  • resize the recovery
 
Last edited:

memontalha

Senior Member
Feb 27, 2013
86
6
0
will you please tell me step by step procedure i am noob and don't know how to do all this stuff

I am currently using CM11 weekly 4 TWRP 2.7 recovery
 

thewisenerd

Senior Member
Sep 27, 2012
1,385
2,995
0
the recovery has been resized, hopefully. get another beta here: http://www.solidfiles.com/d/e6d39a02c4/recovery.img

hopefully, we're just a few commits away from a complete partition table rewrite... I've decided to move devlog to the beginning, then, a 4 mB cache (you shouldn't need more), and the rest for the /system partition. What about /data? sd-ext becomes /data partition by default. I posted a poll on the FB group and most of the people are clearly fed up with internal memory being very low hence, already using sd-ext by default, why "waste" the data partition?

This would stay in beta till some more time.

a new build of twrp should be out any time soon... the kernel, slimmered down to 1.8 mB without affecting actual functionality, *and* adding back support for ext2/3 and exfat and ntfs.

the recovery's size has been slimmered too, by modifying its ramdisk compression. you might not be much interested in all that, but you should see the change in recovery size from this build itself.

and finally, here's the boot.img: http://d-h.st/5jK

proof that it works (otherwise which no one's going to download anyways):


as we are going to use complete /data partition + some moar of cache partition, 112 +
 

Attachments

Last edited:

thewisenerd

Senior Member
Sep 27, 2012
1,385
2,995
0
here's mine.
it's working.
instead of
Code:
http://dl-1.va.us.xda-developers.com/2/8/1/6/0/8/7/Screenshot_from_2014-06-24_202911.png?key=Kg2Fp7s86-Ky-H0UFugPJQ&ts=1403661729
use
Code:
http://forum.xda-developers.com/attachment.php?attachmentid=2816087
as your image link... that should fix image not showing.

will you please tell me step by step procedure i am noob and don't know how to do all this stuff

I am currently using CM11 weekly 4 TWRP 2.7 recovery
how do I made this work bro?

I mean how many partitions should I have particularly their each sizes too

thanks in advance :eek:
Its for this very reason that I wrote this patch. Users don't have to do anything, except what's written in OP.

This is out of beta now, (read: stable), and can be used.

For others: I'll be shifting to a "no userdata", "4 mB cache", and all for "system" policy soon. Hopefully, you can revert back those commits if you don't need. As a user, 150 mB has never been adequate since I had used 40int2ext, and so, I'll be making sd-ext for /data as default.
This would also involve refactoring the entire table, I've decided to move devlog a few places ahead and order it as boot:devlog:cache:system, which should, technically, make stuff easier (for me) if done in the kernel side.


Don't want that? Revert back those commits, simple as that.

p.s. after thinking about this for sometime, this would also involve editing fstabs, recoveries, etc, etc, so, we'll rather stick to the current partitioning, i.e. mtdblock3/4/5, and keep it as 1 mB userdata, rather than no userdata, so that all the mess of having to edit other stuff would be minimised.

welcome working pa's micro modular gapps (soon), which takes up about 135 mB of /system partition, which should fit, after we bring in some more space from userdata partition.
 
Last edited:

Black_Eyes

Senior Member
Apr 11, 2013
1,362
940
0
Lahore
Perfect Setup Bro :)
Well it will be good if we use sd-ext as data default
Much System Space :laugh:

Regards : Bilal

Sent from my One M8 using XDA-FORUM, powered by appyet.com
 

theBstep

Senior Member
Dec 1, 2012
219
38
0
noobified

I tried and so far so good with cm weekly 5 :laugh: I flashed latest cody kernel by the way
the thing is how can I increase internal memory cus only system increased? do I need to add a partition again so that makes 4: fat32, sd-data, sd-cache and sd-ext :confused:
 
Last edited:

thewisenerd

Senior Member
Sep 27, 2012
1,385
2,995
0
I tried and so far so good with cm weekly 5 :laugh: I flashed latest cody kernel by the way
the thing is how can I increase internal memory cus only system increased? do I need to add a partition again so that makes 4: fat32, sd-data, sd-cache and sd-ext :confused:
nah, fat32, and sd-data would do :)

Its really messed up (all the "extra" partitions), considering we don't need to worry about the /cache partition.

The only partitions you need are a fat32 partition, and *one* ext2/3 partitions. Any more partitions, your headaches.

Also, I would suggest that mmcblk0p2 is directly mounted as /data, instead of using any scripts :angel: