Post Reply

[MOD][RECOVERY] Custom MTD Partitions (resize Data,System and Cache ) [ 2010-10-29 ]

OP Firerat

4th July 2010, 07:05 PM   |  #1  
OP Account currently disabled
Thanks Meter: 193
 
3,848 posts
Join Date:Joined: Feb 2009
Donate to Me
More
Custom MTD Partitions
This is an implimentaion of lbcoder's Custom partition layouts
be sure to checkout that thread for the full history

What does it do?
Well, basically Custom MTD Partitions resizes your MTD partitions
for instance this is a CM6.1RC1, ( heavily customised )
Code:
Filesystem                Size      Used Available Use% Mounted on
/dev/block/mtdblock3     73.0M     72.8M    236.0K 100% /system
/dev/block/mtdblock5    134.8M    107.2M     27.6M  80% /data
/dev/block/loop0        896.0K    896.0K         0 100% /system/lib/modules
/dev/block/loop1          4.0M      4.0M         0 100% /system/xbin
/dev/block/mmcblk0p2    457.4M    201.8M    231.1M  47% /sd-ext
/dev/block/mmcblk0p2    457.4M    201.8M    231.1M  47% /cache
/dev/block/mtdblock4      2.0M    776.0K      1.2M  38% /dev/cache
most of my Apps are on sd-ext, dalvik-cache is on data
/dev/cache is where the real cache partition is mounted, /cache is actually a bind mount from /sd-ext/cache


Applicable to..
Probably any device that uses the same kind of nand as the G1 MT3G ( msm_nand )
The intial scripts are geared towards G1 / MT3G. however I have 'rewritten' much of the script for v1.5 , it now reads the partition table in dmesg, so it _should_ be universal **
v1.5.3 confirmed to work on heroc
v1.5.6 confirmed to work on bravo + bravoc ( with S-OFF )
unsure if it will work with your device? checkout the source on github ( or ask your favourite dev to take a look )


The 'Tech' in Breif
This method is beautifully simple...
When booting we give the kernel the mtd partition table we want to use..
Thats it
In practice we need to do this when booting to recovery, and booting the rom.
below are files to make this as simple flashing a rom.

Credits :-
  • Lbcoder - for coming up with the idea
  • Skraw ( CM forums ) - for getting lbcoder interested
  • Koush - for AnyKernel
  • Cyanogen & Co - for giving us all such great ROMs to play with
  • Amon_RA and Koush - for giving us something to patch
  • Techjosh - for fixing the patchers for use with Rogers (EBi1)
  • Mblaster - for pointing out my nasty habit of using -r zip flag at the end of command ( breaks compatibility with older zip versions, fixed in AutoPatcher v1.5 )

Safety First
This method is safe, however it is not without risk
Two things could potentially go wrong
  • Recovery flash corrupt
    This is extremely unlikely, and tbh could happen anytime you flash recovery
    If in the very unlikely event that you find you can't reboot to recovery you have three options
    • re-flash recovery via fastboot ( the preferred option )
    • re-flash recovery via ROM ( not a great option with cm5.0.x/cm6 )
    • Do the whole root thing all over again ( no one wants to do that )
  • system, cache , data partition unmountable - corrupt
    Under the right ( or wrong ) circumstances it is possible to get 'junk' files stuck in system ( or cache,data ), and in such away that recovery can not delete them, more serious corruption can render the partitions unmountable.
    It is actually quite straight forward to fix this, but it does require fastboot
    Code:
    fastboot erase system -w
    Clockwork Recovery 2.0.2.0 and later has erase_image binary, if you can adb shell in then
    Code:
    for i in system cache userdata;do erase_image $i;done
    and reboot
I would advise you seek out how to 'fastboot', which tbh is a good thing to have regardless of using this 'hack' as it can get you out of so much trouble

*NB* don't use a patched recovery to flash SPL or RADIO ( you should avoid using recovery to flash these anyway, feel free to ask for current advice on spl / radio flashing )

Prevention is better than cure
I have only managed to corrupt partitions when switching partition layouts while having files on cache or data, for example going from System 67.5 Cache 67.5 to System 90 cache 5 with
cache approx 80% 'used'...
I have not been able to repeat this if I wipe Cache before rebooting,
therefore I advise that you wipe both cache and data * after patching recovery ( and rebooting )
* along with system if you are using clockwork

OK, now lets patch recovery and a ROM

Install Instuctions
It really is quite simple
  1. download FR-recovery-v1.5.6-CustomMTD_S.zip and FR-boot-v1.5.6-CustomMTD_S.zip
  2. create mtdpartmap.txt and put on /sdcard/ see configuration *
  3. reboot to recovery
  4. nandbackup
  5. wipe cache + data
  6. flash FR-recovery-v1.5.6-CustomMTD_S.zip
  7. reboot to recovery ( reboot and hold Home )
  8. Either : -
    • Nandrestore
    • Flash ROM + extras
  9. flash FR-boot-v1.5.6-CustomMTD_S.zip
  10. reboot

* configuration
The script in the recovery patcher checks for /sdcard/mtdpartmap.txt and reads that to override the default sizes.
e.g. for system 90mb and cache 2mb
NB make sure you mount sdcard first, else you won't write to sdcard/mtdpartmap.txt !

Code:
echo "mtd 90 2" > /sdcard/mtdpartmap.txt
data would be 117.8mb ( 116.7 useable )
e.g. for system 55mb and cache 2mb ( Purhaps a nice 'sugar free' Donut )
Code:
echo "mtd 55 2" > /sdcard/mtdpartmap.txt
data would be 152.8mb ( 151.7 useable )
NB, above data sizes are for G1s, MT3Gs should add ~78mb


All In One Patch runner ( New to v1.5.3 )
new option to run the All In One Patch script ( versions 1.3.6 and higher )
the format is
Code:
aio <option1> <option2> <option3>...
e.g.
Code:
aio swap remount shabang lwp
By default it will install the sd-ext mount ( option sdext )
so a line just reading aio will be fine

Note: the patch must be on the root of the sdcard, and its file name must start with "fr-patch" and end with ".txt"
if you have several versions the newest ( as per files datetime stamp ) will be used

Faking your SPL

If your using an SPL that is not officially supported by your ROM and that ROM checks your SPL you can 'patch' to fake it.
Code:
echo "spl 1.33.2005" >> /sdcard/mtdpartmap.txt
note that we are using ">>" here, this is to append to the file ( ">" would overwrite it )
you can by all means use any text editor you like, the script will automatically convert to unix format
NOTE : you are dodging the checks the ROM dev put in place, do not complain to them if this doesn't workout for you
in post 2 I have some 'CM6' Kernels I compiled for (1)0.95.xxxx SPLs
but I will only likely do these for RCs and Finals ( and there maybe a delay ).
In that post I point you towards the 'SafeSPL' ( 1.33.2003 ) this SPL is compatible with the current CM6 kernels ( so no need to wait for me or someone else to compile with bluetooth as modules ), but 1.33.2003 is not 'officially' supported ( its stock at 67.5mb system ) so you need to Fake your SPL ( say 1.33.2005 ) and resize to 90mb system ( or whatever you feel is optimum )


.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.: *~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~ *:._.:*~*:._.:*~*:._.:*~*:.
Downloads
.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.: *~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~ *:._.:*~*:._.:*~*:._.:*~*:.

http://tinyurl.com/customMTD
aab0fadf658ed275954aea8d0aed9c8f FR-AutoMTD_partitionPatcher_v1.5.6.tar.bz2
8857194cdbe34a52d173def4441ad2ae FR-AutoMTD_partitionPatcher_v1.5.6.zip
1f84a5ec50684a7830a93a8d455bc159 FR-boot-rpp-v1.5.6-CustomMTD_S.zip
bca0360f91aed0acf6e2dc82dfe01b56 FR-boot-v1.5.6-CustomMTD_S.zip
94b4238c2668cbe7cd52fb8ad5a2ee12 FR-recovery-v1.5.6-CustomMTD_S.zip

5404f1a41dbc60105d59c7fa0c335a70 FR-remove-v1.5.6-CustomMTD_S.zip




NB New Config option !!! to automatically run fr-patch136+
e.g.
Code:
mtd 90 2
spl 1.33.2005
aio swap remount shabang lwp a2sd
.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.: *~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~ *:._.:*~*:._.:*~*:._.:*~*:.
ROM Zip Patcher for Devs
To make life even simpler for end users it is possible to integrate the 'patch' within a ROM
AutoMTD_partitionPatcher_v1.5.6.zip
currently Linux only,
within the zip is a tarball, untar that.
get that directory into your PATH, ( or just cd into it )
and then execute
Code:
PatchUpdateScript.sh <zip file to patch>
it will then
  1. create a temp directory ( in your current directory )
  2. copy your zip to it
  3. extract required files
  4. patch update(r)-script
  5. zip and sign.
It simply saves the user from flashing the boot patch after flashing your ROM

The Future....
lbcoder has already suggested ways in which we can implement this 'on the fly'
so for instance it would be possible for a ROM , to instruct recovery what MTD partition layout is required, reload mtd kernel modules, and then flash ROM + boot.img

for those with huge partitions
Hey, you could go all silly and dual boot between ROMs ..


Anyway, enjoy and feel free to modify/improve on these


Changelog

v1-5-6 : 2010-10-28
  • Calculate userdata size, greatly improves compatibility
  • Added a version to patch a boot.img ( boot-rpp ) with run-parts
    didn't want to , but some are using roms which don't have run-parts, so the 06BindCache script wasn't running
  • Added a remove version ( remove )
    flashing this will return the recovery to SPL's layout
    I may well integrate that better, so you don't need a separate zip
  • AutoMTD_partitionPatcher can convert a recovery.img to a AutoMTD flashable zip
    PatchUpdateScript.sh <full path to>/recovery.img
  • boot patcher is much cleaner, it just uses the cmdline of the running recovery
  • removed the default 90 2 sizing, you *must* set your own size in mtdpartmap.txt
v1-5-4/5
  • added stuff
  • removed stuff
  • moved stuff
  • see v1-5-6
v1-5-3 : 2010-08-13
This should be last version we need v1-5-2 : 2010-08-0
  • Bug fixes
    • recovery was getting written to boot ( flash_image <partition> is now a variable )
    • typo in env variable was causing cache and data calculations to fail
  • AutoMTD now prints version number ( when flashing patched Zip )
  • tided up system "0x" 'fudge' ( to be compatible with trout/sapphire fall back )
  • removes temp files from memory when done

v1-5-1 : 2010-08-06
  • Bug fixes, had an extra '0x' on the system start + functions had wrong env var for the location of dmesg derived partition map
v1-5 : 2010-08-06
Version numbers brought into sync
  • Zipe Filename - 'reordered' ( easier to see version numbers in CWR )
  • AutoMTD Patcher - changes as per boot Patcher + zip recursion fix ( my bad habit, thanks go to mblaster for pointing this out )
  • Boot Patcher - cleaned up cache bind mount
    • Now supports leagcy /system/sd mount point
    • supports ROM Manager ( real cache partition mounted on /dev/cache, recovery dir symlinked from 'fake' to 'real' cache )
  • Recovery Patcher - can 'fake' your SPL ( see configuration )
  • Recovery Patcher - creates more noise ( advise wipe and reboot )
  • Single Patcher script ( so I don't have make the same changes to three different files that essentially do the same job )
  • No longer 'Hardcoded' to 32[a/b] Partition Layout ( figures out SPL layout via dmesg ) **
    uses original boot/recovery img's base configuration ( i.e. EBi0 and EBi1 compatible )


=< v1.4
  • Recovery Patcher v1.3, added SPL faker
  • Boot Patcher v1.2, fixed oversite where boot.img was not 'dumped'
  • Recovery Patcher v1.1 initial
  • Boot Patcher v1.1 ( was a fail, it didn't patch boot.img on CM roms as the tmp boot.img was deleted, my fault for just using the AutoMTD Patcher's script ( which runs before the tmp boot.img is deleted )
  • Boot Patcher v1 initial

Todo
  • - 2010-08-13 redundent ( launching AIO script ) - I might add some stuff to cm5/6's backup routine via the Auto patcher, things like the "All in One" installed scripts: 05mountsd and remount
  • windows compatible AutoMTD script ( meh, I hate batch scripts )
  • - 2010-08-13 DONE - thinking of adding a config option to launch the "all in one" script, but have to make that recovery compatible first


.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.: *~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~ *:._.:*~*:._.:*~*:._.:*~*:.
Downloads
.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.: *~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~*:._.:*~ *:._.:*~*:._.:*~*:._.:*~*:.

mediafire downloads
http://tinyurl.com/customMTD
aab0fadf658ed275954aea8d0aed9c8f FR-AutoMTD_partitionPatcher_v1.5.6.tar.bz2
8857194cdbe34a52d173def4441ad2ae FR-AutoMTD_partitionPatcher_v1.5.6.zip
1f84a5ec50684a7830a93a8d455bc159 FR-boot-rpp-v1.5.6-CustomMTD_S.zip
bca0360f91aed0acf6e2dc82dfe01b56 FR-boot-v1.5.6-CustomMTD_S.zip
94b4238c2668cbe7cd52fb8ad5a2ee12 FR-recovery-v1.5.6-CustomMTD_S.zip
5404f1a41dbc60105d59c7fa0c335a70 FR-remove-v1.5.6-CustomMTD_S.zip



The attached files are OLD
Last edited by Firerat; 11th December 2010 at 05:36 PM. Reason: changed example from 85 to 90 ( since we no longer have the default 90, and newbs are getting confused )
The Following 11 Users Say Thank You to Firerat For This Useful Post: [ View ]
4th July 2010, 07:06 PM   |  #2  
OP Account currently disabled
Thanks Meter: 193
 
3,848 posts
Join Date:Joined: Feb 2009
Donate to Me
More
Custom MTD FAQ
FAQ
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q my SPL starts with 0 or 10, and I want to try CM5.0.8 or/and CM6. DO I need to do anything extra?[/b]
A yeap, it seems the newer kernels are just a bit too big for x0.95.x00x SPLs, you can get round it with the below kernels. And to flash CM6 you need to 'fake' your SPL ( see configuration in OP ) or edit the updater-script. faking is easier.
However, I would recommend the 1.33.2003 SPL ( you still need to fake your SPL, but you won't need the 'special' kernels )
guide for flashing 1.33.2003 SPL by Ezterry

#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#
2010-07-11
I have compiled a kernel, which I hope is NoneDanger compatible
the source is simply CyanogenMod's github, I have taken the config from cm6rc1, and simply changed the bluetooth to modules
this approach has worked in the past
It is pre-patched with AutoMTD, so just flash cm6rc1, then flash this
FR-CM6RC1-bootimg4NoneD-AutoMTD.zip(MD5: 386D9A05A3C0FFC08E5B3F844D437AA7)
mirrors
http://rapidshare.com/files/40640201...eD-AutoMTD.zip
http://www.mediafire.com/?152jnqwyme3
#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#
2010-07-27
NoneDanger compatible Kernel for RC2
FR-CM6RC2-bootimg4NoneD-AutoMTD.zip (MD5: 7858a8a8d126919318d1718c6e5167ec )

http://www.mediafire.com/file/ttxfco...eD-AutoMTD.zip

I'll have to dig out the src
#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#
2010-08-15
NoneDanger compatible Kernel for RC3
2010-08-17 ( Sorry, old one had a status6 bug )
New one here
87F160F08FCD2233DDD40FBFC50D3711 FR-CM6RC3-bootimg4NoneD-AutoMTD.zip
src = http://github.com/CyanogenMod/cm-ker...4cfe2637251184
no modifications its straight cm ( besides the config, which you can get from the compiled kernel or zcat /proc/config.gz )

#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#*#
*NB* don't use a patched recovery to flash SPL or RADIO ( you should avoid using recovery to flash these anyway, feel free to ask for current advice on spl / radio flashing )

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q do I need to flash both of the files each time I install a new rom?

A No, you only need to flash the recovery patcher once, unless you want to resize or you install a new recovery.
the boot patcher *must* be flashed after you have installed a new ROM or Kernel update
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q my cache looks odd in df, I see two and its the same size as data or sd-ext, is something wrong?

A everything is fine, since we shrunk cache its no good for things like the Market, so a script is installed to 'bind mount' cache with /sd-ext/cache or data if sd-ext is not mounted.
it actually turns out that /cache is not actually used in CM6, so I might adapt the script a little in a future release.
EDIT: as of version 1.5 the cache bind mount script mount 'real cache' separately, which should reduce confusion
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q When I tell ROM Manger to do something in recovery, it just reboots to recovery and does nothing. it used to do the action but not since I installed CustomMTD

A yeah, nearly forgot about that, I was going to fix it last week, basically ROM Manger writes commands to /cache, but its writing it to our bind mount so.. when recovery boots it doesn't see the commands.
In all honesty that one hasn't been pointed out to me yet, but yeah I can fix it..
EDIT: as of version 1.5 the cache bind mount is compatible with ROM Manager
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Attached Files
File Type: zip FR-MTD-PartHack-recovery-v1-3_S.zip - [Click for QR Code] (250.5 KB, 543 views)
Last edited by Firerat; 17th August 2010 at 08:38 AM.
The Following 3 Users Say Thank You to Firerat For This Useful Post: [ View ]
4th July 2010, 07:07 PM   |  #3  
OP Account currently disabled
Thanks Meter: 193
 
3,848 posts
Join Date:Joined: Feb 2009
Donate to Me
More
The Scripts
'source' is now available on github

http://github.com/Firerat/CustomMTD

errm, tbh I'm not all that clued up on it yet
Last edited by Firerat; 23rd August 2010 at 02:11 PM.
The Following 3 Users Say Thank You to Firerat For This Useful Post: [ View ]
4th July 2010, 07:14 PM   |  #4  
asb123's Avatar
Senior Member
Flag Vancouver
Thanks Meter: 34
 
1,210 posts
Join Date:Joined: Dec 2009
More
looks very promising

So, will i need to change the values everytime i flash a different rom? or can i just set them to a large size and everything will work?
4th July 2010, 07:24 PM   |  #5  
OP Account currently disabled
Thanks Meter: 193
 
3,848 posts
Join Date:Joined: Feb 2009
Donate to Me
More
Quote:
Originally Posted by asb123

looks very promising

So, will i need to change the values everytime i flash a different rom? or can i just set them to a large size and everything will work?

most roms have a target of 90mb system ( DangerSPL )

so 90 2 config will basically give you an extra 28mb on data than you would have with stock DangerSPL MTD map

so yeah

90 2 is a good all rounder
4th July 2010, 07:48 PM   |  #6  
Ace42's Avatar
Senior Member
Flag New York
Thanks Meter: 1,132
 
9,183 posts
Join Date:Joined: Jul 2009
More
Quote:
Originally Posted by Firerat

most roms have a target of 90mb system ( DangerSPL )

so 90 2 config will basically give you an extra 28mb on data than you would have with stock DangerSPL MTD map

so yeah

90 2 is a good all rounder

is it okay if I make system 70mb?
The rom im using is only using 66 out of 90, i want more space for data instead. I remember you said something about it being a miltiple of 128K.
Last edited by Ace42; 4th July 2010 at 08:07 PM.
4th July 2010, 08:02 PM   |  #7  
asb123's Avatar
Senior Member
Flag Vancouver
Thanks Meter: 34
 
1,210 posts
Join Date:Joined: Dec 2009
More
With this, could Devs stop skimping on stuff like wallpapers and ringtones or additional apps, and surpass the 90MB danger spl mark? It would seem so. Also, I do not make roms so I do not know how it works but if there is compression or lower quality stuff they would now be able to use up as much space as wanted correct?
4th July 2010, 08:06 PM   |  #8  
OP Account currently disabled
Thanks Meter: 193
 
3,848 posts
Join Date:Joined: Feb 2009
Donate to Me
More
Quote:
Originally Posted by Ace42

is it okay if I make system 70mb?
The rom im using is only using 66 out of 90, i want more space for data instead.

yeap, should be fine

personally I would be tempted to use clockwork, since you can erase system ( part of partition options )

.img files can be found here
http://www.koushikdutta.com/2010/02/...ery-image.html

I don't recommend flashing via ROM Manager with CM5 or CM6

fastboot it over, or flash via recovery


I may be a little over cautious recommending a full wipe, but I have never had problems when system, cache and data are clean.

if you are 'growing' system make sure cache and data are clean
if your 'shrinking' make sure system is clean ( so 'hangovers don't mess up /cache or data )
4th July 2010, 08:10 PM   |  #9  
OP Account currently disabled
Thanks Meter: 193
 
3,848 posts
Join Date:Joined: Feb 2009
Donate to Me
More
Quote:
Originally Posted by asb123

With this, could Devs stop skimping on stuff like wallpapers and ringtones or additional apps, and surpass the 90MB danger spl mark? It would seem so. Also, I do not make roms so I do not know how it works but if there is compression or lower quality stuff they would now be able to use up as much space as wanted correct?

yes, you can 'grow' or 'shrink' at will

just hope it doesn't get used to be lazy and not trim bloat
4th July 2010, 08:41 PM   |  #10  
Senior Member
Thanks Meter: 124
 
511 posts
Join Date:Joined: Sep 2009
Hi firerat,
you mentioned once before something about market data on the cahe... or something so downloads would be ok if a certain partition was big enough?....something like that... I am having a prob that might be related since it started when I tried to change from default to system 80 5 for data. now I cant sign in to google and after flashing gaaps there is no market? everything else in the gaaps zip is there...? It could be google i know but i remembered you saying that somewhere.

Post Reply Subscribe to Thread

Tags
cm6.1, custom mtd, dream, froyo, low storage space
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes