K, time to give this a proper OP, if anyone wants any of the info that was here before you can look here
_________________________________________________________________________________________________
So, the whole idea here started with me reading an article on how part of the whole I/O problem with the transformer is partially caused by the hardware used as internal storage. I wanted to find out if this had any merit and I figured the best way to do it would be to "replace" the internal storage. I did this by mounting the /data partition to the exteral SD (which according to my research, my specific SD Card is better at writing speeds - allegedly the main problem with the transformer's internal storage hardware wise). Then I ran a bunch of benchmarks and have been running it that way for about 24 hours and so far it feels great. Anyone is welcome to give it a try, and hopefully with help, suggestions and feedback from the community, we can all take as much advantage of this idea as possible.
Before I go any further I want to give credit to those who helped me so far, because without them I would still be completely clueless, and not only have they helped be accomplishing what I got so far, but thanks to them I've also learned a bunch of things I didn't know before. So here it goes:
Rayman - For suggesting the method for mounting /data to the external SD.
lilstevie - For helping me get the new kernel flashed right.
Turge - For showing me how to properly repack the kernel.
Parastie - For suggesting doing the same thing to /cache (working on that now).
dagrim1 - For SQL patch and for suggesting a temporary remount (even though it didn't work it was a good thing to try).
_motley - For all his work on his awesome kernel.
_________________________________________________________________________________________________
Updates:
Update 1 (08-01-2012 File boot-data+cache+internal-AOKP6.1base.zip) :
Now both /data and /cache are moved to the external SD card. This means you need a third partition mmcblk1p3 in order to use this modification.
It will also mount the internal storage (previously inaccessible) to /mnt/sdcard_internal
It also attempts to (fail at this point) to mount the internal sd partition (what used to be /sdcard) to /sdcard/Internal_SD (which is why you will always see that folder get created but stay empty). If anybody knows how to make it work please advise.
Modified Lines:
init.cardhu.rc
Update 2 (08-01-2012 About backing up/restoring in recovery) : - READ UPDATE 5
One thing that worried me was that by using this mod people wouldn't be able to backup their data partition properly, but now I know that it's possible to do it. It will only work on TWRP though since it has basic terminal access and keyboard. To do it, go into Advanced > Terminal and in there type:
umount /dev/block/mmcblk0p8
mount /dev/block/mmcblk1p2 /data
And until you reboot, any backup/restore should use the external SD data partition instead of the internal. The same should be doable with the cache partition in case you want to backup/restore that.
Update 3 (08-02-2012 File flashme-kernel-motley305-aokp-data+cache2SD.zip) :
Put together a flashable zip that will install motley's 3.0.5 aokp kernel using this mod. Works like a charm so far though I only tried flashing on TWRP. Also, internal storage can be accessed in /data2 and internal sd can be accessed in /sdcardi . Current changes are as follow:
Update 4 (08-03-2012 File boot-cm10-unofficial-data+internal.zip) :
Running the unofficial CM10 (no cherrypicks one) using this mod and so far it's pretty amazing. The rom itself is pretty stable and even snappier with /data mounted to external SD. Benchmarks are at the bottom. Current modifications:
fstab.cardhu:
init.cardhu.rc:
Update 5 (08-03-2012 About backing up/restoring in recovery) :
So after doing some tests, and paying more attention to TWRP, I noticed something rather useful:
When you have this mod enabled, or whenever you have a mmcblk1p2 partiion, TWRP will have the sd-ext menu enabled. This means that to backup your data you can simply backup the sd-ext partition and to restore your data you can simply restore your sd-ext partition. No need to worry about manually switching the mount point for /data in recovery. I guess it was a whole lot easier than I thought.
Also congratulations and thanks to everyone who has contributed with this so far.
WE MADE IT TO FRONT PAGE ON XDA (08-03-2012)
_________________________________________________________________________________________________
Requirements:
There are a few things you will need to do in order for this to work right for you, and a couple of things you'll have to research before you even try it.
#1. Obviously, you have to be rooted/unlocked because you're not gonna be able to change much around otherwise.
#2. You MUST repartition your external SD. The kernel I've put together so far WILL ONLY mount /data to mmcblk1p2, which basically says "mount /data to the second partition in the external SD." also, the ramdisk expects that partition to be ext4, so essentially:
Make sure you have an external SD with at least two partitions and that the second partition is formatted to ext4. I personally use Gparted to repartition my stuff, but feel free to use whatever rocks your boat. Even if you're on windows you can still use gparted by using virtualbox, so I'm not gonna go look for a different windows solution.
#3. This is the research part... This will be beneficial or detrimental to each user depending on the SD card used. If you have a slow SD card this probably will do you no good. However, just because you have a class 10 SD card, that doesn't mean it will benefit you either. On my own research I have found that some class 6-10 SD Cards have extremely slow random write speeds, so if you happen to have one of those, even if it's a class 10, this might not be for you. This means that you're gonna have to do some research to find out if your SD Card will benefit you or not. You can always just give it a try, as far as I know this is entirely reversible, how easy or hard being just a matter of how bad you mess up on meeting the requirements and following the instructions.
#4. At this point (07-30-2012) I'm doing all this stuff using the AOKP milestone 6.1 kernel as base for my modified kernel, so if you're not using AOKP milestone 6.1, flashing my kernel might borke your system. You've been warned, feel free to proceed otherwise at your own peril.
_____________________________________________________________________________________________________________
Installation:
#1. Download attached file (boot-data2SD-AOKP6.1base.zip) and extract it to the root of your internal storage (/sdcard).
#2. Open a terminal.
#3. Type the following:
#4. Wait for it to complete.
#5. Reboot.
Upon rebooting you will know that it worked because it will look just as if you just flashed a new rom, that is, you'll get the device setup screen (assuming that the tablet booted at all lol). If you're planning to use TB to restore your apps, you'll probably want to copy the TB folder to your external SD's first partition so that you can copy it back once you're done with the device setup (at this point you will have no access - unless you manually mount it - to your internal storage).
_______________________________________________________________________________________
Reverting:
Follow the same exact steps for installation but use boot-default-AOKP6.1.zip instead.
_______________________________________________________________________________________
Optional:
#1. If you want to have access to everything you had on your data partition in the new data partition, you'll have to clone everything from one to the other. To do this, make sure that your new data partition (the one in your external SD) has enough storage space to fit everything you currently have in your data partition (the one in your internal storage). Then run the following command in your terminal.
BEWARE that if you have a lot of stuff this can take quite a while and even though I've read a way of getting the progress for this in Linux I'm not sure that you can check the progress on Android.
_______________________________________________________________________________________
Next steps in development:
#1. Move /cache as well.
#2. Find out what happens with recovery backups when the partitions are changed.
#3. Attempt to apply mod to motley's kernel.
#4. Create a script that is run on boot to eliminate need for replacing the kernel.
#5. With help from the community, find the best SD Card for this.
#6. Run the modified system for a while to have a good feel for performance benefits
#7. Come up with other interesting uses for this other than getting better I/O (maybe an easy - kinda easy - way to dual boot with ubuntu, maybe other stuff, dunno).
_______________________________________________________________________________________
How can I set my kernel to do this?
I didn't do a whole lot, and it's not like I want it to be a secret, so as I modify things I'll try to keep the steps listed here so that anyone modifying their own kernel who would like to try this modification can go ahead and do it.
You'll need to know how to unpack/repack a kernel. Turge has a SUPER EASY explanation here on how to do it on windows (I'll pack together the necessary binaries for linux later, maybe).
Mount /data to the second partition in your external SD (formatted as ext4 filesystem):
After unpacking the kernel navigate to the folder that has the ramdisk and open it
(DON'T USE ANY ASCII BASED TEXT EDITOR BECAUSE IT WILL PROBABLY MESS THINGS UP I USE NOTEPAD++)
Around line 26 change:
to
_______________________________________________________________________________________________________
SD Cards tested:
Samsung 32GB Class 10 MicroSDHC High Speed Memory Card - Very Good Results
SanDisk® microSDHCTM 8GB Memory Card - Very Good Results
"Either way, with this mod, the tablet feels like it should have right from the start. It's speedy and responsive, and apps being installed don't stall the system." - Turge - Post #59
_______________________________________________________________________________________________________
Benchmarks:
/data mounted to mmcblk0p8 (Internal Storage):
/data mounted to mmcblk1p2 (External SD):
RL Benchmark WITHOUT dagrim1's sql patch as per request:
/data mounted to mmcblk0p8 (Internal Storage):
/data mounted to mmcblk1p2 (External SD):
*Important thing to note: When running the benchmark with data in the internal SD it was about 220 seconds in the first run, then about 180, then about 130 and finally 118; whereas running it from the external SD was consistenly between 63 and 65 seconds every time. I think this more than proves that A) Asus used a cheap I/O storage, B) No matter what software changes are made, more than likely running the rw partitions from a better I/O storage, i.e. an external SD is a good idea.
As promised here are benchmarks on a JB rom (Unofficial CM10). Also sorry it took me a while to get these, I was going to use eos3 but I started getting random reboots. Then I decided to try cm10, and I messed up a flash and had to redo a bunch of things. Anyway, the only change here is the mod itself (no custom kernel or anything). Though one thing to note is that I moved /cache back to the internal partition after some thought that this allows /data and /cache to be written at the same time to different locations thus lowering the bottleneck.
/data mounted to mmcblk0p8 (internal storage):
/data mounted to mmcblk1p2 (external storage):
Now, as you can see, JB did bring a major improvement to I/O, bringing the benchmark down from about 115sec to 68 (almost reaching the modded ICS at 60 seconds). But as I expected, better software works better on better hardware and now the modded JB is running at 50 seconds instead of 60. Next I'm going to put dagrim1's sql patch and see how low the benchmark goes. Also will be posting the modded blob in just a little bit for anyone who wants to use it on CM10.
_________________________________________________________________________________________________
So, the whole idea here started with me reading an article on how part of the whole I/O problem with the transformer is partially caused by the hardware used as internal storage. I wanted to find out if this had any merit and I figured the best way to do it would be to "replace" the internal storage. I did this by mounting the /data partition to the exteral SD (which according to my research, my specific SD Card is better at writing speeds - allegedly the main problem with the transformer's internal storage hardware wise). Then I ran a bunch of benchmarks and have been running it that way for about 24 hours and so far it feels great. Anyone is welcome to give it a try, and hopefully with help, suggestions and feedback from the community, we can all take as much advantage of this idea as possible.
Before I go any further I want to give credit to those who helped me so far, because without them I would still be completely clueless, and not only have they helped be accomplishing what I got so far, but thanks to them I've also learned a bunch of things I didn't know before. So here it goes:
Rayman - For suggesting the method for mounting /data to the external SD.
lilstevie - For helping me get the new kernel flashed right.
Turge - For showing me how to properly repack the kernel.
Parastie - For suggesting doing the same thing to /cache (working on that now).
dagrim1 - For SQL patch and for suggesting a temporary remount (even though it didn't work it was a good thing to try).
_motley - For all his work on his awesome kernel.
_________________________________________________________________________________________________
Updates:
Update 1 (08-01-2012 File boot-data+cache+internal-AOKP6.1base.zip) :
Now both /data and /cache are moved to the external SD card. This means you need a third partition mmcblk1p3 in order to use this modification.
It will also mount the internal storage (previously inaccessible) to /mnt/sdcard_internal
It also attempts to (fail at this point) to mount the internal sd partition (what used to be /sdcard) to /sdcard/Internal_SD (which is why you will always see that folder get created but stay empty). If anybody knows how to make it work please advise.
Modified Lines:
init.cardhu.rc
Code:
# TweakerL MOD > original mount = mmcblk0p8 /data | mmcblk0p3 /cache
mount ext4 /dev/block/mmcblk1p2 /data wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
mount ext4 /dev/block/mmcblk1p3 /cache wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
# TweakerL MOD > added mounts for internal storage
mkdir /mnt/sdcard_internal 0000 system system
mount ext4 /dev/block/mmcblk0p8 /mnt/sdcard_internal wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
# TweakerL MOD > give access to internal SD from /sdcard
mkdir /data/media/Internal_SD 0755 media_rw media_rw
mount /mnt/sdcard_internal/media /data/media/Internal_SD wait noatime nodiratime nosuid nodev nodelalloc,errors=panicUpdate 2 (08-01-2012 About backing up/restoring in recovery) : - READ UPDATE 5
One thing that worried me was that by using this mod people wouldn't be able to backup their data partition properly, but now I know that it's possible to do it. It will only work on TWRP though since it has basic terminal access and keyboard. To do it, go into Advanced > Terminal and in there type:
umount /dev/block/mmcblk0p8
mount /dev/block/mmcblk1p2 /data
And until you reboot, any backup/restore should use the external SD data partition instead of the internal. The same should be doable with the cache partition in case you want to backup/restore that.
Update 3 (08-02-2012 File flashme-kernel-motley305-aokp-data+cache2SD.zip) :
Put together a flashable zip that will install motley's 3.0.5 aokp kernel using this mod. Works like a charm so far though I only tried flashing on TWRP. Also, internal storage can be accessed in /data2 and internal sd can be accessed in /sdcardi . Current changes are as follow:
Code:
# TweakerL MOD > move /data and /cache to external SD card || original mount = mmcblk0p8 /data | mmcblk0p3 /cache
mount ext4 /dev/block/mmcblk1p2 /data wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
mount ext4 /dev/block/mmcblk1p3 /cache wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
# TweakerL MOD > create mount for internal storage
mkdir /data2 0000 system system
mount ext4 /dev/block/mmcblk0p8 /data2 wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
# TweakerL MOD > symlink internal sd to a couple of easily accessible locations
symlink /data2/media /mnt/sdcard_internal
symlink /data2/media /sdcardiUpdate 4 (08-03-2012 File boot-cm10-unofficial-data+internal.zip) :
Running the unofficial CM10 (no cherrypicks one) using this mod and so far it's pretty amazing. The rom itself is pretty stable and even snappier with /data mounted to external SD. Benchmarks are at the bottom. Current modifications:
fstab.cardhu:
Code:
#TweakerL MOD > Move /data to external SD and internal /data to /data2
/dev/block/mmcblk1p2 /data ext4 rw,nosuid,nodev,noatime,user_xattr,acl,barrier=1,journal_async_commit,nodelalloc,data=writeback wait
/dev/block/mmcblk0p8 /data2 ext4 rw,nosuid,nodev,noatime,user_xattr,acl,barrier=1,journal_async_commit,nodelalloc,data=writeback waitinit.cardhu.rc:
Code:
# TweakerL MOD > create mount for internal storage
mkdir /data2 0000 system system
mount_all /fstab.cardhu
# TweakerL MOD > symlink internal sd to a couple of easily accessible locations
symlink /data2/media /mnt/sdcard_internal
symlink /data2/media /sdcardiUpdate 5 (08-03-2012 About backing up/restoring in recovery) :
So after doing some tests, and paying more attention to TWRP, I noticed something rather useful:
When you have this mod enabled, or whenever you have a mmcblk1p2 partiion, TWRP will have the sd-ext menu enabled. This means that to backup your data you can simply backup the sd-ext partition and to restore your data you can simply restore your sd-ext partition. No need to worry about manually switching the mount point for /data in recovery. I guess it was a whole lot easier than I thought.
Also congratulations and thanks to everyone who has contributed with this so far.
WE MADE IT TO FRONT PAGE ON XDA (08-03-2012)
_________________________________________________________________________________________________
Requirements:
There are a few things you will need to do in order for this to work right for you, and a couple of things you'll have to research before you even try it.
#1. Obviously, you have to be rooted/unlocked because you're not gonna be able to change much around otherwise.
#2. You MUST repartition your external SD. The kernel I've put together so far WILL ONLY mount /data to mmcblk1p2, which basically says "mount /data to the second partition in the external SD." also, the ramdisk expects that partition to be ext4, so essentially:
Make sure you have an external SD with at least two partitions and that the second partition is formatted to ext4. I personally use Gparted to repartition my stuff, but feel free to use whatever rocks your boat. Even if you're on windows you can still use gparted by using virtualbox, so I'm not gonna go look for a different windows solution.
#3. This is the research part... This will be beneficial or detrimental to each user depending on the SD card used. If you have a slow SD card this probably will do you no good. However, just because you have a class 10 SD card, that doesn't mean it will benefit you either. On my own research I have found that some class 6-10 SD Cards have extremely slow random write speeds, so if you happen to have one of those, even if it's a class 10, this might not be for you. This means that you're gonna have to do some research to find out if your SD Card will benefit you or not. You can always just give it a try, as far as I know this is entirely reversible, how easy or hard being just a matter of how bad you mess up on meeting the requirements and following the instructions.
#4. At this point (07-30-2012) I'm doing all this stuff using the AOKP milestone 6.1 kernel as base for my modified kernel, so if you're not using AOKP milestone 6.1, flashing my kernel might borke your system. You've been warned, feel free to proceed otherwise at your own peril.
_____________________________________________________________________________________________________________
Installation:
#1. Download attached file (boot-data2SD-AOKP6.1base.zip) and extract it to the root of your internal storage (/sdcard).
#2. Open a terminal.
#3. Type the following:
Code:
su
dd if=/sdcard/boot.blob of=dev/block/mmcblk0p4 bs=1#4. Wait for it to complete.
#5. Reboot.
Upon rebooting you will know that it worked because it will look just as if you just flashed a new rom, that is, you'll get the device setup screen (assuming that the tablet booted at all lol). If you're planning to use TB to restore your apps, you'll probably want to copy the TB folder to your external SD's first partition so that you can copy it back once you're done with the device setup (at this point you will have no access - unless you manually mount it - to your internal storage).
_______________________________________________________________________________________
Reverting:
Follow the same exact steps for installation but use boot-default-AOKP6.1.zip instead.
_______________________________________________________________________________________
Optional:
#1. If you want to have access to everything you had on your data partition in the new data partition, you'll have to clone everything from one to the other. To do this, make sure that your new data partition (the one in your external SD) has enough storage space to fit everything you currently have in your data partition (the one in your internal storage). Then run the following command in your terminal.
Code:
dd if=dev/block/mmcblk0p8 of=dev/block/mmcblk1p2 bs=4096 conv=notrunc,noerrorBEWARE that if you have a lot of stuff this can take quite a while and even though I've read a way of getting the progress for this in Linux I'm not sure that you can check the progress on Android.
_______________________________________________________________________________________
Next steps in development:
#1. Move /cache as well.
#2. Find out what happens with recovery backups when the partitions are changed.
#3. Attempt to apply mod to motley's kernel.
#4. Create a script that is run on boot to eliminate need for replacing the kernel.
#5. With help from the community, find the best SD Card for this.
#6. Run the modified system for a while to have a good feel for performance benefits
#7. Come up with other interesting uses for this other than getting better I/O (maybe an easy - kinda easy - way to dual boot with ubuntu, maybe other stuff, dunno).
_______________________________________________________________________________________
How can I set my kernel to do this?
I didn't do a whole lot, and it's not like I want it to be a secret, so as I modify things I'll try to keep the steps listed here so that anyone modifying their own kernel who would like to try this modification can go ahead and do it.
You'll need to know how to unpack/repack a kernel. Turge has a SUPER EASY explanation here on how to do it on windows (I'll pack together the necessary binaries for linux later, maybe).
Mount /data to the second partition in your external SD (formatted as ext4 filesystem):
After unpacking the kernel navigate to the folder that has the ramdisk and open it
(DON'T USE ANY ASCII BASED TEXT EDITOR BECAUSE IT WILL PROBABLY MESS THINGS UP I USE NOTEPAD++)
Around line 26 change:
Code:
mount ext4 /dev/block/mmcblk0p8 /data wait noatime nodiratime nosuid nodev nodelalloc,errors=panic
Code:
mount ext4 /dev/block/mmcblk1p2 /data wait noatime nodiratime nosuid nodev nodelalloc,errors=panicSD Cards tested:
Samsung 32GB Class 10 MicroSDHC High Speed Memory Card - Very Good Results
SanDisk® microSDHCTM 8GB Memory Card - Very Good Results
"Either way, with this mod, the tablet feels like it should have right from the start. It's speedy and responsive, and apps being installed don't stall the system." - Turge - Post #59
_______________________________________________________________________________________________________
Benchmarks:
/data mounted to mmcblk0p8 (Internal Storage):
/data mounted to mmcblk1p2 (External SD):
RL Benchmark WITHOUT dagrim1's sql patch as per request:
/data mounted to mmcblk0p8 (Internal Storage):
/data mounted to mmcblk1p2 (External SD):
*Important thing to note: When running the benchmark with data in the internal SD it was about 220 seconds in the first run, then about 180, then about 130 and finally 118; whereas running it from the external SD was consistenly between 63 and 65 seconds every time. I think this more than proves that A) Asus used a cheap I/O storage, B) No matter what software changes are made, more than likely running the rw partitions from a better I/O storage, i.e. an external SD is a good idea.
As promised here are benchmarks on a JB rom (Unofficial CM10). Also sorry it took me a while to get these, I was going to use eos3 but I started getting random reboots. Then I decided to try cm10, and I messed up a flash and had to redo a bunch of things. Anyway, the only change here is the mod itself (no custom kernel or anything). Though one thing to note is that I moved /cache back to the internal partition after some thought that this allows /data and /cache to be written at the same time to different locations thus lowering the bottleneck.
/data mounted to mmcblk0p8 (internal storage):
/data mounted to mmcblk1p2 (external storage):
Now, as you can see, JB did bring a major improvement to I/O, bringing the benchmark down from about 115sec to 68 (almost reaching the modded ICS at 60 seconds). But as I expected, better software works better on better hardware and now the modded JB is running at 50 seconds instead of 60. Next I'm going to put dagrim1's sql patch and see how low the benchmark goes. Also will be posting the modded blob in just a little bit for anyone who wants to use it on CM10.
Attachments
Last edited:
