FORUMS
Remove All Ads from XDA

[DEV][DATA2SD] More space (and more I/O) for your Desire (Updated/EXT4/Flashable ZIP)

2,055 posts
Thanks Meter: 376
 
By sibere, Retired Recognized Developer on 26th November 2010, 01:02 PM
Post Reply Email Thread
HI all.

I know it has been already done but I didn't find any detailed tutorial for the hTc Desire so I wrote one for you

Howto for a HTC desire with clockworkmod recovery:
I did this with a stock HTC sense 2.2 froyo rooted rom with init.d support and my own kernel, forked from richardtrip sense kernel v5e. I don't know for other ROMs or Kernels. Your ROM must support init.d boot scripts.

What you need is:

* Fast µSD card (Class 4 or 6 recommended. Lots of issues reported with class 10 on the Desire)
* busybox (http://www.busybox.net/downloads/binaries/) on c:\
* Kernel with EXT4 support. The kernel in original ROM does not provide ext4 support!!
* Rom with init.d script support (StarBurst (Froyo) or Supernova (Gingerbread) have a native DATA2SD support)
* If you already have app2sd+ or a similar script, don't do the following hack (or at your own risks) !!
* If you still want to try it, backup your A2SD scripts and roll-back to a pre-A2SD state. Don't ask me why it's not working, I never used APPs2SD so I can't help.
* To help you with ADB, you can use the GUI

If you feel a bit lazy or you're scared to make any mistake and want to skip all the manual installation, then go straight to the bottom of this post and download the flashable script made by droidzone. Many thanks to him!! You can also flash Supernova (Stock HTC gingerbread release) and you'll be good to go within no time.

If you already have this running but you flashed another rom, have a look here. droidzone posted another script to just update the rom you just flashed to get DATA2SD work properly

1/Get yourself prepared

A/ backup everything (nandroid).
B/ Make sure everything is backed-up
C/ Copy the content of you SD to your computer and keep it handy for a roll-back.
D/ make a copy !!!! (I think know that you've been warned )

2/ Make 2 primary Mb aligned partitions on your SD

Primary partition 1 will be your usual storage, must be FAT32.
To optimize it, I really recommend to have a look here
Primary partition 2 will be your new storage (/data).
You can use gparted or fdisk (I do prefer fdisk and wrote a guide here ) to partition your SD; but it's up to you and Gparted is working great too.
Use the MB align option of gparted to align your partitions. This helps big time to get a better R/W speed.
With the latest busybox, you can do that on your phone in recovery mode.
You will need also e2fs progs. They're all attached at the bottom of this howto.

3/ Boot your phone in recovery and get it prepared

Connect to your phone with "adb shell"
You should get a ~# prompt now.

mount /system
mount /data
mount /sdcard
exit

if mount /sdcard fails, try mount /dev/block/mmcblk0p1 /sdcard

You should get this as a result of a "mount" command:
Code:
~ # mount
mount
rootfs on / type rootfs (rw)
tmpfs on /dev type tmpfs (rw,relatime,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
/dev/block/mtdblock4 on /cache type yaffs2 (rw,nodev,noatime,nodiratime)
/dev/block/mmcblk0p1 on /sdcard type vfat (rw,relatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,s
hortname=mixed,errors=remount-ro)
/dev/block/mtdblock3 on /system type yaffs2 (rw,relatime)
/dev/block/mtdblock5 on /data type yaffs2 (rw,relatime)
Replace you old busybox exec with the new one. It's located in /system/xbin/
adb push c:\busybox-armv6l /system/xbin/busybox

Add the script to init.d folder:
adb push c:\01data.txt /system/etc/init.d/01data

Add the fstab to /system/etc/
adb push c:\fstab.txt /system/etc/fstab

Extract e2fsprogs-arm.zip to c:\

Push all e2fs progs you've extracted from the zip to /system/bin/ the same way:
adb push c:\e2fsprogs\system\bin\e2fsck /system/bin/
adb push c:\e2fsprogs\system\bin\mke2fs /system/bin/
(etc.... for all apps)

Push all e2fs libs you've extracted from the zip to /system/lib/ the same way:
adb push c:\e2fsprogs\system\lib\libext2_blkid.so /system/lib/
adb push c:\e2fsprogs\system\lib\libext2_com_err.so /system/lib/
(etc... for all libs)

adb shell
cd /system/bin
chmod 755 e2fsck
chown 0.2000 e2fsck
(Do the same for all pushed apps.)
cd /system/lib
chmod 644 libext2*

chmod 755 /system/xbin/busybox
chown 0.0 /system/xbin/busybox
chmod 755 /system/etc/init.d/01data
chown 0.0 /system/etc/init.d/01data

Finally type this:
:> /system/etc/mtab

4/ Instant of creation

cd /

For a non-journaled version:

/system/bin/mke2fs -T ext4 -b 4096 -E stride=64,stripe-width=64 -O ^has_journal,extent,^huge_file -m 0 -L userdata /dev/block/mmcblk0p2

For a journaled version (safety/AOSP ROMs):

/system/bin/mke2fs -T ext4 -b 4096 -E stride=64,stripe-width=64 -O extent,^huge_file -m 0 -L userdata /dev/block/mmcblk0p2

stride and stripe-width are RAID options but they are used here to align the filesystem to the max erase block size of an SD card. In conjunction with EXT4 extents feature (not present in ext3), it gives a bit of extra performance thanks to block allocation. It's used and recommended for SSDs, so it should help our poor little cheap (SSDs) SD cards
I use 64 because 64x4 (4kb block size)=256Kb. 256Kb is the max erase block size per the SD standards.
Your card may have a smaller erase block size but 256K is at least compatible with every card.
If you want to change the block allocation size, change the "64" parameter.
Just change it to - say - 32 (32x4=128kb) etc ...
Tune this to your actual erase block size if you know it.

If you want journaling (better data integrity) at the cost of performance and probably sd card wear, use the journaled option when you format the ext4 partition. Journal will also use some space on your EXT partition.

Read this article or my tutorial for more details on aligning partition and file system on linux.

I'd like to mention at this stage that fortunately, HTC has implemented a nice and clean unmount of /data at shutdown. Unless you remove your phone battery while it is running or android completely crashes and the cache hasn't been flushed to the sd card, you shouldn't suffer from data loss. In that case though, I've implemented a forced check of the data partition to try to recover any errors on the ext4 partition before booting the phone. You can expect to have a fair amount of safety without journal as long as you turn off your phone with standard HTC menu on sense ROMs
Anyway, if you want to try DATA2SD on a AOSP ROM, you *must* use a journal!


5/ verbatim

type this:

mkdir /sd
/system/xbin/busybox mount -t ext4 /dev/block/mmcblk0p2 /sd
cd /data
cp -a * /sd
cd /
sync

All you data should copied now.
check with a "ls -l /sd" that you have a least the directory structure.
"df" should report the space used on the new filesystem and should be close to your /data.

/system/xbin/busybox umount /sd

6/ Reboot

Reboot.
Move all you apps (if you have used the Froyo AP2SD) back to the phone, which is now the SD.
Reboot to get a clean and stable phone
Make sure that you don't install any app in the future to the SD. Some apps are installed automatically to SD, move them back to the phone if it is so.

7/ Comments

IMHO, a readahead of 128kb could be a bit excessive and can penalise read speed when reading small amounts of data. I've reduced it on my phone to 64kb and noticed a slight improvement when for instance the phone reads data to show apps, icons and size when you open the application list in the settings/applications/manage applications menu. Some kernels have a default value of 4K. This is really bad for performance and it must be changed to get good read performance.

To change it, type this while android is running:
echo 64 > /sys/block/mmcblk0/queue/read_ahead_kb

This setting will be lost on reboot. Just add this line to the script if you want to keep it on reboot.

ownhere did a fantastic job at hacking the libsqlite.so library. I patched the original sense lib and I strongly recommend to replace it with the one attached below.
To do so, just boot into recovery mode, mount /system and backup your /system/lib/libsqlite.so file to your sdcard. Then push the patched lib to /system/lib/ and do a chmod 644 on the pushed file.

Reboot and VOILA!

Updates of the flashable ZIP are available here

If you're coming from previous version (2 FAT32 partitions, ext2 file and loop mount) and want to move to this new one, have a look at this post. You don't have to do everything from start and loose your data. You should get there all the needed info to backup your data and restore it to the new EXT4 partition.

ownhere posted a very interesting option to play with in this post. Apparently nodelalloc mount option is giving better score than delalloc on flash memories. I recently tested it and I had better Quadrant and TAP scores so check-it out!

If you want to impress your friends with Quadrant score, just add "noauto_da_alloc" in the mount option of the script or do this manually with adb shell : busybox mount -o remount,noauto_da_alloc /data
The effect is pretty similar on the final score than nodelalloc (at least on my phone/sdcard )
To get back to normal: busybox mount -o remount,auto_da_alloc /data
(Got this setting from here).
It is simply making the system (thus Quadrant) working in cache (RAM) instead of the SD card.
I'm not sure that this settings improves that much real life usage, but at least you'll get what you're looking for: A super (fake) Quadrand I/O score
I may add this setting in future release of DATA2SD. I'm testing it ATM on my phone.

Many thanks to ownhere and melethron for all their thinkings at bringing issues, help and updates to get to this new version.
Many thanks to too droidzone for his great work on a flashable ZIP and integrating DATA2SD in his ROM to make your life easier you flash maniacs


I hope that I didn't forget something. It's working for me so it should work for you.
Read the thread before asking a question !
Attached Files
File Type: txt fstab.txt - [Click for QR Code] (80 Bytes, 1496 views)
File Type: zip e2fsprogs-arm.zip - [Click for QR Code] (711.6 KB, 1809 views)
File Type: zip libsqlite-patched.zip - [Click for QR Code] (232.5 KB, 1196 views)
File Type: txt 01data.txt - [Click for QR Code] (279 Bytes, 1695 views)
File Type: zip busybox.zip - [Click for QR Code] (589.7 KB, 1712 views)
The Following 26 Users Say Thank You to sibere For This Useful Post: [ View ] Gift sibere Ad-Free
 
 
26th November 2010, 02:02 PM |#2  
Senior Member
Thanks Meter: 31
 
More
This sounds really great ... 2 things :

1. your links are broken
2. Can't we generate the file for /data on the first fat32 partition ... would be a lot easier to format the sdcard with only one partition ...

thank you very much for this great information ...
26th November 2010, 02:04 PM |#3  
sibere's Avatar
OP Retired Recognized Developer
Flag Paris
Thanks Meter: 376
 
More
Quote:
Originally Posted by fogbav

This sounds really great ... 2 things :

1. your links are broken
2. Can't we generate the file for /data on the first fat32 partition ... would be a lot easier to format the sdcard with only one partition ...

thank you very much for this great information ...

No because you won't be able to use your SD as an external drive when connected to USB.

Just checked the links and they're OK
26th November 2010, 02:14 PM |#4  
Senior Member
Thanks Meter: 36
 
More
i vouch for sibere's script... i'm one of first "live testers" for his script and it works great and very, very stable

and from what have been tested by several testers , it can work in these roms :

without a2sd+ :
- rooted stock
- leedroid no-a2sd

with a2sd+ :
- auraxtsense
- defrost
- leedroid
- alex-v

Sibere, allow me to add notes from AndroHero on using the script with a2sd+ rom :
- remove the data2sd (or in auraxt, stuff2sd) from system/etc/init.d in the rom zipfile, or if you have install the rom, just remove from system/etc/init.d
- do the sibere script

Regards
BL
26th November 2010, 02:24 PM |#5  
mattiadj's Avatar
Senior Member
Flag Francavilla al mare
Thanks Meter: 1,774
 
Donate to Me
More
I have follow this easyest tutorial and i'm ok.
All credits to AndroHero for tutorial and to sibere for the hack! I'm only copy and paste from the previous thread!
This tutorial is for all app2sd sense rom so you can follow this tutorial not onfly for lee droid.
As attachment the file that tutorial need.
I have made a 2gb partition for data and is an awesome to see 1.80gb internal memory at first boot!!!!
WOOOOW

Now let's start with tutorial!

------TUTORIAL BEGIN-------

WHAT YOU NEED:
1)Root
2)Android SDK
3)15minutes of time

ALWAYS MAKE A NANDROID BACKUP AND SD BACKUP!!!


LET'S START NOW!

1) download leedroid 2.2f (apps2sd version)

2) unzip the .zip file, open the folder, open the system folder, and then inside that open the folder called etc, you will see a folder called init.d, open that and delete the file called 40a2sd, now just navigate back to the root of the rom and re-zip it.

3) now you need to partition your sd card, use gparted to make 2 fat partitions, a large primary partition and a secondary one about 1gb in size.

4) once you have done that copy the leedroid.zip to the sd card, boot into recovery, do all your wipes and flash the rom, one fashed boot it up and set it up as normal.

5) once set up, hold the power button and choose the option to reboot into recovery.

6) once in recovery connect your phone to your pc, open a command prompt and navigate to the tools folder inside the android SDK.

7) now in the command prompt type the following one at a time:

adb shell
mount /system
mount /data
mount /sdcard
exit

8) now leave the command prompt open and download the file at the bottom of the screen, (i have had to zip it, just open the .zip and the 01data file is inside) and copy it to the root of the c:\ drive on your pc.

9) in the command prompt type (one at a time and always wait for the # to appear again before entering a new command)

adb push c:\01data /system/etc/init.d
adb shell
chmod 755 /system/etc/init.d/01data
cd /
mount /dev/block/mmcblk0p2 /sd-ext
dd if=/dev/zero of=/sd-ext/ext2 bs=1048576
/system/xbin/busybox mke2fs -F -L userdata /sd-ext/ext2

mkdir /sd
/system/xbin/busybox mount -o loop /sd-ext/ext2 /sd
cd /data
cp -a * /sd
cd /
/system/xbin/busybox umount /sd

exit
adb reboot

10) your phone will now reboot with a much bigger /data/ partition.

------TUTORIAL END-------

I suggest to activate the boost!

type in adb shell window with android running:

busybox mount -o remount,noatime,nodiratime /mnt/asec/ext/ext2 /data
Attached Files
File Type: zip data.zip - [Click for QR Code] (282 Bytes, 1479 views)
The Following 3 Users Say Thank You to mattiadj For This Useful Post: [ View ] Gift mattiadj Ad-Free
26th November 2010, 02:27 PM |#6  
Senior Member
Thanks Meter: 31
 
More
Quote:
Originally Posted by sibere

No because you won't be able to use your SD as an external drive when connected to USB.

Just checked the links and they're OK

Sounds real logical ...8) ... Ups ... shortcut in brain ...8)
26th November 2010, 02:29 PM |#7  
sibere's Avatar
OP Retired Recognized Developer
Flag Paris
Thanks Meter: 376
 
More
Quote:
Originally Posted by bluelavender

i vouch for sibere's script... i'm one of first "live testers" for his script and it works great and very, very stable

and from what have been tested by several testers , it can work in these roms :

without a2sd+ :
- rooted stock
- leedroid no-a2sd

with a2sd+ :
- auraxtsense
- defrost
- leedroid
- alex-v

Sibere, allow me to add notes from AndroHero on using the script with a2sd+ rom :
- remove the data2sd (or in auraxt, stuff2sd) from system/etc/init.d in the rom zipfile, or if you have install the rom, just remove from system/etc/init.d
- do the sibere script

Regards
BL

Year sure, you're free to add notes mate
26th November 2010, 02:45 PM |#8  
Senior Member
Thanks Meter: 6
 
More
will this increase the battery usage?
26th November 2010, 02:51 PM |#9  
Senior Member
Thanks Meter: 36
 
More
Quote:
Originally Posted by kakit526

will this increase the battery usage?

No effect at all on the battery usage. the usage is around the same .

BL
26th November 2010, 03:20 PM |#10  
alessandro_xda's Avatar
Senior Member
Flag Bologna
Thanks Meter: 46
 
More
only one question:

is it better than using a2sd+?

what is the difference?

i had a lot of memory issues with a2sd+ (internal memory decrease very quickly compared to a2sd storage until i get the 'low memory' error message), does this solve my problem?

thanks

bye
Alessandro
26th November 2010, 03:23 PM |#11  
Senior Member
Thanks Meter: 53
 
More
Does this really increase IO speed? Can anyone who managed to set this up please post ur Quadrant benchmark.
Post Reply Subscribe to Thread

Guest Quick Reply (no urls or BBcode)
Message:
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes