This script only for ROM Developers!
EDIT:2010/12/13 update explain:
1. Why three partition?
Answer:for get best balance of stabilty and performance.
2. Why don't use two partition and loopfile on 2nd partition(my script V1)?
Answer:Loopfile on 2nd partition need enable journaling on 2nd partittion file system to get stability after Unexpected loss of power or Unsafe reboot(without umount). And, enable journaling on a large partition is slower than small partition. In my phone, 1.3G /data/app don't need journal, only 300M /data/data need journal, so I use a small parition for safe and performance in V4.
3. Why not use ext4 directly at 3rd partition?
Answer:for speed up sqlite3, ext4 loopfile on ext4 device, make a huge improve for sqlite3 insert.
4.Why not NILFS2 or BRTFS?
Answer: I have already tried nilfs2 before I use ext4. nilfs2 crash my system. Brtfs marked 'EXPERIMENTAL' in kernel source(2.6.32.15), I like stability more than speed.
________________________________________________________________________________________
EDIT:2010/12/12 update V4 script:
/data/:ext4, disable journaling
/data/data:ext4 loopfile on ext4 device, enable journaling on ext4 device, disable journaling on ext4 loopfile, for fast and stable sqlite3 access.
really smooth this time. please trying...
________________________________________________________________________________________
EDIT:2010/12/11 FULL BENCHMARK: (all test use my sandisk C2 16G TF card, run in SuSE Linux 11.2 desktop)
I did a full benchmark for ext2/ext4, with loop or without loop, write, read, sqlite insert, this is result:
I do follow tests, umount after every test to prevent from kernel cache.
Sqlite3 db benchmark is insert 2000 records to db.
500 50k-2M files(random size) to 2G ext4 loopfile on ext4 device, enable journaling on ext4, sync after every file write.
write: 355 seconds
read: 52 seconds
db:76 seconds
500 50k-2M files(random size) to 2G ext4 loopfile on ext4 device, disable journaling on ext4, sync after every file write.
write: 212 seconds
read: 52 seconds
db:3 seconds
500 50k-2M files(random size) to 2G ext2 loopfile on ext4 device, with sync-in-mount-option, sync after every file write.
write: 380 seconds
read: 45 seconds
db: 3 seconds
500 50k-2M files(random size) to 2G ext2 loopfile on ext4 device, without sync-in-mount-option, sync after every file write.
write: 390 seconds
read: 45 seconds
db: 3 seconds
500 50k-2M files(random size) to ext4 device, enable journaling, sync after every file write.
write: 208 seconds
read: 47 seconds
db: 95 seconds
500 50k-2M files(random size) to ext4 device, disable journaling, sync after every file write.
write: 191 seconds
read: 48 seconds
db: 50 seconds
500 50k-2M files(random size) to ext4 device mount with loop option, enable journaling, sync after every file write.
write: 198 seconds
read: 51 seconds
db: 72 seconds
500 50k-2M files(random size) to ext4 device mount with loop option, disable journaling, sync after every file write.
write: 191 seconds
read: 50 seconds
db: 2 seconds
As you see, the best speed is: mount real ext4 device with loop option, disable journaling. The best balance of performance and safe is mount real ext4 device with loop option, enable journaling! We don't need a loopfile but just change mount option.
I attach my testsuit file below.
___________________________________________________________________
EDIT:2010/12/10 V3 changelog:
1.change loopback device to real partition for powersave and better write performance.
2.disable second partition's journaling.
3.You need 3rd partition for /data/data. But even without the third partition of this script can also work(not stable). My partition settings:second partition:1.5G, thrid partition:0.5G
Explain:
1.about NAND and SD: Desire have poor NAND memory for userdata, just 150M(this can be enlarged by MTD or HBOOT, but still small for me). This MTD partition need store app, dalvik-cache, app-data, it is too small to fill many apps.So someone think may be someting can be symbol link to SD, this is original Cyanogen A2SD, in this case , the /data/data directory is not moved to SD, because the battery reason and the speed need. This is what the HDROM 2NAND version do. So, the original A2SD is totally same as 2NAND. I don't known why the 2NAND be named. The NAND is not fast because it is yaffs2(journaling) filesystem, so if install many apps the phone will be slow(every app's sqlite have write demand)
2.about ext2/ext3/ext4: Ext2 have no journal feature, so lost power or just normal reboot will be crashed it, ext3 is journaling version of ext2. Ext4 is diffrent from ext2/ext3, it included Extends feautre, which provide better performance for large file(large than block size:4K). I analyze android system files, 4/5 is large than 4k, so the Extends feature is very useful. all EXTX system is base on Block, and the default block size is 1024 for small partition. Larger block size has better performance, so I force block size to 4096 when format.
3.about loopback device: The original Data2SD script make a huge loopback file for /data parition, then get better IO score. The reason is loopback device provide better Read performance. But the Write performance is wrose, so the phone got lag when install app or copy file.And the lag will occur suddenly. If system want to write data to loopback device, it is not real write, just write to kernel buffer/cache, the data will be flush to real disk when real filesystem(where loopback device stay on) got sync or commit command. So the real filesystem must enable journaling or do sync frequently。
4.about my old data2ext script: I try to splite directory to diffrent FS for the balance of performance and lag. Why I formated partition to EXT4? the reason can be found at item 2. Why create a small loopback device? the reason is item 3. But I found some bug with this case, any data write to loopback device need twice kernel IO command, one virtual one real, so the cpu usage will be high. Real EXT4 partition don't need journaling if no loopback device on it. For data integrity on virtual EXT4 loopback device, I had to enable journal on it.So I design new disk layout for better performance, it is DATA2EXT V3.
__________________________________________________________________
V2 change log:
1.change loopback device FS to EXT4 for journal, more stable
2.change mount option, add commit=large Prime number, reduce real IO, avoid concentration of commit. change journaling scheduler from Ordered to WriteBack to improve performance.
I would like to share my Data2EXT script, a collection of data2sd(Thanks to sibere) and the advantages of ext4, you can let the phone run at peak efficiency.
I use EXT4 for whole SD 2nd partition, and create a ext2 loopback device with 1/5 size of 2nd partition.
The files in /data/app,/data/.systemapp it not updated frequently. so store them in EXT4.
In /data/data and /data/app-private, a lot of small files and sqlite database, is updated frequently. so store thenm in EXT2 loopback device.
/data/dalvik-cache is stored in phone memory /mnt/asec/mtddata for distributed IO pressure.
/data/misc, /data/system, /data/property is stored in phone memory /mnt/asec/mtddata for system settings restore(or you will lost your timezone/language settings after reboot).
use this script, I can get quandrant score:2000+, IO:4000+, my phone is smooth and quickly.
EDIT:2010/12/13 update explain:
1. Why three partition?
Answer:for get best balance of stabilty and performance.
2. Why don't use two partition and loopfile on 2nd partition(my script V1)?
Answer:Loopfile on 2nd partition need enable journaling on 2nd partittion file system to get stability after Unexpected loss of power or Unsafe reboot(without umount). And, enable journaling on a large partition is slower than small partition. In my phone, 1.3G /data/app don't need journal, only 300M /data/data need journal, so I use a small parition for safe and performance in V4.
3. Why not use ext4 directly at 3rd partition?
Answer:for speed up sqlite3, ext4 loopfile on ext4 device, make a huge improve for sqlite3 insert.
4.Why not NILFS2 or BRTFS?
Answer: I have already tried nilfs2 before I use ext4. nilfs2 crash my system. Brtfs marked 'EXPERIMENTAL' in kernel source(2.6.32.15), I like stability more than speed.
________________________________________________________________________________________
EDIT:2010/12/12 update V4 script:
/data/:ext4, disable journaling
/data/data:ext4 loopfile on ext4 device, enable journaling on ext4 device, disable journaling on ext4 loopfile, for fast and stable sqlite3 access.
really smooth this time. please trying...
________________________________________________________________________________________
EDIT:2010/12/11 FULL BENCHMARK: (all test use my sandisk C2 16G TF card, run in SuSE Linux 11.2 desktop)
I did a full benchmark for ext2/ext4, with loop or without loop, write, read, sqlite insert, this is result:
I do follow tests, umount after every test to prevent from kernel cache.
Sqlite3 db benchmark is insert 2000 records to db.
500 50k-2M files(random size) to 2G ext4 loopfile on ext4 device, enable journaling on ext4, sync after every file write.
write: 355 seconds
read: 52 seconds
db:76 seconds
500 50k-2M files(random size) to 2G ext4 loopfile on ext4 device, disable journaling on ext4, sync after every file write.
write: 212 seconds
read: 52 seconds
db:3 seconds
500 50k-2M files(random size) to 2G ext2 loopfile on ext4 device, with sync-in-mount-option, sync after every file write.
write: 380 seconds
read: 45 seconds
db: 3 seconds
500 50k-2M files(random size) to 2G ext2 loopfile on ext4 device, without sync-in-mount-option, sync after every file write.
write: 390 seconds
read: 45 seconds
db: 3 seconds
500 50k-2M files(random size) to ext4 device, enable journaling, sync after every file write.
write: 208 seconds
read: 47 seconds
db: 95 seconds
500 50k-2M files(random size) to ext4 device, disable journaling, sync after every file write.
write: 191 seconds
read: 48 seconds
db: 50 seconds
500 50k-2M files(random size) to ext4 device mount with loop option, enable journaling, sync after every file write.
write: 198 seconds
read: 51 seconds
db: 72 seconds
500 50k-2M files(random size) to ext4 device mount with loop option, disable journaling, sync after every file write.
write: 191 seconds
read: 50 seconds
db: 2 seconds
As you see, the best speed is: mount real ext4 device with loop option, disable journaling. The best balance of performance and safe is mount real ext4 device with loop option, enable journaling! We don't need a loopfile but just change mount option.
I attach my testsuit file below.
___________________________________________________________________
EDIT:2010/12/10 V3 changelog:
1.change loopback device to real partition for powersave and better write performance.
2.disable second partition's journaling.
3.You need 3rd partition for /data/data. But even without the third partition of this script can also work(not stable). My partition settings:second partition:1.5G, thrid partition:0.5G
Explain:
1.about NAND and SD: Desire have poor NAND memory for userdata, just 150M(this can be enlarged by MTD or HBOOT, but still small for me). This MTD partition need store app, dalvik-cache, app-data, it is too small to fill many apps.So someone think may be someting can be symbol link to SD, this is original Cyanogen A2SD, in this case , the /data/data directory is not moved to SD, because the battery reason and the speed need. This is what the HDROM 2NAND version do. So, the original A2SD is totally same as 2NAND. I don't known why the 2NAND be named. The NAND is not fast because it is yaffs2(journaling) filesystem, so if install many apps the phone will be slow(every app's sqlite have write demand)
2.about ext2/ext3/ext4: Ext2 have no journal feature, so lost power or just normal reboot will be crashed it, ext3 is journaling version of ext2. Ext4 is diffrent from ext2/ext3, it included Extends feautre, which provide better performance for large file(large than block size:4K). I analyze android system files, 4/5 is large than 4k, so the Extends feature is very useful. all EXTX system is base on Block, and the default block size is 1024 for small partition. Larger block size has better performance, so I force block size to 4096 when format.
3.about loopback device: The original Data2SD script make a huge loopback file for /data parition, then get better IO score. The reason is loopback device provide better Read performance. But the Write performance is wrose, so the phone got lag when install app or copy file.And the lag will occur suddenly. If system want to write data to loopback device, it is not real write, just write to kernel buffer/cache, the data will be flush to real disk when real filesystem(where loopback device stay on) got sync or commit command. So the real filesystem must enable journaling or do sync frequently。
4.about my old data2ext script: I try to splite directory to diffrent FS for the balance of performance and lag. Why I formated partition to EXT4? the reason can be found at item 2. Why create a small loopback device? the reason is item 3. But I found some bug with this case, any data write to loopback device need twice kernel IO command, one virtual one real, so the cpu usage will be high. Real EXT4 partition don't need journaling if no loopback device on it. For data integrity on virtual EXT4 loopback device, I had to enable journal on it.So I design new disk layout for better performance, it is DATA2EXT V3.
__________________________________________________________________
V2 change log:
1.change loopback device FS to EXT4 for journal, more stable
2.change mount option, add commit=large Prime number, reduce real IO, avoid concentration of commit. change journaling scheduler from Ordered to WriteBack to improve performance.
I would like to share my Data2EXT script, a collection of data2sd(Thanks to sibere) and the advantages of ext4, you can let the phone run at peak efficiency.
I use EXT4 for whole SD 2nd partition, and create a ext2 loopback device with 1/5 size of 2nd partition.
The files in /data/app,/data/.systemapp it not updated frequently. so store them in EXT4.
In /data/data and /data/app-private, a lot of small files and sqlite database, is updated frequently. so store thenm in EXT2 loopback device.
/data/dalvik-cache is stored in phone memory /mnt/asec/mtddata for distributed IO pressure.
/data/misc, /data/system, /data/property is stored in phone memory /mnt/asec/mtddata for system settings restore(or you will lost your timezone/language settings after reboot).
use this script, I can get quandrant score:2000+, IO:4000+, my phone is smooth and quickly.
Last edited:
