[DEV] data2"XYZ" general filesystem discussion (Updated Dec 20 - Benchmarking)

[woti23]

Question at everyone that is better in scripting than me. I want to merge my ext4 and btrfs script so that both are compatible (for my ROM):

If i do:

busybox mount -o noatime,nodiratime,[COLOR="Red"]ssd_spread  [/COLOR]/dev/block/mmcblk0p2 /data;

....will the mount fail if done with ext partition because of the option ssd_spread? Or will it simply mount without that option?

I also browsed "/proc" and "/sys" to see if i can "cat" the partition type somewhere to make a "if-then-else" mount part depending on the type... but i didn't find anything ....

Other option would be:

mount (auto)
mounts | grep btrfs
umount

then mount depending on type ... but i think mount -> umount -> mount is "bad" coding. Since mount recognizes the partition there must be some way to get the type of it .....

maybe
-s Tolerate sloppy mount options rather than failing. This will ignore mount options not supported by a filesystem type. Not all filesystems support this option.

or --fake

 

[woti23]

maybe
-s Tolerate sloppy mount options rather than failing. This will ignore mount options not supported by a filesystem type. Not all filesystems support this option.

or --fake

or mount readonly first without options, check which filesystem is mounted, and make readwritable with options after

 

[balthor80]

Ok !

I have ReflexTSenseHD + data2btrfs + couttstech kernel

All seems ok
What should I try ?

edit :
Quadrant freeze in "Database writes" in I/O..

 

[AdamG]

I dont suppose anyone has tried formatting /data directly and modifying init to support mounting as a different file type other than yaffs?

 

[woti23]

Ok !

I have ReflexTSenseHD + data2btrfs + couttstech kernel

All seems ok
What should I try ?

maybe you could download tap benchmark from here

http://forum.xda-developers.com/showthread.php?t=850454

its "details on performance on /data"


and we could compare with someone with a other data2sd/ext/whatever

 

[mosolicious]

Guess you'll be faster, my pc is very slow at times

Sent from my HTC Desire using XDA App

Could you please post the vork kernel when you've built it? I'd like to test it for MIUI and Oxygen

 

[snq-]

anyone has tried formatting /data directly [..] as a different file type other than yaffs?

i really hope nobody has

 

[woti23]

Ok !

I have ReflexTSenseHD + data2btrfs + couttstech kernel

All seems ok
What should I try ?

edit :
Quadrant freeze in "Database writes" in I/O..

i have 2050 at first run, rcdhmix, no overclocking, uv, cfq

it takes a while for it to go over database writes

 

[balthor80]

maybe you could download tap benchmark from here

http://forum.xda-developers.com/showthread.php?t=850454

its "details on performance on /data"


and we could compare with someone with a other data2sd/ext/whatever

(I have a class 6 sdcard)
First :
Overall Performance: 2531(139)
Database Performance: 307(12)
FileRead/Write Performance: 1391(58)
StressTest Performance: 833(126)

DB writes: 10(2) Writes/s
DB reads: 205(2) Reads/s
DB deletes: 244(18) Deletes/s
File writes 10k: 2079(321) kB/s
File writes 1M: 5226(899) kB/s
File reads 10k: 4471(172) kB/s
File reads 1M: 9496(103) kB/s
File write Overhead: 2.895(0.812) ms
File read overhead: 1.183(0.087) ms
StressWrite 1M: 3933(378) kB/s
StressRead 1M: 5681(102) kB/s
StressWrite 10k: 852(160) kB/s
StressRead 10k: 1518(439) kB/s


Second :
Overall Performance: 2509(111)
Database Performance: 342(12)
FileRead/Write Performance: 1374(29)
StressTest Performance: 793(106)

DB writes: 8(2) Writes/s
DB reads: 234(18) Reads/s
DB deletes: 271(3) Deletes/s
File writes 10k: 2117(241) kB/s
File writes 1M: 5180(102) kB/s
File reads 10k: 4507(80) kB/s
File reads 1M: 9060(264) kB/s
File write Overhead: 2.793(0.538) ms
File read overhead: 1.115(0.051) ms
StressWrite 1M: 4305(640) kB/s
StressRead 1M: 5975(156) kB/s
StressWrite 10k: 930(174) kB/s
StressRead 10k: 1213(317) kB/s

Thrid :
Overall Performance: 2395(105)
Database Performance: 362(3)
FileRead/Write Performance: 1265(22)
StressTest Performance: 768(103)

DB writes: 8(4) Writes/s
DB reads: 265(2) Reads/s
DB deletes: 270(2) Deletes/s
File writes 10k: 1875(70) kB/s
File writes 1M: 4486(346) kB/s
File reads 10k: 4382(84) kB/s
File reads 1M: 8308(197) kB/s
File write Overhead: 3.103(0.263) ms
File read overhead: 1.078(0.052) ms
StressWrite 1M: 4167(575) kB/s
StressRead 1M: 6189(219) kB/s
StressWrite 10k: 861(182) kB/s
StressRead 10k: 1176(300) kB/s

I have seen some galaxy S with 4000+...

And quadrant :
no oc : 1927 - 2285 - 2355 - 2255
oc @1190 : 2600 - 3058 - 2535 - 2772

 

[HTF]

Details of Performance on /data
Iterations: 6 - Single Test Duration: 1000
Timestamp: 20101216T191909

Overall Performance: 2528(106)
Database Performance: 320(9)
FileRead/Write Performance: 1201(20)
StressTest Performance: 1007(104)

DB writes: 40(1) Writes/s
DB reads: 207(10) Reads/s
DB deletes: 233(10) Deletes/s
File writes 10k: 892(8) kB/s
File writes 1M: 2310(65) kB/s
File reads 10k: 4829(191) kB/s
File reads 1M: 10282(161) kB/s
File write Overhead: 6.879(0.159) ms
File read overhead: 1.098(0.083) ms
StressWrite 1M: 1007(23) kB/s
StressRead 1M: 9892(97) kB/s
StressWrite 10k: 530(52) kB/s
StressRead 10k: 2409(405) kB/s

This is what I got using oxygen rom using just standard apps2sd+ and no data2sd/ext etc. Also using a class 6 card.

Comparing this with the one above, I would say that this will gain increased write speeds across the board with database performance being very similar to stock.

I will say though that the rom I am using is not sense based so there are less background services which effect this benchmark a lot. This leaves me feeling hopeful, now if only I could get a working AOSP kernel so I can do a proper comparison test.

 

[bernabap]

- Oxygen 1.0.3
- 40data2ext4 script
- uSD class 6 "Turbo" ( I dont know what that Turbo mean), 1 fat32 and 1 ext4

Tab Benchmark

latest:

Details of Performance on /data
Iterations: 6 - Single Test Duration: 1000
Timestamp: 20101216T043732

Overall Performance: 3043(114)
Database Performance: 399(11)
FileRead/Write Performance: 1596(50)
StressTest Performance: 1048(102)

DB writes: 29(1) Writes/s
DB reads: 283(17) Reads/s
DB deletes: 286(2) Deletes/s
File writes 10k: 2976(229) kB/s
File writes 1M: 7532(900) kB/s
File reads 10k: 4634(55) kB/s
File reads 1M: 9178(35) kB/s
File write Overhead: 2.032(0.304) ms
File read overhead: 1.068(0.026) ms
StressWrite 1M: 4545(554) kB/s
StressRead 1M: 5576(299) kB/s
StressWrite 10k: 1543(225) kB/s
StressRead 10k: 1636(265) kB/s

all others I did:

20101216T193641: Benchmark on /data (6 iterations)
DB writes: 28(1) Writes/s
DB reads: 267(21) Reads/s
DB deletes: 300(2) Deletes/s
File writes 10k: 3068(102) kB/s
File writes 1M: 7587(342) kB/s
File reads 10k: 4577(58) kB/s
File reads 1M: 8946(84) kB/s
File write overhead: 1.94(0.123) ms
File read overhead: 1.067(0.03) ms
Database overall: 397(14)
File overall: 1591(21)
StressWrite 1M: 4542(447) kB/s
StressRead 1M: 5613(296) kB/s
StressWrite 10k: 1288(203) kB/s
StressRead 10k: 1596(291) kB/s
Stress overall: 975(102)

20101216T193500: Benchmark on /data (6 iterations)
DB writes: 20(12) Writes/s
DB reads: 276(3) Reads/s
DB deletes: 285(27) Deletes/s
File writes 10k: 3189(281) kB/s
File writes 1M: 4975(3010) kB/s
File reads 10k: 4738(32) kB/s
File reads 1M: 9274(48) kB/s
File write overhead: 1.126(1.247) ms
File read overhead: 1.032(0.015) ms
Database overall: 388(19)
File overall: 1505(153)
StressWrite 1M: 3518(911) kB/s
StressRead 1M: 5598(278) kB/s
StressWrite 10k: 1206(198) kB/s
StressRead 10k: 1786(317) kB/s
Stress overall: 976(117)

20101216T193301: Benchmark on /data (6 iterations)
DB writes: 25(2) Writes/s
DB reads: 244(19) Reads/s
DB deletes: 283(11) Deletes/s
File writes 10k: 3177(100) kB/s
File writes 1M: 4763(3130) kB/s
File reads 10k: 4674(16) kB/s
File reads 1M: 9693(81) kB/s
File write overhead: 1.048(1.383) ms
File read overhead: 1.108(0.011) ms
Database overall: 369(14)
File overall: 1508(156)
StressWrite 1M: 4296(1403) kB/s
StressRead 1M: 5739(329) kB/s
StressWrite 10k: 1300(222) kB/s
StressRead 10k: 1863(360) kB/s
Stress overall: 1042(141)

20101216T043855: Benchmark on /data (6 iterations)
DB writes: 29(1) Writes/s
DB reads: 283(17) Reads/s
DB deletes: 286(2) Deletes/s
File writes 10k: 2976(229) kB/s
File writes 1M: 7532(900) kB/s
File reads 10k: 4634(55) kB/s
File reads 1M: 9178(35) kB/s
File write overhead: 2.032(0.304) ms
File read overhead: 1.068(0.026) ms
Database overall: 399(11)
File overall: 1596(50)
StressWrite 1M: 4545(554) kB/s
StressRead 1M: 5576(299) kB/s
StressWrite 10k: 1543(225) kB/s
StressRead 10k: 1636(265) kB/s
Stress overall: 1048(102)

20101216T043520: Benchmark on /data (6 iterations)
DB writes: 28(1) Writes/s
DB reads: 223(39) Reads/s
DB deletes: 294(15) Deletes/s
File writes 10k: 4069(313) kB/s
File writes 1M: 4534(2429) kB/s
File reads 10k: 6221(183) kB/s
File reads 1M: 9519(245) kB/s
File write overhead: 0.252(1.197) ms
File read overhead: 0.557(0.054) ms
Database overall: 364(27)
File overall: 1731(127)
StressWrite 1M: 6087(473) kB/s
StressRead 1M: 6858(222) kB/s
StressWrite 10k: 1816(297) kB/s
StressRead 10k: 1989(369) kB/s
Stress overall: 1275(131)

20101216T042713: Benchmark on /data (6 iterations)
DB writes: 26(1) Writes/s
DB reads: 240(22) Reads/s
DB deletes: 305(9) Deletes/s
File writes 10k: 3305(111) kB/s
File writes 1M: 4222(2144) kB/s
File reads 10k: 4783(51) kB/s
File reads 1M: 9769(89) kB/s
File write overhead: 0.657(1.207) ms
File read overhead: 1.067(0.024) ms
Database overall: 381(15)
File overall: 1508(107)
StressWrite 1M: 4096(1782) kB/s
StressRead 1M: 5954(457) kB/s
StressWrite 10k: 1476(547) kB/s
StressRead 10k: 1731(327) kB/s
Stress overall: 1053(205)

20101215T180829: Benchmark on /data (6 iterations)
DB writes: 22(8) Writes/s
DB reads: 249(13) Reads/s
DB deletes: 288(20) Deletes/s
File writes 10k: 3386(156) kB/s
File writes 1M: 5180(2414) kB/s
File reads 10k: 4190(404) kB/s
File reads 1M: 9693(67) kB/s
File write overhead: 1.022(0.91) ms
File read overhead: 1.355(0.23) ms
Database overall: 374(16)
File overall: 1501(128)
StressWrite 1M: 3937(1301) kB/s
StressRead 1M: 5671(86) kB/s
StressWrite 10k: 1391(251) kB/s
StressRead 10k: 1814(352) kB/s
Stress overall: 1042(140)

20101215T175801: Benchmark on /data (6 iterations)
DB writes: 28(0) Writes/s
DB reads: 286(1) Reads/s
DB deletes: 292(31) Deletes/s
File writes 10k: 2973(81) kB/s
File writes 1M: 4726(3229) kB/s
File reads 10k: 4567(59) kB/s
File reads 1M: 8482(86) kB/s
File write overhead: 1.247(1.448) ms
File read overhead: 1.011(0.031) ms
Database overall: 405(20)
File overall: 1414(161)
StressWrite 1M: 3332(904) kB/s
StressRead 1M: 5702(451) kB/s
StressWrite 10k: 1083(253) kB/s
StressRead 10k: 1637(325) kB/s
Stress overall: 906(128)

20101215T175252: Benchmark on /data (6 iterations)
DB writes: 29(0) Writes/s
DB reads: 289(4) Reads/s
DB deletes: 292(8) Deletes/s
File writes 10k: 3091(261) kB/s
File writes 1M: 6173(825) kB/s
File reads 10k: 4529(16) kB/s
File reads 1M: 8782(97) kB/s
File write overhead: 1.615(0.349) ms
File read overhead: 1.069(0.015) ms
Database overall: 407(5)
File overall: 1509(49)
StressWrite 1M: 4162(418) kB/s
StressRead 1M: 5530(130) kB/s
StressWrite 10k: 1166(235) kB/s
StressRead 10k: 1963(377) kB/s
Stress overall: 1025(122)

20101215T172609: Benchmark on /data (6 iterations)
DB writes: 22(8) Writes/s
DB reads: 257(19) Reads/s
DB deletes: 291(2) Deletes/s
File writes 10k: 3218(89) kB/s
File writes 1M: 4963(2485) kB/s
File reads 10k: 4665(63) kB/s
File reads 1M: 9491(74) kB/s
File write overhead: 1.093(1.012) ms
File read overhead: 1.09(0.03) ms
Database overall: 380(13)
File overall: 1511(124)
StressWrite 1M: 4134(667) kB/s
StressRead 1M: 5605(318) kB/s
StressWrite 10k: 1425(244) kB/s
StressRead 10k: 1629(299) kB/s
Stress overall: 1007(114)

20101215T050252: Benchmark on /data (6 iterations)
DB writes: 27(0) Writes/s
DB reads: 256(14) Reads/s
DB deletes: 286(5) Deletes/s
File writes 10k: 3186(174) kB/s
File writes 1M: 5599(1832) kB/s
File reads 10k: 4564(140) kB/s
File reads 1M: 9556(91) kB/s
File write overhead: 1.353(0.609) ms
File read overhead: 1.144(0.068) ms
Database overall: 380(9)
File overall: 1532(94)
StressWrite 1M: 4167(1026) kB/s
StressRead 1M: 5784(180) kB/s
StressWrite 10k: 1253(229) kB/s
StressRead 10k: 1488(298) kB/s
Stress overall: 934(119)

 

[HTF]

Nice benchmarks, they would suggest that the new data2ext does improve speed etc however I am slightly concerned about the database benchmarks. The database write speed is a lot lower than I was getting even though the database read speed was higher.

A 25% database write speed cut may explain some of the lags people are getting.

 

[sibere]

but logcat on boot is "clean".

I//system/xbin/busybox(   65):  +++ Changing scheduler:
I//system/xbin/busybox(   65): [noop] cfq bfq 
I//system/xbin/busybox(   65):  +++ Mounting devices
I//system/xbin/busybox(   65):  +++ Moving app from internal to SD-Card

Well at boot unmount command gives me also the "umount: can't umount /data: Device or resource busy" message seems like an app or OS component is having access to the /data NAND partition before the SD mount. That doesn't sound good to me.

 

[bernabap]

Nice benchmarks, they would suggest that the new data2ext does improve speed etc however I am slightly concerned about the database benchmarks. The database write speed is a lot lower than I was getting even though the database read speed was higher.

A 25% database write speed cut may explain some of the lags people are getting.

I saw that too. But how many times did you run the bench? I am not having any lags at all. My average of DB write speed is 26 (27+22+29+28+22+26+28+29+25+20+28+29 / 12 = 26.083333). Please try run at least 10 times and see your average.

 

[mattiadj]

Can anyone tells how implement this on rcmixhd?are present lags or all is very very smooth?

sent from the future

 

[woti23]

class 2 micro sdhc
a2sd (ext4), /data/data btrfs, dalivik nand, cfq scheduler, no overclocking, undervolt


Overall Performance: 2373(208)
Database Performance 240(18)
FileRead/Write Performance: 1174(47)
StressTest Performance: 959(202)

DB writes: 6(1) Writes/s
DB reads: 160(22) Reads/s
DB deletes: 193(18) Deletes/s
File writes 10k: 2432(116) kB/s
File writes 1M: 2953(492) kB/s
File reads 10k: 3224(261) kB/s
File reads 1M: 9216(578) kB/s
File write overhead: 0.726(0.597) ms
File read overhead: 2.016(0.26) ms
Database overall: 240(18)
File overall: 1174(47)
StressWrite 1M: 4245(1853) kB/s
StressRead 1M: 6731(340) kB/s
StressWrite 10k: 850(185) kB/s
StressRead 10k: 1888(594) kB/s
Stress overall: 959(202)

 

[HTF]

I saw that too. But how many times did you run the bench? I am not having any lags at all. My average of DB write speed is 26 (27+22+29+28+22+26+28+29+25+20+28+29 / 12 = 26.083333). Please try run at least 10 times and see your average.

My average DB write speed is 43 writes per second: (40+44+39+46+46+43+46+46+43+45+46+41 / 12 = 43.75)

 

[bernabap]

My average DB write speed is 43 writes per second: (40+44+39+46+46+43+46+46+43+45+46+41 / 12 = 43.75)

Thanks HTF. Actually there is a considerable difference. But as I said I have no lags at all. I will try now with my uSD class 10 and I will align the partitions as I didnt on my other uSD. Lets see...

 

[melethron]

@All: It's nice that you do all this benches, but im rather interested if you have a smother expirience (considering lags ...). If the phone has quadrant 9000 score but feals like underclocked to 245 it sucks.

maybe
-s Tolerate sloppy mount options rather than failing. This will ignore mount options not supported by a filesystem type. Not all filesystems support this option.

or --fake

Thanks.
Fake doesn't show up in mounts then so that's no help.
And -s doesn't help either. It's not accepted by busybox mount.

It's this for now :

    busybox mount -o noatime,nodiratime  /dev/block/mmcblk0p2 /data;
    mounttest=`busybox mount | grep btrfs`;
    if [ -n "$mounttest" ];
    then 
	  echo "btrfs Partition found - remounting with optimizations";
	  busybox mount -t btrfs -o remount noatime,nodiratime,ssd_spread  /dev/block/mmcblk0p2 /data;
    else
	  echo "EXT partition found";
    fi;

Im not happy with this.

EDIT: Oh ..... did you mean fake mount with ssd and then real mount with ssd if there is no error and mount without if there is one? That would be an option if fake mount gives an error message on fake mount.

i really hope nobody has

Why not?! Remove the ****ty mtdblock03 device. Create a UBI device and format with UBIFS. xD

...well i myself won't mess with mtd yet. at least not before UBIFS is implemented in a recovery kernel. Having no backup of btrfs partition is enough for me.

(I have a class 6 sdcard)
I have seen some galaxy S with 4000+...

(quote snipped)

Galaxy S has like 9mb/s on the internal SD (a dev made a quick dd while i was in irc).

 

[bernabap]

@All: It's nice that you do all this benches, but im rather interested if you have a smother expirience (considering lags ...). If the phone has quadrant 9000 score but feals like underclocked to 245 it sucks.

For me it seems really smoth with 40data2ext4. My goal is have a FS that can support a lot of applications without lag on startup or loading icons, low battery consuption and no data corruption. Until now I think I got my goal. My quadrant score average is ~1500 with I/O of ~650 but this does not mean nothing for me! I dont have lags on thing like "super user permissions", "click buttons", playing games or scrolling screen.