[DEV] data2"XYZ" general filesystem discussion (Updated Dec 20 - Benchmarking)

woti23

Senior Member
May 27, 2010
920
173
0
i moved to compare

/data/data/com.Peaceman.Benchmark

on the fly to nand and made a symlink to the directory from /data/data/ to the directory in nand


so my database writes got from 6 to 52/s

and overal to 2487 from 2373


but the filewrite overhead is now (with in nand not btrfs) 6.66 ms (from 0.72ms), could that be a reason for lags?
 

HTF

Senior Member
Apr 18, 2009
126
32
0
The lower the overhead the better so from what you said nand has a larger overhead. I wonder why the database performance is degraded so much for writes, that imo must be the reason for any lags it is the only time nand does better.

I wonder what version of sqlite android uses because I remember reading something about efficiency improvements in newer versions. I suppose the issue is as usual cross compilation and android may use a modified version of sqlite.
 

mattiadj

Senior Member
May 22, 2010
4,089
1,779
253
Francavilla al mare
For me it seems really smoth with 40data2ext4. My goal is have a FS that can support a lot of applications without lag on startup or loading icons, low battery consuption and no data corruption. Until now I think I got my goal. My quadrant score average is ~1500 with I/O of ~650 but this does not mean nothing for me! I dont have lags on thing like "super user permissions", "click buttons", playing games or scrolling screen.
Which Rom?which SD?how many apps installed?

sent from the future
 

woti23

Senior Member
May 27, 2010
920
173
0
The lower the overhead the better so from what you said nand has a larger overhead. I wonder why the database performance is degraded so much for writes, that imo must be the reason for any lags it is the only time nand does better.

I wonder what version of sqlite android uses because I remember reading something about efficiency improvements in newer versions. I suppose the issue is as usual cross compilation and android may use a modified version of sqlite.
yes just compared with unjournaled ext2
46 database writes / s


i think there is also some strange thing goin on with this test/benchmark:


shouldn't i get close to SGS (3600) or even more, when i have the test in RAM,
in RAM i get only about 2900
 
Last edited:

HTF

Senior Member
Apr 18, 2009
126
32
0
Interesting but way above my head to figure out the cause for the huge database write speed loss.

Just an fyi to my previous post, android uses sqlite 3.6.22, the current sqlite version is 3.7.4 so android is some way behind. I expect there have been performance improvements since 3.6.22 the problem is how to install a newer version and will it break the android sqlite lib? I personally think it might break things otherwise google would have included it in gingerbread I am sure.
 

woti23

Senior Member
May 27, 2010
920
173
0
Interesting but way above my head to figure out the cause for the huge database write speed loss.

Just an fyi to my previous post, android uses sqlite 3.6.22, the current sqlite version is 3.7.4 so android is some way behind. I expect there have been performance improvements since 3.6.22 the problem is how to install a newer version and will it break the android sqlite lib? I personally think it might break things otherwise google would have included it in gingerbread I am sure.
i put it in RAM - just to compare, and get only 2900 overall
but 253writes/sec - strange
 

bernabap

Senior Member
Jul 25, 2009
217
16
0
Woow!!!211apps are terrific!
Can you explain me how set it up on sense hd based Rom?

sent from the future
I didnt tested on any HD sense based ROM but I think should work the same way. All I did was: 1- flashed Oxygen ROM, 2- before reboot (from recovery): 2a- replace my busybox with the new one, 2b- remove A2SD script and replace with 40data2ext4 script, 2c- set permissions, 2d- create the folder /system/data_internal. That`s it! 3- reboot.
 

mattiadj

Senior Member
May 22, 2010
4,089
1,779
253
Francavilla al mare
I didnt tested on any HD sense based ROM but I think should work the same way. All I did was: 1- flashed Oxygen ROM, 2- before reboot (from recovery): 2a- replace my busybox with the new one, 2b- remove A2SD script and replace with 40data2ext4 script, 2c- set permissions, 2d- create the folder /system/data_internal. That`s it! 3- reboot.
I'm a newbie....can you explain and write what commad have you done?
It's required that Rom is only on nand?

sent from the future
 

anko184

Senior Member
Jan 29, 2009
892
195
43
Volos
In a working data2ext rom if i copy all files from ext partition and then make again the partition formated at btrfs,
add the script with the right permissions and copy back the files to the new btrfs partition will it work?
 

bernabap

Senior Member
Jul 25, 2009
217
16
0
I'm a newbie....can you explain and write what commad have you done?
It's required that Rom is only on nand?

sent from the future
Most of this was quoted from where everything started: http://forum.xda-developers.com/showthread.php?t=852219

What you need is:

* Rom with init.d script support
* Latest busybox build (http://www.busybox.net/downloads/binaries/)

1.17.2> busybox-arm6l

* Boot your phone in recovery
* Flash your ROM (does not reboot!!!)
* Plug on computer

Connect to your phone with "adb shell"
You should get a ~# prompt now.

#mount /system
#exit

Replace you old busybox exec with the new one. It's located in /system/xbin/
$adb push c:\busybox-armv6l /system/xbin/busybox

Add the script to init.d folder:
$adb push c:\40data2ext4.txt /system/etc/init.d/40data2ext4

back to the shell:
$adb shell

#chmod 755 /system/xbin/busybox
#chgrp shell /system/xbin/busybox
#chown root /system/xbin/busybox

#chmod 755 /system/etc/init.d/40data2ext4
#chgrp shell /system/etc/init.d/40data2ext4
#chown root /system/etc/init.d/40data2ext4

#mkdir /system/data_internal

#exit

$adb reboot

Test your self and see if works! Make BACKUPS of EVERYTHING FIRST!!!

Melethron chaged his script today and you dont need to create the data_internal folder any more, the script will create for you!
 
Last edited:

melethron

Senior Member
Sep 13, 2010
854
193
0
i moved to compare

/data/data/com.Peaceman.Benchmark

on the fly to nand and made a symlink to the directory from /data/data/ to the directory in nand


so my database writes got from 6 to 52/s

and overal to 2487 from 2373


but the filewrite overhead is now (with in nand not btrfs) 6.66 ms (from 0.72ms), could that be a reason for lags?
The overhead would fit to my assumption that the reason the lags are better with btrfs is the filesystem. So its not the ACTUAL write speed to /data/data that matters but it's the difference between ext4 journal and btrfs copy on write.

Woow!!!211apps are terrific!
Can you explain me how set it up on sense hd based Rom?

sent from the future
mattiadj. I said in the thread, i said it in ownheres thread and i said it per PM. Just WAIT a bit for ROMs that have everything ready. Explaining the installation for every user just takes time.

I just fixed the last bug of my ROM and then i upload it...... All you need now is either a second ext4 or btrfs partition. And if you don't have linux you can't make btrfs partition now. So just make an ext partition as for app2sd and wait till i uploaded my ROM.
 

melethron

Senior Member
Sep 13, 2010
854
193
0
In a working data2ext rom if i copy all files from ext partition and then make again the partition formated at btrfs,
add the script with the right permissions and copy back the files to the new btrfs partition will it work?
this could mess up the permission of the files on the external but im not sure....
 

woti23

Senior Member
May 27, 2010
920
173
0
this could mess up the permission of the files on the external but im not sure....
this works only in an *nix like enviroment
with cp -ap /part1..../* /part2/

you must also preserve the ownerships of files and directories and keep symlinks
cp -ap ... does