[DEV] data2"XYZ" general filesystem discussion (Updated Dec 20 - Benchmarking)

[melethron]

@mattiadj

Rom takes more time ....

it somehow wont flash /data/local .... everthing else in /data is flashed but the local folder is missing ...

 

[HTF]

Make sure if there are no files in data/local that you put a placeholder file in there which you can delete afterwards or it will not create the folder.

 

[yukkio]

Hi, I've modified Aurax 8.1.3 to support btrfs with this script but 8.1.3 hasnt a kernel with btrfs support so all things in /data/ arent copied. Is this a recovery problem? Do I need a boot.img with btrfs support to copy all data when I install the rom?

BR.

 

[melethron]

Make sure if there are no files in data/local that you put a placeholder file in there which you can delete afterwards or it will not create the folder.

Thanks for that.

I already tried this. It is more and issue that there ARE files in there. To be specific: That there are 2 ZIP files in there. The bootaninmation and the downanimation. Im pretty sure now that they are the reason for the issue, because /system/customize/resource is also not flashed (complete folder with the stuff in there is missing after flash) because there is the shutdown.zip.

I tried to ONLY flash this folder with the zips and it is working then.

So i think it could be something with compression, because the archive doesn't get compressed if there are only zip's in. But i really would wonder about this because the other roms ARE ALSO compressed.

Edit: uncompressed doesn't work also....

 

[woti23]

Thanks for that.

I already tried this. It is more and issue that there ARE files in there. To be specific: That there are 2 ZIP files in there. The bootaninmation and the downanimation. Im pretty sure now that they are the reason for the issue, because /system/customize/resource is also not flashed (complete folder with the stuff in there is missing after flash) because there is the shutdown.zip.

I tried to ONLY flash this folder with the zips and it is working then.

So i think it could be something with compression, because the archive doesn't get compressed if there are only zip's in. But i really would wonder about this because the other roms ARE ALSO compressed.

Edit: uncompressed doesn't work also....

take a look in the updater-script of the rom?

META-INF / com / google / android / updater-script

so you see what the rom install deletes or install or mounts or does symlinks

maybe it mounts another partition to data

mount("MTD", "userdata", "/data");
package_extract_dir("data", "/data");

 

[melethron]

Bit off topic now, but if someone wonders why i didnt make a ROM yet:

I still didn't figure out why this file wont get written on the mtd but while trying to flash over and over again i managed to get an USB BRICK.....

Wouldnt be a problem but:
-i had no working rom at this time on the phone bcs i only flashed /data
-i couldn't flash leedroid from sd because i had no kernel to flash afterwards and btrfs was not recognized ....
-and i couldnt get access the sd because i had no card reader ....
-and couldn't get a card reader bcs i had other stuff to do and apart from that i have no driver license (atm) bcs they made a nice photo of me earlier this year......XD

Got someone that drove me and bought a card reader and now the brick is fixed .....

... but i still don't have solved the mtd problem.....

 

[yukkio]

Incredible... http://twitpic.com/3gqfnp

 

[melethron]

I check #alphrev to get some help on my flash issue:

I did:

fastboot oem partition_test userdata

to check for bad blocks:

INFOTime cost = 3748405 ms
INFOPASS rate = 1181 / 1181 = 100 %
INFOFAIL rate = 0 / 1181 = 0 %

None bad block and my /system is also clean.

But i found my issue:

/dev/block/mtdblock5    151168    151128        40 100% /data

LOL ....

But at least i have some news about UBIFS or messing with NAND in generall:

[19:31] <melethron> would be nice if we could use like UBIFS
[19:31] <@IEF> simply because of the interface
[19:31] <melethron> or something
[19:31] <@IEF> lol
[19:32] <melethron> but i wont mess with nand partitions ...
[19:32] <@IEF> oh, I've no problem to do that
[19:32] <@IEF> as HBOOT is S-OFF anyway
[19:32] <@IEF> if any problems occur -> fastboot erase <partition>
[19:32] <@IEF> done, back to yaffs2
[19:33] <melethron> btw is there anyway to check bad blocks on yaffs2 it has no fsck i think ...
[19:33] <melethron> ?
[19:33] <@IEF> yes, there i
[19:34] <@IEF> is
[19:34] <@IEF> through fastboot
[19:34] <melethron> i had a problem with a file that was not flashed properly .... it was only written partial
[19:34] <melethron> would erase partition help on this?
[19:35] <@IEF> uh why was it written only partially?
[19:35] <melethron> cause amonra recovery only does rm when wiped and no real format
[19:35] <melethron> i have no idea ....
[19:35] <@IEF> oh
[19:35] <melethron> no bad block warning no nothing
[19:35] <@IEF> then don't use amon-ra
[19:36] <melethron> ^^
[19:36] <melethron> obviously ^^
[19:36] <@IEF> anyway
[19:36] <@IEF> try 'fastboot oem partition_test'
[19:37] <@IEF> on alpharev hboot, obviously ;p
[19:37] <melethron> ^^
[19:37] <@IEF> perhaps I'll try UBIFS
[19:37] <@IEF> or LogFS
[19:38] <@IEF> UBIFS is weird tho
[19:38] <melethron> well if it is that easy to erase the partition again ....
[19:38] <@IEF> it is
[19:38] <@IEF> that's a real yaffs2 format
[19:38] <melethron> logfs is also nice.... i like the idea of logbased of cow filesystems
[19:38] <@IEF> you won't be able to cross MTD partition boundaries anyawy
[19:38] <@IEF> so not really any risk of overwriting something
[19:38] <@IEF> yeah
[19:39] <@IEF> UBIFS <> LogFS are competitors
[19:39] <@IEF> both successors to JFFS2
[19:39] <@IEF> i'm not sure if it does compression
[19:39] <melethron> ok so the radio/hboot wont get "hurt" and you have always acces to hboot?
[19:39] <@IEF> i'd like to try a bit of FS compression tho
[19:39] <@IEF> radio/hboot is protected anyway
[19:39] <@IEF> you won't be able to overwrite that from the FS
[19:39] <@IEF> er
[19:39] <@IEF> OS
[19:40] <@IEF> besides, the order of the partition also will prevent that
[19:40] <melethron> ok .... i didn't really understand all you did to get acces to hboot .... so im just cautious
[19:41] <@IEF> radio -> hboot -> recovery -> boot -> system -> cache -> userdata
[19:41] <@IEF> yeah, you have to be cautious about fastboot use only
[19:41] <@IEF> fastboot flash hboot is completely unchecked
[19:41] <melethron> and messing with mtd will keep radio hboot and recovery still save ....
[19:41] <@IEF> yeah
[19:42] <@IEF> mtd is just a block layer
[19:42] <@IEF> you're just putting information inside of it

So if anyone wants to give it a try do what you like with nand as long as you are S-OFF.

So actually flashing a radio is more risky than formating nand to another fs....

But don't forget that you need recovery support of the filesystem you use.

But who knows. Maybe we some soon:

[20:34] <melethron> IEF if you like logFS or ubifs when you try it will you implement support in recovery kernel ???
[20:35] <@IEF> yeah, that would be easy enough
[20:35] <@IEF> just have a seperate recovery for it

 

[rafi300]

could some one advise on the fact that this script can be replaced in the init.d folder of ReflextSenseHD v1.4.1 rom with fat32 partition and a 2gb btrfs partition?

 

[rafi300]

I'm doing something wrong that this script is not running correctly in reflextsenseHD v1.4.1. Could someone point out what im doing wrong

Here are my steps:

First parition my sdcard with gparted.
1. Create Primary Partition of 12.9gb Fat32
2. Create 2.00gb btrfs partition.

Do complete wipe.
Flash ReflextHDv1.4.1 A2SD+ version
Flash btrfs kernel of coutts.
Plug phone into usb.
update the busybox with command.
"adb shell"
You should get a ~# prompt now.

#mount /system
#exit

Replaced old busybox exec with the new one.
$adb push c:\busybox-armv6l /system/xbin/busybox

Removed 03stuff2sd
$adb rm /system/etc/init.d/03stuff2sd

Added the script to init.d folder:

$adb push c:\40data2ext4 /system/etc/init.d/40data2ext4

back to the shell:
$adb shell

#chmod 755 /system/xbin/busybox

#chmod 755 /system/etc/init.d/40data2ext4

#reboot

and wait....

Than after 10min i get only lockscreen and no rosie?

Please help.

 

[bernabap]

I'm doing something wrong that this script is not running correctly in reflextsenseHD v1.4.1. Could someone point out what im doing wrong

Here are my steps:

First parition my sdcard with gparted.
1. Create Primary Partition of 12.9gb Fat32
2. Create 2.00gb btrfs partition.

Do complete wipe.
Flash ReflextHDv1.4.1 A2SD+ version
Flash btrfs kernel of coutts.
Plug phone into usb.
update the busybox with command.
"adb shell"
You should get a ~# prompt now.

#mount /system
#exit

Replaced old busybox exec with the new one.
$adb push c:\busybox-armv6l /system/xbin/busybox

Removed 03stuff2sd
$adb rm /system/etc/init.d/03stuff2sd

Added the script to init.d folder:

$adb push c:\40data2ext4 /system/etc/init.d/40data2ext4

back to the shell:
$adb shell

#chmod 755 /system/xbin/busybox

#chmod 755 /system/etc/init.d/40data2ext4

#reboot

and wait....

Than after 10min i get only lockscreen and no rosie?

Please help.

I dont get it, why this 40data2ext4 script is in this thread?

 

[bernabap]

Melethron that 40data2ext4 script on this OP is right? Should not be 40data2btrfs?

 

[melethron]

I'm doing something wrong that this script is not running correctly in reflextsenseHD v1.4.1. Could someone point out what im doing wrong

Here are my steps:

First parition my sdcard with gparted.
1. Create Primary Partition of 12.9gb Fat32
2. Create 2.00gb btrfs partition.

Do complete wipe.
Flash ReflextHDv1.4.1 A2SD+ version
Flash btrfs kernel of coutts.
Plug phone into usb.
update the busybox with command.
"adb shell"
You should get a ~# prompt now.

#mount /system
#exit

Replaced old busybox exec with the new one.
$adb push c:\busybox-armv6l /system/xbin/busybox

Removed 03stuff2sd
$adb rm /system/etc/init.d/03stuff2sd

Added the script to init.d folder:

$adb push c:\40data2ext4 /system/etc/init.d/40data2ext4

back to the shell:
$adb shell

#chmod 755 /system/xbin/busybox

#chmod 755 /system/etc/init.d/40data2ext4

#reboot

and wait....

Than after 10min i get only lockscreen and no rosie?

Please help.

Hmm, from the first look at the update-script and the init.d scripts i think it should work. Symlinks should be alright from what i see.

Make sure your sure your 2nd partition is mounted with "df" in "adb shell" and check the logcat (adb logcat) of the first boot. If you are unsure just post the results (for logcat till the the btrfs finish message)

OR if you are not that focused on Neophytes HD Rom you wait a bit. I'm just doin the last test with my HD rom (checking both btrfs and ext4). I used Pays HD Rom for that so if you are fine with that one too just wait some min. I'll upload it in the next couple of minutes.

 

[melethron]

Melethron that 40data2ext4 script on this OP is right? Should not be 40data2btrfs?

Oops ... sry ....

Well i updated both the same time.... only difference between them is this line

busybox mount -t [COLOR="Red"]btrfs[/COLOR] -o noatime,nodiratime,[COLOR="Red"]ssd_spread[/COLOR]  /dev/block/mmcblk0p2 /data;

Well i made 1 script for both for my ROM. But there are ROM specific stuff in. When im finished i add one script for both which detects the partition type and then mounts with specific options....

sry again ....

EDIT: I really wondered why that happened bcs im sure i went to btrfs dir... this wont happen again .....

 

[melethron]

Btrfs ready HD rom can be found here.

 

[melethron]

I tested the I/O speed with mount options "compress". While it increases speed on large files it slows things down on small files. Especially on db writes which leads to lags. So spare yourself the time if you wanted to try that.

Also i compared bfq / cfq / noop. Bench its almost the same. But noop feels more smooth. Didn't try "deadline" (not in coutts kernel) but i think it can't be improve from noop.

About noop note this:

NOOP scheduler is best used with solid state devices such as flash memory or in general with devices that do not depend on mechanical movement to access data (meaning typical "hard disk" drive technology consisting of seek time primarily, plus rotational latency). Such non-mechanical devices do not require re-ordering of multiple I/O requests, a technique that groups together I/O requests that are physically close together on the disk, thereby reducing average seek time and the variability of I/O service time.

 

[melethron]

Benchmarks

I did some sqlite benchmark to estimate performance of filesystems on /data/data writes. To make a "clean" benchmark i did the following before changing the filesystem:

- Full Wipe (+format the sd partition)
- fresh ROM install (avaritia HD v 0.2 of course)
- No setup of wifi and apn (so that no other writes happen from any sync)
- Wait 1 min after rosie loaded
- run 5 tap benchmarks in a row

The results (not completly finished but rest will come) are contrary to what i found before. The reason i had a smoother feeling on btrfs than on ext4 was that i switched to noop scheduler. The speed increase i noticed was due to the scheduler.


Part1: SCHEDULER
The first Benchmark is on EXT4 only and is only about scheduler. While i talked to people about scheduling there where 2 arguements against noop scheduler:

1. Noop means "NO OPereations" and means the scheduler doesn't do anything at all. No scheduling is BAD and reduces performance.

2. While "No-op" is definitly the best scheduler for sd-cards since there are 2-4 parallel channels which makes scheduling completly useless it is still a matter of the controller used. If the controller of the sd-card is "bad" it can't handle all the unsorted I/O requests.

About the argument 1:

Thís argument is absolutly true on a normal HDD (rotational) because of the seek times of the "actuator arm" to move the "head" and noop fails completly on stress test and is like 25 times slower than cfq. On the other side this argument is absolutly WRONG on any kind of flash based device. There is no actuator arm, no head and also NO seek time. Optimizing for seek time is thus complete nonsense. All scheduling does then is use CPU. (so much for the theory)

About argument 2:

This is a practical issue and can only be tested to see if the controller can handle the I/O. (since we have no information on controllers used in microSD all we can do is testing here.

Here are the Bench results for EXT4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)

1. Noop:

DB overall: 402,4
Stress overall: 915,4

2. CFQ:

Db overall: 397,4
Stress test: 909,2


These results AREN'T speeking for themselfs and have to be explained. The results are relativly even there doesn't seem to be a "real improvement" from "noop". But there is something this bench doesn't show: The "smoothness":

Ill make an example. You save something with an app (like clicking on superuser accept button) while another app is syncing in background and also writes to the database. The write request are the following:

App1: write to sector 301,302
App2: write to sector 101,102,103,104,105

The timeline of the I/O request in wich these request occur are these and this is also the the way the I/O request are sent to SD with NOOP:

101,102,103,301,104,302,105

The seek time on HDD would then be 103->301 , 301->104,104->302, 302->105

The scheduler will now sort this request most likely to this:

101,102,103,104,105,301,302

So the result in only one seek: 105->301

And this will GREATLY increase the time needed on random I/O request BUT ONLY ON A HDD (Check how much longer the NOOP scheduler needs for random reads/writes: http://www.linux-magazin.de/var/lin...bb4_jpg/18703-1-ger-DE/abb4_jpg_reference.jpg)

But again. I dont have a HDD in my phone but an SSD. What would this sorting mean: The results above mean that (OH WHAT A WONDER - i proved that an sd-card is no harddisk ^^) that there are no seek times. The random read/writes of the stress test have the same results. This means it DOESNT MATTER IN ANY KIND IF THE I/O is like this if you only consider the time needed:

101,102,103,104,105,301,302

or like this

101,102,103,301,104,302,105

Both need the same time (and cfq is even slightly slower bcs it needs cpu time to sort the requests). But as i said there is one thing you don't see in the bench values:

As you are the user who just clicked on the "superuser" accept button you need to WAIT for the write of sector 104,105 until the app even starts to write the data. And this results in ....
LAGS.


EDIT: Some might think now that they can interpret the results the other way around. If the user-app is the other app from the example. This would really apply to this example. But simply think of the number of processes that are running. If you doubt this posting test it yourself.


Part 2: with Benchmarks of the filesystems will follow.....

Attachment: "Raw" results from the "TAP benchmark" log.

 

[Fightspit]

If I understand, the Noop scheduler can improve the smoothness despite of the time needed on I/O request which is true on HDD because it takes a few time to change from a sector to another one. But it seems it is totally irrelevant on a SSD/Sd-card because the access time is really smaller than the HDD's. So the SSD/Sd-card can handle the noop scheduler without being affected by the time needed on I/O request.

EDIT:
I found an interesting paper (which confirms your statement) about the current schedulers (noop, anticipatory, deadline & cfq) and the new ones on SSD.
Look at the page 3 (figures 1 & 2) and 8-10 for the benchmarks
http://dropzone.tamu.edu/techpubs/2009/TAMU-ECE-2009-02.pdf

 

[HTF]

Here is an interesting blog post from one of the android devs, in it there is an explanation of why YAFFS on the nand is a big area of speed loss. From what he is saying we could get a decent improvement in "stuttering" and "lag" by switching it to either EXT4 like the Nexus S or some other nand filesystem (I would personally go for EXT4 due to its high availability in kernels).

Obviously this does mean data2ext can help as well as it means any writes that would normally go to data on the nand go to the sd card on ext.

 

[melethron]

If I understand, the Noop scheduler can improve the smoothness despite of the time needed on I/O request which is true on HDD because it takes a few time to change from a sector to another one. But it seems it is totally irrelevant on a SSD/Sd-card because the access time is really smaller than the HDD's. So the SSD/Sd-card can handle the noop scheduler without being affected by the time needed on I/O request.

EDIT:
I found an interesting paper (which confirms your statement) about the current schedulers (noop, anticipatory, deadline & cfq) and the new ones on SSD.
Look at the page 3 (figures 1 & 2) and 8-10 for the benchmarks
http://dropzone.tamu.edu/techpubs/2009/TAMU-ECE-2009-02.pdf

Yep. On a hdd it sucks:

As you can see a "stress test" increses the time needed by almost factor 20. But on the sd-card there is no difference at all. Also it makes more use of the parallel channels on a ssd. This is like a tiny raid-0 device between some nand banks.

The only reason for scheduling in the first was to reduce the time that a the mechanical parts need to move the head. Thats also the reason you do a defrag on a HDD so that files are at the same place. Defragmentation doesnt matter at all on a ssd.

Here is an interesting blog post from one of the android devs, in it there is an explanation of why YAFFS on the nand is a big area of speed loss. From what he is saying we could get a decent improvement in "stuttering" and "lag" by switching it to either EXT4 like the Nexus S or some other nand filesystem (I would personally go for EXT4 due to its high availability in kernels).

Obviously this does mean data2ext can help as well as it means any writes that would normally go to data on the nand go to the sd card on ext.

Hmm i didn't mess with internal memory yet. I first do all the bench marks of sd card and then i make some test by formating the cache partition of the internal memory. As IEF from alpha rev said there is no risk on this if you have fastboot. To revert back to normal you can just do "fastboot erase <partition>" and the partition is clean again.

Also note that internal memory works a bit different than an microSD. Internal is mtd RAW nand chips and sd-card is nand with a controller that emulates an LBA. The difference now between our desire and Nexus S is that Nexus is actually the same as the samsung galaxy S. And this has an internal sd-card. Apart from that an sd-card can also be very different (controller, FTL etc). So there might be differences. But apart from that i agree. Of the "common" filesystems EXT4 is the best choice.