[Benchmarks] File System Performance: F2FS vs EXT4

I have recently implemented Samsung's Flash Friendly File System (F2FS) into my kernel for the Z1 (Pimped Kernel).

In order to see what this file system really brings to the table I decided to run some benchmarks, in a controlled environment, trying to eliminate as many fluctuation factors as possible.
The methodology is described below.

For this first round, only the /data & /cache partitions were formatted as f2fs, in order to see what kind of improvement we would be able to get out of formatting /system as f2fs too later on.


Benchmarks Used

• CF-Bench
• 0xBenchmark
• Androbench
• RL Bench
• AnTuTu
• Quadrant

Methodology

• Fully wiped and fresh install of the 26/03 CM nightly
• PA Gapps installed (pa_gapps-modular-full-4.4.2-20140301-signed) & updated
• Additionnal apps installed:
  • Pimp My Z1
  • Disk Info
  • Benchmark apps
  
  
• No other app active in the background during benchmarks
• Fresh reboot before running the batch of benchmarks
• Airplane mode enabled (except for quadrant which requires an internet connection to show the results)
• Order of the benchmarks:
  • RL Bench
  • Androbench
  • CF Bench
  • 5 minutes pause with powersave governor to cool down the device
  • 0xBenchmark
  • AnTuTu
  • Quadrant
  
  
• Pimp My Z1 Settings:
  • CPU Governor: performance (in order to eliminate differences due to frequency scaling, as this governor just stays at max freq all the time).
  • GPU Governor: performance
  • GPU Max Freq: 600MHz
  • Intelliplug: disabled
  • Dynamic Fsync: enabled
  • I/O Scheduler: FIOPS for both emmc & sdcard
  • Readahead: 1024kb for both emmc & sdcard
  • Entropy contribution: disabled for both emmc & sdcard
  • PowerSuspend: disabled
  • KSM: enabled, 512 pages-to-scan, 500ms timer
  • Thermal throttling: disabled

Disk Usage
One thing I noted is that F2FS has a much higher usage of the /cache partition than ext4, which is to be expected coming from a log-based file system such as F2FS.
As you can see in the screenshots below (ext4 on the left & f2fs on the right), only 4mb out of the 192mb /cache partition is used by ext4, while f2fs uses 68% of that same partition with 114mb used.
Another thing to note is that under f2fs, the reported size of the /cache partition is apparently 2mb larger than on ext4, with 198mb.

EXT4 / F2FS




1. RL Bench
RL Bench is an SQLite benchmark, it only tests database read/writes with various SQL commands in various amounts.
As we can see in the results below, f2fs trumps ext4 with an overall time of only 14.417 seconds versus the 21.403 seconds of the latter.

EXT4


F2FS




2. Androbench
Androbench is supposedly an all-around I/O benchmark.
On this one, the results were so incredibly appart that I think we can safely assume a big part of these results are due to poor coding on the benchmark side.
Still, if we believe the results, f2fs provides slightly slower reads (51mbps vs 66mbps) but over 120x faster sequential writes & 300x faster random write speeds!
I had to reboot and run it a second time to make sure this wasn't an error, but the second run results were even a bit better, as you can see in the screenshots below.

EXT4


F2FS
1st Run


2nd Run




3. CF-Bench
CF-Bench is an overall benchmark by Chainfire which measure everything except graphics.
Here again, f2fs is the winner with a better overall score.
That being said, the native writes are a little inferior to the ext4 native write score with 529 vs 535, but has a higher native read score with 996 vs 913.

EXT4


F2FS




4. 0xBenchmark
This reliable open-source benchmark by 0xlab is an overall benchmark used by the Linaro android team during their famous toolchain demonstration.
It has the advantage to output the results in text, xml and json, which you can download below.

On the SunSpider javascript benchmark run, f2fs gets a better score with 744.5ms vs 792.0ms for ext4.
Strangely, on Linpack, ext4 gets a better score with 156mflops/s vs 122mflops/s for f2fs.
On the other hand, Scimark2 reports better mflops/s scores for f2fs in all 6 of its tests, as seen below:

EXT4 Scimark2
Results in Mflops/s

F2FS Scimark2
Results in Mflops/s

EXT4 full results: http://d-h.st/iER
F2FS full results: http://d-h.st/iOf



5. AnTuTu Benchmark
No need for an introduction, it's probably the most popular benchmark these days.
On this one again, f2fs is the clear winner with a difference of over 1200 points.

EXT4


F2FS




6. Quadrant Benchmark
Another overall mainstream type of benchmark.
F2FS wins on this one too, with an advantage of 1100+ points.

EXT4 / F2FS





Conclusion, F2FS actually is an improvement over Ext4 for flash devices, and considering its very young age it will most likely be improved over time.

Poesini said:

How are you getting 600 Mhz on the GPU?

I have

PHP:

root@honami:/sys/devices/fdb00000.qcom,kgsl-3d0/kgsl/kgsl-3d0 # cat gpu_available_frequencies
450000000 389000000 320000000 200000000 27000000 320000000 320000000 200000000 200000000 
root@honami:/sys/devices/fdb00000.qcom,kgsl-3d0/kgsl/kgsl-3d0 # echo 600000000 > max_gpuclk
root@honami:/sys/devices/fdb00000.qcom,kgsl-3d0/kgsl/kgsl-3d0 # cat gpu_available_frequencies
450000000 389000000 320000000 200000000 27000000 320000000 320000000 200000000 20000000
root@honami:/sys/devices/fdb00000.qcom,kgsl-3d0/kgsl/kgsl-3d0 # cat max_gpuclk
450000000

And its still showing 450 on Pimp and Trickster

Click to expand...

Click to collapse

You're probably just not running Pimped Kernel, otherwise you'd have 389 & 600 too.

EDIT: weird, you do have 389, but not 600.

Poesini said:

I am running Pimped Kernel and I have reflashed it also (I have /cache and /data with f2fs)

Click to expand...

Click to collapse

That's really weird. Which rom are you running ?
I'm also running the last F2FS build and here's my output :

You have some messed-up steps, too, the 270MHz shouldn't be there, and certainly not after a 320MHz step, that's really weird.

Poesini said:

I am using Omni 3/15 lol

Click to expand...

Click to collapse

I made some changes to the GPU frequency table to try and fix that for next build, although I really don't see what caused this on your device.
Please report back after next build if the issue persists.

Let's take this discussion back to the kernel thread though, please.

Chainfire said:

I would like to point out that CF-Bench does not take disk reads and writes into account for its final device score for one very important reason:

Android's base Linux kernel (at least when I wrote CF-Bench) does not respect the flag to turn off low-level buffering. As such, the measured results do not reflect actual performance, and it is unknowable how big the unreliability of the results is.

That being said, if one option benchmarks consistently and significantly faster than the other, on the same device with the same setup, then it is likely to actually be faster - you just don't know how reliable the produced benchmark index is, and I certainly wouldn't compare them to scores from other devices.

Click to expand...

Click to collapse

Thanks a lot for the heads-up, that's good to know.
I probably should have taken the time to do at least 4 or 5 runs of each benchmark and calculate the average score in order to get more reliable results.
As you said, I wouldn't give too much importance to these results, but the fact the the f2fs runs came ahead every time is at least a pointer that there is some kind of improvement.

I will do another round of benchmarks with the /system partition in f2fs as well soon, I'll try to do several runs this time around.
Maybe I'll also add this cross-compiled Bonnie++ to the mix (won't do full runs though, just test a few hundred mbs, takes way too long otherwise)

->--- said:

I am on ext4 and my GPU clocks are also very different from yours. I tried to rewrite the file manually, but RootExplorer want overwrite the file.It was strict read-only. I changed that to rw-rw-r--, but not a chance. Not by creating a new file and copying into the folder, not by editing the original file. I will try to edit the file in the flashable zip and overwrite it hereby.

Edit:
Its in the boot.img, am I right? So no chance to correct it.

Click to expand...

Click to collapse

Like any file under /sys, it' a sysfs interface, those "files" permissions are handled by the kernel.
What you input into the sysfs interfaces is processed by the kernel, and it usually has safeguards in case the value is too high/low or invalid, same for the permissions, if a sysfs node is defined as read-only then you won't be able to change this unless you recompile the kernel and change this behaviour in the module/driver itself.
Either way you wouldn't be able to just modify it like that, there's a lot more to defining a GPU frequency than just the frequency itself.
This interface is simply here to expose which frequencies are available to userspace.

This isn't related to the filesystem type, I had the same freq table on ext4 before I implemented f2fs in the kernel. Can't seem to figure out what is the cause of this issue, as it seems to be different from a device to another on the same build, and isn't related to the CPU's pvs binning level.

Please keep discussion about this issue in the kernel thread.