Development

42 Vote(s)
64.62%
General

23 Vote(s)
35.38%

# [REF] Kernel Features and Benchmarks

n/a posts
Thanks Meter: 0

By bedalus, Guest on 12th January 2012, 03:48 PM
New to this stuff? Read this.

Features: plain text or FEATURE TABLE

All ICS Kernel BENCHMARKS

Where did the other benchmarks go?

All ICS ROM Benchmarks: this thread

CPU Governors and I/O Schedulers: this thread

Thanks to all the developers.

bedalus
12th January 2012, 03:48 PM |#2
Guest
Thanks Meter: 0

More
This information regards the Kernel Benchmark spreadsheet.

Sheet 1: The Overall Scores

I realised that looking at all the raw data was going to give people a headache. (For those who enjoy headaches, like me, the raw data is still visible lower down.)

How did I arrive at the percentages?

First, I looked at an individual result, e.g. the AnTuTu 2D graphics score for simplekernel. I use it as the numerator of a fraction where the denominator is the average of all the AnTuTu 2D results from all the kernels. If above 1, this score is better than the average. If less than 1, this score is below. I do the same thing for all the graphics results, and then calculate the mean. This provides equal weighting for all the various graphics benchmark results. In other words, it prevents a bad quadrant result from skewing the overall picture if the rest of the graphic results are good.

How to interpret these percentages? At the time of writing, if we compare morfic's RAM result of 109% to the terryhau's stock(ish) simplekernel RAM result of 96%, we can say with some certainty that morfic's RAM performs 13% better than stock.

Sheet 2: Statistics for Variance

Firstly, thanks are due to morfic and ezekeel whose scepticism of benchmarks made me want to investigate the issue of variance in the results.

The most important column is n. Check out these rows:
Row 8: AnTuTu's Database I/O benchmark can vary by as much as 7.55%
Row 16: Quadrant Advanced Memory benchmark can vary by as much as 6.66%
Row 21: Smartbench 2012's Gaming Index can vary by a massive 24.42%
Row 25: Vellamo's Aquarium Canvas can vary by 12.11%
Row 26: Vellamo's Pixel Blender can vary by 5.66%
Row 28: Vellamo's Aquarium Canvas can vary by 5.59%

These figures are calculated within two standard deviations of the mean, or in other words, a 95% certainty. For example, we can say that 95% of the time, the Quadrant Advanced Memory benchmarks we run will give a result within 6.66% of a genuinely accurate figure. This is okay, but just not good enough for me!

By eliminating these test results, I have improved the accuracy of my overall scores to within 2% of the genuine mean, with a 95% certainty. In the spreadsheet, you can see the individual accuracies for each of the overall scores: CPU, RAM, I/O, GRAPHICS. Red highlighting of the figures takes these individual variances into account, and shows which tests are definitely at least 1% better than stock, even assuming stock tested at it's maximum, and the compared kernel tested at it's minimum.

Testing Methodology
To test the kernels I want a fair environment, so any differences in the results are down to the kernel, and nothing else. To achieve this I first make sure I have a clean system:
1) Format /system and /cache and wipe dalvic
2) Install the ROM, install the kernel
3) Boot up, use nstools to select deadline for I/O scheduler
4) Then select ondemand governor and set the minimum frequency to equal the maximum, which is not the same as performance, but close enough if you see my CPU governor comparison chart. CPU cycles are wasted as the governor tries to decide what frequency to jump to, even if it has no option! Thanks to Steve Garon for the info. This method was suggested by morfic, thanks again! I/O scheduler is set to deadline. Thanks tchaari!
5) Restore all my benchmark programs. Set everything to off, no gps, no sync, no BT, airplane mode. Force GPU rendering is selected. Wifi is on for connecting to the benchmark servers. A power adaptor is in use so the battery is always full.

Then I begin testing:
6) Power off and power back up. When lock screen arrives, wait one minute to settle the system, i.e. until screen auto-off. Count to three. Unlock, and begin testing, recording all scores at the bottom of my spreadsheet. I copy them up (from the old data to the current raw data section) once complete. This avoids the overall score constantly altering for any current viewers, and means I keep a record of all previous tests under the heading Old Data.
bedalus
12th January 2012, 03:50 PM |#3
Guest
Thanks Meter: 0

More
Source Threads For Kernels and Stated Features

r_data (Air Kernel): http://forum.xda-developers.com/show....php?t=1469842
- I/O Schedulers: cfq, deadline, noop, sio (Default is cfq)
- CPU governors: conservative, ondemand, performance, lazy, lulzactive(v2), smartass, smartassV2, intellidemand, interactiveX, scary, lagfree Lionheart (Default is ondemand)
- BLN, BLD, Touch Wake and custom voltage by Ezekeel
- Overclockable to 1.2Ghz
- SQLB
- BIGMEM
- init.d support
- Voodoo sound and colour
- Deep Idle
- Live OC
- CPUIDLE
- 65 fps
- Blue Colors
- BIGMEM
- init.d support
- USB OTG support
- Great Battery Life!
- CIFS, TUN support
- SLQB memory allocator
- Optimized CRC32 algorithm
- ARM RWSEM algorithm

netarchy: http://forum.xda-developers.com/show....php?t=1413117
- CPU governors: interactive, conservative, userspace, powersave ondemand, performance (Default is ondemand)
- Overclocking up to 1.4ghz enabled
- Voodoo Sound driver
- Voodoo Colour
- CIFS support
- TUN support
- NFS support
- Updated kernel base (Currently 3.0.14)
- Cpu policy fix to avoid crashes with SetCPU and screen off profiles
- EXT4 remount tweak*
- BLN
- Custom Voltage Interface
- Deep Idle
- Backlight Dimmer

mathkid95: http://forum.xda-developers.com/show....php?t=1361477
- I/O Schedulers: cfq, bfq, deadline, noop, SIO, V(R) (Default is deadline)
- CPU governors: lulzactive, interactiveX, minmax, smartassV2, lazy, ondemand, userspace, performance, intellidemand, lionheart, wheatley (Default is ondemand)
- Based on Linux kernel version 3.0.14
- OC to 1.46 Ghz
- Voodoo sound and colour
- BLN
- BFS
- SLQB memory allocator
- Stochastic Fair Blue (internet packet queue management algorithm)
- CIFS, TUN, NFS
- INIT.D*
- Deep Idle
- LiveOC
- BLX
- BLD
- Ezkeel's voltage mod
- Touch Wake
- Bigmem

Ezekeel: http://forum.xda-developers.com/show....php?t=1336529
- CPU governors: interactive, conservative, ondemand, userspace, powersave, lazy, performance (Default is ondemand)
- Based on stock Android kernel 3.0.8 IML74K
- CPUfreq governor 'lazy'
- Live OC version 1
- Custom Voltage version 1
- Battery Life eXtender (BLX) version 1
- Power saving state DEEP IDLE version 2
- Touch Wake version 1
- Backlight Dimmer (BLD) version 3
- Backlight Notification (BLN) version 9
- Seperate BFS and CFS versions available
- SLQB memory allocator
- Voodoo Color version 2
- Voodoo Sound version 10
- CIFS
- TUN
- Bigmem (+20 MB more available RAM)

Steve.Garon: http://forum.xda-developers.com/show....php?t=1427352
- I/O Schedulers: cfq, V(R), deadline, noop, sio (Default is cfq)
- CPU governors: lulzactive(v2), smartassV2, lazy, ondemand, conservative, performance (Default is conservative)
- Boots @ 1.1Ghz Overclock (Bus 220MHZ)
- Undervolted to maximize battery life
- Voodoo Sound V10
- BIGMEM for more available memory
- Kernel updated to 3.0.16
- Increased refresh rate 65fps (alternate version 56fps/improved colour available)
- Backlight Dimmer
- Battery Life Extender
- TouchWake
- Custom voltage
- CIFS Module built-in
- Cold colors
- Cpuidle backport from 3.2 Kernel
- up to 1320 Mhz overclock
- init.d support

Morfic: http://forum.xda-developers.com/show....php?t=1036996
- CPU governors: conservative, ondemand, userspace, powersave, interactive, performance (Default is ondemand)
- BAREBONES i.e. no features
- TEUV version has BLN and BLD

Terryhau: http://forum.xda-developers.com/show....php?t=1420542
- I/O Schedulers: cfq, deadline, noop (Default is cfq)
- CPU governors: conservative, ondemand, userspace, powersave, interactive, performace (Default is ondemand)
- BLN
- BLD
- CIFS module
- Init.d
- Insecure (ro.secure=0)
- Voodoo (*optional)

Reddv1: http://forum.xda-developers.com/show....php?t=1173797
- I/O Schedulers: conservative, ondemand, userspace, powersave, interactive, performance (Default is ondemand)
- CPU governors: cfq, deadline, noop (Default is cfq)
- UV
- Voodoo Sound V.10
- init.d Support
- ext4 optimizations
- BLN
- Compatible w/ NS & NS4G
- BLD
- Touch Wake
- Deep Idle
- Live OC
- Custom Voltage
- ICS version, ADB/ UMS/ Wifi tethering/ USB tethering/ Data stats & limits all work
- ICS version, ext4 optimization, init.d support and all mods from GB version
- Lazy CPU governor (but not in testing version)

Eugene373: http://forum.xda-developers.com/show....php?t=1401455
- I/O Schedulers: cfq, bfq, deadline, sio, noop, V(R) (Default is cfq)
- CPU governors: minmax, lazy, lulzactive, lagfree, interactive, interactiveX, conservative, ondemand, userspace, powersave, smartass, smartassv2, scary, performance, brazilianwax, intellidemand, lionheart, ondemandx, savagedzen, Wheatley (Default is ondemand)
- OTA ICS Compatible
- ro.secure=0 ( So you can push file's via adb )
- Voodoo Sound
- Voodoo Color
- Touch-wake
- BT Sleep
- LCD voltage drop
- Bigmem patch
- BLN
- TUN
- CIFS
- Battery 100%
- ext4 edits for smoother rom
- init.d Supported
- OC now Stable up to 1.44GHz
- Live OC is limited to 115%

novic_dev: http://forum.xda-developers.com/show....php?t=1525257
- I/O Schedulers: cfq, bfq, deadline, sio, noop, V(R) (Default is V(R))
- CPU governors: minmax, lazy, lulzactive, lagfree, interactive, interactiveX, conservative, ondemand, userspace, powersave, smartass, smartassv2, scary, performance, brazilianwax, intellidemand, lionheart, ondemandx, savagedzen, Wheatley (Default is ondemand)
• Linux Kernel version 3.1.10
• GRSecurity version 2.2.2 with different security features enabled by default (see GRSecurity Working Feature List for details)
• OC up to 2.0 GHz (still need to be tested!!!)
• Advanced LiveOC!!! (ALOC) support with Selective LiveOC MOD (thanks to byeonggonlee) - use this feature with caution now due to LiveOC unlimited up to 150% for testing purposes!!!
• EXT4, NTFS, CIFS support
• init.d support
• Compatible with any OTA-based and AOSP ICS Roms
• Can be managed with NSTOOLS
• Provides Galaxy Nexus boot animation
• Voodoo Sound and Vodoo Colors
• BIGMEM for more available memory
• Backlight Dimmer(BLD, disabled by default)
• Battery Life Extender (BLX, not active for now, need testing)
• TouchWake (TW, enabled by default)
• Custom Voltage Interface
• Deep Idle version 2 (thanks to Ezekeel, disabled for now due to stability issues!)
• CPUIdle from Linux 3.2.x backport
• SLQB Memory Allocator with GRSecurity integration (using SLUB for now due to stability issues!)
• USB OTG (Build 5) support (use it with this app)
• Blue Colors support for Screen Color Scheme
• 50 FPS and 65 FPS Screen Refresh Rate support

Franciscofranco: http://forum.xda-developers.com/show....php?t=1357333
- CPU governors: interactive, conservative, ondemand, performance (Default is ondemand)
- 3.0.8 kernel version
- Full ramdisk with the proper mounting flags for extreme speed
- tweaked Conservative governor
- SLQB memory allocator
- Kernel stripped of debug options +others like cgroups, autogroup, swap, etc.
- TinyRCU as being the obvious choice
- CRC32 lib updated
- I/O-less dirty throttling (backport: kernel 3.2: http://kernelnewbies.org/Linux_3.2)
- Ext4 with 3.2 patches (see link above)
- Proportional Rate Reduction for TCP (see link above)
- Fsync() to oblivion just like I had in the GB kernels
- Init.d support in the ramdisk
- BLN
- Voodoo Sound

Peter Alfonso: http://forum.xda-developers.com/show...&postcount=265
- I/O Schedulers: cfq, deadline, noop (Default is cfq)
- CPU governors: conservative, ondemand, userspace, powersave, interactive, performance (Default is ondemand)
- https://github.com/peteralfonso
- Optimised Stock Kernel

Thalamus (Part of Oxygen): http://goo.gl/o9bxK
- CPU governors: ondemand, performance (Default is ondemand)
- Optimised Stock Kernel

Link to stock ics kernel: http://forum.xda-developers.com/show...&postcount=120
12th January 2012, 03:54 PM |#4
Senior Member
Thanks Meter: 113

More
This is nice. Also, it could be a good idea for roms. The only thing is to redo the benchmarks every once in a while, as the developers keep updating. Great work!

Transmitted from somewhere in space... from my Nexus S... and Tapatalk.
12th January 2012, 03:54 PM |#5
Senior Member
Somewhere in South Korea
Thanks Meter: 172

More
Steve Garen's kernel seems to have the best benchmark results. Anyways thanks for this I was curious about this but too lazy to do it myself.

Sent from my Nexus S using Tapatalk
12th January 2012, 03:55 PM |#6
Senior Member
Thanks Meter: 83

More
So it's a test for the performance of kernels against Slim ICS ROM.
bedalus
12th January 2012, 03:57 PM |#7
Guest
Thanks Meter: 0

More
Many thanks to krarvind for Slim ICS, the host ROM.

Testimonials

Kernel developer
Quote:
Originally Posted by mathkid95

Good job with this! It can be very helpful for ppl wanting to see how the kernels stack up...

Kernel developer
Quote:
Originally Posted by eugene373

It's nice to see somebody put raw facts behind something & not just bloated scores from Quadrant..

XDA user
Quote:
Originally Posted by mobile_pc

Your great research inspires the kernel devs to further efforts
Thanks

Rom developer
Quote:
Originally Posted by Cybergr

Things are getting more clearer now..thanks for taking your time testing and posting such a useful comparison of different ICS kernels...Helps everyone who is really intrested and sometimes confused...devs or not...but specially the lazy one! Very cool and interesting idea to post a Kernel comparison in here bedalus..!!

XDA veteran and kernel crackflasher
Quote:
Originally Posted by simms22

... when things differ and deviate from what you expect that draws attention. thats why benchmarks can be very useful to developers. they will know right away if they messed something up or did something very right... impressed with your effort. thank you

Kernel developer
Quote:
Originally Posted by steve.garon

Cool comparison. Tells me that I can improve 2D and 3D score somehow...
My crackflashing/crackbuilding habbits are surfacing again.

bedalus
12th January 2012, 03:59 PM |#8
Guest
Thanks Meter: 0

More
Quote:
Originally Posted by suksit

So it's a test for the performance of kernels against Slim ICS ROM.

I'm not certain, but I believe the performance of the kernels may be compared directly so long as the testing circumstances remain absolutely equivalent.
12th January 2012, 04:04 PM |#9
Senior Member
San Diego
Thanks Meter: 111

More
Very cool to see.. Nice work bedalus
12th January 2012, 04:04 PM |#10
Senior Member
Vienna
Thanks Meter: 213

More
Thank you!
12th January 2012, 04:07 PM |#11
Senior Member
Thanks Meter: 113

More
Quote:
Originally Posted by bedalus

Oh no, what have I let myself in for!

You're in for a lot of work, but I don't think it's necessary to update after each of the kernels in updated, just on a periodical base (of course, you have a life and that comes first)