5,606,409 Members 33,506 Now Online
XDA Developers Android and Mobile Development Forum
View Poll Results: Which section does this belong in?
Development 42 64.62%
General 23 35.38%
Voters: 65. You may not vote on this poll

[REF] Kernel Features and Benchmarks

Tip us?
 
bedalus
Old
(Last edited by bedalus; 20th March 2012 at 12:22 PM.)
#1  
bedalus's Avatar
Recognized Contributor - OP
Thanks Meter 7787
Posts: 5,969
Join Date: Jun 2011
Location: Chester
Default [REF] Kernel Features and Benchmarks

New to this stuff? Read this.

Features: plain text or FEATURE TABLE

All ICS Kernel BENCHMARKS

Where did the other benchmarks go?

All ICS ROM Benchmarks: this thread

Battery drain: this thread

CPU Governors and I/O Schedulers: this thread

Power Saving Governors: this thread

Thanks to all the developers.
For various benchmarks, see my threads: >here<
The Following 105 Users Say Thank You to bedalus For This Useful Post: [ Click to Expand ]
 
bedalus
Old
(Last edited by bedalus; 3rd February 2012 at 10:03 AM.)
#2  
bedalus's Avatar
Recognized Contributor - OP
Thanks Meter 7787
Posts: 5,969
Join Date: Jun 2011
Location: Chester
This information regards the Kernel Benchmark spreadsheet.

Sheet 1: The Overall Scores

I realised that looking at all the raw data was going to give people a headache. (For those who enjoy headaches, like me, the raw data is still visible lower down.)

How did I arrive at the percentages?

First, I looked at an individual result, e.g. the AnTuTu 2D graphics score for simplekernel. I use it as the numerator of a fraction where the denominator is the average of all the AnTuTu 2D results from all the kernels. If above 1, this score is better than the average. If less than 1, this score is below. I do the same thing for all the graphics results, and then calculate the mean. This provides equal weighting for all the various graphics benchmark results. In other words, it prevents a bad quadrant result from skewing the overall picture if the rest of the graphic results are good.

How to interpret these percentages? At the time of writing, if we compare morfic's RAM result of 109% to the terryhau's stock(ish) simplekernel RAM result of 96%, we can say with some certainty that morfic's RAM performs 13% better than stock.

Sheet 2: Statistics for Variance

Firstly, thanks are due to morfic and ezekeel whose scepticism of benchmarks made me want to investigate the issue of variance in the results.

The most important column is n. Check out these rows:
Row 8: AnTuTu's Database I/O benchmark can vary by as much as 7.55%
Row 16: Quadrant Advanced Memory benchmark can vary by as much as 6.66%
Row 21: Smartbench 2012's Gaming Index can vary by a massive 24.42%
Row 25: Vellamo's Aquarium Canvas can vary by 12.11%
Row 26: Vellamo's Pixel Blender can vary by 5.66%
Row 28: Vellamo's Aquarium Canvas can vary by 5.59%

These figures are calculated within two standard deviations of the mean, or in other words, a 95% certainty. For example, we can say that 95% of the time, the Quadrant Advanced Memory benchmarks we run will give a result within 6.66% of a genuinely accurate figure. This is okay, but just not good enough for me!

By eliminating these test results, I have improved the accuracy of my overall scores to within 2% of the genuine mean, with a 95% certainty. In the spreadsheet, you can see the individual accuracies for each of the overall scores: CPU, RAM, I/O, GRAPHICS. Red highlighting of the figures takes these individual variances into account, and shows which tests are definitely at least 1% better than stock, even assuming stock tested at it's maximum, and the compared kernel tested at it's minimum.

Testing Methodology
To test the kernels I want a fair environment, so any differences in the results are down to the kernel, and nothing else. To achieve this I first make sure I have a clean system:
1) Format /system and /cache and wipe dalvic
2) Install the ROM, install the kernel
3) Boot up, use nstools to select deadline for I/O scheduler
4) Then select ondemand governor and set the minimum frequency to equal the maximum, which is not the same as performance, but close enough if you see my CPU governor comparison chart. CPU cycles are wasted as the governor tries to decide what frequency to jump to, even if it has no option! Thanks to Steve Garon for the info. This method was suggested by morfic, thanks again! I/O scheduler is set to deadline. Thanks tchaari!
5) Restore all my benchmark programs. Set everything to off, no gps, no sync, no BT, airplane mode. Force GPU rendering is selected. Wifi is on for connecting to the benchmark servers. A power adaptor is in use so the battery is always full.

Then I begin testing:
6) Power off and power back up. When lock screen arrives, wait one minute to settle the system, i.e. until screen auto-off. Count to three. Unlock, and begin testing, recording all scores at the bottom of my spreadsheet. I copy them up (from the old data to the current raw data section) once complete. This avoids the overall score constantly altering for any current viewers, and means I keep a record of all previous tests under the heading Old Data.
For various benchmarks, see my threads: >here<
The Following 17 Users Say Thank You to bedalus For This Useful Post: [ Click to Expand ]
 
bedalus
Old
(Last edited by bedalus; 14th March 2012 at 05:10 AM.)
#3  
bedalus's Avatar
Recognized Contributor - OP
Thanks Meter 7787
Posts: 5,969
Join Date: Jun 2011
Location: Chester
Source Threads For Kernels and Stated Features

Feature Comparison Table: https://docs.google.com/spreadsheet/...jl6TURCVFR0cWc

r_data (Air Kernel): http://forum.xda-developers.com/show....php?t=1469842
- I/O Schedulers: cfq, deadline, noop, sio (Default is cfq)
- CPU governors: conservative, ondemand, performance, lazy, lulzactive(v2), smartass, smartassV2, intellidemand, interactiveX, scary, lagfree Lionheart (Default is ondemand)
- BLN, BLD, Touch Wake and custom voltage by Ezekeel
- Overclockable to 1.2Ghz
- SQLB
- BIGMEM
- init.d support
- Voodoo sound and colour
- Deep Idle
- Live OC
- CPUIDLE
- 65 fps
- Blue Colors
- BIGMEM
- init.d support
- USB OTG support
- Great Battery Life!
- CIFS, TUN support
- SLQB memory allocator
- Optimized CRC32 algorithm
- ARM RWSEM algorithm

netarchy: http://forum.xda-developers.com/show....php?t=1413117
- I/O Schedulers: cfq, deadline, noop (Default is deadline)
- CPU governors: interactive, conservative, userspace, powersave ondemand, performance (Default is ondemand)
- Overclocking up to 1.4ghz enabled
- Voodoo Sound driver
- Voodoo Colour
- CIFS support
- TUN support
- NFS support
- Updated kernel base (Currently 3.0.14)
- Cpu policy fix to avoid crashes with SetCPU and screen off profiles
- EXT4 remount tweak*
- BLN
- Custom Voltage Interface
- Deep Idle
- Backlight Dimmer

mathkid95: http://forum.xda-developers.com/show....php?t=1361477
- I/O Schedulers: cfq, bfq, deadline, noop, SIO, V(R) (Default is deadline)
- CPU governors: lulzactive, interactiveX, minmax, smartassV2, lazy, ondemand, userspace, performance, intellidemand, lionheart, wheatley (Default is ondemand)
- Based on Linux kernel version 3.0.14
- OC to 1.46 Ghz
- Voodoo sound and colour
- BLN
- BFS
- SLQB memory allocator
- Stochastic Fair Blue (internet packet queue management algorithm)
- CIFS, TUN, NFS
- INIT.D*
- Deep Idle
- LiveOC
- BLX
- BLD
- Ezkeel's voltage mod
- Touch Wake
- Bigmem

Ezekeel: http://forum.xda-developers.com/show....php?t=1336529
- I/O Schedulers: deadline, noop (Default is deadline)
- CPU governors: interactive, conservative, ondemand, userspace, powersave, lazy, performance (Default is ondemand)
- Based on stock Android kernel 3.0.8 IML74K
- CPUfreq governor 'lazy'
- Live OC version 1
- Custom Voltage version 1
- Battery Life eXtender (BLX) version 1
- Power saving state DEEP IDLE version 2
- Touch Wake version 1
- Backlight Dimmer (BLD) version 3
- Backlight Notification (BLN) version 9
- Seperate BFS and CFS versions available
- SLQB memory allocator
- Voodoo Color version 2
- Voodoo Sound version 10
- CIFS
- TUN
- Bigmem (+20 MB more available RAM)

Steve.Garon: http://forum.xda-developers.com/show....php?t=1427352
- I/O Schedulers: cfq, V(R), deadline, noop, sio (Default is cfq)
- CPU governors: lulzactive(v2), smartassV2, lazy, ondemand, conservative, performance (Default is conservative)
- Boots @ 1.1Ghz Overclock (Bus 220MHZ)
- Undervolted to maximize battery life
- Voodoo Sound V10
- BIGMEM for more available memory
- Kernel updated to 3.0.16
- Increased refresh rate 65fps (alternate version 56fps/improved colour available)
- Backlight Notification
- Backlight Dimmer
- Battery Life Extender
- TouchWake
- Custom voltage
- CIFS Module built-in
- Cold colors
- Cpuidle backport from 3.2 Kernel
- up to 1320 Mhz overclock
- init.d support

Morfic: http://forum.xda-developers.com/show....php?t=1036996
- I/O Schedulers: deadline, noop (Default is deadline)
- CPU governors: conservative, ondemand, userspace, powersave, interactive, performance (Default is ondemand)
- BAREBONES i.e. no features
- TEUV version has BLN and BLD

Terryhau: http://forum.xda-developers.com/show....php?t=1420542
- I/O Schedulers: cfq, deadline, noop (Default is cfq)
- CPU governors: conservative, ondemand, userspace, powersave, interactive, performace (Default is ondemand)
- BLN
- BLD
- CIFS module
- Init.d
- Insecure (ro.secure=0)
- Voodoo (*optional)

Reddv1: http://forum.xda-developers.com/show....php?t=1173797
- I/O Schedulers: conservative, ondemand, userspace, powersave, interactive, performance (Default is ondemand)
- CPU governors: cfq, deadline, noop (Default is cfq)
- UV
- Voodoo Sound V.10
- init.d Support
- ext4 optimizations
- BLN
- Compatible w/ NS & NS4G
- BLD
- Touch Wake
- Deep Idle
- Live OC
- Custom Voltage
- ICS version, ADB/ UMS/ Wifi tethering/ USB tethering/ Data stats & limits all work
- ICS version, ext4 optimization, init.d support and all mods from GB version
- Lazy CPU governor (but not in testing version)

Eugene373: http://forum.xda-developers.com/show....php?t=1401455
- I/O Schedulers: cfq, bfq, deadline, sio, noop, V(R) (Default is cfq)
- CPU governors: minmax, lazy, lulzactive, lagfree, interactive, interactiveX, conservative, ondemand, userspace, powersave, smartass, smartassv2, scary, performance, brazilianwax, intellidemand, lionheart, ondemandx, savagedzen, Wheatley (Default is ondemand)
- OTA ICS Compatible
- ro.secure=0 ( So you can push file's via adb )
- Voodoo Sound
- Voodoo Color
- Added BLD
- Touch-wake
- BT Sleep
- LCD voltage drop
- Bigmem patch
- BLN
- TUN
- CIFS
- Battery 100%
- added dm-cache
- kernel:sched: LOAD_FREQ (4*HZ+61) avoids loadavg
- readahead edits
- ext4 edits for smoother rom
- init.d Supported
- OC now Stable up to 1.44GHz
- Live OC is limited to 115%

novic_dev: http://forum.xda-developers.com/show....php?t=1525257
- I/O Schedulers: cfq, bfq, deadline, sio, noop, V(R) (Default is V(R))
- CPU governors: minmax, lazy, lulzactive, lagfree, interactive, interactiveX, conservative, ondemand, userspace, powersave, smartass, smartassv2, scary, performance, brazilianwax, intellidemand, lionheart, ondemandx, savagedzen, Wheatley (Default is ondemand)
  • Linux Kernel version 3.1.10
  • GRSecurity version 2.2.2 with different security features enabled by default (see GRSecurity Working Feature List for details)
  • OC up to 2.0 GHz (still need to be tested!!!)
  • Advanced LiveOC!!! (ALOC) support with Selective LiveOC MOD (thanks to byeonggonlee) - use this feature with caution now due to LiveOC unlimited up to 150% for testing purposes!!!
  • EXT4, NTFS, CIFS support
  • init.d support
  • Compatible with any OTA-based and AOSP ICS Roms
  • Can be managed with NSTOOLS
  • Provides Galaxy Nexus boot animation
  • Voodoo Sound and Vodoo Colors
  • BIGMEM for more available memory
  • Backlight Notification(BLN, enabled by default with blinking)
  • Backlight Dimmer(BLD, disabled by default)
  • Battery Life Extender (BLX, not active for now, need testing)
  • TouchWake (TW, enabled by default)
  • Custom Voltage Interface
  • Deep Idle version 2 (thanks to Ezekeel, disabled for now due to stability issues!)
  • CPUIdle from Linux 3.2.x backport
  • SLQB Memory Allocator with GRSecurity integration (using SLUB for now due to stability issues!)
  • USB OTG (Build 5) support (use it with this app)
  • Blue Colors support for Screen Color Scheme
  • 50 FPS and 65 FPS Screen Refresh Rate support

Franciscofranco: http://forum.xda-developers.com/show....php?t=1357333
- I/O Schedulers: deadline, noop (Default is deadline)
- CPU governors: interactive, conservative, ondemand, performance (Default is ondemand)
- 3.0.8 kernel version
- Full ramdisk with the proper mounting flags for extreme speed
- tweaked Conservative governor
- SLQB memory allocator
- Kernel stripped of debug options +others like cgroups, autogroup, swap, etc.
- TinyRCU as being the obvious choice
- CRC32 lib updated
- I/O-less dirty throttling (backport: kernel 3.2: http://kernelnewbies.org/Linux_3.2)
- Ext4 with 3.2 patches (see link above)
- Proportional Rate Reduction for TCP (see link above)
- Fsync() to oblivion just like I had in the GB kernels
- Init.d support in the ramdisk
- BLN
- Voodoo Sound

Peter Alfonso: http://forum.xda-developers.com/show...&postcount=265
- I/O Schedulers: cfq, deadline, noop (Default is cfq)
- CPU governors: conservative, ondemand, userspace, powersave, interactive, performance (Default is ondemand)
- https://github.com/peteralfonso
- Optimised Stock Kernel

Thalamus (Part of Oxygen): http://goo.gl/o9bxK
- I/O Schedulers: deadline, noop (Default is deadline)
- CPU governors: ondemand, performance (Default is ondemand)
- Optimised Stock Kernel

Link to stock ics kernel: http://forum.xda-developers.com/show...&postcount=120
For various benchmarks, see my threads: >here<
The Following 19 Users Say Thank You to bedalus For This Useful Post: [ Click to Expand ]
 
flodb113
Old
#4  
flodb113's Avatar
Senior Member
Thanks Meter 113
Posts: 165
Join Date: Apr 2011
This is nice. Also, it could be a good idea for roms. The only thing is to redo the benchmarks every once in a while, as the developers keep updating. Great work!

Transmitted from somewhere in space... from my Nexus S... and Tapatalk.
The Following User Says Thank You to flodb113 For This Useful Post: [ Click to Expand ]
 
ljordan2
Old
#5  
ljordan2's Avatar
Senior Member
Thanks Meter 171
Posts: 734
Join Date: Jun 2011
Location: Somewhere in South Korea
Steve Garen's kernel seems to have the best benchmark results. Anyways thanks for this I was curious about this but too lazy to do it myself.

Sent from my Nexus S using Tapatalk
Device : Nexus S(M200K)
Android Version : 4.1.1
Rom : CM10 Kang Jellybro
Kernel : CM Kernel
 
suksit
Old
#6  
suksit's Avatar
Senior Member
Thanks Meter 83
Posts: 410
Join Date: Dec 2010
So it's a test for the performance of kernels against Slim ICS ROM.
 
bedalus
Old
(Last edited by bedalus; 3rd February 2012 at 10:13 AM.)
#7  
bedalus's Avatar
Recognized Contributor - OP
Thanks Meter 7787
Posts: 5,969
Join Date: Jun 2011
Location: Chester
Many thanks to krarvind for Slim ICS, the host ROM.

Testimonials

Kernel developer
Quote:
Originally Posted by mathkid95 View Post
Good job with this! It can be very helpful for ppl wanting to see how the kernels stack up...
Kernel developer
Quote:
Originally Posted by eugene373 View Post
It's nice to see somebody put raw facts behind something & not just bloated scores from Quadrant..
XDA user
Quote:
Originally Posted by mobile_pc View Post
Your great research inspires the kernel devs to further efforts
Thanks
Rom developer
Quote:
Originally Posted by Cybergr View Post
Things are getting more clearer now..thanks for taking your time testing and posting such a useful comparison of different ICS kernels...Helps everyone who is really intrested and sometimes confused...devs or not...but specially the lazy one! Very cool and interesting idea to post a Kernel comparison in here bedalus..!!
XDA veteran and kernel crackflasher
Quote:
Originally Posted by simms22 View Post
... when things differ and deviate from what you expect that draws attention. thats why benchmarks can be very useful to developers. they will know right away if they messed something up or did something very right... impressed with your effort. thank you
Kernel developer
Quote:
Originally Posted by steve.garon View Post
Cool comparison. Tells me that I can improve 2D and 3D score somehow...
My crackflashing/crackbuilding habbits are surfacing again.
For various benchmarks, see my threads: >here<
The Following 6 Users Say Thank You to bedalus For This Useful Post: [ Click to Expand ]
 
bedalus
Old
#8  
bedalus's Avatar
Recognized Contributor - OP
Thanks Meter 7787
Posts: 5,969
Join Date: Jun 2011
Location: Chester
Quote:
Originally Posted by suksit View Post
So it's a test for the performance of kernels against Slim ICS ROM.
I'm not certain, but I believe the performance of the kernels may be compared directly so long as the testing circumstances remain absolutely equivalent.
The Following 2 Users Say Thank You to bedalus For This Useful Post: [ Click to Expand ]
 
brenix
Old
#9  
brenix's Avatar
Senior Member
Thanks Meter 111
Posts: 204
Join Date: Mar 2011
Location: San Diego
Very cool to see.. Nice work bedalus
Nexus 4
The Following 2 Users Say Thank You to brenix For This Useful Post: [ Click to Expand ]
 
Fir3start3r
Old
#10  
Fir3start3r's Avatar
Senior Member
Thanks Meter 200
Posts: 380
Join Date: Jan 2012
Location: Vienna
Thank you!

The Following User Says Thank You to Fir3start3r For This Useful Post: [ Click to Expand ]
Tags
benchmark ics kernel
THREAD CLOSED
Subscribe
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes