Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,809,924 Members 45,150 Now Online
XDA Developers Android and Mobile Development Forum

Rom Benchmarking For Topaz |'Official' Benchmarks |User Submitted Benchmarks|Accurate

Tip us?
 
ThaDeanesta
Old
(Last edited by ThaDeanesta; 9th October 2010 at 07:43 PM.)
#1  
ThaDeanesta's Avatar
Retired Forum Moderator - OP
Thanks Meter 162
Posts: 413
Join Date: Apr 2009

 
DONATE TO ME
Post Rom Benchmarking For Topaz |'Official' Benchmarks |User Submitted Benchmarks|Accurate

This thread is for the submission and viewing of ROM benchmarks. There are two sections of this post: One for the Official ROM Benchmarks, and one for user submitted benchmarks. This will help us get a more accurate representation of the 'Real' benchmark.


'Official' Benchmarks
-----------------------------------------------------------------------------------------------------
These benchmarks are taken by Scabes24. He fallows a more strict set of rules and posts a more exact representation of ROM speeds. Please use this in conjunction with the User Submitted benchmarks to compare ROMs for speed.

This won't be updated constantly,(Will be updated Monthly) so we encourage you to submit your benchmarks in order to keep things fresh around here.

Click here to view the 'Official' benchmarks

User Submitted Benchmarks
-----------------------------------------------------------------------------------------------------

These Benchmarks are submitted by other XDA-Developers users for us to read. These Benchmarks will be up to date (Hopefully), and the rom date will be on the benchmark submission. These spreadsheets will be STRICTLY Moderated so no bogus gets in there. Remember, some of these benchmarks may not be 100% accurate, because it may differ from device to device (but most likely won't). If you need help, just shoot me a PM.


Please Note: The numbers in this spreadsheet are the time in Milliseconds, and not a 'Score'. The lower the number, the better.
Spreadsheet Link (Results)


IF YOU SUBMIT A BENCHMARK, PLEASE "TEST" THE ROM WITHOUT ANY MODS!
Submitting a Benchmark
1. Flash a new ROM and do a hard reset.
2. Go into airplane mode
3. Disable switching screen off and turning the topaz off
4. DO NOT CHANGE ANY SETTINGS!
5. Install SPB Benchmark (Download HERE, and it IS freeware!)
6. Reboot (soft-reset)
7. Run SPB Benchmark : Select the "Main Test" option, and disable external software tests like word and IE or file explorer (whole category)
8. Submit results using this form.

Have i Helped You? Donate 250MB without spending a penny, and get 2.25GB for yourself.
Retired Moderator
The Following User Says Thank You to ThaDeanesta For This Useful Post: [ Click to Expand ]
 
Scabes24
Old
(Last edited by Scabes24; 7th September 2010 at 03:44 AM.) Reason: For Scabes24
#2  
Scabes24's Avatar
Recognized Themer
Thanks Meter 787
Posts: 2,374
Join Date: Aug 2008
Location: Seattle, WA

 
DONATE TO ME
Default Changes for 'Official' Benchmarks

Code:
Changelog

Changes 9/6/10 *Will be updating over the next few days*

•If anyone would like any stock/shipped ROMs tested let me know.

Changes 7/30/10

•Added Energy CHT - DinikGlass 23673/21911 (July 17) - 23121 (July 21)
•Added Energy Standard Sense 23673/21911 (July 17)
•Added Energy Titanium and Sense 2.1 23569/21911 July 14
•Added Energy Standard Reference 23673/21911 (July 17)
•Added Energy GTX 23121 (July 21)
•Added NhatHoa 4.0 23128 (July 18)
•Added Dutty's HG R15 23569/21905 (May 31)
•Added Dutty's Nu Era V1 Redemption 23128 (July 23)
•Added LouMeiYin Yin.09 23568/21905 (June 11)
•Added QBUS22 23569/21909 (July 12)
•Added Melissa v0.3A 21887 (July 8)
•Added MaryOne 23128/23569/21907 (July 15)
•Added Tess Prime V 21907 (June 30)
•Added NullPointer Evo v3.5 23569/21907 (June 28)
•Added Cheetah 4.3r 21901 (July 20)
•Added Cancer TIR-2 (July 2) and TIR-3 (July 10)
•Added Avatar IV 23128 (July 19) and 21911 (July 20 - in progress)
•Added TopazNihon 21911 (July 19) (in progress)
•Added Googy ROM v3.4 219xx (June 14) (w/ RAM Cab)
•Added Column - 'Average' of columns H-Z (NOT an overall score)
•Added Column - 'Sum' of columns H-Z (NOT an overall score)
•Added Column - 'Overall Score' - (still calculating)
•Added Column - 'Free RAM on first boot'
•Added Column - 'Free Storage on first boot'
Search XDA using Google
 
joooe
Old
#3  
joooe's Avatar
Senior Member
Thanks Meter 88
Posts: 375
Join Date: Sep 2007
And it would be great to have some kind of spreadsheet formula to calculate the rating of each test (average function, maybe?). Just for us to know which ROM is the fastest, overall.
This initiative is great!
 
BumAround
Old
#4  
Senior Member
Thanks Meter 59
Posts: 2,729
Join Date: Apr 2009
Guys, a benchmark is a benchmark, you must try the ROM for a week and decide for yourself.
Sensation (ENG S-Off) | MIUI 1.11.4 a la TripNRaver (S01072B) | K TripNKernel ENDTIME-01[/I]| R 10.58.9035.00U_10.15.9035.02_2 | Recovery CWM 5.0.2.0 |
(Phunkycow's variant of the CM7, Rey-619's & a user's MIUI were AWESOME! Lord ClockAn's kernels are FABULOUS!)

Desire (AMOLED S-Off N1 Table)|Oxygen 2.0.3 "the ONE" | R 32.48.00.32U_5.10.05.30 | Recovery AmonRa 2.0.1
 
Scabes24
Old
#5  
Scabes24's Avatar
Recognized Themer
Thanks Meter 787
Posts: 2,374
Join Date: Aug 2008
Location: Seattle, WA

 
DONATE TO ME
I think most people that aren't very interested in each specific benchmark test would much rather see some type of 'Overall score' for each ROM, so if someone knows of a simple formula or way to average the benchmark scores and place them on a 1-10ish scale (based on speed and possibly graphics) please PM me and I will add the stats.
Search XDA using Google
 
bnm7bnm
Old
#6  
bnm7bnm's Avatar
Senior Member
Thanks Meter 5
Posts: 784
Join Date: Mar 2008
Quote:
Originally Posted by Scabes24 View Post
I think most people that aren't very interested in each specific benchmark test would much rather see some type of 'Overall score' for each ROM, so if someone knows of a simple formula or way to average the benchmark scores and place them on a 1-10ish scale (based on speed and possibly graphics) please PM me and I will add the stats.
you can try make it like in previous benchmarking thread - relate to the fastest score of each column as 100% and others as "current"*100/fastest but for overall score need to know which of the colums really effects performance feeled by user to give em priority in overall score
 
Scabes24
Old
#7  
Scabes24's Avatar
Recognized Themer
Thanks Meter 787
Posts: 2,374
Join Date: Aug 2008
Location: Seattle, WA

 
DONATE TO ME
Quote:
Originally Posted by bnm7bnm View Post
you can try make it like in previous benchmarking thread - relate to the fastest score of each column as 100% and others as "current"*100/fastest but for overall score need to know which of the colums really effects performance feeled by user to give em priority in overall score
Yeah I was trying to figure out how to determine what the 'best/max' score would be, going with the current highest/fastest score for each column (or even groups of columns) should work. As far as overall I was thinkin of seperating it into 2 or 3 scores. one for read/write/copy speed and maybe another for graphics.. but you are right, I would need to know which ones effect performance. Also I'm not quite sure how much of an affect the different (21xxx vs 23xxx) builds have on the stats.

I put the average and sum of the columns (except free RAM and storage) for a few energy ROMs, Dutty's Redemption, and Cheetah to see if it they were comparable and accurate to performance but havn't looked at them enough yet, most likely meaningless.
Search XDA using Google
 
mondilv
Old
#8  
mondilv's Avatar
Senior Member
Thanks Meter 13
Posts: 1,517
Join Date: Jul 2008
Location: Riga
Quote:
Originally Posted by ThaDeanesta View Post
I will do this. First, however, we need benchmarks to make the score. I will handle everything =].
Not think, thats right way: all b/m must be at the same device, this the same condition (like disabled manila, sound, phone, phone canvas, the same time after flash/boot etc.).
Devices use different nand sometimes, with diff FS speed. So for compare roms, you need to use the same device.
It's creazy work.
Pietrucci did use it before. Ask him, how right to interpret results.
(the every test element have the own "weight" in the test. Need the special formula for calculate).
Diamond/Topaz - Project "Nataly"
Legend/Wildfire/Desire/OneS/OneSV/One/XperiaV/ZenFone5 custom roms & kernels.
(Don't see my "thanks" count: forum moderators may be too stupid. Even idiots).
 
ljankok
Old
#9  
ljankok's Avatar
Retired Recognized Developer
Thanks Meter 6
Posts: 628
Join Date: Mar 2006
Location: Berkel en Rodenrijs

 
DONATE TO ME
Quote:
Originally Posted by mondilv View Post
Not think, thats right way: all b/m must be at the same device, this the same condition (like disabled manila, sound, phone, phone canvas, the same time after flash/boot etc.).
Devices use different nand sometimes, with diff FS speed. So for compare roms, you need to use the same device.
It's creazy work.
Pietrucci did use it before. Ask him, how right to interpret results.
(the every test element have the own "weight" in the test. Need the special formula for calculate).
Yes, this is true. There is a special formula for this. Piertrucci knows this, so it is a good idea to ask him instead of doing the same work twice.
--
Cheetah ROM. -- Now looking at Android Kernel's Fun
 
ThaDeanesta
Old
#10  
ThaDeanesta's Avatar
Retired Forum Moderator - OP
Thanks Meter 162
Posts: 413
Join Date: Apr 2009

 
DONATE TO ME
i have scabes24 doing everything on the same device

Have i Helped You? Donate 250MB without spending a penny, and get 2.25GB for yourself.
Retired Moderator

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Lock Apps and Functions Easily with Droid Protector

Access and security are some of the key issues facingmobile technology. Whether … more

ARChon Lets You Run Android Apps in Google Chrome Browser

Not too long ago, Google added the ability torun select Android applications on … more

Flyme OS Finds Its Way to the Google Nexus 5

Flyme OS is a heavily modified Android variantdeveloped by the Meizu team for their MX … more

MultiROM Makes Its Way to the LG G2

Over the last few weeks, we’ve seen quite a fewflagship (and former flagship) devicesget to … more