Android App Review: Manage Your Connections Automagically – XDA TV

Material Design is all the buzz in the Android world right now. … more

Official TWRP Recovery Lands on Micromax Canvas Magnus

With the recent release of Android One, Micromax and MediaTek released fully buildable … more

Update Overload: SuperSU Heavily Updated

There are apps and apps out there. Very few can reach the level of popularity required to stay at … more

Lollipop Leak Available for the Sprint Galaxy S5

Since the Android 5.0Lollipop’s official release not too long ago, we’ve seen … more

Welcome to XDA

Search to go directly to your device's forum

Register an account

Unlock full posting privileges

Ask a question

No registration required
Post Reply

Rom Benchmarking For Topaz |'Official' Benchmarks |User Submitted Benchmarks|Accurate

OP ThaDeanesta

8th July 2010, 03:23 AM   |  #1  
ThaDeanesta's Avatar
OP Retired Forum Moderator
Thanks Meter: 162
 
413 posts
Join Date:Joined: Apr 2009
Donate to Me
More
This thread is for the submission and viewing of ROM benchmarks. There are two sections of this post: One for the Official ROM Benchmarks, and one for user submitted benchmarks. This will help us get a more accurate representation of the 'Real' benchmark.


'Official' Benchmarks
-----------------------------------------------------------------------------------------------------
These benchmarks are taken by Scabes24. He fallows a more strict set of rules and posts a more exact representation of ROM speeds. Please use this in conjunction with the User Submitted benchmarks to compare ROMs for speed.

This won't be updated constantly,(Will be updated Monthly) so we encourage you to submit your benchmarks in order to keep things fresh around here.

Click here to view the 'Official' benchmarks

User Submitted Benchmarks
-----------------------------------------------------------------------------------------------------

These Benchmarks are submitted by other XDA-Developers users for us to read. These Benchmarks will be up to date (Hopefully), and the rom date will be on the benchmark submission. These spreadsheets will be STRICTLY Moderated so no bogus gets in there. Remember, some of these benchmarks may not be 100% accurate, because it may differ from device to device (but most likely won't). If you need help, just shoot me a PM.


Please Note: The numbers in this spreadsheet are the time in Milliseconds, and not a 'Score'. The lower the number, the better.
Spreadsheet Link (Results)


IF YOU SUBMIT A BENCHMARK, PLEASE "TEST" THE ROM WITHOUT ANY MODS!
Submitting a Benchmark
1. Flash a new ROM and do a hard reset.
2. Go into airplane mode
3. Disable switching screen off and turning the topaz off
4. DO NOT CHANGE ANY SETTINGS!
5. Install SPB Benchmark (Download HERE, and it IS freeware!)
6. Reboot (soft-reset)
7. Run SPB Benchmark : Select the "Main Test" option, and disable external software tests like word and IE or file explorer (whole category)
8. Submit results using this form.
Last edited by ThaDeanesta; 9th October 2010 at 08:43 PM.
The Following User Says Thank You to ThaDeanesta For This Useful Post: [ View ]
9th July 2010, 05:33 AM   |  #2  
Scabes24's Avatar
Recognized Themer
Flag Seattle, WA
Thanks Meter: 787
 
2,374 posts
Join Date:Joined: Aug 2008
Donate to Me
More
Changes for 'Official' Benchmarks
Code:
Changelog

Changes 9/6/10 *Will be updating over the next few days*

•If anyone would like any stock/shipped ROMs tested let me know.

Changes 7/30/10

•Added Energy CHT - DinikGlass 23673/21911 (July 17) - 23121 (July 21)
•Added Energy Standard Sense 23673/21911 (July 17)
•Added Energy Titanium and Sense 2.1 23569/21911 July 14
•Added Energy Standard Reference 23673/21911 (July 17)
•Added Energy GTX 23121 (July 21)
•Added NhatHoa 4.0 23128 (July 18)
•Added Dutty's HG R15 23569/21905 (May 31)
•Added Dutty's Nu Era V1 Redemption 23128 (July 23)
•Added LouMeiYin Yin.09 23568/21905 (June 11)
•Added QBUS22 23569/21909 (July 12)
•Added Melissa v0.3A 21887 (July 8)
•Added MaryOne 23128/23569/21907 (July 15)
•Added Tess Prime V 21907 (June 30)
•Added NullPointer Evo v3.5 23569/21907 (June 28)
•Added Cheetah 4.3r 21901 (July 20)
•Added Cancer TIR-2 (July 2) and TIR-3 (July 10)
•Added Avatar IV 23128 (July 19) and 21911 (July 20 - in progress)
•Added TopazNihon 21911 (July 19) (in progress)
•Added Googy ROM v3.4 219xx (June 14) (w/ RAM Cab)
•Added Column - 'Average' of columns H-Z (NOT an overall score)
•Added Column - 'Sum' of columns H-Z (NOT an overall score)
•Added Column - 'Overall Score' - (still calculating)
•Added Column - 'Free RAM on first boot'
•Added Column - 'Free Storage on first boot'
Last edited by Scabes24; 7th September 2010 at 04:44 AM. Reason: For Scabes24
12th July 2010, 04:15 PM   |  #3  
joooe's Avatar
Senior Member
Thanks Meter: 91
 
384 posts
Join Date:Joined: Sep 2007
More
And it would be great to have some kind of spreadsheet formula to calculate the rating of each test (average function, maybe?). Just for us to know which ROM is the fastest, overall.
This initiative is great!
17th July 2010, 07:31 AM   |  #4  
Senior Member
Thanks Meter: 59
 
2,729 posts
Join Date:Joined: Apr 2009
Guys, a benchmark is a benchmark, you must try the ROM for a week and decide for yourself.
23rd July 2010, 05:04 AM   |  #5  
Scabes24's Avatar
Recognized Themer
Flag Seattle, WA
Thanks Meter: 787
 
2,374 posts
Join Date:Joined: Aug 2008
Donate to Me
More
I think most people that aren't very interested in each specific benchmark test would much rather see some type of 'Overall score' for each ROM, so if someone knows of a simple formula or way to average the benchmark scores and place them on a 1-10ish scale (based on speed and possibly graphics) please PM me and I will add the stats.
24th July 2010, 03:56 AM   |  #6  
bnm7bnm's Avatar
Senior Member
Thanks Meter: 5
 
784 posts
Join Date:Joined: Mar 2008
More
Arrow
Quote:
Originally Posted by Scabes24

I think most people that aren't very interested in each specific benchmark test would much rather see some type of 'Overall score' for each ROM, so if someone knows of a simple formula or way to average the benchmark scores and place them on a 1-10ish scale (based on speed and possibly graphics) please PM me and I will add the stats.

you can try make it like in previous benchmarking thread - relate to the fastest score of each column as 100% and others as "current"*100/fastest but for overall score need to know which of the colums really effects performance feeled by user to give em priority in overall score
24th July 2010, 06:09 AM   |  #7  
Scabes24's Avatar
Recognized Themer
Flag Seattle, WA
Thanks Meter: 787
 
2,374 posts
Join Date:Joined: Aug 2008
Donate to Me
More
Quote:
Originally Posted by bnm7bnm

you can try make it like in previous benchmarking thread - relate to the fastest score of each column as 100% and others as "current"*100/fastest but for overall score need to know which of the colums really effects performance feeled by user to give em priority in overall score

Yeah I was trying to figure out how to determine what the 'best/max' score would be, going with the current highest/fastest score for each column (or even groups of columns) should work. As far as overall I was thinkin of seperating it into 2 or 3 scores. one for read/write/copy speed and maybe another for graphics.. but you are right, I would need to know which ones effect performance. Also I'm not quite sure how much of an affect the different (21xxx vs 23xxx) builds have on the stats.

I put the average and sum of the columns (except free RAM and storage) for a few energy ROMs, Dutty's Redemption, and Cheetah to see if it they were comparable and accurate to performance but havn't looked at them enough yet, most likely meaningless.
26th July 2010, 01:40 PM   |  #8  
mondilv's Avatar
Senior Member
Flag Riga
Thanks Meter: 13
 
1,517 posts
Join Date:Joined: Jul 2008
More
Quote:
Originally Posted by ThaDeanesta

I will do this. First, however, we need benchmarks to make the score. I will handle everything =].

Not think, thats right way: all b/m must be at the same device, this the same condition (like disabled manila, sound, phone, phone canvas, the same time after flash/boot etc.).
Devices use different nand sometimes, with diff FS speed. So for compare roms, you need to use the same device.
It's creazy work.
Pietrucci did use it before. Ask him, how right to interpret results.
(the every test element have the own "weight" in the test. Need the special formula for calculate).
26th July 2010, 05:23 PM   |  #9  
ljankok's Avatar
Retired Recognized Developer
Flag Berkel en Rodenrijs
Thanks Meter: 6
 
628 posts
Join Date:Joined: Mar 2006
Donate to Me
More
Quote:
Originally Posted by mondilv

Not think, thats right way: all b/m must be at the same device, this the same condition (like disabled manila, sound, phone, phone canvas, the same time after flash/boot etc.).
Devices use different nand sometimes, with diff FS speed. So for compare roms, you need to use the same device.
It's creazy work.
Pietrucci did use it before. Ask him, how right to interpret results.
(the every test element have the own "weight" in the test. Need the special formula for calculate).

Yes, this is true. There is a special formula for this. Piertrucci knows this, so it is a good idea to ask him instead of doing the same work twice.
5th August 2010, 04:33 PM   |  #10  
ThaDeanesta's Avatar
OP Retired Forum Moderator
Thanks Meter: 162
 
413 posts
Join Date:Joined: Apr 2009
Donate to Me
More
i have scabes24 doing everything on the same device

Post Reply Subscribe to Thread
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes