[REF][15-05-2009: NEW ROMs]Some benchmarking...

[pietrucci]

--------------------------------------------------------------
New ROMs:
Nataly 2.5 Final (210436)

--------------------------------------------------------------
YOU WANT MY OPINION WHICH ROM TO TAKE???
LOOK HERE: http://forum.xda-developers.com/showpost.php?p=2601858&postcount=199


--------------------------------------------------------------
Hi guys and dolls,

I've been busy doing some performance testing using SPB Benchmark and VSBenchmark.
Not very scientific, these benchmarks don't seem to work that perfect on WM6.1. But for comparing reasons it should say sómething.
Furthermore: a good testresult it no guarantee your device really FEELS fast, often the feeling is different from cold measurements...

All ROMs tested right after flashing, I disabled TF3D, set power to max, no radio, no phoneskin, no SIMcard, tested on batterypower.

--------------------------------------------------------------
Tested ROMs so far:
(shortened list because post gets to big; see attached Excel sheet)
.....
Panosha Shine-on 1.7 Light
SpeedUP TOR_ROM V2.1 NetCF3.5
Panosha Shine-on 1.7 Full
Swtos WWE 20271 v1.0 Naked
SpeedUP TOR_ROM V2.2
Dutty's 20748 v1.9 UltraLite
Dutty's 20748 v1.9 Full
Swtos WWE 20271 v1.0 Vaniila
SpeedUP TOR_ROM V3
SpeedUP TOR_ROM V5
SC 3.0NLD EXTREME Colour (20721)
SwiftBL 6.5 Lite 12MB PP (20721)
SwiftBL 6.5 Lite 16MB PP (20721)
Panosha Shine on 1.8
Panosha Shimple 0.8 Clean (20275)
HaVeN 1.0.1 build 1999
Dutty v2 Full (20749)
Dutty v2 Extreme Lite (20749)
Speedup A.Z.T.O.R. v6
Swtos WWE V1.2 Full (20748)
Dutty V2.2 Full CVSD (20753)
Wildberry V1 (20755)
AZTOR SprintFlavor (20755)
Swtos V1.4 Full (20755)
SwiftBL WWE V7.0 Full 0MB PP (20753)
SwiftBL WWE V7.0 Full 12MB PP (20753)
SC 3.1 NLDTT7 Extreme (20753)
Dutty V2.3 Full CVSD (20755)
SwiftBL WWE V7.5 Full 16MB PP (20755)
Swtos v1.6 Full (20755)
Wildberry V1 Light (20755)
Dutty V2.5 Full CVSD (20759)
AZTOR WWE V7 (20755)
Panosha ShineOn V2.2 Light 12MB PP (20755)
Dutty V2.6 Full CVSD (20758)
udK Syrius HD RO beta 1 (19971)
Mary One (20758)
Dutty V2.7 Full RC2 CVSD (20921)
seatone's Cupra v1.4 xLT
udK Syrius HD R0 beta 2 (20757)
udK Syrius HD R0 beta 2 (19971)
VIC Diamond 3.0 LT Dyn (20924)
Dutty V2.8 RTM CVSD private beta (20921)
Dutty V2.8 RTM CVSD private beta (19974)
Dutty V2.8 RTM CVSD (19974)
Diamentowy ERA Lite (Polish ROM)
O.R.D.E. Kitchen v1.95.404.9 SMD NLD
Dia Duit v0.5 (1.96.7.xx)
Dutty v2.9 RTM CVSD (20275)
udK Syrius HD R0 beta 3 (20757)
Dutty v3 Full CVSD (20931)
Org HTC Vodafone CH-WWE 1.97.170.4
Dutty v3.2 RW Full CVSD 8PP (20931)
Dutty v3.3 APR Full CVSD 8PP (20931)
Own version Kitchen Tree 1.97 (20755)
Mondilv version Kitchen Tree 1.96 (20755)
Deiviniven D&D 1.93.412 SPN Final
Vic Hybrid WWE (20755)
udK Syrius HD R0 beta 5 (20759)
Mary One WWE 05-12-2008 (20764)
AZTOR HD WWE V8 (20759)
Dia Duit v0.9 pre4 PP4Mb based on WWE 2.0 Test
mondilv KitchenTree 1.96/20755 Grey
Dutty v3.6 Full CVSD (20764)
CT's V6 HD TFlo Full
swtos WWE v2.0 Almost Full
AZTOR V9preview (22-12-2008)
Swtos V2.2 Naked (20931)
Swampy395 2.00.421 v2 (19965)
CT's V6.1 Full classic TF3D 16PP (19965)
CUPRA v2.3 Full (19965)
Nataly V1.1 Light (20764)
udK Syrius HD RO beta6 (20764)
TestROM seatone
Anja ROM HD Manila
Anja ROM Diamond Manila
DIB V1.28 Full WWE (20940)
udK Syrius HD RO beta7 based on 2.03 (20764)
AZTOR X based on 2.03 (20764), radio .23
AZTOR X based on 2.03 (20764), radio .05
Dutty V4 CVSD Full (20764)
Swtos V2.5 WWE Full (20764)
Shaks.P V3.00 Full (20764)
Shaks.P V3.10 Full (20764)
HTC 2.03.401 Ship (20764)
Dia Duit V0.9-pre8 (20764)
Vic V3.3.1b Classic TF3D (21009)
CT's V6.4 Full Original TF3D
CT's V6.4 Lite Original TF3D
SONE V2.1 WWE (21109)
HTC Official 2.03 NLD (20764)
udK Syrius HD RO Beta 8 (20765)
Nataly v2.0 RC1 Full WWE (20769)
Nataly v2.0 Final Full WWE (20769), radio 1.08.25.20
Cloudyfa_6.5.01 Full WWE (21159)
Nataly v2.0 Final Full WWE (20769), radio 1.00.25.05
swtos V2.6 Light (20764)
udK Syrius Topaz R2 (21032)
Nataly V2.1 Final (20771)
Cloudy WM 6.1 V1.0 (21018)
PDACornerIllusion V3 (21039)
udK Syrius Topaz R4 (21040, 2.04)
Cloudy WM 6.1 V1.1 (21040, 2.04)
Nataly V2.2 Test (21042)
swtos V2.71 Full (21042)
Shaks.P V4.00 (21042)
AZTOR X2 WWE (20764)
Nataly 2.5 beta test1 (21043)
HTC NLD 1.39.404.1, radio 3.43.25.19, DIAMOND 2!!!
Nataly 2.5 Final (210436)

Upcoming:
Not so much I fear, I stepped over to the diamond 2...

--------------------------------------------------------------
Reason for editing: new ROM(s)

[Sergio PC]

Dude im Honored to be the first one to congratttt you on such a fine job!
Good one man!

[pietrucci]

Test of PDAVIET 1.0 and Panosha lite II added (1st post).
Panosha's doesn't seem to be as fast as I hoped for... (according to the 2 benchmarks).

[duttythroy]

Nice one mate, Glad to see youre bringing your ideas form Kaiser forums to us, I havent done a bench yet but will when I finish my rom

[pietrucci]

Nice one mate, Glad to see youre bringing your ideas form Kaiser forums to us, I havent done a bench yet but will when I finish my romcan't wait to test again...

[panosha]

Great job, thanks. :)

[R. DevilR]

Really great job !!!

It's amazing to see that the orignal ROM generally has one of the (or THE) best results ^_^

[zimonia]

could u maybe benchmark the rom in this thread:
http://forum.xda-developers.com/showthread.php?p=2392283

I heard from various people its pretty fast and it packs newest editions of a lot of programs. It's a dutch rom, though the cooker wil prolly make a WWE one also.

Would be greatly appreciated.

[pietrucci]

could u maybe benchmark the rom in this thread:
http://forum.xda-developers.com/showthread.php?p=2392283

I heard from various people its pretty fast and it packs newest editions of a lot of programs. It's a dutch rom, though the cooker wil prolly make a WWE one alsoOk, will test it.
I also tested the 6.1.1.15 NLD and I'm afraid it was the slowest ROM I tested on my device...Find out if this one's faster.

[Hornet331]

great thread :)

[bepe]

you got me, I love Excel tables :D ... btw, top test conditions you have there!

[pietrucci]

hi, new version in post 1

fixed a calculation error and layout a little bit
added Surface 6.1.1.16 NLD

[sonictrend]

Very Nice work you got there.

[cyphol]

It's Nice, Man.

[naser404]

All ROMs tested right after flashing, I disabled TF3D and all startup programs, stopped running programs, no SIMcard, device on AC adapter.


Speaking from a computer POV, i think it might actually make better sense to "not" disable TF3D & especially startup appz, why?

Let just say that once you disable all those & compare 2 roms A & B, and A happens to be faster then once TF3D & appz r running then some might hinder the performance & you come out with B as a winner.

Startup appz on most roms are essential & therefore they are needed for that rom & performance should be tested using each rom settings & configurations.

I can have a Vista setup that flies by disabling startup appz & services but to "use" that setup with all its glory, then many of these are required.


ps: just my opinion.

[mikkelnl]

Thanks for the effort! Appreciate it!

[zimonia]

Ok, will test it.
I also tested the 6.1.1.15 NLD and I'm afraid it was the slowest ROM I tested on my device...Find out if this one's faster.

Thanks for the effort.
there are 2 roms called surface atm, i suppose the slow one is the standard rom and the faster one is the cooked one?

Even that one is not as fast as i hoped though

[panosha]

Speaking from a computer POV, i think it might actually make better sense to "not" disable TF3D & especially startup appz, why?

Let just say that once you disable all those & compare 2 roms A & B, and A happens to be faster then once TF3D & appz r running then some might hinder the performance & you come out with B as a winner.

Startup appz on most roms are essential & therefore they are needed for that rom & performance should be tested using each rom settings & configurations.

I can have a Vista setup that flies by disabling startup appz & services but to "use" that setup with all its glory, then many of these are required.
I absolutely agree. :)

[pietrucci]

Speaking from a computer POV, i think it might actually make better sense to "not" disable TF3D & especially startup appz, why?

Let just say that once you disable all those & compare 2 roms A & B, and A happens to be faster then once TF3D & appz r running then some might hinder the performance & you come out with B as a winner.

Startup appz on most roms are essential & therefore they are needed for that rom & performance should be tested using each rom settings & configurations.

I can have a Vista setup that flies by disabling startup appz & services but to "use" that setup with all its glory, then many of these are required.


ps: just my opinion.You have a point... (although there are also many people who dón't use TF3D).

It takes too much time for me to test every ROM with and without TF3D, so I have to make a choice for future tests. I guess I'll test with TF3D because most of us do use it.
Perhaps if I find some extra time I will test some of the current ROMs again with startup and TF3D.

By the way could be that activating startup and TF3D doesn't make that much of a difference. I mean: yes, you do load a lot of extra's in memory but I guess there will be more then enough memory avialable for the tests. And these extra programs don't take that much of CPU capacity.
IBut that's just a guess, time to test and compare!

later...

[pietrucci]

You have a point... (although there are also many people who dón't use TF3D).

It takes too much time for me to test every ROM with and without TF3D, so I have to make a choice for future tests. I guess I'll test with TF3D because most of us do use it.
Perhaps if I find some extra time I will test some of the current ROMs again with startup and TF3D.

By the way could be that activating startup and TF3D doesn't make that much of a difference. I mean: yes, you do load a lot of extra's in memory but I guess there will be more then enough memory avialable for the tests. And these extra programs don't take that much of CPU capacity.
IBut that's just a guess, time to test and compare!

later...Well first comparisation between bold and fully loaded ROM.
I took the Diamond Surface 6.1.1.16 NLD ROM and tested twice wíth and twice withóut startup programs and TF3D. All in all no much difference (about 1%) except for one graphics test, one file read and Arkaball.
Big differences in red.

Very strange: the VSBenchmark tool was faster when everything was loaded!!!

I think it shows again that these test are not very reliable in absolute sence, only in relative sence for comparisation between ROMs.

[JanDaMan]

hey, how did the TLR 2.0 rom perform? i'm a fan of it because it has all the alarms etc and AppToDate too,

[naser404]

Very strange: the VSBenchmark tool was faster when everything was loaded!!!

I think it shows again that these test are not very reliable in absolute sence, only in relative sence for comparisation between ROMs.

Hi,

It might be, but again there might "also" be a chance that its actually a better test, let me explain.

lets take a windows based machine again: If you disable 2 services for example:
1-File indexing
2-Super fetch

Things might "seem" snappier, & "some" benchmarks might give better performances, cause it cuts on using cpu cycles & obviously valuable ram but once you start using your machine on a day to day basis then these two really start coming into effect & actually help performance, File indexing for example will make your search a LOT faster, instant search & SF (depending on the settings 1,2 or 3) will PRE-fetch your appz, boot into ram to give a faster bootime and/or applications startup.

Again, different configuration will give different results, but with quad core cpus & 2GB ram i would say these are two services that should be enabled & you will defiantly benefit from them.

PDA are made equally in configuration so that should help to determine a setting that should in general work best for all Diamonds.

Again im not in the know how on pda but i think it makes the same sense & again i might be wrong & id be glad to hear anyone else opinion.

once again thank you for doing a great job testing different roms with different settings.

[pietrucci]

hey, how did the TLR 2.0 rom perform? i'm a fan of it because it has all the alarms etc and AppToDate too,Like every ROM: right after install I remove the startup folder in windows, disable TF3D, install SPB Benchmark and VSBenchmark, remove sim-card, connect to AC-adapter, restart twice (because sometimes it does some post-installing and post-settings things), remove running programs from memory (like SAPSettings, BioTouch, Opera) , then run SPB Benchmark, then restart and then run the VSBenchmark.
Takes about about 1.5 hour.

[pietrucci]

Hi,

It might be, but again there might "also" be a chance that its actually a better test, let me explain.

lets take a windows based machine again: If you disable 2 services for example:
1-File indexing
2-Super fetch

Things might "seem" snappier, & "some" benchmarks might give better performances, cause it cuts on using cpu cycles & obviously valuable ram but once you start using your machine on a day to day basis then these two really start coming into effect & actually help performance, File indexing for example will make your search a LOT faster, instant search & SF (depending on the settings 1,2 or 3) will PRE-fetch your appz, boot into ram to give a faster bootime and/or applications startup.

Again, different configuration will give different results, but with quad core cpus & 2GB ram i would say these are two services that should be enabled & you will defiantly benefit from them.

PDA are made equally in configuration so that should help to determine a setting that should in general work best for all Diamonds.

Again im not in the know how on pda but i think it makes the same sense & again i might be wrong & id be glad to hear anyone else opinion.

once again thank you for doing a great job testing different roms with different settings.Thnx, primary I'd like to test everything for myself, but I'm happy to share it with others.

After some testing with the startup programs and TF3D loaded I think I'll stick to my original teststrategy: remove all startup programs and disable TF3D.
Why? Because that's the most objective test I think. With everything loaded the ROMs are often very different, some are light some are heavy loaded. When I test without extra loadings the configurations are more (or less) the same.
When people want to know TF3D or other extras they know the ROM will be slower. I assume when using TF3D the decrease in performance will be the same for every ROM because there are not much differences between the TF3D versions (yet).
Another reason to stick to my original plan is that I can continue comparing with the already tested ROMs.

[sev7en]

Thanks for the sheet. Benchmarks are with the first release of TLR? It would be good to test the TLR2: I just flashed the device with this and it seems faster than original Diamond ROM.
I haven't tested other ROMs but the Elite v2 seems a good alternative.

I was expecting better performaces for the Surface roms...

[JanDaMan]

Like every ROM: right after install I remove the startup folder in windows, disable TF3D, install SPB Benchmark and VSBenchmark, remove sim-card, connect to AC-adapter, restart twice (because sometimes it does some post-installing and post-settings things), remove running programs from memory (like SAPSettings, BioTouch, Opera) , then run SPB Benchmark, then restart and then run the VSBenchmark.
Takes about about 1.5 hour.

no w8 i think you misunderstood me, i meant how did the TLR2 ROM do compared to the Elite 2.0 ROM, thats what i ment by 'how did the TLR2 perform' lol soz for the trouble

[athanaso77]

New dutty's and Tom's ROMs must be added in this benchmark! ;)

[pietrucci]

Thanks for the sheet. Benchmarks are with the first release of TLR? It would be good to test the TLR2: I just flashed the device with this and it seems faster than original Diamond ROM.
I haven't tested other ROMs but the Elite v2 seems a good alternative.

I was expecting better performaces for the Surface roms...TLR2 is in the list...

[pietrucci]

New dutty's and Tom's ROMs must be added in this benchmark! ;)Must???
I'll think about it after getting some sleep...

[pietrucci]

Added the first Dutty's and Tom's.

Another good example of the difference between cold testing and the feel of a ROM during use: both ROM's are very responsive and feel very fast.
But testresults show these ROMs perform only mediate...

Let's test the new one from Dutty (downloading now...)!

[duttythroy]

Added the first Dutty's and Tom's.

Another good example of the difference between cold testing and the feel of a ROM during use: both ROM's are very responsive and feel very fast.
But testresults show these ROMs perform only mediate...

Let's test the new one from Dutty (downloading now...)!

Your right but all roms I tested before including my last this new one feels the fastest but I guess we cant exactly know the truthabout benchmarking with wm6.1 roms with spb

[bepe]

Your right but all roms I tested before including my last this new one feels the fastest but I guess we cant exactly know the truthabout benchmarking with wm6.1 roms with spb

It is strange; because of the new ROM base I had much better graphic results with your rom
Something must be wrong... I'll do the benchmarks again.

[duttythroy]

It is strange; because of the new ROM base I had much better graphic results with your rom
Something must be wrong... I'll do the benchmarks again.

very true I tested my roms on 4 different occassion with same atmosphere and get 4 different results the highest I got was 475 but still the rom felt faster:confused:

[pietrucci]

very true I tested my roms on 4 different occassion with same atmosphere and get 4 different results the highest I got was 475 but still the rom felt faster:confused:dutty and bepe, I also had only 450 or so on the new dutty. then I set the system caches back to disable and auto, glyphcache to 8k. now it flies through the tests: 475!
also VSBenchmark much beter results.
but I fear in daily use it will be slower with these low cache values...

[bepe]

I'm getting very stable results like this:

http://img390.imageshack.us/img390/6498/compfb8.png (http://imageshack.us)
http://img390.imageshack.us/img390/6498/compfb8.7de201203b.jpg (http://g.imageshack.us/g.php?h=390&i=compfb8.png)

But have to say that I had problems with "duttythroy test 2"... forgot to disable the auto display off. Anyway, I've done the tests many times and got almost the same results.

... And never got graphic results with my rom that were better than yours.

Somehow that is very strange, could you redo the benchmarks for this roms?
And could some ppl please do the same with different devices, would be really interesting...

[duttythroy]

I'm getting very stable results like this:

http://img390.imageshack.us/img390/6498/compfb8.png (http://imageshack.us)
http://img390.imageshack.us/img390/6498/compfb8.7de201203b.jpg (http://g.imageshack.us/g.php?h=390&i=compfb8.png)

But have to say that I had problems with "duttythroy test 2"... forgot to disable the auto display off. Anyway, I've done the tests many times and got almost the same results.

... And never got graphic results with my rom that were better than yours.

Somehow that is very strange, could you redo the benchmarks for this roms?
And could some ppl please do the same with different devices, would be really interesting...

Its all strange bepe, not too confident in spb right, :confused: seems the new build has very good graphics

[bepe]

Its all strange bepe, not too confident in spb right, :confused: seems the new build has very good graphics

Yes, it has... but I completely disagree on the confidence of spb.
As I said, I get stable results with the same test conditions... also with your rom.
I'm making many benchmarks like this, for example to boost the file system performance I'm changing hardcoded values in dll's. To find a good value I sometimes need to make more than 5 benchmarks, so I know what I'm talking about.

I know about the fluctuations of some results, but not like this.
For me there is no way to declare this results implausible, bc I know better.

Anyway, let’s do other ppl some tests too, to see the credibility of the device used for these benchmarks.

... or would you like to end this benchmarks???

[pietrucci]

I'm getting very stable results like this:

http://img390.imageshack.us/img390/6498/compfb8.png (http://imageshack.us)
http://img390.imageshack.us/img390/6498/compfb8.7de201203b.jpg (http://g.imageshack.us/g.php?h=390&i=compfb8.png)

But have to say that I had problems with "duttythroy test 2"... forgot to disable the auto display off. Anyway, I've done the tests many times and got almost the same results.

... And never got graphic results with my rom that were better than yours.

Somehow that is very strange, could you redo the benchmarks for this roms?
And could some ppl please do the same with different devices, would be really interesting...Bepe my testresults with Dutty's 2 are almost the same as yours, but only after disabling/lowering the system and graphic caches! (see next post)

[duttythroy]

Yes, it has... but I completely disagree on the confidence of spb.
As I said, I get stable results with the same test conditions... also with your rom.
I'm making many benchmarks like this, for example to boost the file system performance I'm changing hardcoded values in dll's. To find a good value I sometimes need to make more than 5 benchmarks, so I know what I'm talking about.

I know about the fluctuations of some results, but not like this.
For me there is no way to declare this results implausible, bc I know better.

Anyway, let’s do other ppl some tests too, to see the credibility of the device used for these benchmarks.

... or would you like to end this benchmarks???

iT WOULD BE GOOD to see how others devices do on the same roms with benchmark scoring this would give us a rough idea on how we all doing on roms, but like i said sometimes devices performs different wether the benchmarks score are high or low. so we still continue

[bepe]

Bepe my testresults with Dutty's 2 are almost the same as yours, but only after disabling/lowering the system and graphic caches! (see next post)

Great that means the results are accurate :)

Anyway, don’t really understand why you changed back the caches, because I think benchmarks are about testing exactly things like cache settings.

[pietrucci]

Ok here my little theory, try to explain in English...

When you set the system cache and glyphcache, the system loads everthing in cache first and that takes a little time. The larger the cache the more time you need to put everything in it.
For normal use that's no problem because you re-use the caches so everything works faster.
BUT... in these benchmarks everything is tested only once so the caches aren't really re-used. But still the tests do need some time to put it in cache. And therefore the testresults are SLOWER when caches are larger...

How about this theory (hope you understand what I mean?)
I tested it with Dutty 2: the results are much faster with small caches!

[duttythroy]

Ok here my little theory, try to explain in English...

When you set the system cache and glyphcache, the system loads everthing in cache first and that takes a little time. The larger the cache the more time you need to put everything in it.
For normal use that's no problem because you re-use the caches so everything works faster.
BUT... in these benchmarks everything is tested only once so the caches aren't really re-used. But still the tests do need some time to put it in cache. And therefore the testresults are SLOWER when caches are larger...

How about this theory (hope you understand what I mean?)
I tested it with Dutty 2: the results are much faster with small caches!


That makes lots of sense and I think ur right probably, as if u notice in my roms if u open a foler e.g multimedia it takesaround 2 seconds but next time its like instant

[hallatore]

Run the test 3 times and select the results from the last run.
Will give a much better result for "overall usage" :)

[pietrucci]

Great that means the results are accurate :)

Anyway, don’t really understand why you changed back the caches, because I think benchmarks are about testing exactly things like cache settings.well, I could do two things: testing the ROMs as they are. That means that cache settings are different because cooks often use their own settings...
Or I could set cache values to a default value before testing. But then I don't test the ROM as it is.
So what's the best strategy?

Why I changed Dutty's settings? Well I got very bad testresults and couldn't understand this because it feels so fast. After many setup changes I finally decided to lower the caches. And then Dutty was very fast, both in SPB and VSBenchmark!

So I figured out the cache theory a few posts back...
Don't know how or what exactly, fact is I get very different (= better) results with small caches!

[pietrucci]

Run the test 3 times and select the results from the last run.
Will give a much better result for "overall usage" :)Yes I guess it will...
But maybe you understand flashing and testing a ROM with both SPB Benchmark and VSBenchmark takes about an hour.
Doing this 3 times for each ROM...

Last week I tested about 20 ROMs (my poor device...).

[hallatore]

Yes I guess it will...
But maybe you understand flashing and testing a ROM with both SPB Benchmark and VSBenchmark takes about an hour.
Doing this 3 times for each ROM...

Last week I tested about 20 ROMs (my poor device...).

Ah, 2 times and take the last result then :p
3 times is only to get a "perfect" result.

[bepe]

well, I could do two things: testing the ROMs as they are. That means that cache settings are different because cooks often use their own settings...
Or I could set cache values to a default value before testing. But then I don't test the ROM as it is.
So what's the best strategy?

Why I changed Dutty's settings? Well I got very bad testresults and couldn't understand this because it feels so fast. After many setup changes I finally decided to lower the caches. And then Dutty was very fast, both in SPB and VSBenchmark!

So I figured out the cache theory a few posts back...
Don't know how or what exactly, fact is I get very different (= better) results with small caches!

I'm voting for the first strategy, bc most of the users do not know about this settings or are not changing them... and I think it is part of making a rom to set this to the best value.

One interesting thing to add that would be the bootup memory, but just a suggestion.

I know how much work you are doing there with flashing and testing every rom, thank you very much for that!

[pietrucci]

I'm voting for the first strategy, bc most of the users do not know about this settings or are not changing them... and I think it is part of making a rom to set this to the best value.

One interesting thing to add that would be the bootup memory, but just a suggestion.

I know how much work you are doing there with flashing and testing every rom, thank you very much for that!Ok, I can live with that strategy. Only problem is that sometimes tests will be bad (with large caches) while in daily use these ROMs are fast (also because of the large caches).

I already have the bootup memory, but left it out in the published file. Here a version WITH the bootup memory (on the bottom).

[panosha]

Only problem is that sometimes tests will be bad (with large caches).
I think Diamond works better (the first PPC) and with faster feeling with standard caches. ;)

[pietrucci]

Ok, second ROM from Dutty added.
Tested with default cache settings, make testresults better (writing to cache slows down)

From now on I will test every ROM with default/disabled cache settings so I compare the ROMs with same cachesettings.

[kurgandk]

could you guys include standard roms in your tests? It would be nice ti see how much reason there is to venture into the undiscovered land of cooked roms... ;)

[murks_muell]

bepe has a new rom, v0.93...

[pietrucci]

bepe has a new rom, v0.93...Yes I know, I tested it already yesterday.
But to be honest: I was so disappointed with the graphic testresults I hadn't the courage to publish it.
I tried several cache settings to make it perform better in the tests but nothing really worked, it stayed slow.
I'd really wanted it to be fast(er) because Bepe's 0.54 also was very fast (in fact: overall the fastest around)

Although it feels pretty fast during normal use (not really superfast but "pretty" fast) I just can't make more of the tests...

Post #1 updated...

[saveferris]

Wow, bepe has gone from the fastest Rom to the slowest in his next ROM, thats quite interesting. Looks like his exams are getting ther better of him. Keep plugging away bepe. :)

[pietrucci]

Wow, bepe has gone from the fastest Rom to the slowest in his next ROM, thats quite interesting. Looks like his exams are getting ther better of him. Keep plugging away bepe. :)I'm sure it's NOT because of Bepe, I think its the new ROM itself.
Also the tests don't fully represent how we users experience the ROMs in daily use.

[walshieau]

I'm sure it's NOT because of Bepe, I think its the new ROM itself.
Also the tests don't fully represent how we users experience the ROMs in daily use.

1.93 has MAJOR issues.

[pietrucci]

1.93 has MAJOR issues.perhaps time to test Elite 2.5 beta...
:D:cool:

[walshieau]

perhaps time to test Elite 2.5 beta...
:D:cool:

fingers crossed tonight :D

two more flashes left and i reckon im done

[pietrucci]

fingers crossed tonight :D

two more flashes left and i reckon im donepssssst... (don't wake up anybody...)

[saveferris]

I'm sure it's NOT because of Bepe, I think its the new ROM itself.
Also the tests don't fully represent how we users experience the ROMs in daily use.

I know that, I was just doing a bit of shit stirring, I 'm a big fan of bepe to the point i'm still using his 0.54 Rom. :)

[ceevee369]

1.93 has MAJOR issues.

Don't understand with all respect to the chef above.
1.93.4656.2 WWE cooked by Diamond Project team is running stable without any significant bugs and hits the highest graph scoring...?
Are we talking about the same 1.93 Core?

I am confused ...

[tom_codon]

1.93 has MAJOR issues.

Which one ?
Funny post :)
Remember it's shiped version by htc

[ceevee369]

Which one ?
Funny post :)
Remember it's shiped version by htc

Hoped you did not found that remark - statement :D

We don't want a "Battle of the Chefs" :rolleyes:

[walshieau]

Which one ?
Funny post :)
Remember it's shiped version by htc

activesync issues

[pietrucci]

REMOVED...

[saveferris]

activesync issues

That really isn't major issues is it? I was under the impression that major issues meant that the ROM all round had some serious flaws.

[KukurikU]

That really isn't major issues is it? I was under the impression that major issues meant that the ROM all round had some serious flaws.

you must be joking or you are using yr diamond for mp3 only....
without sync you can not sync yr contacts, calendar, tasks. there are programs that can be installed only when in sync mode. you can use yr diamond as modem for yr pc\laptop.
if this is not a serious flaw than what is? today screen animation not snappy enough? bad sound quality of sound from the speaker when listening to music?:D

[SecureGSM]

Could you please be a little bit more specific? Thank you very much!


activesync issues

[duttythroy]

k just to add I sync with vista no issues for 3 days, battery is good with .05 radio, so not sure really if its a major bug, But this build significant faster and much stable than all builds so far:)

[pizzachips]

elite 3.0 beta is out....
hopefully scores will be much better than the amazing ones for 2.0 =)

[pietrucci]

elite 3.0 beta is out....
hopefully scores will be much better than the amazing ones for 2.0 =)
see post #1
pretty fast, benchmarks like Dutty's latest!
even a little bittie better!

[pietrucci]

Will test to new (pre-)ship 1.93 now...

[pietrucci]

Added off. WWE 1.93.456.2

[ceevee369]

activesync issues

I do have to agree with this despite my postings on DPT V2 thread.
Active sync is indeed for some a hastle and has a bug vs 1.37 builds.

Continously dropping connection with and without "enable advanced network" checked -unchecked.

If a Sync connection is lost while flashing a ROM, we might have some bricked Diamonds..

[SecureGSM]

ActiveSync is not involved in firmware update (fortunatelly). I can confirm succesfull sync with Win XP and Vista over last 3 days. NOTE: sync between 2 computers still does not work and causes sync errors. That is not something specific to this firware though.




... If a Sync connection is lost while flashing a ROM, we might have some bricked Diamonds..

[saveferris]

you must be joking or you are using yr diamond for mp3 only....
without sync you can not sync yr contacts, calendar, tasks. there are programs that can be installed only when in sync mode. you can use yr diamond as modem for yr pc\laptop.
if this is not a serious flaw than what is? today screen animation not snappy enough? bad sound quality of sound from the speaker when listening to music?:D

So you're calling that major issues with the1.93 ROM, well it doesn't seem to be a major issue considering that the ROM is now used in all of the current cook releases. if they had an issue with it it didn't take them long to figure it out. Personally i'd call it a bug.

Major issues I would classify as multiple issues within a ROM to the point its unsuable, whether it be speed stability etc........ Time to get over yourself bud.

[ceevee369]

NOTE: sync between 2 computers still does not work and causes sync errors. .

I guess this responds indeed to my findings though partely.
Yes, I believe that most problems are reported when using 2 PC's (more specific one XP and the other VISTA.

No, the same "bug" was not found in any 1.37 or 1.39 build.

Maybe we better make a new Active Sync thread as it does not belong here :o

[Ce]

Is it possible to test the new Panosha clean 2 because the versionnumber didn't change....the core ROM did...it's now 1.39xxxxxx. TIA!!

[bepe]

I've found now what the problem is related to!

Test 1: Disconnect your device from the PC, make a soft reset and run spb graphic test
Result will be very good
e.g.: 6 - 25 - 6

Test 2: Connect the device to the PC, run spb graphic test again
Result will be bad!!
e.g.: 7 - 30 - 26

Test 2: Disconnect your device from the PC, run spb graphic test again
Result will stay bad!!
e.g.: 7 - 30 - 26

So this is why the results were that different.
btw, you don’t need to disable any apps like manila for this

This is really bad; I think it might be drivers related

Perhaps some ppl would have time to try this on different roms and post a result like this
bElite 0.94 - 1.93 based (not public)
6 - 23 - 6
7 - 30 - 26 (after connecting to PC)

Heard that would also happen on 1.37
18 - 37 - 6
18 - 40 - 14 (after connecting to PC)

[pietrucci]

I've found now what the problem is related to!

Test 1: Disconnect your device from the PC, make a soft reset and run spb graphic test
Result will be very good
e.g.: 6 - 25 - 6

Test 2: Connect the device to the PC, run spb graphic test again
Result will be bad!!
e.g.: 7 - 30 - 26

Test 2: Disconnect your device from the PC, run spb graphic test again
Result will stay bad!!
e.g.: 7 - 30 - 26

So this is why the results were that different.
btw, you don’t need to disable any apps like manila for this

This is really bad; I think it might be drivers related

Perhaps some ppl would have time to try this on different roms and post a result like this
bElite 0.94 - 1.93 based (not public)
6 - 23 - 6
7 - 30 - 26 (after connecting to PC)

Heard that would also happen on 1.37
18 - 37 - 6
18 - 40 - 14 (after connecting to PC)Don't know...
After flashing I never connected to my PC and still got bad results.
And good... and bad...

Really don't know. Someone suggested to disable the Diamond dialer since it uses a lot of memory. But then he tested it and the results stayed the same.
My only positive find till now is testing on battery: my results on battery are better than on AC-power.

[KukurikU]

ActiveSync is not involved in firmware update (fortunatelly). I can confirm succesfull sync with Win XP and Vista over last 3 days. NOTE: sync between 2 computers still does not work and causes sync errors. That is not something specific to this firware though.
sync with 2 pc is a wm 6.1 related problem. look here:
http://forum.xda-developers.com/showthread.php?t=377823
i can confirm that i am able to sync with 2 pc my athena with wm 6 but not my trinity or diamond with wm 6.1

[bepe]

Don't know...
After flashing I never connected to my PC and still got bad results.
And good... and bad...

Really don't know. Someone suggested to disable the Diamond dialer since it uses a lot of memory. But then he tested it and the results stayed the same.
My only positive find till now is testing on battery: my results on battery are better than on AC-power.

But soft reset your device and just do the benchmark, so you will get good ones for sure.

[sandro.bertini]

But soft reset your device and just do the benchmark, so you will get good ones for sure.

I tried with my device (Using bElite 0.93) and got very very close results to yours, Bepe.

It seems to be repeatable.

Regards,

[bepe]

I tried with my device (Using bElite 0.93) and got very very close results to yours, Bepe.

It seems to be repeatable.

Regards,

All Diamond roms have that bug 1.93, 1.39, 1.37, ... and all cooked roms
But I think I've found a way to solve the problem

[pietrucci]

But soft reset your device and just do the benchmark, so you will get good ones for sure.I softresetted many times, but still bad.
I just tried Dutty's new hybrid. First test it tested not so good (but VSBenchmark did, fastest ever!).
Then tried very different settings, including softresets. The test keep getting worse and worse, I never reached the results I got the first time.

However, we keep on trying...

[pietrucci]

I tested Dutty's new Hybrid, see post #1.
Very strange testresults: SPB Benchmark very bad (graphics and CPU fast, but very bad file system index).
BUT: VSBenchmark superfast! (this doesn't really test the file system).
Did several tests in SPB, couldn't get is better. Now I'll do a hardreset and test again once more. Still don't know exactly what the best testprocedure is.

EDIT: Did a hardreset, disabled TF3D, installed SPB Benchmark, set saving settings off on battery power, no SIM-card, softresetted and ran Dutty's Hybrid: exactly the same bad results I got the first time.

[ceevee369]

Hi Pietrucci,

yes, I do know it is rather impossible to test all ROM releases as there are now more ROM's than Diamond owners..

Though, TLR's Roms seen having some fans and he uses some tweaks since his first release, so any chance to benchmark his?

Thanks

[saveferris]

Hi Pietrucci,

yes, I do know it is rather impossible to test all ROM releases as there are now more ROM's than Diamond owners..

Though, TLR's Roms seen having some fans and he uses some tweaks since his first release, so any chance to benchmark his?

Thanks

Thats his next ROM he will be testing, he confirmed it in a private message.

[pietrucci]

Tested the new TLR 3 twice: once with TF3d installed (but not active) and once with TF3D uninstalled (took ages to uninstall...).
The version without TF3D is pretty fast, as expected.

See post #1

[flashmp3]

Hi pietrucci,

I love your thread ! Very useful! I'm sure everyone had been waiting for this kind of thread for ages! CONGRATULATION
Could you add last swiftblade's please?
Do you know why there is such a gap between sound performance in stock and cooked roms?

Thanks and thanks again for this thread and your initiave

[walkerwalk]

can add panosha light 5? this ROM seems interesting... thanks. :)

[pietrucci]

Hi pietrucci,

I love your thread ! Very useful! I'm sure everyone had been waiting for this kind of thread for ages! CONGRATULATION
Could you add last swiftblade's please?
Do you know why there is such a gap between sound performance in stock and cooked roms?

Thanks and thanks again for this thread and your initiaveThnx!
Today I'll test Panosha Light 5
And Swiftblade's
And Cyphol's

I hope...

[panosha]

Certain spb benchmark measurements in this Device are perfectly irregular. Other you realise at Diamond using and other show in measurements. This is for all ROMs, maybe Qualcomm, maybe something else, I don't know... :(

[pietrucci]

Certain spb benchmark measurements in this Device are perfectly irregular. Other you realise at Diamond using and other show in measurements. This is for all ROMs, maybe Qualcomm, maybe something else, I don't know... :(Yes you're absolutely right. SPB Benchmark isn't fully reliable for WM6.1.
But haven't a better alternative yet.

[milleniumbug89]

i've edited it for excel 2007, easier consultation...
i did it for me, but i think is right share my work with the comunity... :P

http://img367.imageshack.us/img367/4893/testaa3.th.jpg (http://img367.imageshack.us/my.php?image=testaa3.jpg)

thanks again, pietrucci :D

[gurkburk]

Thnx!
Today I'll test Panosha Light 5
And Swiftblade's
And Cyphol's

I hope...

Very exciting and see what results you get :D

[pietrucci]

Tested Cyphol's GM360D.3, see post #1
Installed Opera and TF3D like all the ROMs I test, but disabled TF3D (like all the ROMs I test).

[cyphol]

Tested Cyphol's GM360D.3, see post #1
Installed Opera and TF3D like all the ROMs I test, but disabled TF3D (like all the ROMs I test).

As I predicted it would be since I did not tweak any cache parts in device registry. As I said in the ROM thread, I leave that parts to users liking and efforts... for now.:D
Right now I'm playing with multiple conditions for preparing a good ROM for fans.

Very nice benchmark pietrucci. The value is what I need to accomplished my future missions since I don't like to do the benchmarking myself. You know why Pietrucci.:) I hope you will always do the benchmarking for all my future ROMs.

Cyphol.

[pietrucci]

As I predicted it would be since I did not tweak any cache parts in device registry. As I said in the ROM thread, I leave that parts to users liking and efforts... for now.:D
Right now I'm playing with multiple conditions for preparing a good ROM for fans.

Very nice benchmark pietrucci. The value is what I need to accomplished my future missions since I don't like to do the benchmarking myself. You know why Pietrucci.:) I hope you will always do the benchmarking for all my future ROMs.

Cyphol.Ok, thnx. I'll see what I can do.
As long as the wife and kids aren't complaining and at night I need some sleep sometimes).

[pietrucci]

New test: Panosha Light 5

[pietrucci]

Last one for today: SwiftBL-WWE 2.0

[VisitorSam]

Hello,

i want to flash rom on my O2 Diamond, but I don't know what ROM is better in what point. So I decided to make a Benchmark of my orginal ROM and I am surprise.
My Orginal Rom is in some points faster than other ROMs? Why it is so? I don't understood why the copy of 1MB file and the dir list is very faster but the read and write time is many slower.


Sorry for my bad english I am german.

PS: in the Excel table is one point wrong: Arkaball frames per seconds (higher values are better)

PPS: why is the attachment limit four PDF only 20kb?

[pietrucci]

Hello,

i want to flash rom on my O2 Diamond, but I don't know what ROM is better in what point. So I decided to make a Benchmark of my orginal ROM and I am surprise.
My Orginal Rom is in some points faster than other ROMs? Why it is so? I don't understood why the copy of 1MB file and the dir list is very faster but the read and write time is many slower.


Sorry for my bad english I am german.

PS: in the Excel table is one point wrong: Arkaball frames per seconds (higher values are better)

PPS: why is the attachment limit four PDF only 20kb?With the benchmarks we use nowadays you can't compare our devices that good. It seems that every device gets (some) differente results.
The only thing you can do if you want to know which ROM is fast is test different ROMs on the same device and under the same conditions. And these conditions change very often en very fast: if you do a test, then do a simple softreset and then test again you get different results already. Not too much most of the time but sometimes I wonder why results are suddenly that different...
Hope you understand what I try to say: just test different ROMs on your device immediately after flashing.

My tests are all on the same device more or less under the same conditions, so they do say something about speed I think...

[VisitorSam]

I have flashed bebe 9.4 premium in german,

and it is very faster. Show the Bench :)

[pietrucci]

I have flashed bebe 9.4 premium in german,

and it is very faster. Show the Bench :)Yes you're right, Bepe's new ROM is fast!
I got about the same figures you got, see post #1.

[Huey85]

Yes you're right, Bepe's new ROM is fast!
I got about the same figures you got, see post #1.

I wonder how Bepe's latest 0.95 will stack up against the pack ;)

[ceevee369]

I wonder how Bepe's latest 0.95 will stack up against the pack ;)

i bet slower than 0.94. Opera is set as default browser and takes more background CPU power , en if idle.
But, yes, I am curious.

0.95 is a for me a candidate release to test further on stepping (temporarely) away from DP V2 which was/is perfect still for me.

Piet kerel, you took a hell of a job on your shoulders.
Maybe you should take a rest, and analyze the collected data a bit.
Like pointing out to flashers which score's they should loook at if they want speed - quality- functionalit - super graphics - best audio etc.

Thanks for your valuable contribution to XDA and the Diamond Forum!

[pietrucci]

i bet slower than 0.94. Opera is set as default browser and takes more background CPU power , en if idle.
But, yes, I am curious.

0.95 is a for me a candidate release to test further on stepping (temporarely) away from DP V2 which was/is perfect still for me.

Piet kerel, you took a hell of a job on your shoulders.
Maybe you should take a rest, and analyze the collected data a bit.
Like pointing out to flashers which score's they should loook at if they want speed - quality- functionalit - super graphics - best audio etc.

Thanks for your valuable contribution to XDA and the Diamond Forum!Thnx for the advice to rest. But I believe my Diamond is one of my resting points... (sad? o no, not for me). I wonder of my device ever wants a break! Always flashing to a new identity, heavy attacks on his battery...

I don't think it makes much a difference between Bepe's 0.94 and 0.95 because of the Opera browser. I mean in 0.94 links to Opera also were loaded in memory (Opera9.exe and OperaPreL.exe) and as long as you don't use Opera there won't be much difference. Ok, there are some extra registry settings.
Best way yo find out is to test it!

About your remark to start some analyzing: I was just about to do that!
I was thinking to do some other ranking: now I rank the graphics twice: with SPB and with VS. Furthermore VS ranks the sound, what's the use? And why is that ranking so low in the latest ROMs? Will play around in Excel with weights and stuff.
My first obvious conclusion is that the 1.93.xxx ROMs have the future (till there's a better one), the "older" ROMs seem dead (hey what's old in this world, we're talking about 1-2 month old ROMs).
Althought: why score the first ROMs so high on CPU? Why scored Bepe's 0.54 so high on the file system? And why scored his 0.93 so low on graphics in SBP Benchmark and rather well in VS Benchmark?
Questions questions...

In fact why don't YOU all people do some analysing too???


One final thought, alrady mentioned many times: don't take this shit too serious. These benchmark do say something about (relative) speed but certainly not all of it! And are certainly not 100% reliable in WM6.1.
The responsiveness in daily use, the stability, the look, those factors are much more important.
That's my word.

[meow0623]

OMGOSH

Dude you're the man!!!!! you completed the mission impossible (for most ppl) LOL

[pietrucci]

hi,
added Bepe's bElite 0.95 WWE Premium: fast and much free memory...
see post #1

[pietrucci]

Now we've collected such an enormous amount of figures it's time to do something with it...
First my question: which ROM overall performs best according to these two benchmarks (SPB en VS)???
This is difficult to answer: do you like file system speed, do you want CPU speed or speedy graphics?
Well probably all of them I guess...
So this is what I did:

> I considered all individual tests from SPB Benchmark and VS Benchmarktool. VS only tests graphics and sound, so I divided VS in two parts: one part was the sound test, the other all other (graphics) tests. Doing this I came to these individual indices:
1. SPB File system
2. SPB CPU
3. SPB Graphics
4. VS Graphics (so all trest excl. the sound test)
5. VS Sound

> Now I normalized all indices to the same score level. This means I made the scores for the CPU index as high as the index for the graphics test ect.
To achieve this I took the average of every index and calculated the weightfacter for that index. Then I multiplied every individual score with that indexfactor. Doing so every score got about the same height (for example: the File system index got a factor of 14,82 because the average value of the file system index was 183,56 and the average value of the highest index (VS graphics) was 2720,46.

> Then I added all results with factor 1 except for the sound index, this got only factor 0.5. Because I don't think the sound is that important. (by the way: the speed of sound, what is that...?).

> Results are scores from about 10380 till 12844.


AND THE RESULTS ARE............................ (20th July)

1. Walshieau Elite 2.0 12844,60
2. Bepe Elite 0.95 Premium 12807,22
3. Bepe Elite 0.94 Premium 12719,19
4. Walshieau Elite 3.0 12692,22
5. Diamond Project 1.5.2.19965 12653,24
6. Panosha Light 5 12602,59
7. Org WWE 1.93.456.2 12600,91
8. Dutty's Hybrid Publ. Beta 12583,62
9. Panosha Light 4 12578,48
10.GM360D.3 incl. Opera and TF3D installed 12548,34
11. Diamond Surface 6.1.1.16 NLD 12513,51
12. GM360D.2 12481,73
13. SwiftBL WWE 2.0 12460,26
14. TLRDiamond 3.08.07.16 TF3D 12433,78
15. PDAVIET 1.0 12382,50
16. Bepe Elite 0.54 12187,26
17. Org Dutch 1.35.4.. 12109,03
18. Diamond Project 1.5.2.19593 12083,93
19. TLR Diamond 2.08.07.01 11995,58
20. Org WWE 1.37.401.3 11963,52
21. Panosha light 3 1.37.401.3 11895,40
22. Diamond Project 1.5.2.19960 11873,79
23. Panosha light II 1.37.401.3 11848,97
24. Panosha light 1.37.401.3 11839,43
25. Diamond Project Lite 1.5.2.19593 10604,90
26. Bepe Elite 0.93 10381,30

IS THIS SCIENTIFIC??? NO
IS THIS OBJECTIVE? NO
IS THIS OF ANY PRACTICAL USE? HARDLY

(it's just fun...)


Note: If anyone is interested in the individual results of the above ranking I could post a detailed sheet.

[panosha]

pietrucci

The better friend or our better enemy ? :p

[pietrucci]

pietrucci

The better friend or our better enemy ? :pCould you explain, I'm not that bright...

[mgear356]

one question..how do u benchmark sound?

as in the loading of the song OR the sound quality?
weird to see all cooked roms getting lousy score for sound from Stock..

[Swiftblade]

Hi pietrucci.... what a nice benchmarking works, bro and you throw in your analysis too.:) Thanks for the hard works and now I can save doing the benchmarking myself for comparison purposes.:D

Well Done and Keep Up the Good Works!

[zjsteven]

thanks! Great job!

[knives79]

Wow, really tons of work! Thanks, I hope this help who want install new rom and also who want cook :)
Bai bai!

[ceevee369]

Note: If anyone is interested in the individual results of the above ranking I could post a detailed sheet.

Hoi Piet,

Seems you had the same idea about some analyzes - paralyzes.:D
Yes, you're right about the scoring, but me , not a Tech guy understand vagely what the scoring per criteria mean.
I do not know if you have somewhere a " guide" from within the 2 tools You used to point out what the scoring does.
CPU I-II-III for example, what resources are used. etc

I would like to share some time to analyze further on, but without more indepth meaning of score it is diffcult.
FE, Sound , where you gave a lower score might maybe important for some drivers which take a lot of resources (High sound score -lower graph index maybe?)

The good thing is -there is no right or wrong and you used your own device, with same settings so we got ourselves a very very good benchmark.

I am also convinced that the Chefs themselves are looking into your scoring and adapt their tweaks as such :)

One thing I am a bit afraid for is that some people would look at the scorings as "best ROM" meaning overseeing that some of thsoe ROM's might still in BETA mode where bug's might appear. mmmhh. maybe I think too much.

Glad your post become a sticky as it is more than worth!
Kei-goed man! - You Rock pall!

Followin further your thread in my subscribed list, and yes, I might give BePe 0.95 a try , but it hard to step away from DP V2 now that it is being tweaked to MY almost perfection standard :)

[egal02]

Hi pietrucci,
can you bench this (http://www.ppc-welt.info/community/showthread.php?t=132797)rom to?

It's great I think, maybe it has good results..

[pietrucci]

Hi pietrucci,
can you bench this (http://www.ppc-welt.info/community/showthread.php?t=132797)rom to?

It's great I think, maybe it has good results..I would be glad to test it, but... in a few hours I'm going on a vacation for a 2 weeks.
Perhaps in Sharm EL Sheikh there's internet somewhere... and hope I can find taht.

But no more testing for 2 weeks...

Bye bye

[pietrucci]

Hoi Piet,
Seems you had the same idea about some analyzes - paralyzes.:D
Yes, you're right about the scoring, but me , not a Tech guy understand vagely what the scoring per criteria mean.
I do not know if you have somewhere a " guide" from within the 2 tools You used to point out what the scoring does.
CPU I-II-III for example, what resources are used. etc

I would like to share some time to analyze further on, but without more indepth meaning of score it is diffcult.
FE, Sound , where you gave a lower score might maybe important for some drivers which take a lot of resources (High sound score -lower graph index maybe?)

The good thing is -there is no right or wrong and you used your own device, with same settings so we got ourselves a very very good benchmark.

I am also convinced that the Chefs themselves are looking into your scoring and adapt their tweaks as such :)

One thing I am a bit afraid for is that some people would look at the scorings as "best ROM" meaning overseeing that some of thsoe ROM's might still in BETA mode where bug's might appear. mmmhh. maybe I think too much.

Glad your post become a sticky as it is more than worth!
Kei-goed man! - You Rock pall!

Followin further your thread in my subscribed list, and yes, I might give BePe 0.95 a try , but it hard to step away from DP V2 now that it is being tweaked to MY almost perfection standard :)Thnx for you reaction ceevee (is that your name?),

Yes, you pointed out to do some analysis, I was just planning to do so.
I do think these test say something about speed. When I feel a ROM is fast and snappy, it always gives me a good benchmark. But you're right that doesn''t always mean it's a good ROM. It can have freezes, bugs, not the programs we want or programs we dón't want...

I was in doubt publishing a ranking because people might conclude it's an absolute truth. Well it isn't and I think people know by now.

Well that's it folks. Hope to hear from you in a few weeks after my vacation. Ready to test...!!!

[mgear356]

one question..how do u benchmark sound?

as in the loading of the song OR the sound quality?
weird to see all cooked roms getting lousy score for sound from Stock..
:) bump bump :)

[suiller]

hi guys,
just installed bElite.95, after having reinstalled all the software I need I tried to run vbench... and results are 2875 :eek: with Vbench?? tried 3 times, waited 1min after soft reset... why a value so high??

2nd set test I've enabled all common tweaks and result are about 2600 :confused:

so all those tweaks are unuseful?? :cool:

bye,
davide

[Gissan]

So ... which is the fastest ROM of them all?

[hitman220881]

So ... which is the fastest ROM of them all?

Almost all of the cooked ROMs are fast after applying tweaks.Maybe it more useful what ROM is more energy efficient. Can anybody do some tests with batterystatus to find out? Maybe with and without TouchFlo3D activated?

[suiller]

it depends also by energy saving parameters, 2g or 3g, gprs auto attach, and so on... we have all different setups that just fit our needs, surely radio rom can make the difference but I don't think a so BIG difference.

for example I made this test: I can reach 2 "working" days without recharging by disabling wifi, gprs auto attach/search, g3 and bluetooth.

if I enable all stuffs my battery dies within 12 hours :D

bye,
davide

[wonderiuy]

this thread got sticky but don't get further updates

[pietrucci]

this thread got sticky but don't get further updatesThat's partly because I'm on vacation in Egypt. I do all the testing in my free time, it's hard to find the means to do this in Egypt.
In a week I will test all the newer ROMs' I hope...

[flashmp3]

That's partly because I'm on vacation in Egypt. I do all the testing in my free time, it's hard to find the means to do this in Egypt.
In a week I will test all the newer ROMs' I hope...

Enjoy your trip ;) this thread has to remain sticky altough it isn't updated evrey day !

[zyborg]

Please add a date in front of the thread headline, so that we can see when it's updated with new benchmark results. Thank you.

[DarkRay_]

what are conditions used for the tests?

with SIM or whitout SIM?

I read that the test is launch after the flashing a rom. But can you more specifics in conditions?
Where install VS and SPB Benchmarks, in main memory or internal?

I will test all Spanish roms, for the moment: Original 1.37, Original 1.93 , Vodafone 1.37, Movistar ??, Diamond Project Team Rom V2 Lite ESN , Vauss 1.2 (from HTCmania forum).

[pokevitek]

Hi, very good work, but I noticed one "bug" here. From your test results, it seems that graphical performance of orig 1.93 ROM is terrible, but it is not. I noticed days ago that device is very slow after active sync connection, if you not restart device after that. So, I tried orig rom 1.93 before restart - get same results as you, and after restart, and I get 2536 points. I dont know how, it is a mystery for me, but active sync terribly slows device.

[iliescu902]

well great job on the testing. however, like suiller said, because we have different needs the ROMs , radios and battery life will perform differently each time being tested.

all in all these benchmark results are good for giving you roughly an ideea of what ROM to try out.
anyway, keep up the good work

[Knoore]

Could you maybe update this with the newest roms?

[Udi]

Nice work.
It would be great if you could benchmark latest official ROM. I think that would be the best comparison over all.

Edit: Sorry, I see it was already made. :)

[creatix5]

Can you test Mossis Rom 2.0? Its my favorite, very stable, fast, but I think only in German

Link: http://download.stanigel.de/

greetings
CreaTiX

UPDATED JULY 20TH:
* Added testresults Bepe 0.95
--------------------------------

OK, TIME TO DO SOME SCIENTIFIC ANALYZING STUFF...
(now this is the part where you all should start laughing...)

I STARTED A DISCUSSION IN POST http://forum.xda-developers.com/showpost.php?p=2427975&postcount=111
HOW ABOUT IT???

-------------------------------


Hi guys and dolls,

I've been busy doing some performance testing using SPB Benchmark and VSBenchmark.
Not very scientific, these benchmarks don't seem to work that perfect on WM6.1. But for comparing reasons it should say sómething.
Furthermore: a good testresult it no guarantee your device really FEELS fast, often the feeling is different from cold measurements...

I've tested most recent ROMs, maybe add some soon if anyone is interested.
All ROMs tested right after flashing, I disabled TF3D, set power to max., no radio, no SIMcard, tested on battrypower.

--------------------------
Tested ROMs so far:
Org Dutch 1.35.4.xxx
Org WWE 1.37.401.3
Diamond Surface 6.1.1.15 NLD
GM360D.2
Panosha light 1.37.401.3
Bepe Elite 0.54
Walshieau Elite 2.0
TLR Diamond 2.08.07.01
PDAVIET 1.0
Panosha light II 1.37.401.3
Diamond Surface 6.1.1.16 NLD
Diamond Surface 6.1.1.16 NLD (with startup and TF3D enabled)
Diamond Project 1.5.2.19960
Diamond Project 1.5.2.19593
Diamond Project Lite 1.5.2.19593
Diamond Project 1.5.2.19965 (low cache settings)
Panosha light 3 1.37.401.3
Bepe Elite 0.93 (low cache)
Walshieau Elite 3.0
Org WWE 1.93.456.2
Dutty's Hybrid Publ. Beta
TLR Diamond 3.08.07.16 (TF3D uninstalled)
TLR Diamond 3.08.07.16 (TF3D installed but disabled)
Panosha light 4
------------------------

Most recent:
GM360D.3 (Opera and TF3D installed but disabled)
Panosha Light 5
SwiftBL WWE 2.0
Bepe's 0.94 WWE Premium
Bepe's 0.95 WWE Premium
-------------------------

[pietrucci]

Can you test Mossis Rom 2.0? Its my favorite, very stable, fast, but I think only in German

Link: http://download.stanigel.de/

greetings
CreaTiXOk, I usually don't test other language ROMs, only WWE and (sometimes) Dutch. But hey, it's thursday...

See post #1 for some recent ROMs:

Dutty's 20273 V1.5
Dutty's priv beta V1.6
Dutty's 20273 V1.6
Panosha Shine On 1.1
Tom Diam Prj v3.3 WWE
Mossis ROM 2.0 GER
HTC WWE_EE 1.93.479.3
SpeedUP TOR_ROM 1.93.456.2

[yiannosk2]

Why you didn't test any ROMs from SwiflBL?

[ninja.rogue]

Yes, this was also my question.
Just as a suggestion, obviously, but as SwifBL are amongst the most viewed topics as far as Roms are concerned, I am quite curious about comparing them.....
Could you just try it out - a 6.0 release has just turned out to be the newest one!
Pllllleeeeeaaaasssseeeeeee.....

[duttythroy]

Ok, I usually don't test other language ROMs, only WWE and (sometimes) Dutch. But hey, it's thursday...

See post #1 for some recent ROMs:

Dutty's 20273 V1.5
Dutty's priv beta V1.6
Dutty's 20273 V1.6
Panosha Shine On 1.1
Tom Diam Prj v3.3 WWE
Mossis ROM 2.0 GER
HTC WWE_EE 1.93.479.3
SpeedUP TOR_ROM 1.93.456.2


Nice to see you back here friend updating and giving us insight look of our rom performance,

[pietrucci]

Yes, this was also my question.
Just as a suggestion, obviously, but as SwifBL are amongst the most viewed topics as far as Roms are concerned, I am quite curious about comparing them.....
Could you just try it out - a 6.0 release has just turned out to be the newest one!
Pllllleeeeeaaaasssseeeeeee.....I DID test a SwiftBL ROM 2.0.
But hey, I can't test them all.
I will test some more recent ROMs next days (for starters with Dutty's 1.7 beta!)

[alphazero]

Thank you very much for testing my TOR ROm. It shows im on the right way leaving the Official rom untouched and just adding netcf 3.5. I will do much better next time even my actual rom did what it should be.

[panosha]

Ok, I usually don't test other language ROMs, only WWE and (sometimes) Dutch. But hey, it's thursday..
Wellcome back mate. :)

[pietrucci]

Thank you very much for testing my TOR ROm. It shows im on the right way leaving the Official rom untouched and just adding netcf 3.5. I will do much better next time even my actual rom did what it should be.Ok man, good luck with your cookin'!

[pietrucci]

Wellcome back mate. :)Thanx mate...

[pietrucci]

Ok, I updated with three new ROMs: Dutty's v1.7 beta, Bepe's 0.99dx and SwiftBL's 6.0...

[duttythroy]

:)Ok, I updated with three new ROMs: Dutty's v1.7 beta, Bepe's 0.99dx and SwiftBL's 6.0...


nice one bro, the final should be hopefully better, great work and glad to see u back agin, realy like ur work:)

[pietrucci]

:)


nice one bro, the final should be hopefully better, great work and glad to see u back agin, realy like ur work:)Thanx bro, like your work more!

[christonge]

Nice work my friend.... welcome back. You have been missed ;)

[Swiftblade]

Welcome back! Nice to see you back with your benchmarking updates. We were totally lost for the last few weeks that you were away on holidays..:)

Keep up the excellent job! Cheers

[pietrucci]

Welcome back! Nice to see you back with your benchmarking updates. We were totally lost for the last few weeks that you were away on holidays..:)

Keep up the excellent job! CheersNice reaction...
Hope my device stays happy with all these flashes, hardresets, softresets and testing of its components...

B.t.w. I wan't on holidays all that long, was also busy with some other things (like wife and kids...)

[pietrucci]

HI,

Added two new ROMs: Swtos GREEK and Swtos WWE.
VERY fast ROMs with very large memory, don't know how Swtos did this!

ROMs are (really) light versions, the WWE version is without HTC Keyboards, only default keyboard.
Had some issues though with this WWE ROM, see my post in his thread.

Congratz Swtos!

[Swiftblade]

Nice reaction...
Hope my device stays happy with all these flashes, hardresets, softresets and testing of its components...

B.t.w. I wan't on holidays all that long, was also busy with some other things (like wife and kids...)

LOL.. with all those flashings and hard resetings, it's really a great sacrifice on your parts. That bring up a pointof doubt, is there a limit to many times we can flash a device, I wonder?

You do desire your holidays and more times with your family. Cheers :D

[swtos]

HI,

Added two new ROMs: Swtos GREEK and Swtos WWE.
VERY fast ROMs with very large memory, don't know how Swtos did this!

ROMs are (really) light versions, the WWE version is without HTC Keyboards, only default keyboard.
Had some issues though with this WWE ROM, see my post in his thread.

Congratz Swtos!

ThanX alot for your Tests.
Keep Up your eXcellent work !

[pietrucci]

Added three more ROMs:
PDAVIET V02 20275
SC 2.8NLD EXTREME 20273
Dutty's 20275 v1.7beta2 Lite

Dutty's 1.7beta2 a bit disappointing, ROM is slower than 1.7beta. The CPU figures are not so good (tested and reflashed over and over again, no better results...)

[pietrucci]

Added the newest Swtos v0.95 WWE: very fast!

[skippycentraal]

nice to see the results of scv2.8 :D

thnx my friend

[GSeeker]

Added the newest Swtos v0.95 WWE: very fast!

will try this one, thanks!:D

[GriFolle]

Just a courious think:
Why didn't you tried any panosha's shine over the 1.1 version?

[pietrucci]

Just a courious think:
Why didn't you tried any panosha's shine over the 1.1 version?See first post at the bottom.

I'm a bit tied at the moment, my Vista lost it's spirit...
Reinstalling it, then do further benchmark tests including Panosha.
Our friends from the Greece (Panosha, Swtos) are active!

[christonge]

See first post at the bottom.

I'm a bit tied at the moment, my Vista lost it's spirit...
Reinstalling it, then do further benchmark tests including Panosha.
Our friends from the Greece (Panosha, Swtos) are active!


Sounds familar!! Vista always gives you that "added value" :p

[GriFolle]

See first post at the bottom.

I'm a bit tied at the moment, my Vista lost it's spirit...
Reinstalling it, then do further benchmark tests including Panosha.
Our friends from the Greece (Panosha, Swtos) are active!

About your Vista issues: LINUX RULES..! :D
About the Greek activities..don't know, maybe the Olympic Spirit has some benefical effects on Swtos & Panosha! :D

Anyway, seriously...THANKS A LOT FOR YOUR WORK MAN! :cool:

[pietrucci]

Couldn't wait to test Dutty's new UltraLite, Geistteufel got some incredible SPB Benchmark results (see Dutty's thread), used my work's laptop to flash.

I'm sorry to say that I didn't get his super results...
Tried three times, inclusing hardresets, just didn't get it...
See first post.

[pietrucci]

Tested CT's 20275 V5 Lite. Results comparable with Dutty's latest UltraLite

[christonge]

Tested CT's 20275 V5 Lite. Results comparable with Dutty's latest UltraLite

Thanks mate.... I appreciate you took the time and effort to do all these test! It´s a whole load of flashing!!!

[pietrucci]

Thanks mate.... I appreciate you took the time and effort to do all these test! It´s a whole load of flashing!!!Yes it is! It's not only flashing, it's also hardresets and sometimes in between reflashing to an original ROM (seemd to make a difference in the past, I don't if it still does). But I kind of automated the preocess of disabling everything and setting it up for tests, so that's not so big a time problem.
I still wonder if flashing and hardrestting that much effects the lifetime of the device... (I guess by the time my device gives up I already have another one...)

But hey, you cooks do also a lot of flashing, do you!

[christonge]

Yes it is! It's not only flashing, it's also hardresets and sometimes in between reflashing to an original ROM (seemd to make a difference in the past, I don't if it still does). But I kind of automated the preocess of disabling everything and setting it up for tests, so that's not so big a time problem.
I still wonder if flashing and hardrestting that much effects the lifetime of the device... (I guess by the time my device gives up I already have another one...)

But hey, you cooks do also a lot of flashing, do you!

Certainly do.... on one hand I enjoy it but sometimes it would be nice just to "sit back and enjoy" the Diamond. Certainly when a ROM is working well but unfortunately :o I soon get bored and I want to experiment with the kitchen :p

[ninja.rogue]

New Roms from Panosha and Swtos are out!!!
Looking forward to reading your benchmarks updated! :)

[pietrucci]

No more testing for a few days... I bricked my Diamond, my GPS is broken. Don't know how or what, but it's totally dead.

Will bring it to repair monday, I guess it wont'be back for a week or so.

Grrrrr


Luckily I still have my Kaiser

[panosha]

No more testing for a few days... I bricked my Diamond, my GPS is broken. Don't know how or what, but it's totally dead.

Will bring it to repair monday, I guess it wont'be back for a week or so.

Grrrrr


Luckily I still have my Kaiser
I wish you have Diamond fast back and without expenses. What ROM did you have when did happen this ?

[Blazeitup123]

No more testing for a few days... I bricked my Diamond, my GPS is broken. Don't know how or what, but it's totally dead.

Will bring it to repair monday, I guess it wont'be back for a week or so.

Grrrrr


Luckily I still have my Kaiser
oh thats soooo weak! it was great having you back and now your gone again! oh no!

[duttythroy]

No more testing for a few days... I bricked my Diamond, my GPS is broken. Don't know how or what, but it's totally dead.

Will bring it to repair monday, I guess it wont'be back for a week or so.

Grrrrr


Luckily I still have my Kaiser

really sorry to hear about this bro:confused: wish ur diamond a quick recovery

[pietrucci]

oh thats soooo weak! it was great having you back and now your gone again! oh no!yes I know, weak weak....
I guess the flashing has exhausted my baby...

But no fear, I'll be back!

Arnie

[losdrivare]

yes I know, weak weak....
I guess the flashing has exhausted my baby...

But no fear, I'll be back!

Arnie

Friend, you should have a donate button.

If your Diamond got bricked due to all your flashing and benchmarking, then I'd be happy to make a donation to help with the repairs. Your work with this thread is very much appreciated!

[christonge]

That´s a disaster mate :( I hope you have a quick turn around with HTC. As I previously said to Tom if you need to but another Diamond you can count on my help. Chris ;)

[pietrucci]

Thanx for your support mates, really appriciate it.

Seeing all those new ROMs without playing with them...
(have to find someone else to play with for a while)

[neuro159]

With any ROM I have low GAPI blt score (~30) and low Arcaboll fps in spb benchmark, and all other individual scores are comparable with published results.

Does anybody have any ideas on how this is possible?

also VSBenchmark score is almost the same. All results are very stable.

[pietrucci]

Hi,

My device is up again, somehow I managed to get my GPS to work again. As long as it lasts...
So immediately I did some benchmarking. I tested Swtos 1.0 Extreme without any addons. Best results ever...!!! Congratz Swtos.

But... this is a very very clean and basis ROM. No Manila, no Opera, no HTC Keyboards, no Office.
So I also tested this ROM with these programs installed like all other ROMs I benchmarked. Results: still very quick.
I tested 5 versions of Swtos and they're in the top 5!!!
Swtos, you know how to cook fast ROMs (or you know how to influence SPB and VS Benchmark, lol)!!!

One but: by installing addons (Manila ect.) afterwards the device uses more memory then other ROMS which have these pre-installed.
This ROM has only 63 Mb Program memory left, other have more...

[christonge]

Hi,

My device is up again, somehow I managed to get my GPS to work again. As long as it lasts...
So immediately I did some benchmarking. I tested Swtos 1.0 Extreme without any addons. Best results ever...!!! Congratz Swtos.

But... this is a very very clean and basis ROM. No Manila, no Opera, no HTC Keyboards, no Office.
So I also tested this ROM with these programs installed like all other ROMs I benchmarked. Results: still very quick.
I tested 5 versions of Swtos and they're in the top 5!!!
Swtos, you know how to cook fast ROMs (or you know how to influence SPB and VS Benchmark, lol)!!!

Glad your back up and running mate ;)

swtos is on fire!!! He´s cooking up some great stripped down ROMs and I think it´s difficult to compare with things like Manila and office missing. But hey that´s the beauty of XDA something for everyone and he´s doing an excellent job :D

[wontcachme]

Hi,

My device is up again, somehow I managed to get my GPS to work again. As long as it lasts...
So immediately I did some benchmarking. I tested Swtos 1.0 Extreme without any addons. Best results ever...!!! Congratz Swtos.

But... this is a very very clean and basis ROM. No Manila, no Opera, no HTC Keyboards, no Office.
So I also tested this ROM with these programs installed like all other ROMs I benchmarked. Results: still very quick.
I tested 5 versions of Swtos and they're in the top 5!!!
Swtos, you know how to cook fast ROMs (or you know how to influence SPB and VS Benchmark, lol)!!!

One but: by installing addons (Manila ect.) afterwards the device uses more memory then other ROMS which have these pre-installed.
This ROM has only 63 Mb Program memory left, other have more...

I had exactly the same problem. Had the phone for 2 weeks and had just flashed a custom rom with radio .05. Also no gps whatever i did. Also factory defaults didn't work, several resets didn't. So i contacted the repairservice, made a repairnumber. I decided after i hang up to reflash everything back to defaults again and then out of the blue it worked. Even with custom roms now it still works. I still have no clue what was wrong, but it works again.

Thanx very much for testing rom after rom after rom. Like it much :D

[pietrucci]

Glad your back up and running mate ;)

swtos is on fire!!! He´s cooking up some great stripped down ROMs and I think it´s difficult to compare with things like Manila and office missing. But hey that´s the beauty of XDA something for everyone and he´s doing an excellent job :Dweel, I'm afraid I still have a problem. After flashing Swtos my GPS doesn't work again. Don't know, think there's a harware problem.

Like I said a few threads before: I also tested Swtos WITH Manila, HTC Keyboards, Opera and Office. And it still is very fast.

Hope I can get my device to work again else back to repair.

[christonge]

weel, I'm afraid I still have a problem. After flashing Swtos my GPS doesn't work again. Don't know, think there's a harware problem.

Like I said a few threads before: I also tested Swtos WITH Manila, HTC Keyboards, Opera and Office. And it still is very fast.

Hope I can get my device to work again else back to repair.

I´m sure it´s a hardware problem.... probabably better to send it back anyway although from personal experience it´s a real pain.

What would be nice if we could do some benchmarking regard the age old question which type of ROM is best. A full cooked ROM with OEMs or a stripped ROM with applications added as cabs.

On the Spanish site where I post my Spanish ROMs one of the guys has done some benchmarking with my Spanish Full ROM using different radios... I´ve attached the files, it´s very interesting. Thanks to jalozano22 (the Spanish version of pietrucci!!) at HTC Mania for all his hard work on compiling the results

[panosha]

My opinion for the problems that we see these days in some GPS: are owed in 08 radio.

[losdrivare]

Great to hear your GPS is working again, I hope you'll be able to sort it out without too much hassle.

Just want to tell you that I'm really looking forward to seeing the results for the new Panosha Shine On 1.6 ROM with the new 20721 build. I'm curious about what improvements the 20721 will bring to a full ROM. Hope you'll find time to test it!

[pietrucci]

Added two more ROMs: Swtos v1.0 Full version and Speedup TOR v2 WWE.

The first, Swtos Full, is interesting because before I also tested his Extreme lite with addons Manila, HTC Keyboars, Office and Opera. So we have two more or less the same versions, one with programs postinstalled (cab-version) and one with those programs buildin (I made both tests blue in my table). Surprisingly the one with the postinstalled programs is faster and about the same memory than the buildin. That's what both the benchmarks say... NOT COMPLETELY HONEST, because the full version has more programs installed than my addon version (but then again: this shouldn't influence the persormance that much as these programs aren't in memory during tests).

The second ROM is alphazero's Speedup TOR WWE V2, the tuned WWE193 version.
I am a bit disappointed in the becnhmark results, thougt it would be faster (and maybe even beat Swtos...). I tested twice, both with my default settings, but can't get it any faster.
At the moment I still have this ROM installed and it rally feels good and snappy. TouchFlo3D responses quickly, navigation through the menu is fast, starting programs ok. But as said: the benchmarks stay behind.

[alphazero]

Added two more ROMs: Swtos v1.0 Full version and Speedup TOR v2 WWE.

The first, Swtos Full, is interesting because before I also tested his Extreme lite with addons Manila, HTC Keyboars, Office and Opera. So we have two more or less the same versions, one with programs postinstalled (cab-version) and one with those programs buildin (I made both tests blue in my table). Surprisingly the one with the postinstalled programs is faster and about the same memory than the buildin. That's what both the benchmarks say... NOT COMPLETELY HONEST, because the full version has more programs installed than my addon version (but then again: this shouldn't influence the persormance that much as these programs aren't in memory during tests).

The second ROM is alphazero's Speedup TOR WWE V2, the tuned WWE193 version.
I am a bit disappointed in the becnhmark results, thougt it would be faster (and maybe even beat Swtos...). I tested twice, both with my default settings, but can't get it any faster.
At the moment I still have this ROM installed and it rally feels good and snappy. TouchFlo3D responses quickly, navigation through the menu is fast, starting programs ok. But as said: the benchmarks stay behind.

thx for the test pietrucci! i would suggest not only to trust benchmark results. The results can be fooled by tweaks you do very easily. I mean this benchmarks do tests like writing files to ram and back to storage. Of course you will win this test if you rip out all kind of apps to free more storage. This will get you the benchmark lead but compared to a similar rom it will be anyway slower. i hope u understand what i mean =) I hope with V3 and integrating netcf 3.5 the benchmark results will look better too.

greetings alpha

[pietrucci]

What would be nice if we could do some benchmarking regard the age old question which type of ROM is best. A full cooked ROM with OEMs or a stripped ROM with applications added as cabs. See the previous post: I tested Swtos twice: once hite Lite with addons, once his Full. Surprisingly and interesting: the addon version was (a bit) faster!!! And about the same memory.
Nice puzzle for you cooks!

[swtos]

Added two more ROMs: Swtos v1.0 Full version and Speedup TOR v2 WWE.

The first, Swtos Full, is interesting because before I also tested his Extreme lite with addons Manila, HTC Keyboars, Office and Opera. So we have two more or less the same versions, one with programs postinstalled (cab-version) and one with those programs buildin (I made both tests blue in my table). Surprisingly the one with the postinstalled programs is faster and about the same memory than the buildin. That's what both the benchmarks say... NOT COMPLETELY HONEST, because the full version has more programs installed than my addon version (but then again: this shouldn't influence the persormance that much as these programs aren't in memory during tests).



This is the reason that i prefer to cook only light & naked roms. They are
always faster than the same rom precooked with all apps installed.
Personally, i don't like full editions . It's released only for beta testers
who wants all programs preinstalled ...
A rom with all apps preinstalled is only a repackage not a cooking ...
Agree also that "feeling" is better than benchmarks many times ...



ThanX again

[pietrucci]

thx for the test pietrucci! i would suggest not only to trust benchmark results. The results can be fooled by tweaks you do very easily. I mean this benchmarks do tests like writing files to ram and back to storage. Of course you will win this test if you rip out all kind of apps to free more storage. This will get you the benchmark lead but compared to a similar rom it will be anyway slower. i hope u understand what i mean =) I hope with V3 and integrating netcf 3.5 the benchmark results will look better too.

greetings alphaYou're completely right alphazero, that's what I also said a thousand times before. What I also said a thousand times is that these benchmarks aren't 100% reliable. When you do the test right after softreset and do the test again after a new softreset you often some get different results. Not much (the indices thay the same) but it's there... That's why I benchmark twice most of the times.
Also benchmarks and user experience are just different things. As is stability.

[alphazero]

This is the reason that i prefer to cook only light & naked roms. They are
always faster than the same rom precooked with all apps installed.
Personally, i don't like full editions . It's released only for beta testers
who wants all programs preinstalled ...
A rom with all apps preinstalled is only a repackage not a cooking ...

ThanX again

swtos we should really have a talk about our roms. im pretty sure we could combine something to make a very good rom. up to you feel free to contact me

[pietrucci]

This is the reason that i prefer to cook only light & naked roms. They are always faster than the same rom precooked with all apps installed. Can you explian why addons are faster than precooked? I mean when it's there in ROM and integrated with the rest, this should be faster, shouldn't it?

Addons have another big advantage off course: new versions. When there's a new version you will completely remove the previous version. With ROM versions that isn't the case.

So now we focus on naked chicks...eh ROMs with addons??

[losdrivare]

It would be really great to hear everyones experiences about what ROM actually FEELS the fastest and most responsive!

Not to fill up your thread with subjective discussion, pietrucci, I created a new thread for this here (http://forum.xda-developers.com/showthread.php?p=2594087):
http://forum.xda-developers.com/showthread.php?p=2594087

Since you're probably one of few who's tried so many ROMs, I'm sure you'll have a lot to say on the matter!

[pietrucci]

励成科技发展有限公司专业开发、生产、销售智能换画灯箱 (http://www.licheng01.com)/系统及管状电机等系列产品。公司换画灯箱 (http://www.licheng01.com)在整个产品研发、生产过程中全面推行高质量管理体系，滚动灯箱 (http://www.licheng01.com)产品均按照高技术智能控制系统化、开模注塑配件模具化、部件加工组装 精细化来不断的加强提高产品的质量和生产,滚动灯箱 (http://www.licheng01.com)为了满足市场多样化需求，我们不断的进行产品创新，研发、制造出适应 不同客户的需求及满足客户长期、稳定的要求。并大力加强售后服务的规范，滚动灯箱 (http://www.licheng01.com)让每个客户全无后顾之忧。I thought you'd never ask... Well, look in the grey closet, second shelf on the right. Be carefull, it bites!

[pietrucci]

See first post, added Panosha's 1.7 Light.

[panosha]

Thank you again and again and again. I need Full to. :D

[creatix5]

Danke für den Test! Ich weiß, ich sollte englisch schreiben, aber heute ist mein schlechter Tag *vbg* . Tolle Arbeit! Nice job!

Ok, I usually don't test other language ROMs, only WWE and (sometimes) Dutch. But hey, it's thursday...

See post #1 for some recent ROMs:

Dutty's 20273 V1.5
Dutty's priv beta V1.6
Dutty's 20273 V1.6
Panosha Shine On 1.1
Tom Diam Prj v3.3 WWE
Mossis ROM 2.0 GER
HTC WWE_EE 1.93.479.3
SpeedUP TOR_ROM 1.93.456.2

[Addicteddddd]

very very very nice table, i'm currently on panosha 1.6 full but can't decide what to go to :(

[pietrucci]

Added two more Greek babies: Panosha's 1.7 Full and Swtos 1.0 Naked...
And alphazero's V2.1 with NetCF3.5

[pietrucci]

EDIT 14-03: I changed my advise for 1.93 to 2.03
------------------------------------------------

So you Want my opinion which ROM to choose???

Well here it is: just pick one!

Although benchmarks give (some) differences between ROMS, in normal daily use really you won't notice any difference.
People often shout "very fast ROM, speedy ROM" (including me), but I think that are mostly feelings than facts (by the way: there's nothing wrong with these feeling, just fine!).
- Take any 2.03 based cooked ROM: they all are fastest and stabile at the moment; the cooked versions are just better with less bloat and more memory.
- Maybe take a look in my table which has the largest memory left.
- Take one with the programs you like cooked in. Or just a ROM that looks nice.
- Or take a naked/lite version and add the programs you like.
Advantage of a naked ROM is that new program versions will be overwritten, so no extra ROM memory in use.
Also you can just install the programs you need, no extra's.
And they seem to be as fast as ROMs with cooked in programs!
Disadvantage of naked ROM: you have to install everything afterwards (althoug you can automate this process; I postinstall about 20 programs in about 10-15 minutes, fully automated).

That's my opinion....

until there comes a new one...!



Pietrucci

[foolsgold]

Anyone install this official 1.93 HTC ROM "A_RUU_Diamond_HTC_PTG_1.93.410.3_Radio_Signed_Diam ond_52.29.25.12_1.00.25.05_Ship.exe"

It landed yesterday, over here:

http://www.htc.com/www/SupportDownload.aspx?p_id=133&cat=2&dl_id=102

Any good?

[alphazero]

hello thank you very much for your test!

it gives me very good information about what to work on

[udayrulz]

Can you explian why addons are faster than precooked? I mean when it's there in ROM and integrated with the rest, this should be faster, shouldn't it?

Addons have another big advantage off course: new versions. When there's a new version you will completely remove the previous version. With ROM versions that isn't the case.

So now we focus on naked chicks...eh ROMs with addons??

On the downside using add ons take more storage space then the same application already integrated in the ROM ..

[swtos]

On the downside using add ons take more storage space then the same application already integrated in the ROM ..

that's not true ....

[udayrulz]

that's not true ....
Ok this is what dutty said (http://forum.xda-developers.com/showthread.php?t=413782) .., i feel he is right..:)

Just a tip, A clean rom is very good if you going to stick with what you got in the rom and possible a few otther apps by cabs, If your going to installed the same apps which are in a full rom then the clean rom with those apps in will make you lose valuable storage space rather than them cooked in and sometimes device will start to feel sluggish but it all depends on how many cabs youre going to re-install.

TIps , there are certain apps which are cooked in doesnt take up any storage space but if install by cab you will then use up valuable space.

:)
bascally there some programs which ar cooked in for e.g Remote desktop, If cook this in its 1.45mb I think, I will only use up only about .45mb and save 1mb of storage space, If i install this by cab I would use up all of the 1.45mb storage space

[pietrucci]

Two more:
Swtos WWE 20271 v1.0 Naked
SpeedUP TOR_ROM V2.2

[udayrulz]

Two more:
Swtos WWE 20271 v1.0 Naked
SpeedUP TOR_ROM V2.2

Thanks .. great work as always ..:)

[pietrucci]

Two more new ROMs from Dutty:
Dutty's 20748 v1.9 UltraLite
Dutty's 20748 v1.9 Full

Both fast...

[laser21]

Swift has a new 6.2 rom out...would be interesting...:) Thanks

[pietrucci]

Swift has a new 6.2 rom out...would be interesting...:) ThanksDon't know... as far as I can see Swift's changes are about new programs and they have no influence on the benchmarks.
The thing that could make a difference is the updated NetCF3.5. But previous tests have proven there's no real difference in benchmark results between NetCF2.0 and NetCF3.5. So I also don't expect any better results in updated 3.5...

[pietrucci]

Benchmarked Swots 1.0 Vanilla, his most naked
And also the fastest Diamond I've ever tested...!!!

[C.QL]

New one:

HTC DIAMOND TOR V.5.0 Mission Accomplished FINAL Recommended :)

[pietrucci]

New one:

HTC DIAMOND TOR V.5.0 Mission Accomplished FINAL Recommended :)Testing it now... results are not really better as v3.0, don't know why.
Will perform a second test (as always).

[wontcachme]

Perhaps you could test the new swift series full or lite? This since there now are 2 of both with different pagepool. Will be interesting what is faster lite 12mb or lite 32mb

Love your work :)

[charly_mx]

nice workon this thread... ass soon as your diamond gets bricked for all that flashing you will need to ask for donations to buy a new one :)

[pietrucci]

nice workon this thread... ass soon as your diamond gets bricked for all that flashing you will need to ask for donations to buy a new one :)bricked? why should it get bricked???
I have a very special Diamond, it's made from real diamonds, can't be broken...

[pietrucci]

HI, two more ROMs:
SpeedUP TOR_ROM V3
SpeedUP TOR_ROM V5

[scolling]

... you can automate this process; I postinstall about 20 programs in about 10-15 minutes, fully automated

Hi Pietrucci

Great work, thanks for the effort and advice. One question - How do you do automate your postinstall then?

Thanks :)

[lmr2003]

HI, two more ROMs:
SpeedUP TOR_ROM V3
SpeedUP TOR_ROM V5


Hi,

May I aks you if your could perform a benchmarking for the following french version ?

http://ppc.spotzone.fr/diamond/roms/DiamondStone_2.0_FRE_1.93_19965.1.2.3.zip

Seems to be very fast and very stable.

Thanks

[C.QL]

Thanks a lot man!:)

It would be great if you can test also these two rom's (or one of them):

* SC v3.0 NLD CE OS 5.2.20721 (BUILD 20273.1.3.3) 1.93.831.2 without radio and splashscreen
* SC v3.0 NLD LITE CE OS 5.2.20721 (BUILD 20273.1.3.3) 1.93.831.2 without radio and splashscreen

Hartelijk dank van 1 van je zuiderburen! :D

Perfect job!;)

[pietrucci]

Hi Pietrucci

Great work, thanks for the effort and advice. One question - How do you do automate your postinstall then?

Thanks :)I use Mortscript. Initial it's a bit of work, you have to record all your screenactions and then put them into scripts. I've attached a script file for TomTom7, I put some comment in it so I hope you understand what it does.

There are some other tools who do the same (maybe better), search XDA...

[pietrucci]

Hi,

May I aks you if your could perform a benchmarking for the following french version ?

http://ppc.spotzone.fr/diamond/roms/DiamondStone_2.0_FRE_1.93_19965.1.2.3.zip

Seems to be very fast and very stable.

ThanksUsually I don't do foreign ROMs (ok, I did test Greek and Dutch versions but I just can't test them all, sometimes I also have a life...).
Why do you think it's that special?

[pietrucci]

Thanks a lot man!:)

It would be great if you can test also these two rom's (or one of them):

* SC v3.0 NLD CE OS 5.2.20721 (BUILD 20273.1.3.3) 1.93.831.2 without radio and splashscreen
* SC v3.0 NLD LITE CE OS 5.2.20721 (BUILD 20273.1.3.3) 1.93.831.2 without radio and splashscreen

Hartelijk dank van 1 van je zuiderburen! :D

Perfect job!;)Ok zuiderbuur,
I will test one of them (because they're Dutch), tested Skippy's before.

[lmr2003]

Usually I don't do foreign ROMs (ok, I did test Greek and Dutch versions but I just can't test them all, sometimes I also have a life...).
Why do you think it's that special?

I'm asking that because I find this ROM very fast and stable and if benchmarks are good maybe we can ask him to do a WWE ROM :D

[panosha]

Other fast, other stable and other deserves it remains in my device. Better is the last one. :)

[C.QL]

Ok zuiderbuur,
I will test one of them (because they're Dutch), tested Skippy's before.

Great, thanks a lot! :)

De bovenste lijkt me ideaal, maar als je al de onderste had genomen, geen probleem hoor! :p

[lmr2003]

Other fast, other stable and other deserves it remains in my device. Better is the last one. :)

I would like to try your ROM but you don't have a french version

[devilsreject]

Nice post ..thanks!

[pietrucci]

4 More new ROMs:
SC 3.0 NLD EXTREME Colour (20721)
SwiftBL 6.5 Lite 12MB PP (20721)
SwiftBL 6.5 Lite 16MB PP (20721)
Panosha Shine on 1.8


SwiftBL 12Mb PP and SwiftBL 16MB PP give no real differences in benchmarks...

[AndyChecker]

Any new recommendations pietrucci after these latest roms have come out?

[SaP1]

pietrucci, it's in ITA, but can you test this rom, please??

http://www.megaupload.com/it/?d=TYZ57LZ8

thank you

[pietrucci]

pietrucci, it's in ITA, but can you test this rom, please??

http://www.megaupload.com/it/?d=TYZ57LZ8

thank youSaP1, installed this ROM, but after installing the benchmarks and softreset the ROM freezes. And keeps freezing...
So can't test it.

[patterns]

man this is SO SO helpful, thank you and well done on your hard work!!!

[pietrucci]

Tested three more baby's:
Panosha Shimple 0.8 Clean (20275)
HaVeN 1.0.1 build 1999
Dutty v2 Full (20749)

The HaVen ROM frooze right after second softreset, so I couldn't test it...

[pietrucci]

man this is SO SO helpful, thank you and well done on your hard work!!!thanx, glad it is of any use...

[duttythroy]

Tested three more baby's:
Panosha Shimple 0.8 Clean (20275)
HaVeN 1.0.1 build 1999
Dutty v2 Full (20749)

The HaVen ROM frooze right after second softreset, so I couldn't test it...






nice one bro seems this time ur benchmarks and mine are very close did u run one on the ultra lite

[haven80]

Tested three more baby's:
Panosha Shimple 0.8 Clean (20275)
HaVeN 1.0.1 build 1999
Dutty v2 Full (20749)

The HaVen ROM frooze right after second softreset, so I couldn't test it...




:p dunno why... works for all the guys hat have tried it.
You can try testing this:
marsinellazio.altervista.org/Rom_HaVeN_v2_build_19965_DELUXE.rar

copy and paste in your browser.....
Its pretty full of software but i think its usefull for me see how it perform.
thank you for your work.

[pietrucci]

nice one bro seems this time ur benchmarks and mine are very close did u run one on the ultra liteit's coming...

[sev7en]

Great work as usual petrucci.
Really impressed about the latest Dutty: I am with it now and it's as fast as reliable. Highly suggested for any English Diamond :P.

[panosha]

Thanks pietrucci, I have a Raphael ROM now, but you don't have Raphael ? :D

http://img143.imageshack.us/img143/981/pietruccijc3.png (http://imageshack.us)

Avatar gift for you. :p

[duttythroy]

Thanks pietrucci, I have a Raphael ROM now, but you don't have Raphael ? :D

http://img143.imageshack.us/img143/981/pietruccijc3.png (http://imageshack.us)

Avatar gift for you. :p

You got a raphael already bro, you changed devices very quick. touch hd looks great:D

[panosha]

You got a raphael already bro, you changed devices very quick. touch hd looks great:D
Tanks Dutty, but realy I don't have any Raphael. :p

[duttythroy]

Tanks Dutty, but realy I don't have any Raphael. :p

lol, You getting one or waiting for touch HD

[panosha]

I don't like Pro, I need somethig like HD with 4/3, 3,5", 480x640 screen. :rolleyes:

[duttythroy]

I don't like Pro, I need somethig like HD with 4/3, 3,5", 480x640 screen. :rolleyes:

its nice specs but only prob is that ist wvga like sony experia

[pietrucci]

:p dunno why... works for all the guys hat have tried it.
You can try testing this:
marsinellazio.altervista.org/Rom_HaVeN_v2_build_19965_DELUXE.rar

copy and paste in your browser.....
Its pretty full of software but i think its usefull for me see how it perform.
thank you for your work.
Ok, tested it. Publish tomorrow...

[pietrucci]

Thanks pietrucci, I have a Raphael ROM now, but you don't have Raphael ? :D

http://img143.imageshack.us/img143/981/pietruccijc3.png (http://imageshack.us)

Avatar gift for you. :phi Panosha, don't know exactly what you mean. You have a Raphael based ROM for Diamond? Where is it, you want me to bench it?

[panosha]

My bad English... :mad:

http://forum.xda-developers.com/showthread.php?t=426587

[pietrucci]

its nice specs but only prob is that ist wvga like sony experiawhy is that a problem, wvga? because of incompatibility with other devices/ROMs?

[pietrucci]

Added benchmark of Dutty's v2 Extreme Lite.
As expected faster than his full version.
In fact one of the fastest on the market according to SPB and VS benchmarks.

[pietrucci]

New ROM Speedup A.Z.T.O.R. v6