Speed comparision between cooked ROMs (29/11/07 Re-tested with Spb Benchmark!)

[nugift]

1st TEST:

This is the result about the speed between 4 ROMs: Helmi_C (Clean+Bigstorage), Sleuth V1.1, XDA-Live 1.1 and Dutty SupaLite Beta. All have been installed the same programs. :)

The benchmark has been done by SKTools with 3 Kaiser devices! I took the average benchmark.:D

For your information!:p

2nd TEST:

As promised, I have just tested 4 ROMs with Spb Benchmark. The result is very clear and detail (pic 2, 3, 4, 5). All ROMs have not been installed with any soft except Spb Benchmark and SOTI Pocket Controller.

3th TEST:

Pic 6 is the result of CUSTEL v1.0!:p

[kireol]

wow, thanks for the work! That's good stuff to know.

I'd have thought that Dutty's was the fastest, but doesnt look to be so. With those numbers I'd say you'll never "see" a difference though.

[lewcamino]

Nice job man. There a a ton of people always asking that question. This should be a sticky.

Later, Lew

[taiseer999]

Could you elaborate a bit on the meaning of the numbers ;)

[CUSTEL]

SPB Benchmark would give you a more in depth comparison.

[nugift]

SPB Benchmark would give you a more in depth comparison.

Thanks for your suggestion! I will try and report soon!;)

[jychan28]

This is the result about the speed between 4 ROMs: Helmi_C (Clean+Bigstorage), Sleuth V1.1, XDA-Live 1.1 and Dutty SupaLite Beta. All have been installed the same programs. :)

The benchmark has been done by SKTools with 3 Kaiser devices! I took the average benchmark.:D

For your information!:p

The numbers are within the statistic difference and I don't believe it is perceivable in real life operation. In other words, they are all about the same. But each ROM comes with different features. I hope the stability of each ROM are about the same. Stability is the most important factor. I have had machine hung on me during the most critical moment.

[nugift]

The numbers are within the statistic difference and I don't believe it is perceivable in real life operation. In other words, they are all about the same. But each ROM comes with different features. I hope the stability of each ROM are about the same. Stability is the most important factor. I have had machine hung on me during the most critical moment.

Agreed that those numbers could not show the real performance of each ROM. But I do not think that the stability is the same in those ROMs.

As per CUSTEL' suggestion, I am testing those ROMs with Spb Benchmark. The result seems to be different that the one from SKTools!:confused: We all can see the result tonight!

[jakontil]

i'd have to say, when i was using kyphur's XDA LIVE, it was the snappiest among all, only it was designed for non video call device enabled

[jychan28]

i'd have to say, when i was using kyphur's XDA LIVE, it was the snappiest among all, only it was designed for non video call device enabled

It was very difficult to say. I strolled around a phone store and saw a TyTN II and started playing with it. Wow, it was very fast and I was impressed. It was flashed with a CHS ROM. I offer to pay them to sell me a copy of the ROM. However, the machine hang a couple of times during my playing with it. Well that is without tons of other stuff installed and I really didn't think its stability is there yet. So much for my excitement, but it does show the machine can be fast if someone know what to tweak:rolleyes:.

[duttythroy]

Thanks for this thread I realise I messed up somewhere in my performance tweaks.

Can someone do another tests on my Touchflo final Rom 2 which is uploading now and also my SupaLite beta 2 which I'm uploading a fixed version soon. I found out my Rom should be faster thank you guys for putting this tests out, enabling me to found out that I did something wrong in my performance tweaks and now the toucFlo Final rom which is uploading is much faster now:eek: even with more Program memory

[zaharakis]

i have try all roms for kaizer and i agree that helmi_c's roms are still the faster and better roms till now :)

i use the helmi_c clean rom and is far away the best!!

another very good rom is sleuth's 1.1!this rom is too very fast and stable and with all htc official stuff in ;)

both,helmi_c and sleuth,are a step infront all the others at this moment :)

[zaharakis]

Thanks for this thread I realise I messed up somewhere in my performance tweaks.

Cand now the toucFlo Final rom which is uploading is much faster now:eek: even with more Program memory


we'll give it a shot dutty :) keep walking my friend ;)

[duttythroy]

we'll give it a shot dutty :) keep walking my friend ;)

thanks friend you will be in for a surprise:eek: The Final rom which is uploading now is much faster than my supaLite Rom, just fixing the SupaLite beta 2 now

[asfoor]

I think SPEED is second to Stability.... I think a fair test should also have ROM stability measures. i.e. crashes, connectivity, call drops, program load, display and presentation, programs, battery Life, and reception.

[duttythroy]

I think SPEED is second to Stability.... I think a fair test should also have ROM stability measures. i.e. crashes, connectivity, call drops, program load, display and presentation, programs, battery Life, and reception.

TOTALLY AGRRE with that thats why I try to balance this with all my roms .:)

[zaharakis]

I think SPEED is second to Stability.... I think a fair test should also have ROM stability measures. i.e. crashes, connectivity, call drops, program load, display and presentation, programs, battery Life, and reception.

i agree with you my friend.the most wanted in cooking is both,stability and speed!! but those 2 are oposite to battery life :(

[loaferkan]

The best rom that worked for me is Rom Version 1.57.502.2 ,Pr version 22.45.88.07H i got it from one of these form, its very stable, very fast and it was cooked. so try finding this one and get it for tilt, its just remove all the junk leave you with lots of free space and you upload the sotware you want, bye. oh radio version 1.27.12.11

[nugift]

The 'famous' ROMs (Helmi_C Bigstorage, Dutty SupaLite Beta 1, Sleuth 1.1, XDA-Live 1.1) have been re-tested with Spb Benchmark.

The result can be seen at the 1st post!:D

[CUSTEL]

The 'famous' ROMs (Helmi_C Bigstorage, Dutty SupaLite Beta 1, Sleuth 1.1, XDA-Live 1.1) have been re-tested with Spb Benchmark.

The result can be seen at the 1st post!:D
how about a test with CUSTEL v1.0 (http://forum.xda-developers.com/showthread.php?t=347844)? ;)

[nugift]

how about a test with CUSTEL v1.0 (http://forum.xda-developers.com/showthread.php?t=347844)? ;)

Sorry! :p CUSTEL v1.0 will be tested tomorrow!:D

[duttythroy]

Sorry! :p CUSTEL v1.0 will be tested tomorrow!:D

did you tested the latest supalite beta

[CUSTEL]

Sorry! :p CUSTEL v1.0 will be tested tomorrow!:D
Gracias :)

[nugift]

did you tested the latest supalite beta

Only SupaLite beta 1. Is there any difference from SupaLite beta 2?

[Sleuth255]

To make these numbers meaningful, you need to connect the device up and run the windows version of spb benchmark. Then choose "Visualize" and it will pull the xml off the Kaiser and upload to SPB then produce a great chart showing comparisons + the overall benchmark index. This is how I developed the numbers I posted on my ROM thread.

All ROMs uploaded can be saved off then a side by side comparison can be developed.

As I stated earlier, there's something wrong with the Kaiser filesystem (specifically the directory list of 2000 files that runs 10 times longer than it does on other WM devices :eek:) that might really uncork the device if it can be fixed....

[duttythroy]

Only SupaLite beta 1. Is there any difference from SupaLite beta 2?


There was a problem with my storage card affecting my program memory which the memory card inserted took about 10 mb of program memory.

I thought i fixed it in beta 2 but I just finally fixed it so beta 3 will be up. you can o your tests on that one:)

[Sleuth255]

Here's the benchmark for v1.1 Sleuth with the turbomode cab installed. :eek:

IMO all chefs should be just cooking these settings in. I believe only Kyphur is doing this currently.

http://images30.fotki.com/v480/photos/4/43626/167477/1Sleuthspbbenchmarkresults_p2-vi.jpg (ftp://xda:xda@ftp.xda-developers.com/Kaiser/v1.1SleuthBenchmark.jpg)

(Click for full size version if you can't read squintplex)

[CUSTEL]

I'll take a look at the cab to see what is modified...

[zaharakis]

personaly i don't like these tests...

i see every rom in practice with my apps and programs that i use every day..

i have try every rom and i'll keep on doing this till rom cooking stops for kaiser ;)

my opinion ... speed,stability and performance in a rom,is the most wanted things!

helmi_c's clean and

sleuth's 1.1 are the roms that they get all my attention and my respect!!
because they are fast,stable and honest..not big words and noice around them,just work in practice.


now i'm trying custel's 1.0 because it's seems very fast (76mb in cold boot isn't something that we see it every day!;) but isn't look very stable....i gave it 3 HR (build and rebuild for the begining),just to give another oportunity and to be sure..

my friend's dutty new rom is little disapponting for me...it have's only 65mb free ram in cold boot and is so empty....it's no need to be empty to be fast...
sleuth's 1.1 is the big excample for this...

but all you guys who make the roms for us to have to play with our pda's ...don't forget that when we saying ""this rom is fast or this rom isn't""
we mean that we do not respect you!!!!

kind regards Dimitris

[CUSTEL]

personaly i don't like these tests...

i see every rom in practice with my apps and programs that i use every day..

i have try every rom and i'll keep on doing this till rom cooking stops for kaiser ;)

my opinion ... speed,stability and performance in a rom,is the most wanted things!

helmi_c's clean and

sleuth's 1.1 are the roms that they get all my attention and my respect!!
because they are fast,stable and honest..not big words and noice around them,just work in practice.


now i'm trying custel's 1.0 because it's seems very fast (76mb in cold boot isn't something that we see it every day!;) but isn't look very stable....i gave it 3 HR (build and rebuild for the begining),just to give another oportunity and to be sure..

my friend's dutty new rom is little disapponting for me...it have's only 65mb free ram in cold boot and is so empty....it's no need to be empty to be fast...
sleuth's 1.1 is the big excample for this...

but all you guys who make the roms for us to have to play with our pda's ...don't forget that when we saying ""this rom is fast or this rom isn't""
we mean that we do not respect you!!!!

kind regards Dimitris
please elaborate on the instability issue in my thread, I can't do anything if I don't know about it.

[zaharakis]

please elaborate on the instability issue in my thread, I can't do anything if I don't know about it.


i say for your rom custel that is extremely fast!!!

but i must check for good first if is something else that gives me an error who i have in your rom ;)

i have a frozen screen when the device is in sleep mode...and i'm looking to find where is this from..

i put in all roms that i'm trying,the same apps to have an honest comparison between them ;)

[blue_94_trooper]

How do these numbers compare to the stock (AT&T or HTC) ROM's? Are those benchmarks somewhere else on the site?

[CUSTEL]

How do these numbers compare to the stock (AT&T or HTC) ROM's? Are those benchmarks somewhere else on the site?
No comparison....

3rd party roms are generally created to increase the stability and performance of their standard counterparts.

[austinsnyc]

the biggest factor in speed and RAM on boot is going to be what apps open when the OS starts up. Now the Factory shipped HTC rom has the HTC_Home screen which uses lots of memory and system speed just to get going and it is always running in the background. So any rom that does not have the HTC_Home screen is of course going to be faster in boot time and after boot time. These tests almost prove nothing to me because each has different apps that run at start and in the back ground to do this test fair each rom would have to have the same number of files and the same apps at boot and at normal use. If you notice Custel does not have the HTC_Home in his rom which is why it will feel faster and respond better. Its just a matter of what fits you better at this point until a major release is done from HTC or Microsoft.

Austin

[nugift]

The test result with Spb Benchmark of CUSTEL v1.0 can be seen at the 1st post!:D

[zaharakis]

all the benchmarks are not realistick...

when i add all programs in my device and all settings i like (not just me but all of us) only then i see the all true ;)

and all the tests i've made with all roms in here for kaiser,helmi_c's clean is (till now) the most stable and fast rom :) no bugs no problems everything works perfect in it.:)

[_Alex_]

I agree with all the pundits that:


Benchmarks are synthetic. They have little or no value in the real world, because they do not simulate what a human does with their device.
Whatever is resident in memory (today plug-ins, notifications, etc) will slow down the benchmarks.Since my HTC ROM is the first base ROM with all OEM packages included, I figured that alot of people will be using this soon to cook their own ROMs.

So, when in Rome, see attached :D

[panosha]

http://img529.imageshack.us/img529/8712/48035547eh3.gif (http://imageshack.us)

I believe that all ROM 's they have the same roughly speed. The differences that exist are owed in font size & file system cache.

Try this in any ROM with the same applications and tell me. :cool:

[CUSTEL]

http://img529.imageshack.us/img529/8712/48035547eh3.gif (http://imageshack.us)

I believe that all ROM 's they have the same roughly speed. The differences that exist are owed in font size & file system cache.

Try this in any ROM with the same applications and tell me. :cool:
I wouldn't recommend modifying the file system cache, just my .02.

[zaharakis]

http://img529.imageshack.us/img529/8712/48035547eh3.gif (http://imageshack.us)

I believe that all ROM 's they have the same roughly speed. The differences that exist are owed in font size & file system cache.

Try this in any ROM with the same applications and tell me. :cool:

i agree pano but in this value of font cash size you lose ram ;)

a good size of cash for me is 16384kbs or maximum 32768kbs ;)

[panosha]

i agree pano but in this value of font cash size you lose ram ;)

a good size of cashe for me is 16384kbs or maximum 32768kbs ;)
With font cache size you don't lose any RAM. Only with file system cache under the zero losing RAM and i don't think this make the speed better in this device. ;)

[Sleuth255]

File system cache/glyphcache tweaks really help. Compare my "Turbo mode" results posted on page 3 or so with the results on page 1 for my ROM & you will see a noticable improvement (in fact, compare my turbo mode results with any of the results for an interesting surprise ;)). This improvement is reflected in usability as well.

Kaisers have ram-to-burn folks. Why not cook these in? Heck, I've got every app on the planet running & still have 50megs of program memory free

[panosha]

I've got every app on the planet running & still have 50megs of program memory free
OK, i 'll try. Τhat's the best with this machine, i think. :)

[CUSTEL]

i agree pano but in this value of font cash size you lose ram ;)

a good size of cash for me is 16384kbs or maximum 32768kbs ;)
glyphcache is fine, file system will cause stability issues. I have done extensive testing with this...

I'm 95% sure the File System Cache modification you had made in my rom is what is causing your bsod issue.

This is why only one of my hermes rom's had the tweaks enabled, due to instabilities and memory leaks the tweaks were no longer used by myself.

An example of this could be seen in Dutty's fix before his latest fix when users would experience an ~10mb drop in RAM when a memory card was inserted.

[Sleuth255]

Not reflected for me tho CUSTEL. Turbo mode jacks the filesystem cache on my ROM w/o reliability issues. It may be a contributor when combined with something else though...

also: inserting a sd card drops pgm memory in mine by 9mb too. I don't consider this a bug as the system is allocating file system memory for SDHC to increase performance (in fact, my ROM memory screenshots are with SD card inserted to show worse case). I consider the fix to be a potential problem actually...

Hermes is a whole different animal btw. My extensive Kaiser testing showed that the hermes file instability issue doesn't exist on this device.

[CUSTEL]

Not reflected for me tho CUSTEL. Turbo mode jacks the filesystem cache on my ROM w/o reliability issues. It may be a contributor when combined with something else though...

also: inserting a sd card drops pgm memory in mine by 9mb too. I don't consider this a bug as the system is allocating file system memory for SDHC. I consider the fix a problem actually...

Hermes is a whole different animal btw. My extensive Kaiser testing showed that the hermes file instability issue doesn't exist on this device.
If you remove your turbo cab and take a look at RAM when inserting the sd card the issue will more than likely be gone.

[Sleuth255]

agree completely. Without static filesystem caching enabled, its dynamic. With static, memory is pre-allocated and the fat/dir tables loaded into memory. My point is that the Kaiser has gobs of memory so our design paradigms should adjust accordingly.

[jakontil]

in the speed comparison, how'd u see the results? is the lowest in ms the fastest?

[Sleuth255]

For the most part, yes. Lower time=quicker operation. CPU benchmarks such as whetstones don't follow this rule of thumb though.

[lewcamino]

the biggest factor in speed and RAM on boot is going to be what apps open when the OS starts up. Now the Factory shipped HTC rom has the HTC_Home screen which uses lots of memory and system speed just to get going and it is always running in the background. So any rom that does not have the HTC_Home screen is of course going to be faster in boot time and after boot time. These tests almost prove nothing to me because each has different apps that run at start and in the back ground to do this test fair each rom would have to have the same number of files and the same apps at boot and at normal use. If you notice Custel does not have the HTC_Home in his rom which is why it will feel faster and respond better. Its just a matter of what fits you better at this point until a major release is done from HTC or Microsoft.

Austin

I agree with this to an extent but I can confidently say that XDA after I added the home plug-in still absolutely FLYs...NO exaggeration. I have not seen a ROM this quick since WM2003.

Later, Lew

[Sleuth255]

I have not seen a ROM this quick since WM2003....

Later, Lew

:eek::eek: Ahem.. Can I please smoke some of that?.... :)

Seriously though, we all traded the speed of WM2003 for the convenience of persistent storage. Same thing happened to PalmOS users. No other device has ever performed at the speed of my Tungsten T3.

[CUSTEL]

:eek: Ahem.. Can I please smoke some of that?.... :)

Seriously though, we all traded the speed of WM2003 for the convenience of persistent storage. Same thing happened to PalmOS users. No other device has ever performed at the speed of my Tungsten T3.
ewww.... palm os.... :eek: LOL

I have no complaints about the speed of my blackberry.... :D

[Sleuth255]

I came from there heheheheh but was lured to the dark side when WM5 finally became a real os. ;)

[CUSTEL]

I came from there heheheheh but was lured to the dark side when WM5 finally became a real os. ;)
I started there as well, had a palm 7 years ago. It was then realized how much it lacked and switched to an ipaq and there was no looking back since :cool: lol Debating picking up a kaiser as a pda for tasks my blackberry isn't so great at such as doc editing but I can't get away with using it as my primary device :(

[dan13l]

I know it's a little off topic, but I was thinking of getting myself a Blackberry - they're pretty good then? I can keep the Kaiser as a PDA, like you say, but then use the Blackberry as my phone.

[xmoo]

So wich one is better :P

[Sleuth255]

someone will need to post a crackberry spb benchmark to answer that ;)

[CUSTEL]

someone will need to post a crackberry spb benchmark to answer that ;)
haha crackberry would get owned :(

[panosha]

File system cache/glyphcache tweaks really help. Compare my "Turbo mode" results posted on page 3 or so with the results on page 1 for my ROM & you will see a noticable improvement (in fact, compare my turbo mode results with any of the results for an interesting surprise ;)). This improvement is reflected in usability as well.

Kaisers have ram-to-burn folks. Why not cook these in? Heck, I've got every app on the planet running & still have 50megs of program memory free
You are right, my own opinion was not right. Τhanks for all Sleuth. :)

[gf_gollum]

As promised, I have just tested 4 ROMs with Spb Benchmark.


Anyone out there with these tools and an UNMODIFIED Kaiser who can post a result.. we can then see if there is any difference at all with all this flashing about
Thx
Graham

[panosha]

http://img132.imageshack.us/img132/6619/64499876ap2.gif (http://imageshack.us)

HTC_WWE_1.56.405.5 us it came from his mum. :p