Originally Posted by okantomi

these are my antutu scores

AndroBench?

Originally Posted by okantomi

HoushaSen, As you can see, I'm using the unofficial CM10 9/25 build on my Infinity...been using it for several days, no recent reboot, background processes settled in and running...and these are my antutu scores (performance mode on performance governor, so flat-out 1700 all the way):

Total score: 13463
CPU: 9879
GPU: 999
RAM: 2681
I/O: 804

This rom does not have fsync disabled, so no crazy-fast SQlite performance.

I'm not sure how this stacks up against other Infinities (stock ICS/custom ICS/JB).

Despite I have been placing quite few comment recently regarding to this matter, I am not big fan of benchmark, but having said that since that is what this thread is about and if nothing else it's fun to talk about, I posted above comment based on the post from Prime forum.

I think your IO score is pretty good in Antutu as far as I can tell. My Infinity as I said is only 500 and I basically have pure stock with root just to install browser2ram, but nothing else other than bloatware turned off. Not even CPU governor. Based on my google search, even Galaxy Note 10.1 had only 700's IO score. As for Prime it was as low as 250 at one point by Andropolice site.

So if Antutu is more reflective of real life (which I am not saying it is) but if that's the case, then yeah essentially you should not be feeling any IO issue. How is your actual experience? Do you remember some actual user experience that you felt like it was surely due to IO? If so, do you feel like has that got improved?

Barry

Would you mind sharing your Antutu score on Note with IO? Because the Youtube clip I linked may be old one or perhaps had some special thing running in background. I just like to see what you get on your stock rom with latest firmware.

Thanks

Originally Posted by HoushaSen

So if Antutu is more reflective of real life

AnTuTu, Vellamo, and Quadrant are overall performance benchmarks that run four simple IO tests - 1) disk reads, 2) disk writes, 3) SQL reads, and 4) SQL writes. 50% of their IO scores are based on SQL which, so far, is the only score that's been improved with s/w on the TF700. AndroBench is specifically an IO benchmark and runs eleven tests of which only three are SQL. No matter how good the IO scores people have posted in AnTuTu, Vellamo, and Quadrant their non-SQL scores in AndroBench have been the same or worse. So because of its depth, AndroBench is a truer measurement of IO performance. If everyone is going to go by "feel" which is totally subjective and based on individual use there's really no reason to have these detailed discussions. Likewise, if everyone just wants to feel good, pick the benchmark that makes you the happiest and only quote that one. But remember they're all testing the same thing so if the results are off it isn't that the h/w is performing any differently, it's the depth and thoroughness of the test.

Anand's (an engineer) comments pretty much sums up how easy it is for even experts to be unaware of Asus' IO issues in cursory testing.

For the first time, in our Nexus 7 review, I started seriously looking at integrated storage performance of tablets and smartphones. I've casually done this in the past, but users complaining of poor system responsiveness with background writes on ASUS' Transformer Prime/Pad series demanded something a little more thorough.

He added this which is truly the crux of the problem and why some people aren't bothered by it. And whether the simple tests in AnTuTu, Velammo, and Quadrant highlight or not it's still there.

Sequential read/write speed isn't bad, but it's the random write speed that's really a problem. We're talking about write speeds of a couple hundred KB/s. Remember what I said earlier about how multitasking can take an otherwise sequential IO stream and make it look fairly random? I suspect the low random write performance is one reason we're seeing significant slowdowns with background IO. Not all NAND controllers do well with concurrent reads and writes, which could be another contributing factor to poor performance.

Note that for light usage this isn't a problem. Similar to the first generation of affordable MLC SSDs for PCs, as long as you're doing a lot of reading you'll be ok. It's really for the heavier usage models that this is a problem. On a tablet however, simple background installation or downloading of files counts as heavy these days. The real solution to this problem is to integrate better NAND flash controllers on-board, or even onto the SoC itself.

Would you mind sharing your Antutu score on Note with IO? Because the Youtube clip I linked may be old one or perhaps had some special thing running in background. I just like to see what you get on your stock rom with latest firmware.

One X on the left, Note on the right. You'll also notice the test gave the One X and Note the same points (150) for SD card write even though they were 5MB's apart and only a two point difference when they were off 10GB in reads. And SQL is the 430/550 number shown under "Database IO."

Thanks Barry_H for quick check.


This is extremely interesting test result for me. As basically, on the Antutu both of your devices are essentially same as the my infinity or Custom Rom installed infinity of the okantomi's is better than either of your devices.

Just as you said, this is secondary to the fact the IO test Antutu performing is very simplified one. However, as you said before if benchmark test is something to trust of then, Antutu is most commonly used benchmark so why would we not assume this is indeed most reflective of overall benchmark in real life?

In the end, I think all benchmarks are flawed in one way or other simply because if one benchmark is purely testing hardware, then that benchmark score should not be able to be changed no matter what you do on software. So this simply speaks, there is software component on the benchmark that is assessed. This is though exactly expected as no hardware can run without software. So essentially comes back to conclusion, software can change overall performance.

In any event, I hope the improvement in IO score reflects real life experience. I know one test I can do for myself, which is to try to run Horn while downloading 1GB+ file in the background. I wonder if you can try that on Galaxy Note 10.1?

Originally Posted by HoushaSen

Antutu is most commonly used benchmark so why would we not assume this is indeed most reflective of overall benchmark in real life?

Because the problems like Anand described are what caused people to investigate and why we're having conversations like this one. Issues people were experiencing is what started everyone running IO benchmarks, not vice versa. Out of a total score of 12,411 on the Note, only 895 (7%) "points" are the IO numbers. Their weighting of both the importance and differences of IO is obviously very low. If a home pregnancy test gives a false negative are you really not pregnant? Also, consider this. The TF700 is based on a design that's over a year old. Other than a better Teg3 chip, all the other components are the same as the Prime. All the components of the Note have advanced compared to both the OG G-Tab and G-Tab 2. To expect that not to affect a performance comparison isn't practical.

So essentially comes back to conclusion, software can change overall performance. In any event, I hope the improvement in IO score reflects real life experience.

Let's put it this way, no matter what any of the benchmarks say, if the background IO problems go away the problem's solved. If they don't, they aren't. To Anand's point, light users probably never experienced them in the first place. I have two Exchange e-mail accounts and GMail set up to download complete message copy as well as attachments via push. In Anand's example, every time I received an e-mail I'd experience stalls or hesitation in whatever else I was doing. Same thing with social media syncs. The less people do in the background the less of an issue this is. Just because you're not personally experiencing them based on the way you use your device doesn't mean they don't exist and that others won't experience them.

I wonder if you can try that on Galaxy Note 10.1?

I'd love to but with different networks and other variables the results wouldn't mean much. It would be great if someone with a Note and TF700 could test them side-by-side.

P.S. - My One X was below 15% battery life when I ran AnTuTu so the CPU was throttled. Here's the accurate score. I'm surprised no one said anything. That was a pretty piss poor result.

Originally Posted by BarryH_GEG

Because the problems like Anand described are what caused people to investigate and why we're having conversations like this one. Issues people were experiencing is what started everyone running IO benchmarks, not vice versa. Out of a total score of 12,411 on the Note, only 895 (7%) "points" are the IO numbers. Their weighting of both the importance and differences of IO is obviously very low. If a home pregnancy test gives a false negative are you really not pregnant? Also, consider this. The TF700 is based on a design that's over a year old. Other than a better Teg3 chip, all the other components are the same as the Prime. All the components of the Note have advanced compared to both the OG G-Tab and G-Tab 2. To expect that not to affect a performance comparison isn't practical.

I do see your point. I believe our difference in opinion is most likely that you are believing IO as absolute bottleneck. Whereas, I believe IO may had been a bottleneck and may even still be but it can be changed to the degree that bottleneck is shifted, which in hardware it is done all the time. Throughput is determined by the bottleneck if everything is in sequential linear operation.

For instance, nand read -> Memory -> Cache -> CPU/GPU process -> Cache -> Memory -> nand write. Before next cycle of this starts.

In above scenario, yes. no matter what you do bad nand write would not change the work flow. As every other piece be waiting for nand write to finish.

In fact, this might had been a case in a big picture when TF700 was originally released, or for Prime.

However, as you are aware hardware operation is not this simple. Just as an pure example, what if you have memory hold data twice before actually writing to nand? If nand write was assumed to had been bottleneck and in fact more than twice slower than completion of the other piece, you can actually boost the throughput by factor of 2.

The worst scenario had actually been RAM issue or Cache because those are so close to CPU i.e. right before the actual data processing, you cannot do anything. But fortunately RAM performance if Antutu score to be believed is superior to the many devices so they can potentially utilize this.

Using your medical example. Number means only so much. A person may be pregnant with beta hCG level of 2000, or 10,000. The number goes down, is it concerning? Yes. depending on which stage of pregnancy. Again, it really depends on the context. The number may be valuable tool at specific time and situation; however, beyond which point, it may not mean as much.

Another more related example: How project butter works: http://www.androidpolice.com/2012/07...what-it-added/

If you had thought graphic processing is essentially done by CPU/GPU then there is technically no way that you can improve the smoothness of the graphic rendering because it is bounded by CPU/GPU. So how does project butter make it possible to achieve 60FPS? Using parallel processing (occasional triple buffer), and background processing (Vsync).

It is true that this would take your potential CPU/GPU power off of other application when it is in need. So if we had been playing a game that was putting CPU/GPU to their maximum full throttle all the time, those may no longer work. However, probably there are not that many games like that so overall user experience on day to day use is nothing but improvement.

There are people having issue with Jellybean update on Prime forum but I also see positive comment particularly addressing ANR and decreased/improved/resolved system lag/stutter while downloading/installing application. If these comments are to be believed then IO issue for the practical point of view is indeed fixed/solved/improved independent of benchmark score. Obviously other way is the same. Even benchmark score has improved, if these issues continue to show up then IO issue still exist OR afterall the issue may not had been due to IO but rather something else.



Above is my infinity Antutu score in normal/performance mode. No custom rom, build prop tweak. Just have bloatware off, and browser2ram installed. SD write speed is particularly low, which is why I believe in theory make sense to turn of fsync off or call less.

By the way, I truly enjoy this discussion with you Barry_H because it makes me think, and I know we have no right answer here but you are always professional and respective.

Can anyone post AndroBench runs on the One X, Note and/or TF700 pre- and post-JB? I read this is more reliable but there are few AndroBench runs.

Cheers

Originally Posted by HoushaSen

Throughput is determined by the bottleneck if everything is in sequential linear operation.

You're way overthinking this my friend. Simply, regardless of brand, OS version, custom UI, NAND capacity, processor speed, et. al., only Asus TF-series tablets perform this way. Using logic to explain it away would be more appropriate if the problem was wider than Asus products. But it doesn't appear to be from the devices I follow. It certainly isn't with Samsung and HTC devices and they're the two biggest Android device manufacturers. A more apt example of what's going on is "a chain is only as good as its weakest link." For example, the ****ty AnTuTu score I posted for my One X when it was throttled to 1GHz went up dramatically when retested at 1.5GHz whereas the IO portion went up two points. So clearly CPU/GPU has little or no impact on IO, at least as far as AnTuTu’s concerned. At the end of the day nothing anyone's done with s/w (so far) has caused the wrties, especially random which are exceptionally low, to budge. And that's the weakest link.

Originally Posted by JamesRC

Can anyone post AndroBench runs on the One X, Note and/or TF700 pre- and post-JB? I read this is more reliable but there are few AndroBench runs.

I've done some comparisons to the Note but HoushaSen asked me to compare Asus' tablets against each other. Once the TF700 gets JB I’ll do a before and after on all three of them and include Zeus on 4.0.4 also. With JB coming so soon it doesn't make sense to do it now and then do it again later.

Originally Posted by BarryH_GEG

You're way overthinking this my friend. Simply, regardless of brand, OS version, custom UI, NAND capacity, processor speed, et. al., only Asus TF-series tablets perform this way. It certainly isn't with Samsung and HTC devices and they're the two biggest Android device manufacturers. A more apt example of what's going on is "a chain is only as good as its weakest link." For example, the ****ty AnTuTu score I posted for my One X when it was throttled to 1GHz went up dramatically when retested at 1.5GHz whereas the IO portion went up two points. So clearly CPU/GPU has little or no impact on IO, at least as far as AnTuTu’s concerned. At the end of the day nothing anyone's done with s/w (so far) has caused the wrties, especially random which are exceptionally low, to budge. And that's the weakest link.

Perhaps I may be overthinking but there seem to me that what we previously related as IO issue: 1. ANR. 2. System stutter/lag with installing/downloading applications from Google Play, and 3. System stutter/lag with WIFI download have markedly improved since the initial release of Infinity. Again I had infinity initially from day #1 of release, so I know how it performed originally though did not do any benchmarking back then. Also your Anandtech video clearly showed the issue existed in July with original firmware (since then we had 3 firmware updates).

If these issues still exist, I can at least say it has improved because even my type of use I had #1 and #2 previous, which are now essentially gone. At this point, I would truly like to hear from people who are still claiming there are IO issues, what is it that they are experiencing. Not that I am doubting they have issues, but that may not be related to IO.

As I recall you have had Infinity at one point, didn't you? If so, perhaps it may be worth a try to see (if anybody around you have one) that you can replicate your original experience with Infinity. Because I am one of those who had issue and got concerned about infinity and thus went to Note 10.1 for a while. Even before coming back I was hesitant/skeptical because of ANR (mainly) and browser speed so I was going to tell myself infinity just won't cut it despite its great screen and end my trail. But to my surprise (for good), the experience was significantly different.

Using logic to explain it away would be more appropriate if the problem was wider than Asus products. But it doesn't appear to be from the devices I follow

I actually believe it would be opposite if we rely on benchmark. If its purely due to IO, then we should see this being more prevalent across the devices rather than ASUS line specific.

Perhaps you may find this interesting,


This table is nothing but extracted data from Antutu official website. I just picked few machines including Acer, Nexus 7, Note 10.1, Prime and Infinity. As Antutu does not provide total IO score, I simply created the column and summed last three and sorted based on the total IO score.

What is interesting here is TF700 is not too far off from Note 10.1 in total score but more interestingly, Acer Iconia series even the latest version is inferior to the Prime and Infinity. Nexus is far down.

I am by no means expert on Iconia as I didn't really follow the device but if what you have said is true and issue is primarily on ASUS product line then this table really points to couple potential explanations.

1. Issue is NOT IO, but other hardware OR
2. It is software or kernel optimization issue that ASUS has.

I've done some comparisons to the Note but HoushaSen asked me to compare Asus' tablets against each other. Once the TF700 gets JB I’ll do a before and after on all three of them and include Zeus on 4.0.4 also. With JB coming so soon it doesn't make sense to do it now and then do it again later.

Thank you for remembering this. I think it would be a great summary table for benchmark-wise. It would definitely be worth making a dedicated thread.

Performance mode Total 4032, I/O is 1168