[APP] RealBench

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
Okay, I got sick of using Quadrant for benchmarking filesystem improvements, so I spent the last hour or so on this: RealBench!

It's pretty crappy right now, but I plan to add all kinds of different (hopefully) real-world-ish benchmarks.

For now it has just one benchmark -- a benchmark that heavily stresses the filesystem for both reading and writing, as well as heavily stressing ram and the garbage collector. Basically, this is a benchmark designed to do as poorly as possible with the SGS's stock problems. Increases in this benchmark really should correlate with a better, more responsive everyday phone. (Not perfect at this stage though, it still needs work!)

Anyway, it will list the current fps at the top of the screen while it's running. Higher is better! It's also easy to visually see the difference, as it scrolls a bit smoother. If you have any other devices, test this out on them and report back with the scores!

V1.1 Changes
Now writes 1000 files (~30mb of data), and picks randomly from them. This should make it impossible for any caching to occur.
V1.2 Changes
Better cleanup, better initialization.
V1.4 Changes
File writing is now on a separate thread from reading, to give a better estimate of real world usage.
File writing/reading is no longer a 50-50 split, it is now closer to a 80-20 split, for better real world usage.
FC bug fixes.
Black screen on start fixed.
V1.4.1 Changes
Fix for app close.

Notes
Any task which will take focus away from this app will cause it to close. Phone calls, etc.

Please give the version number with any score! :)
 

Attachments

Last edited:

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
The difference between stock and oclf looks way too big, I hope I haven't made some mistake somewhere!

This benchmark copies a bitmap out of its apk into ram, and then saves it to disk with a random file name. It then reads that bitmap back out of the new file, decodes it into a drawable, and puts it on screen.

This should stress read, writing, and memory usage.

OCLF should not be getting a boost that huge though, I would have thought. Something may be wrong.

Anyway, test this out on other devices such as the N1 and report back, please! :)
 

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
XWJM7 + voodoo beta 1

Last Minute FPS: ~12, Longest period without frames: ~0.93 sec

Almost stock values. :confused:

I noticed that in a brief period the current FPS jumps tp 55 and then drops to 11.
OCLF values might be heavily inflated, I'm going to try write a huge whack of files to the disk and pick a random one to read back. That will get rid of any possible caching that might be inflating OCLF scores.

Voodoo being the same as stock is a bit worrying though... hummm...

EDIT: Tried voodoo, and left it running for awhile, and it was getting 13+ which is a pretty good improvement from 10 I guess.


Anyway, new version going up which will get around any caching problems!
 
Last edited:

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
V1.1 is up now! Here is the core of the benchmark for anyone who wants to know what it's doing. ;)

Code:
			int rint = r.nextInt() % 3;
			int bmpres;
			if (rint == 0) {
				bmpres = R.raw.blackbar;
			} else if (rint == 1) {
				bmpres = R.raw.greenbar;
			} else {
				bmpres = R.raw.redbar;
			}

			try {
				InputStream is = getResources().openRawResource(bmpres);
				String Filename = UUID.randomUUID().toString();
				FileOutputStream fos = openFileOutput(Filename, 0);
				byte[] bytebuf = new byte[1024];
				int read;
				while ((read = is.read(bytebuf)) >= 0) {
					fos.write(bytebuf, 0, read);
				}
				is.close();
				fos.getChannel().force(true);
				fos.flush();
				fos.close();
				
				FileNames.add(Filename);

				int pickrandom = r.nextInt(FileNames.size());
				Filename = FileNames.get(pickrandom);
				
				is = openFileInput(Filename);
				ByteArrayOutputStream baos = new ByteArrayOutputStream();
				while ((read = is.read(bytebuf)) >= 0) {
					baos.write(bytebuf, 0, read);
				}
				byte[] rawimage = baos.toByteArray();
				BitmapDrawable bmpdraw = new BitmapDrawable(BitmapFactory.decodeByteArray(rawimage, 0, rawimage.length));
				is.close();
				convertView.setBackgroundDrawable(bmpdraw);

				if (FileNames.size()>1000) {
					pickrandom = r.nextInt(FileNames.size());
					Filename = FileNames.get(pickrandom);
					File file = new File(getFilesDir(), Filename);
					file.delete();
					FileNames.remove(pickrandom);
				}
			} catch (Exception ex) {
				ex.printStackTrace();
			}
 

ed10000

Senior Member
Jun 9, 2010
829
179
0
So I tried v1.1 on my stock JPC w/o any lagfix.
Last Minute FPS - 12.64
Longest Period without frames: 1.19s

During the initial 15 seconds, it was constantly 15+ FPS. Max I saw was 22.

EDIT: That's weird though. My previous setup - JM5 + CFLagFix (which I believe is almost the same as OCLF) was a lot smoother in certain places. I'd have expected the difference in scores to be bigger.
 
Last edited:

lyno

Senior Member
Feb 9, 2005
96
10
0
JPC OCLF apk

Last minute - 54.31 fps
Longest - 2.11

HTC Desire official froyo, unrooted

Doesn't run. Just shows a black screen then forces close after a minute or so


Sent from my GT-I9000 using XDA App
 

ed10000

Senior Member
Jun 9, 2010
829
179
0
v1.2 on my JPC stays on the black screen at start up for about a minute, runs at 4-5 FPS for a few seconds and then FC's.
 

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
v1.4 is up on the first post, with big changes. I'm still a bit confused why EXT2 performs so much better at writing than EXT4. If anybody has any ideas... :)
 

jjwa

Senior Member
Jun 24, 2010
220
10
0
35
Enschede
Version 1.2. XWJM5 Hackeclair, Project Voodoo beta1. With 54 installed apps.

The results do not look great. The first tests were done without restarting, it's been running quite a while, and I used and installed quite a bunch of stuff since last time I restarted the phone. So a nice worst case scenarie.
It takes 30~60 seconds of black screen before it starts, LMFPS 11.72, LPWF 3.53 sec.
Second time it closed after a 30~60 sec black screen, third time it FCs after 30~60 sec black screen. Fourth attempt it simply closed itself after a duration of black screen. Oh, and a FC error comes like 20 seconds after it actually closed...
I guess that at least this means it stresses the phone really well! ;)
(The first run it was plugged into my PC, with the SD mounted, btw. In case we find out that actually matters.)

Time to try it from a fresh boot.
Meh. Last minute 10.22. Longest without frames 4.62 seconds.

Edit:
Version 1.4.
Starts pretty much instantly. FC after running for about 20 seconds. I did have an orientation change or two during the this, no idea of that's related.
Second run, no black screen, initating takes 5 seconds or so.
Longest without frames started out really well, 0.45 sec. And then it froze for 3.14 sec. FPS is all over the place. Between 1 (displayed after a freeze) and 55. Last minute FPS 34.60. Last minute FPS is 23.90 a few minutes later... Total average FPS went from 32 to 27. No crashes, after running for at least 5 minutes.
Oh, and then it gave me an FC when I used the hardware Back button to close it.

Third run with version 1.4.
Longest withuot frames was 0.82, until I had another 3 second freeze. Several freezes. Last minute FPS 22.65. The whole thing is not running smooth at all. Sometimes it runs fine for a few seconds, and that 'normal' FPS seems to be around 50~60, but then it will start become laggy again (many smaller and larger freezes) resulting in 20~26 FPS averages. And FC at close again.
 
Last edited:

ed10000

Senior Member
Jun 9, 2010
829
179
0
v1.4 on stock JPC
Last minute FPS: 26.98
Longest without frames: 1.4s

Crashes after a minute and a half.

EDIT:
Tried again. 30.79 LM FPS. Longest time without frames - 0.7s. Crashed after a minute and a half again though.
 
Last edited:

CazeW

Senior Member
Nov 14, 2008
855
448
83
v1.2 on JM5 + root + voodoo
Last minute: 11-14fps
Longest period: ~0.9s but did jump to over 3s at one point when fps dropped to 1.

From watching the benchmark, as it being a repetative one, I'd expect it give roughly the same fps all the time but it keeps jumping between 7 and +50 fps.
 

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
v1.4 on stock JPC
Last minute FPS: 26.98
Longest without frames: 1.4s

Crashes after a minute and a half.

EDIT:
Tried again. 30.79 LM FPS. Longest time without frames - 0.7s. Crashed after a minute and a half again though.
Crashes are pretty strange, since mine running on JP8 does not crash. Left it running a few minutes. It could be highlighting a bug in JPC for all I know, though. ;)

Anyway I'm going to leave 1.4 up without changes for now, and hopefully people with some time can run it on different phones and post their results. The info from these benchmarks should hopefully prove helpful in the future. Specially when compared with different devices. :D
 

RyanZA

Senior Member
Jan 21, 2006
2,023
777
0
JHB
v1.2 on JM5 + root + voodoo
Last minute: 11-14fps
Longest period: ~0.9s but did jump to over 3s at one point when fps dropped to 1.

From watching the benchmark, as it being a repetative one, I'd expect it give roughly the same fps all the time but it keeps jumping between 7 and +50 fps.
Looks can be deceiving. ;) Each image you see flying past is chosen randomly from a list of 1000 images. This means that buffering can't be playing a role in the results.

As far as the fps jumping, this is from
a) Garbage collection being very heavily stressed out by this. :D
b) filesystem caches/other