OpenGL ES 2.0 games possibility

[robert-qfh]

As far as I understand, the java.library.path property is used by the JVM to load native libraries, just like the JNI library we load with System.loadLibrary(jniLib). However, it does not map to the traditional UNIX environment variables, and therefore does not influence any native code, like the library loader, that's trying to find the dependencies of jniLib.

Modifying the UNIX environment of the process from Java code is not possible (stackoverflow.com/questions/318239/how-do-i-set-environment-variables-from-java). You can read it using System.getenv(), but the map you get is read-only. A workaround might be to use a 2nd native library that modifies the environment using putenv(). But that's way too hackish for my taste, and we don't even know if the library loader will care about LD_LIBRARY_PATH or some other variable - after all, Google has optimized/stripped a lot of functionality we know from a desktop Linux.

Every Unix process that wants to use functionality of a shared library has to load it, either by using dlopen() or by embedding the information about required shared objects into the binary. In the latter case, the library loader will take care of loading the library and patching the addresses of the functions into the code. The OS doesn't really load the library into RAM multiple times if more than one process is using it, nevertheless every process has to load the shared library explicitly.

 

[MarcusMaximus04@gmail.com]

As far as I understand, the java.library.path property is used by the JVM to load native libraries, just like the JNI library we load with System.loadLibrary(jniLib). However, it does not map to the traditional UNIX environment variables, and therefore does not influence any native code, like the library loader, that's trying to find the dependencies of jniLib.

Modifying the UNIX environment of the process from Java code is not possible (stackoverflow.com/questions/318239/how-do-i-set-environment-variables-from-java). You can read it using System.getenv(), but the map you get is read-only. A workaround might be to use a 2nd native library that modifies the environment using putenv(). But that's way too hackish for my taste, and we don't even know if the library loader will care about LD_LIBRARY_PATH or some other variable - after all, Google has optimized/stripped a lot of functionality we know from a desktop Linux.

Every Unix process that wants to use functionality of a shared library has to load it, either by using dlopen() or by embedding the information about required shared objects into the binary. In the latter case, the library loader will take care of loading the library and patching the addresses of the functions into the code. The OS doesn't really load the library into RAM multiple times if more than one process is using it, nevertheless every process has to load the shared library explicitly.

Ah, gotcha(with regards to the environment path). I agree that that does sound like a bit too much of a hack, especially since it'd be just a temporary fix until Google gets their act together.

Right. I think the problem we're having is that it is using the second method(embedding information about required shared objects into the binary) but it's trying to read from /system/lib and ignoring /system/lib/egl.

Argh, trying to change the LD_LIBRARY_PATH env variable so I can try this out but the damn thing is set up in /init.rc, which, as always, is read-only and can't be mounted read/write.

And setting the environment LD_LIBRARY_PATH doesn't appear to work either. From the terminal I did:

export LD_LIBRARY_PATH=/system/lib:/lib:/system/lib/egl
am start -a android.intent.action.MAIN -n com.example.SanAngeles/.DemoActivity

and it still said it couldn't find the library. So either running an activity like this restarts it with its own environment variables(possible, but I would think it would continue using the ones from the shell it was launched from), or the system doesn't bother looking at LD_LIBRARY_PATH at all.

 

[MarcusMaximus04@gmail.com]

By the way, anyone know a place like this for the Droid? I figure they're in a similar boat ES2.0-wise and they might have some talented devs who could help figure this out. Plus, we could try to see if they have a similar library that works.

 

[pr0cl1v1ty]

By the way, anyone know a place like this for the Droid? I figure they're in a similar boat ES2.0-wise and they might have some talented devs who could help figure this out. Plus, we could try to see if they have a similar library that works.

Finally something I can help with. ill go see if I can round up any devs

 

[robert-qfh]

FINALLY! I got a small OpenGL ES 2.0 sample program (based on webreference.com/programming/opengl_es/) running. The only output is a static red triangle, but it uses a vertex and a fragment shader

And the best thing: it's a locked, unrooted Nexus One. The important thing is to load the library at the right moment:

class DemoRenderer implements GLSurfaceView.Renderer {
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
System.loadLibrary("glesv2sample");
nativeInit();
}
...
(code based on the SanAngeles sample)

By some mechanism (probably the initialization of the GLSurfaceView) the ES 2.0 libraries are loaded automatically, BEFORE the JNI lib is loaded, without us doing anything special to request 2.0 support. When the JNI lib gets loaded, the linker does not have to lookup the path of the library again.

All you need in Android.mk is
LOCAL_LDLIBS += -L/path/to/local/copy/of/glesv2/lib -lGLESv2_adreno200

No rpath necessary (I think it's being ignored), and no LD_LIBRARY_PATH modifications. Before I found this simple solution I tried the helper-library approach where putenv() is used to set the variable. I can modify environment variables this way, and the bionic linker actually parses this variable (verified that with the bionic source code) but it still didn't work. Doesn't matter now.

I wonder if Droid users have an OpenGL ES 2.0 library as well, where it's stored, and if its name is GLESv2_adreno200.so as well...

 

[MarcusMaximus04@gmail.com]

FINALLY! I got a small OpenGL ES 2.0 sample program (based on webreference.com/programming/opengl_es/) running. The only output is a static red triangle, but it uses a vertex and a fragment shader

And the best thing: it's a locked, unrooted Nexus One. The important thing is to load the library at the right moment:

class DemoRenderer implements GLSurfaceView.Renderer {
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
System.loadLibrary("glesv2sample");
nativeInit();
}
...
(code based on the SanAngeles sample)

By some mechanism (probably the initialization of the GLSurfaceView) the ES 2.0 libraries are loaded automatically, BEFORE the JNI lib is loaded, without us doing anything special to request 2.0 support. When the JNI lib gets loaded, the linker does not have to lookup the path of the library again.

All you need in Android.mk is
LOCAL_LDLIBS += -L/path/to/local/copy/of/glesv2/lib -lGLESv2_adreno200

No rpath necessary (I think it's being ignored), and no LD_LIBRARY_PATH modifications. Before I found this simple solution I tried the helper-library approach where putenv() is used to set the variable. I can modify environment variables this way, and the bionic linker actually parses this variable (verified that with the bionic source code) but it still didn't work. Doesn't matter now.

I wonder if Droid users have an OpenGL ES 2.0 library as well, where it's stored, and if its name is GLESv2_adreno200.so as well...

Nice. I figured you might have better luck than me at doing that. Alright, so now we're pretty well good. I'm still working on porting that game, but I'll put up some free beta versions of it so everyone at xda can check it out. Also, everyone who posted so far on this thread is welcome to the final game if you want it. My thanks for everyone's help.

Oh and I found a droid owner. Turns out they have libGLESv2_POWERVR_SGX530_121.so. I'm wondering if theres a way to have it link with one or the other, depending which is available?

 

[robert-qfh]

That calls for using dlopen(), in a slightly different way than I sketched before:

Java code
-> System.loadLibrary('glesv2wrapper.so')

glesv2wrapper (NOT linked to any of the gles libs via -l)
-> dlopen('glesv2wrapper_nexus.so'), if that fails
-> dlopen('glesv2wrapper_droid.so')

glesv2wrapper_nexus.so links to libGLESv2_adreno200 via -l
glesv2wrapper_droid.so links to libGLESv2_POWERVR_SGX530_121 via -l

Both export a minimal set of functions, like initialize() and drawFrame().
The main wrapper (glesv2wrapper) uses dlsym() to get pointers to those functions and dispatches the calls from Java world to the device-specific library loaded before.

The downside is that both device specific libraries, even though they share the same source code, will basically be a copy of each other, the only difference being the GL library they're linked to.
Advantage is that you don't have to dlsym() every single gl....() function (as it would be the case if you dlopen()'ed the libGLESv2_* directly).

BTW, which game is it you're porting?

 

[MarcusMaximus04@gmail.com]

That calls for using dlopen(), in a slightly different way than I sketched before:

Java code
-> System.loadLibrary('glesv2wrapper.so')

glesv2wrapper (NOT linked to any of the gles libs via -l)
-> dlopen('glesv2wrapper_nexus.so'), if that fails
-> dlopen('glesv2wrapper_droid.so')

glesv2wrapper_nexus.so links to libGLESv2_adreno200 via -l
glesv2wrapper_droid.so links to libGLESv2_POWERVR_SGX530_121 via -l

Both export a minimal set of functions, like initialize() and drawFrame().
The main wrapper (glesv2wrapper) uses dlsym() to get pointers to those functions and dispatches the calls from Java world to the device-specific library loaded before.

The downside is that both device specific libraries, even though they share the same source code, will basically be a copy of each other, the only difference being the GL library they're linked to.
Advantage is that you don't have to dlsym() every single gl....() function (as it would be the case if you dlopen()'ed the libGLESv2_* directly).

BTW, which game is it you're porting?

Gah, ya was afraid of that. It might actually make sense to do it the second way, using dlsym on every gl function. That way one person would need to do it once and open source it and boom, everybody could just tie that in.

For the game, it's the same one I mentioned in the first post. Video of it here: http://www.youtube.com/watch?v=f7ovEbpOAn4

By the way, in case you're curious: I'm pretty well along in porting it. I have all the actual gameplay ported(except the controls, right now there's no way to control it) but don't have any of the display code moved over. Well, that's not completely true, I've ported over the main vertex shader that is run on pretty much everything except the smoke particles. But I still don't have any of the code implemented to actually use it. That main c drawing code is going to be a pain. C# isn't too much different from java so the main gameplay was easy enough. The Direct X shader language(forget what they call it) is also pretty similar to GLSL, so that hasn't been too bad either. But porting from C# using the XNA libraries to basic C using OpenGL ES 2.0? That's going to be intense. Luckily that drawing code isn't too involved and is mostly just setting up the uniforms for the shaders.

 

[xdarkfirex]

So in simple terms I can use ES 2.0 on the nexus and droid w/o rooting the phone to move over the libraries. If that is so, good work. This deserves some sort of news.

 

[MarcusMaximus04@gmail.com]

So in simple terms I can use ES 2.0 on the nexus and droid w/o rooting the phone to move over the libraries. If that is so good work. This deserves some sort of news.

Indeed. Now to make it work on both the Droid and the N1.

 

[MarcusMaximus04@gmail.com]

Of interest, there doesn't appear to be any of the helper functions you get with the PowerVR(or at least I can't find them), so it looks like we're gonna have to do a lot of stuff on our own(like setting up perspective matrices... ewww)

And looks like texture loading is gonna be a bit intense as well... yay.

 

[robert-qfh]

You're right, the dlopen() approach is probably the better one. The more I think about the more I like it, because it could be done as a drop-in replacement for the standard OpenGL header, plus one function that binds the function pointers to the addresses retrieved by dlsym().

Sort of like this:

Original declaration:
GL_APICALL void GL_APIENTRY glActiveTexture (GLenum texture);

Function pointer declaration:
void (*glActiveTexture) (GLenum texture) = NULL;

void init() {
h = dlopen(...);
glActiveTexture = dlsym(h, "glActiveTexture");
}

Calling the function pointer can be done using glActiveTexture(), so no changes to the code required.

What are those PowerVR helper functions? I also had trouble setting up the projection system. Main problem is probably the lack of good resources about GLES 2.0 development on the web. There are tons of tutorials dealing with GLES 1.1 (glRotate & friends) and Desktop-OpenGL (gl_ModelViewMatrix etc.). I guess you have to implement all that stuff yourself, but I find it somewhat tedious to create my own rotation matrices, projection matrix, etc. There most be an easier way, even if it's a library of shader code snippets to get started with.

 

[MarcusMaximus04@gmail.com]

You're right, the dlopen() approach is probably the better one. The more I think about the more I like it, because it could be done as a drop-in replacement for the standard OpenGL header, plus one function that binds the function pointers to the addresses retrieved by dlsym().

Sort of like this:

Original declaration:
GL_APICALL void GL_APIENTRY glActiveTexture (GLenum texture);

Function pointer declaration:
void (*glActiveTexture) (GLenum texture) = NULL;

void init() {
h = dlopen(...);
glActiveTexture = dlsym(h, "glActiveTexture");
}

Calling the function pointer can be done using glActiveTexture(), so no changes to the code required.

What are those PowerVR helper functions? I also had trouble setting up the projection system. Main problem is probably the lack of good resources about GLES 2.0 development on the web. There are tons of tutorials dealing with GLES 1.1 (glRotate & friends) and Desktop-OpenGL (gl_ModelViewMatrix etc.). I guess you have to implement all that stuff yourself, but I find it somewhat tedious to create my own rotation matrices, projection matrix, etc. There most be an easier way, even if it's a library of shader code snippets to get started with.

Makes sense. For the PowerVR helper functions, they're in the OpenGL ES 2.0 SDK for PowerVR SGX. There seems to be two sets: PVRTools and PVRShell. I haven't done much looking into them because A: they're names include PVR so it seems a fair guess that they're power vr only and B: they're in C++, which one of the docs that comes with the ndk says is incomplete in Android.

That said, last night I bit the bullet and sat down and worked on implementing the matrix functions(the main ones we're missing, aside from texture loading) and wrote up a .h file that should be able to handle most if not all of what we want to do. I'll attach it here. I had to put it in a zip file so xda would let me attach it.

-Oh and by the way, I haven't tested those functions at all except to ensure that they compile. So testing will be necessary(and, if you'd like, a quick spot-check code review would be great).

 

[robert-qfh]

NICE, thanks for sharing! That should help anyone getting started. Especially the gluLookAt() is nice to have.

I can shed some light on the C++ on Android topic. C++ is supported, but the STL (vector, list, string, etc.) is missing. That's not a problem, though, as there's a project called STLport. Build it using the NDK and link it whenever you'd like to use C++/STL:

STLPORT_BASE := /home/robert/devel/android/ndk-wrappers/stlport
LOCAL_C_INCLUDES += $(STLPORT_BASE)/stlport
LOCAL_CFLAGS += -D__NEW__ \
-D__SGI_STL_INTERNAL_PAIR_H \
-DOS_ANDROID
LOCAL_LDLIBS += -L$(STLPORT_BASE)/build/lib/obj/arm-linux-gcc/so -lstlport

This will allow you to compile most C++ code. AFAIR some features are missing, especially exceptions, but they're rarely used in C++ code anyway.

BTW, if you ever put your JNI functions inside a .cpp file, make sure to wrap a extern "C" {} block around it. Took me one or two hours to spot that problem.

 

[MarcusMaximus04@gmail.com]

So I've spent pretty much the whole day trying to play around with it and I seem to be having a problem somewhere, but I can't really figure out what it is. I even modified my project so I could test out some gluLookAt calls and return the resulting matrix to java to poke around. Everything in those returned matrices looks perfectly the way it should, but when I try to actually display using those same gluLookAt calls, everything ends up completely warped. The only thing I could think of is that I was screwing up on whether the matrices are column or row major. But when I change it to row major, I was unable to display anything.

Anyway, I've modified a lot of the Matrices.h code to make it easier to read and fixed some bugs I found so I'll attach that here.

EDIT: oh and the glPerspective call doesn't seem to work right now, so I'd suggest trying to just use glLookAt for now, if you're going to play around with it.

 

[NPS_CA]

GDC and Android

Any of you going to GDC; I used to have to go for work (previous company was a GPU company).

Interested in the C++ discussion they are having; wonder if they are wrapping more around NDK; Really want to push to see if we can see a true OGL ES2.0 environment in the SDK and future ROM's on the more capable GPU devices.

See: http://android-developers.blogspot.com/2010/01/android-at-2010-game-developers.html

Also; GDC conference (paid attendees) can get a Nexus One of Droid:
http://gdconf.com/androidphone.html

 

[robert-qfh]

Marcus, I think a spotted a problem in your glTranslate(). OpenGL uses column-major format, i.e. the base vectors are laid out contiguously, i.e.
0 4 8 12
1 5 9 13
2 6 10 14
3 7 11 15

The M() macro is fine, but
M(3,0) = x;
M(3,1) = y;
M(3,2) = z;
should be
M(0,3) = x;
M(1,3) = y;
M(2,3) = z;

Also in glRotateZ() shouldn't
mat[5] = -sin(rotation);
mat[6] = cos(rotation);
be
mat[4] = -sin(rotation);
mat[5] = cos(rotation);

I didn't check whether sin and cos are used in the right order with the correct sign, but a rotation around Z should leave Z untouched, therefore [2], [6], [10] are independent of the angle.

Keep in mind that OpenGL uses degrees, but C's sin() and cos() use radians.

 

[robert-qfh]

@NPS_CA, I won't be attending GDC, I'm still busy with my master's thesis and actually I'm more interested in using the GPU for doing general-purpose calculations than for game development Haven't done any GPGPU stuff so far, but now that we can use GLES2 it should be possible.

I guess Google is going straight for OpenGL ES 2.0, and support is probably only a few weeks away (next NDK?). I don't expect them to improve the incomplete/non-existing support for ES 1.1, just doesn't make too much sense. Those < 2.0 devices aren't designed for games anyway (GPU-wise). But it's very interesting to see that Google actually puts some decent effort into gaming on their platform...

Just for the records, while doing some research on rooting I came across the following lines in /system/build.prop:

# The OpenGL ES API level that is natively supported by this device.
# This is a 16.16 fixed point number
ro.opengles.version = 131072

For the Nexus One, this means 2.0. Not too interesting, we already knew that. But might come in handy for other/future devices

 

[MarcusMaximus04@gmail.com]

Marcus, I think a spotted a problem in your glTranslate(). OpenGL uses column-major format, i.e. the base vectors are laid out contiguously, i.e.
0 4 8 12
1 5 9 13
2 6 10 14
3 7 11 15

The M() macro is fine, but
M(3,0) = x;
M(3,1) = y;
M(3,2) = z;
should be
M(0,3) = x;
M(1,3) = y;
M(2,3) = z;

Also in glRotateZ() shouldn't
mat[5] = -sin(rotation);
mat[6] = cos(rotation);
be
mat[4] = -sin(rotation);
mat[5] = cos(rotation);

I didn't check whether sin and cos are used in the right order with the correct sign, but a rotation around Z should leave Z untouched, therefore [2], [6], [10] are independent of the angle.

Keep in mind that OpenGL uses degrees, but C's sin() and cos() use radians.

For the translate: Ya, tried that. It ends up displaying nothing and no matter what values I try passing in(I checked to make sure I wasn't culling the vertices both by trying translating them in both direction along the z axis and by reversing the order of them to make sure it wasn't backface culling. Neither worked), it just wouldn't do anything. I agree with you that that's what they should be, I just can't make it display anything with those values, and I'm not sure why.

For rotation: I think you're right. I haven't tried rotating much since I'm trying to get translation working first since that *should* be a simpler operation. Unfortunately, it's just not turning out all that well for that.

Oh and I won't be attenting GDC either. Don't have the money or the time.

EDIT: OMGWTF$(!%((^(^(#^$#)^(#%&$%&$(&!!!!! I have no freaking clue why it worked, but this did. I'm going to kill someone.

M(0,3) = x;
M(1,3) = y;
M(3,2) = z;

Yes, that's right, the perfect way you're supposed to do it except with a screwed up z value. Anyway, I'll attach the updated Matrices.h here.

EDIT: may have spoken too soon. I'm not sure why but now I can't get z translation working(which was always working perfectly before). Now it just can't translate along the z axis if I leave it like that, and if I use the matrix position where the z value should go (2,3) then the whole thing breaks and I can't see anything.

 

[NPS_CA]

I guess Google is going straight for OpenGL ES 2.0, and support is probably only a few weeks away (next NDK?). I don't expect them to improve the incomplete/non-existing support for ES 1.1, just doesn't make too much sense. Those < 2.0 devices aren't designed for games anyway (GPU-wise). But it's very interesting to see that Google actually puts some decent effort into gaming on their platform...

In my previous life I worked on mobile GPU architecture. I'm three years removed from that now. Bottom line is that the 3D Core Qualcomm licensed has OGL ES 2.0 support in the snapdragon product line. So Agree; 1.1 is more casual 3D - NOT gaming centric (e.g. UI, etc). I see if Google is serious they will make a push to show OGL ES 2.0 at GDC in some development form. I like the fact that Droid and Nexus One are two of the giveaway handsets; gives some encouragement that apps will likely be compliant with the embedded GPU's on both for any dev plan Google has.

Google really needs to get off their duff in the Android team and ensure they put together a good dev rel program - bring on board some key ATI/NVIDIA dev role folks who can help bring some key Tier 1 mobile developers on board.... Man even the pathetic Palm guys are showing some good content now on Palm Pre/Pixi.

GPGPU/Mobile CUDA style programability for the GPU is VERY interesting for some non graphics apps; (e.g. image processing, etc)