TUTORIAL: Control issues of Java MIDlets – all secrets of button handling

Lately, with the advent of Opera Mini and the really excellent and highly recommended Gmail MIDlet and some quality games (see their list in the 10/19/2007 update of my 3D MIDlet article), there has been a tremendous increase in the demand for MIDlet-related information. This is certainly shown by the sheer number of MIDlet-related questions asked at XDA-Developers, probably the best, most lively Windows Mobile hardcore user community with the most posts. For example, today, I’ve answered at least 20 different MIDlet-related questions there. Quite a few, isn’t it?

Let me give you all another modest present: in addition to my already-published previews (for example, the 3D Gaming Bible, the 4pda.ru Download Bible etc.), another excerpt – a full chapter – from the forthcoming Bible. Yes, it’s coming and yes, I do try to get it ready tomorrow or the day after – everything, all accompanying screenshots and charts (the main chart; 3D games Compatibility Chart and JBenchmark Chart) are ready, I only need to consolidate all my thoughts into an all-in-one, still-somewhat-comprehensive Bible, which will, I promise, be MUCH better, will contain MUCH more information and MUCH more up-to-date than that of the Russian-only 4pda.ru. Now, look at the length of the 4Pda tutorial (and all the linked-in ones) to see how much information it contains Not very easy to come up with something that has even more info, is it?

Note that the MIDlet managers I refer to in the article can all be found in the main chart of the Bible.

1. Control issues

If you stick to either IBM J9 or TAO Intent (two well-known, established MIDlet managers with versions running even under WM2003(SE)), you’ll inevitably encounter major control problems with several titles. Either the WM5 softkeys or the D-pad won’t work, or none of them. If you, for some reason, stick to these MIDlet managers (also referred to as Kilobyte Virtual Machines, KVM’s), this chapter will be of extreme importance to you. Note that now that all WM5+ devices, Smartphones (Windows Mobile Standard), plain Pocket PC’s (Windows Mobile Classic) and Pocket PC Phone Edition (Windows Mobile Pro) devices alike support the, in general, vastly superior Esmertec and Jblend KVM’s, you should only do this if you really don’t have any chance of running Esmertec’s KVM’s or Jblend – that is, for example, if you have a WM2003 or WM2003SE Pocket PC or Smartphone.

However, even with the, in this regard, flawless Esmertec KVM’s and Jblend, you may still encounter problems with titles (mostly games) not supporting the D-Pad or situations (most importantly, Opera Mini and its modded version) where a MIDlet heavily depends on the hardware buttons. This means the chapter will be useful for non-TAO / J9 users as well.

There are three related (two of them being pretty huge and thorough) articles / tutorials at 4pda.ru explaining these problems and the way you can fix them. They all are linked from the main, first article in the main MIDlet manager thread. In general, they recommend the usage of a Chinese tool JavaMagic, which is supposed to help J9 and/or TAO users run their titles. In addition, the also recommend some always-on-screen numeric keypads to help numeric input and, finally, two other tools as well for MIDlet conversions. We’ll soon see whether JavaMagic is able to help at all and what you should do instead of using on-screen virtual keyboards. In addition, I – a veteran Java programmer and instructor – also explain what the control problems are caused by and why TAO and IBM J9 behave so strangely.

1.1 How it does work behind the curtains & fixing the problems with external hardware button assignment

First, let’s take a look at what problems you will face. This section will contain some references to Java; that is, the language the MIDlets are written in. Note that you can safely skip this section if you don’t know the language. If you do know it, or, at least, have some knowledge of how programming languages work, you might find this section pretty interesting.

A MIDlet, in general, implements the callback method ("function" in the traditional, non-OOP parlance) keyPressed(), which is called back when a button is pressed. This method is passed an integer (numeric) parameter denoting the actual ASCII character (or, with non-character input, scan, denoted by negative numbers) code of the button that has been pressed. If you’ve done any actual programming in any language (or are just computer-savvy), you will certainly recognize the ASCII character code values 48, 49, 50 etc. it typically has: yes, they correspond to the 0, 1 and 2 buttons, respectively (and the list continues).

Most MIDlets don’t directly act on this direct code, but give a call to the getGameAction() (a method in the superclass javax.microedition.lcdui.Canvas) to make the MIDlet manager convert this code to a symbolic constant. This is, in most cases, a much safer way to decide what has happened (what button was pressed). The sole reason for this is the following: D-pad-less phones (and many users) use dialpads (2 for up, 8 for down, 4 for left and 6 for right) instead of D-pads. This, unfortunately, results in different ASCII character / button scancodes returned. Using getGameAction() guarantees that these different codes (for example, except for TAO, the scancode -2 (with TAO, -57378) for the down arrow of the D-pad and the ASCII code 56 for the numeric button on the dialpad) are reported as DOWN events, independent of their original (numeric) value.

There are, unfortunately, some MIDlets that, at least in the menus, don’t adhere to this convention – that is, they don’t give an additional call to getGameAction() but try to directly process the parameter passed to keyPressed(). One example is Andreotti Racing (see "!!!3D_Andretti_ Racing_240x320.jar" in the main game compatibility chart), where the menus MUST be controlled by the dialpad (or numeric hardware keys), NOT the D-pad.

Also note that, regarding the special case of TAO, some games (for example, "!!3D_Micro_Counter_Strike_Beta.jar") might be hard-wired to both, most common D-pad and dialpad codes (again, -2 and 56 for downwards), instead of calling getGameAction(). These games will work just great with most MIDlet managers utilizing; however, TAO, which uses special D-pad scancodes (for example, -57378 for the "down" arrow, as opposed to the standard -2), D-pad-based control won’t work, only that of the (even virtual) dialpad. This is why this particular game can’t be controlled by the D-pad, only the hardware dialpad (or any software button input solution) under TAO, while, under all the other M3G-capable MIDlet managers (Nokia N95, Jblend, Jeodek M3G), you can use the D-pad for steering the car.

Fortunately, there is a very easy solution for all these problems. If you encounter a MIDlet that can only be controlled by (virtual) numeric keys (because, again, the game doesn’t use the additional getGameAction() call to be compatible with as many different KVM’s as possible), you might still add D-pad controllability by just using a button enhancer tool capable of redefining the four D-pad directions (currently, they’re PQzII, AEBPlus and the non-WM5-compliant buttonMax) and either the (slower) MortScript or the faster VJKeyPress to generate the actual keypresses – or, if you go with PQzII, the built-in keypress simulation feature.

This also applies to the great Web browser MIDlets Opera Mini and Opera Mini Mod, which add a lot of (with the Mod version, freely redefinable) shortcut functionality to (numeric) phone buttons. As they are inaccessible on Pocket PC’s without any kind of keyboard and pretty hard to access on phones with a slide-out keyboard (you always need to slide it open to be able to access the numeric row), in these cases, you WILL want to use external tools to simulate the dialpad button press using the hardware application buttons.

You can find extensive information on all this (assigning the simulation of dialpad keypresses to hardware application buttons or D-pad arrows) in the Button Enhancer Bible. Please do make sure you read it very thoroughly. Here, therefore, I don’t spend more time on this question.

1.1.1 On-screen dialpads – are they of any use?

As has already been mentioned, one of the 4pda tutorials (original (volta_john, 26.11.06 10:19:44); Babelfish; Google Translate) recommends installing on-screen dialpads for inputting numerals. I don’t think this is a good idea. First, you’ll need to run significantly smaller MIDlets than the native resolution of your screen so that the active MIDlet area and the always-displayed keyboard don’t overlap. This means you’ll need to stick to (at most) 176*220 MIDlets on your QVGA and QVGA or 352*416 (the hi-res version of Nokia’s traditional 176*208 screen) titles on your VGA Pocket PC’s. And, of course, you won’t be able to run MIDlets that stretch themselves dynamically to fill in the entire screen. This alone would make you forget the entire thing. Second, you’ll need to click on-screen buttons instead of just pressing the D-pad directions – as you otherwise would prefer. A pretty awkward way to control particularly games, isn’t it?

Note that you can avoid having to stick to significantly smaller MIDlets than your screen estate really is if you, as opposed to what the 4Pda tutorial recommends, a numeric input panel (or, if you don't mind the horizontal layout in no way representing a real phonepad) only brought up when necessary. I recommend assigning it to a hardware button (just assign <Input Panel> to a hardware app button in Settings / Buttons; this doesn't even require that you read and understand the Button Enhancer Bible); this will be particularly important with MIDlet managers that, on Pocket PC's, don't dispay the bottom bar and/or the icon to bring up the on-screen input panel any time (not only when a text input field has the focus.)

1.1.2 A full chart of all MIDlet manager scan/character codes & the return values of getGameAction()

I’ve also written a small MIDlet to display what the passed code before and after the getGameAction()-based translation are. As usual, I also provide you with the source of the MIDlet. The deployable JAR file is HERE. It’ll, first, display the numeric, and, then, the symbolic name of the return value of getGameAction(); finally, the original, unprocessed, original keycode. It also makes the system dynamically assign "Exit" to one of the two softkeys; the scancode of the other softkey (and, with Jblend, also the one which results in exiting; this is because of the differences in the callback processing order) will be displayed.

For example, upon pressing "2" on the dialpad, hardware or on-screen keyboard, if you see "1: UP: 50" printed on the screen, this means the following: getGameAction() returned the integer 1, which corresponds to the constant "UP". The original character code (it’s a positive number; therefore, it’s a real ASCII character code) 50 ("2" in ASCII). This is what you will see in all cases, with all MIDlet managers, except IBM J9, where this button, as with all other keypad numeric buttons, are unhandled and, therefore, no matter what numeric dialpad button you press, getGameAction() returns 0 ("Unknown"). Yes, you’ve read this right: the dialpad handling in IBM J9 is severely messed up: IBM J9 can ONLY be controlled via the D-pad, unlike with all the other MIDlet managers.

If you press the button 1, "0: Unknown: 49" means getGameAction() returned 0, which corresponds to the unhandled (unknown) case and the original character ASCII code was 49. This is the case with all MIDlet managers except Esmertec’s managers and Jblend, which assign one of the four gaming buttons, GAME_A, to the first dialpad button.

If you press the up direction on your D-pad, you’ll see the following with all MIDlet managers except TAO (the latter uses special scancodes):

"1: UP: -1"

This means getGameAction() returns 1, which, as we’ve already seen, corresponds to the "UP" symbolic constant. Finally, the scancode (it’s negative so it’s not a "real" character code) of the given D-pad direction is -1.

As has already been stated, TAO is behaving differently in this case (too); it’ll return

"1: UP: -57377"

instead. This, as has already been pointed out, is caused by TAO’s using entirely different scancodes than all the other MIDlet managers. Fortunately, this is correctly translated by getGameAction(). Still, if you try to play a game (for example, the already-mentioned Micro Counter Strike 3D Beta) that doesn’t give a call to getGameAction() but directly parses the actual argument of the keyboard handler (for "up", the ASCII code (50) of the dialpad 2 and/or the D-pad "up" scancode -1), it will most likely not work under TAO.

Note that the chart below, which shows how the different Windows Mobile (and, in addition, as a well-implemented, high-quality reference, the MIDlet manager coming with the Nokia’s current top-of-the-line, flagship model, the N95 and its newer versions like the N95-3 and the N95 8GB) MIDlet managers fare in this respect. Now, based on all explanations above, you’ll be able to understand what this all means.



1.2 Strictly softkey issues & JavaMagic results

OK, now, from the D-pad and dialpad-related questions, let’s move on to the question of the WM5 softkeys. As you’ve probably realized if you’ve ever tried to deploy MIDlets under TAO, the right/left softkeys simply won’t work with most of them.

Note that the 4pda.ru folks state the TAO 11.x series, finally, fixed at least the softkey problems (as opposed to the 10.1 series); in this regard, I don’t think they are right.) Unfortunately, this is impossible to fix using the same method as above - that is, just assigning a scancode generator to the two soft buttons with a softkey-assignment-capable button enhancer or, if you don’t necessarily want them to be assigned to hardware softkeys, with any button enhancer – and even the built-in Buttons settings applet. The sole reason for this is that MortScript is only able to pass positive "scancodes" (that is, real characters) and the other two solutions, namely, PQzII and VJKeyPress, aren’t able to pass these particular scancodes either.

Unfortunately, these softkeys,

1. with several MIDlet managers (namely, the one coming with the Nokia N95, IBM J9, later – non-M3G-enabled – Esmertec Jeodek versions and TAO running on Pocket PC’s (but not on Smartphones)) don’t cause a callback to keyPressed() and,
2. in the rest of the MIDlet managers (that is, the ones where keyPressed() is called back) getGameAction() returns zero ("Unknown") upon trying to identifying them,

unlike with "traditional" direction, action and/or fire buttons where getGameAction() is the preferred way to see what really has happened and to guarantee compatibility with as many different MIDlet managers as possible. This all means MIDlets themselves must deal with direct scancodes and there're no helper methods (no getGameAction()-alikes) to guarantee cross-KVM compliance. As these codes are pretty much standardized, they have no problems with the Nokia / Esmertec / Jblend ones; this is why they do work. TAO, on the other hand, is, again, an exception, as it uses vastly different scancodes than any else MIDlet manager. This is why the softkeys of the vast majority of MIDlets, unless they have specifically been developed keeping TAO compliance in mind (the case with high-quality business / productivity apps like Opera Mini, the Gmail MIDlet etc.), will NOT work under TAO.

1.2.1 JavaMagic – is it really any good?

The related 4PDA.ru tutorial ((volta_john's post at 23.05.07 08:25:07) ; Babelfish; Google Translate) recommends the use of JavaMagic to fix this problem.

I’ve thoroughly tested JavaMagic with several games running on TAO with TAO unable to handle their softkeys (see their list in the main game compatibility chart). As it has turned out, in no configuration did TAO Intent become compatible with these titles. It was only in one case that the Action button became usable after the translation (TAO Intent uses a different keycode – it passes the ASCII code 13 instead of the industry-standard -5 scancode). In no other cases did any of the keys, let alone the softkeys become usable. Again, these tests were VERY thorough and, in most cases, I’ve tried to make the MIDlets compatible with TAO Intent with at least two (Nokia and Sony-Ericsson source compatibility mode) input configurations. The output configuration, of course, was that of TAO.

Because of these very bad results, I don’t think you should waste any time with JavaMagic. It simply won’t work. This is why I don’t present you any tutorial for JavaMagic either. Even if you don’t know Russian, you’ll be able to follow the really thorough tutorial in the above-linked post, particularly now that I’ve provided you with two different "translations". But, again, it’s highly unlikely it’ll work with you.

Following is the chart of my JavaMagic tests:



Note that there are two other MIDlet conversion suites, Motomidman and FullJava, explained HERE (rendor, 30.10.06 00:35:31) (Babelfish; Google Translate). They, however, seem to help even less than JavaMagic.

1.3 Summary

In this chapter, I’ve given you a detailed description of why

1. the left/right softkeys of the vast majority of MIDlets refuse to work under TAO Intent and the fact that this can't be helped
2. there are MIDlets that can only be controlled with dialpads but NOT D-pads in all KVM's
3. there are MIDlets not controllable with anything, not even dialpads, under J9 (again, J9’s getGameAction() returns "Unknown" for all dialpad buttons – a very bad KVM implementation in this respect)
4. why some MIDlets don’t support the D-pad under TAO, while they do support it under all the alternative KVM’s

To fix all these problems (except for the softkey problems of TAO Intent, which, based on my tests, do seem to be unfixable), I’ve also explained you’ll need external button enhancer and/or ASCII and/or keycode "injectors", as opposed to what, so far, has been recommended (that is, on-screen keyboards / dialpads).

Artlan, you're GREAT! Thanks for the tutorial!

aminfri said:

hi
first thanks for the good job
now i did as you said but in the part that i save the modified myclass.class after that i don't see any canvas.java files being created, i even tried the saved as but the created file is not readable
can you help me out thanks
(trying to run burnout on my i-mate jasjar)

Click to expand...

Click to collapse

IIRC (can't really chck it now because I'm writing this on my N95, using a slooooow GPRSD connection) several KVM's run Burnout without key problems. that is, you don't need to use TAO. Did you read the MIDlet Bible?