[Beta, x86/ARM] Onscreen Joystick

[netham45]

Onscreen Joystick
By Netham45, Version Beta 1​

I've been working on this for Windows RT/Windows 8 off and on (more off than on), and I think it's ready for a beta tag.
I'm eventually planning on selling this somewhere for $5 or something, but it'll always be free on XDA.

Features:
Configurable for positions/keys pressed
Multi-touch support
Written in .Net, so one binary works on ARM and x86/x64
Open-Source


Documentation on how to use the configs should be at the top of the provided sample config.

Since this -is- the first public release, please let me know of any bugs you have with it, by posting in this thread and not by PMing me.

Known bugs:
Does not work with fullscreen applications (games/apps must run windowed)
Does not update smoothly (due to me using forms to render)
Quake 2 on Windows RT does not accept input from this app

 

[Bec07]

Very nice! I don't know if it can be decompiled or serve as inspiration but comfort keyboard pro can stay on screen in almost any game I threw at it. Problem is it's kinda buggy and doesn't take well resolution changes, also the screen behind the keyboard gets pressed too.

Anyway, great app! this is certainly a must have for Win8!

 

[mamaich]

I was thinking about making such program myself, but I don't know how to draw my own controls above fullscreen DirectX or D3D programs, so I have not even started.

 

[netham45]

I was thinking about making such program myself, but I don't know how to draw my own controls above fullscreen DirectX or D3D programs, so I have not even started.

You'd basically have to inject into the DirectX process and hook the drawing and input functions.

 

[mamaich]

You'd basically have to inject into the DirectX process and hook the drawing and input functions.

Yes, I'm doing that in my project (private build for now) - as I emulate DX anyway. But I've thought about an easier method. Maybe overlays, or something like that.

 

[rockkilic]

Very nice, looking forward to use this with various emulators

 

[unvaluablespace]

Great idea! I don't own Windows 8 with a touchscreen, but I can imagine the convenience of having this for light-to-moderate gaming, especially on the go! It already looks like it, but if not it would be cool to show each button as there respective keyboard command. So if button binded to keyboard "X" key were changed to keyboard "P" instead, the button itself would change, respectively. Good luck with it.

 

[themac]

Thanks, great app !!! Did you use it with epsxe (playstation emulator) ? It works ?

(doesn't have my win8 here with me)

Thanks !

 

[netham45]

I don't actually own a Windows 8 tablet, only a Windows RT one, so I can't test it with ePSXe

 

[themac]

Ok, netham45. I'll test it as soon as i'll get my Asus vivotab and report here. Want me to do some kind of beta-test ? Thanks, again !!

TheMac

 

[jonthe838]

Right I need some help here, I want to create a second stick so that I can use mouse controls. couöd you help me set this up? i want to play minecraft heh

 

[pvyParts]

Right I need some help here, I want to create a second stick so that I can use mouse controls. couöd you help me set this up? i want to play minecraft heh

not sure that will work as the clicks on the screen "are the mouse"

was sorta thinking of the same sorta thing for FTL, as the touchscreen dont respond properly in game.

Also works great on asus vivotab (me400) full win 8. With FTL.

Have you thought about Putting the sources up in github so ppl can push commits and help you develop it

I grabbed the zip and made a few changes. Want me to send you a zip back?

I made a basic program to test the above theory, and i could not get it to work
the "Mouse Thumbstick" makes the app you are trying to control loose focus.

Pvy.

 

[LilCthulhu]

DEV Feedback

Hey guys,

First I would like to apologize to Netham45, I did contact him earlier today directly when I did not yet have permission to post here. Second, I would like to thank XDA to allow me to post my first post here.

I've been working very hard on and off for the past few weeks from the original sources. I have completely rewrote a vast majority of the original source code, but kept the underlying principles. Everything is becoming a little bit more object-oriented.

Here's what I've been up to:

- Direct X support
- Multiple analog joystick
- Mouse support (analog joystick)
- Touchscreen as a mouse device
- Preliminary Edit mode to move buttons around
- System action buttons ( popup keyboard, edit mode, exit, etc)

Regarding "Touchscreen as a mouse device", In FPS or MMO, touching the screen is badly interpreted and the "mouse-view" system becomes chaotic and uncontrollable. By re-emulating mouse input from the touchscreen, I've been able to control mouse-view in game and mouse "mechanics".

I'm currently re-writing again some part of the software, I'm taking the edit mode out of the gameplay mode, separation purposes. I need to address some little issues and complete my mouse emulation on the touchscreen. It's been a lot of fun adding support, a lot of learning how to interrupt and inject back into windows os.

Now all of this is done on my spare time, on and off. It's something that I like to do, but I have a family, kids and game commitment (SW:TOR Guildmaster).

Right now, here are the games I've been playing with a fair amount of success :

SW:The Old Republic (MMO)
Team Fortress 2 (FPS)
Alien Swarm (shoot-em up)

As soon as I get a feedback from Netham45 and I feel I can release an update, I'll do.

not sure that will work as the clicks on the screen "are the mouse"

was sorta thinking of the same sorta thing for FTL, as the touchscreen dont respond properly in game.

Also works great on asus vivotab (me400) full win 8. With FTL.

Have you thought about Putting the sources up in github so ppl can push commits and help you develop it

I grabbed the zip and made a few changes. Want me to send you a zip back?

I made a basic program to test the above theory, and i could not get it to work
the "Mouse Thumbstick" makes the app you are trying to control loose focus.

Pvy.

 

[pvyParts]

Hey guys,

First I would like to apologize to Netham45, I did contact him earlier today directly when I did not yet have permission to post here. Second, I would like to thank XDA to allow me to post my first post here.

I've been working very hard on and off for the past few weeks from the original sources. I have completely rewrote a vast majority of the original source code, but kept the underlying principles. Everything is becoming a little bit more object-oriented.

Here's what I've been up to:

- Direct X support
- Multiple analog joystick
- Mouse support (analog joystick)
- Touchscreen as a mouse device
- Preliminary Edit mode to move buttons around
- System action buttons ( popup keyboard, edit mode, exit, etc)

Regarding "Touchscreen as a mouse device", In FPS or MMO, touching the screen is badly interpreted and the "mouse-view" system becomes chaotic and uncontrollable. By re-emulating mouse input from the touchscreen, I've been able to control mouse-view in game and mouse "mechanics".

I'm currently re-writing again some part of the software, I'm taking the edit mode out of the gameplay mode, separation purposes. I need to address some little issues and complete my mouse emulation on the touchscreen. It's been a lot of fun adding support, a lot of learning how to interrupt and inject back into windows os.

Now all of this is done on my spare time, on and off. It's something that I like to do, but I have a family, kids and game commitment (SW:TOR Guildmaster).

Right now, here are the games I've been playing with a fair amount of success :

SW:The Old Republic (MMO)
Team Fortress 2 (FPS)
Alien Swarm (shoot-em up)

As soon as I get a feedback from Netham45 and I feel I can release an update, I'll do.

any chance of having a look at some of it? or the current functioning binary?

i didn't put that much time into it

be pretty happy to help where i can.

Pvy.

 

[netham45]

Feel free to post your developments on it.

 

[LilCthulhu]

Thanks Netham,

I'll try to consolidate some part of the code, I'd like my mouse support to be better that what it is right now and I'll try to have a tentative release in a few days minus the edit mode that I'm basically tearing apart right now

Right now, my mouse support works well for a while and then becomes a little bit chaotic, I'd like to explain what I mean by that, but anybody who have played a FPS or MMO and touched the screen must know what I'm talking about. So right now, sometimes my code will trigger properly or not with not effect really... when it does 99% of the time it does well, with 1% with the same chaotic movement. It works well in MMOs, in FPS, it's different. For example in Team Fortress, for the first few seconds, it doesn't, then it works... it's not really related to how I code AFAIK. In SWTOR, it works very well... using the touchscreen or a 2nd analog joystick to mouse view stuff is awesome

Now I'll try to explain how mouse view works in a vast majority of games. When you use mouse-view, it kinda hide the mouse cursor and calculates movement from the center of the screen. Micro movement on the mouse means major camera movement. If you would click anywhere on the screen without my version of the on-screen joystick, the camera would go crazy and spin several time. What I do in code is center the mouse cursor then, slightly move it based on the touch mouvement, and voila works well

Now I wouldn't raid with that, but for leveling and doing solo quest, it works well...

 

[pvyParts]

Thanks Netham,

I'll try to consolidate some part of the code, I'd like my mouse support to be better that what it is right now and I'll try to have a tentative release in a few days minus the edit mode that I'm basically tearing apart right now

Right now, my mouse support works well for a while and then becomes a little bit chaotic, I'd like to explain what I mean by that, but anybody who have played a FPS or MMO and touched the screen must know what I'm talking about. So right now, sometimes my code will trigger properly or not with not effect really... when it does 99% of the time it does well, with 1% with the same chaotic movement. It works well in MMOs, in FPS, it's different. For example in Team Fortress, for the first few seconds, it doesn't, then it works... it's not really related to how I code AFAIK. In SWTOR, it works very well... using the touchscreen or a 2nd analog joystick to mouse view stuff is awesome

Now I'll try to explain how mouse view works in a vast majority of games. When you use mouse-view, it kinda hide the mouse cursor and calculates movement from the center of the screen. Micro movement on the mouse means major camera movement. If you would click anywhere on the screen without my version of the on-screen joystick, the camera would go crazy and spin several time. What I do in code is center the mouse cursor then, slightly move it based on the touch mouvement, and voila works well

Now I wouldn't raid with that, but for leveling and doing solo quest, it works well...

That's pretty much what I was doing. But... I was having problems passing clicks through.

Pvy

Sent from my Galaxy Nexus using Tapatalk 2

 

[LilCthulhu]

Quick update...

In the past few days, I've written a complete overhaul of the touch system, I had made some design mistakes on purpose along the way. Mainly to allow controls to be move/resized on screen... I'm now starting to implement an Edit Mode that works outside of the Gameplay mode. I've been working on serializing/deserializing object to save/load templates. So I guess, I'm about to work on the edit mode...

The hardest part is keeping everything working with the passing of events (mouse mostly) to the application, and trying to copy the cursor back to the application itself just to maximise a kind of blend in between the multiple layers on screen. One slight changes can break everything...

I'm going to explain how things works :

The Gamepad Mode is either active or inactive.

When inactive, the windows soft keyboard can be opened and use as usual and you can control your table with the usual touch interface. However, no control appears on screen.

When active, the software takes over every touch input and interprets it according to two modes:

The Gamepad mode where touch events are only interpreted by on screen visual objects. In this state, outside of the gamepad controls, the touch input is totally disabled.

The Touchscreen mode where touch events are interpreted by the on screen visual objects and the rest of the surface can be used as cursor/mouse view input, basically it completely reproduce a touchscreen device... Right now you can tap the screen to move the cursor to the tap location and left click OR you can move the cursor around. With that you have on screen buttons that replicate the mouse buttons.

All of that works pretty well.

This week I will try to implement a quick Edit Mode with load/save capabilites and the possibility to add controls on screen and to edit them. This will allow you to add Analog Gamepad to control the Mouse or Keyboard (WASD, QWES, Arrow Keys), mouse button and mouse scroll buttons (scroll up or down) and "simple" keyboard to replicate number and letters. Eventually I will add Shift/Control/Alt Keyboard command.

I'd like to have a small list of alpha testers, so if you're interested, let me know. Basically with Alpha, I'm not going to try to support a particular game, I'm looking for major issues that make it so that the software becomes unusable. If a software doesn't work, I'm not "yet" going to spend time trying to fix it, so it's mainly to figure out major bugs.

With Beta, I will start implementing other functions that I have may not thought in advance and that will cover multiple games.

Once it's released, then I'll look into providing support for particular games based on popularity and demands...

 

[pvyParts]

Quick update...

In the past few days, I've written a complete overhaul of the touch system, I had made some design mistakes on purpose along the way. Mainly to allow controls to be move/resized on screen... I'm now starting to implement an Edit Mode that works outside of the Gameplay mode. I've been working on serializing/deserializing object to save/load templates. So I guess, I'm about to work on the edit mode...

The hardest part is keeping everything working with the passing of events (mouse mostly) to the application, and trying to copy the cursor back to the application itself just to maximise a kind of blend in between the multiple layers on screen. One slight changes can break everything...

I'm going to explain how things works :

The Gamepad Mode is either active or inactive.

When inactive, the windows soft keyboard can be opened and use as usual and you can control your table with the usual touch interface. However, no control appears on screen.

When active, the software takes over every touch input and interprets it according to two modes:

The Gamepad mode where touch events are only interpreted by on screen visual objects. In this state, outside of the gamepad controls, the touch input is totally disabled.

The Touchscreen mode where touch events are interpreted by the on screen visual objects and the rest of the surface can be used as cursor/mouse view input, basically it completely reproduce a touchscreen device... Right now you can tap the screen to move the cursor to the tap location and left click OR you can move the cursor around. With that you have on screen buttons that replicate the mouse buttons.

All of that works pretty well.

This week I will try to implement a quick Edit Mode with load/save capabilites and the possibility to add controls on screen and to edit them. This will allow you to add Analog Gamepad to control the Mouse or Keyboard (WASD, QWES, Arrow Keys), mouse button and mouse scroll buttons (scroll up or down) and "simple" keyboard to replicate number and letters. Eventually I will add Shift/Control/Alt Keyboard command.

I'd like to have a small list of alpha testers, so if you're interested, let me know. Basically with Alpha, I'm not going to try to support a particular game, I'm looking for major issues that make it so that the software becomes unusable. If a software doesn't work, I'm not "yet" going to spend time trying to fix it, so it's mainly to figure out major bugs.

With Beta, I will start implementing other functions that I have may not thought in advance and that will cover multiple games.

Once it's released, then I'll look into providing support for particular games based on popularity and demands...

if your still looking for testers let me know happy to help you debug any quirks.

Pvy.

 

[netham45]

Have you managed to retain Windows RT compatibility?