Have Any Developments Been Made On This
Is anyone working on this to make it run faster and/or more stable.
Is anyone working on this to make it run faster and/or more stable.
[Beta] Win86emu: Running x86 apps on WinRT devices
- Thread starter mamaich
- Start date
If I don't update the post - this does not mean that there is no progress.
I've integrated no2chem's DosBox dynamic core patches into my own code, added several specific optimizations, and here is the video showing the current emulation speed:
http://www.multiupload.nl/N6N0JYV3IK - it is HOMM 3 with the hires patch.
The build is still unstable - you can see that it hangs in the video end. But of cause I'm working on that.
I'll publish the binaries when I'll fix current problems and would add more functionality - for example the registry virtualization layer needed by old apps that write to HKLM (that layer is present on desktop Win8, but is removed from RT as there is no need in compatibility). You may see the need of this layer in HOMM 3 video - as I have to reset all game settings on every launch. There are more things to do, fortunately I'm on vacation and have enough time for coding.
Last edited:
Fantastic I am actually a developer myself so I understand the way that this generally works.
I was just checking up on the development status, I like hearing about all the behind the scenes development work that is going on you know. The emulation speed seems to be much faster now, is that all from the dynamic core and is that all going to be coming in the next update?
anyway thanks heaps for the update and keep me posted.
Dynamic core improves speed a lot, but there were some other hacks - for example DosBox emulates CPU page tables, TLB and other features that are unnecessary in my project. Removing them and making direct RAM access from the generated code gave the best speedup. I'll try to speedup things a bit more, but currently increasing stability has a higher priority.
This is fantastic work!
Personally (i.e. do whatever you like, it's not like my opinion matters here), my priorities would be Stability > Compatibility > Speed. It's already fast enough, especially after removing the emulation for CPU parts (like TLBs) that aren't needed, to run non-real-time apps and very old real-time ones happily; given that, I'd love to see support for a wider variety of software. Running Heroes3 on my Surface, as awesome as that is, isn't quite as useful as running, say, Irfanview.
With that said, a possible performance improvement: I've noticed it appears to hammer one core very hard, but not use the others much if at all. Is there any chance that the dynamic recompilation could be moved to another thread, and basically do a JIT recomp of the stuff right ahead of the current program counter, while the program executes "natively" on the other core(s, if it's a multithreaded program)? This may be complex as hell (I'm not a DOSBox hacker in any way) but if possible, it could give considerable speedup given the Tegra 3's multiple but relatively slow cores.
Also, any parts of this you want to open source, that would be awesome. There are a handful of small changes that I would like to make (and I know that I could), and it would also be good to spread the process of adding support for additional libraries.
Personally (i.e. do whatever you like, it's not like my opinion matters here), my priorities would be Stability > Compatibility > Speed. It's already fast enough, especially after removing the emulation for CPU parts (like TLBs) that aren't needed, to run non-real-time apps and very old real-time ones happily; given that, I'd love to see support for a wider variety of software. Running Heroes3 on my Surface, as awesome as that is, isn't quite as useful as running, say, Irfanview.
With that said, a possible performance improvement: I've noticed it appears to hammer one core very hard, but not use the others much if at all. Is there any chance that the dynamic recompilation could be moved to another thread, and basically do a JIT recomp of the stuff right ahead of the current program counter, while the program executes "natively" on the other core(s, if it's a multithreaded program)? This may be complex as hell (I'm not a DOSBox hacker in any way) but if possible, it could give considerable speedup given the Tegra 3's multiple but relatively slow cores.
Also, any parts of this you want to open source, that would be awesome. There are a handful of small changes that I would like to make (and I know that I could), and it would also be good to spread the process of adding support for additional libraries.
I agree with the priorities you've wrote. Now I'm busy with stability issues. Unfortunately due to ASLR I can't reliably reproduce bugs and this slows down the whole process :-\
Regarding multithreaded recompilation - DosBox is using only one thread and does not use any synchronization inside. I've tried to make it multithreaded in the builds published earlier - and that caused random crashes. I have an idea of making my own CPU emulation engine in a very distant future, and it would be optimized for multiple cores, would have an option to store recompiled code on the disk for later aggressive optimizations, would use MEM_WRITE_WATCH feature of VirtualAlloc to track self-modifying code. But making my own CPU core would be in a very distant future, may be even not this year.
I'm planning to release the modified DosBox and Bochs sources and publish an interface of adding Win API wrapper DLLs, but I'll do it a bit later, when those interfaces would be stabilized.
Regarding multithreaded recompilation - DosBox is using only one thread and does not use any synchronization inside. I've tried to make it multithreaded in the builds published earlier - and that caused random crashes. I have an idea of making my own CPU emulation engine in a very distant future, and it would be optimized for multiple cores, would have an option to store recompiled code on the disk for later aggressive optimizations, would use MEM_WRITE_WATCH feature of VirtualAlloc to track self-modifying code. But making my own CPU core would be in a very distant future, may be even not this year.
I'm planning to release the modified DosBox and Bochs sources and publish an interface of adding Win API wrapper DLLs, but I'll do it a bit later, when those interfaces would be stabilized.
By the way - found the reason why HOMM stopped responding in the end of the posted video. It was due to the lack of "LOCK" prefix emulation in DosBox. As DosBox is only single-threaded, it does not need "lock" command at all, so it was not implemented.
Binkw32.dll (video decoder) heavily uses "lock inc [addr] / lock dec [addr]" for synchronization, and without "lock" the BINK decoder quickly deadlocks itself on a multicore machine.
Binkw32.dll (video decoder) heavily uses "lock inc [addr] / lock dec [addr]" for synchronization, and without "lock" the BINK decoder quickly deadlocks itself on a multicore machine.
Last edited:
This is awesome, and the fact that you've got HOMM3 (one of my favourite games) running very close to lag free is amazing. Great work!
I'm excited to play HOMM3 on my Surface. One question though, would this emulation be eventually able to emulate proper touch events? As HOMM3 is basically a mouse only game, with touch it could definitely work as a proper game, just need to change the way the game reads touch events (rather than move the cursor, click). Would this be possible?
Plug in a mouse to the USB port.
Not really waht this thread is about, but for the record, HOMM3 works fine for me using just touch. Simply tap where you want to click, or press-and-hold to right click. Dragging doesn't work correctly but the game doesn't use dragging anyhow.
Hello mamaich, it's been ages I haven't connected to XDA and the last time was when I was playing with Windows Mobile devices and I was doing some api hooking. At this time you already were on this forums and I am bit surprised to see that you are still here. Anyway these last month I had to port a big c++ sdk to windows rt and I had to develop some missing features like registry emulation or that kind of thing. I have a small lib for that so if you need something like that I might share it.I am not saying this is great code but might be useful...
Thank you for the offer - but I've already implemented all the registry manipulation functions, including the registry key redirection for HKLM that is readonly now for programs that are not run as admin.
Currently I'm working on generic WinAPI - like shell32 interfaces (IMalloc, IShellFolder and so on), implement windows messages that pass callback functions in LParam and so on.
There is a good progress, but need time to make workarounds for bugs in the existing programs. For example WinRar sometimes returns pointers to strings allocated on the stack of the returning function, other programs, HOMM3 for example - like to access the heap after it was freed, other programs use SuspendThread or "lock inc [addr] / lock dec [addr]" for synchronizations instead of normal WinAPI.
On desktop Windows we have existing shims with workarounds, but here I have to emulate them myself as there was no sense for MS to port them to RT.
I really appreciate the amount of work Microsoft did in recent desktop Windows versions for keeping the compatibility with such old programs. This layer is invisible for the end-user, but you really can't live without it.
I'll post a new test build here after a set of a test programs like WinRAR would run correctly. This would be soon enough.
Great work. Looking forward to seeing further developments (and helping if I could).
Just out of interest was reading up on the vaguely similar older project at http://wiki.winehq.org/ARM (WINE on Android/LInux ARM)
Update - Note WINE on ARM emulates the cpu and catches the apis and runs their equivalent natively
Just out of interest was reading up on the vaguely similar older project at http://wiki.winehq.org/ARM (WINE on Android/LInux ARM)
Update - Note WINE on ARM emulates the cpu and catches the apis and runs their equivalent natively
Last edited:
This, technologically speaking, is pretty much the opposite of WINE.
WINE implements the API and runs the code natively
This emulates the code and runs the API natively
Yes, I'm looking at their sources and to the code of ReactOS (they have some ARM support too). I can't directly borrow their code as we have too much differences in the core, but I'm looking on the ways how they've implemented some non obvious/undocumented things. And I'm also heavily studying the leaked Windows NT4 and 2000 sources (of cause I don't use any part of them in my project, but they contain really useful information).
I'll publish a new version this weekend.
Last edited:
And when do you sleep
?Just curious to know how you can find time to code so many things...
I'm Windows Server admin in a real life. And good admins have enough free time at work
By the way, I've tried to run pinball from Windows 98 under emulation. It runs, but the ball is stuck in the top-left corner. This is due to floating-point emulation problems that would be hard to find and fix.
By the way, I've tried to run pinball from Windows 98 under emulation. It runs, but the ball is stuck in the top-left corner. This is due to floating-point emulation problems that would be hard to find and fix.
Hmm... is 80387 (x86 floating point instruction set) emulation not available, or is it justa bug in the ARM port of DOSBox? While FP math is kind of crazy and many processors seem to handle the instructions a bit differently (dedicated registers or not, different FP modes, etc.), it would be good to have. Many games, media decoders, and tools will rely on this... Let us know how it goes, thanks!
The second - incorrect emulation. Or maybe it is my fault, I could initialize emulated FPU control word with a wrong value, for example I could set rounding to the wrong direction (I've taken the value from a desktop Windows 8, and it may differ for Win98 programs). I've tried the Bochs core - the same problem there, so it really may be my bug.
Edited:
It was my bug. I've set FPU control word to 0x37F, while Win98 value was 0x27F. That one bit controls the FPU precision and caused problems in Pinball.
Last edited:
Wow this sounds like it is coming along great!
Any idea when you can drop a new version?
Similar threads
Top Liked Posts
-
There are no posts matching your filters.
-
96The project is abandoned.
As I no longer own a Windows RT device and I'm not willing to use Windows RT anymore, unless Microsoft would make it more open (at least to run your own desktop apps) - I've decided to stop working on this project.
As usual I'm publishing complete sources of this tool. Feel free to use them in your own projects or to continue developing this one - only leave my copyrights somewhere.
Don't ask me how to build the sources or to explain anything in them. Figure that out yourself.
The project is abandoned. Sorry.
I'm presenting a tool that allows running a set of x86 Windows applications on Windows RT (ARM) tablets. Its goal is to support all apps except for those that:
- require much CPU power,
- use complex features that were cut out from WinRT like D3D9 extensions or OpenGL,
- require drivers or specific services,
- make heavy use of COM interfaces,
- use undocumented windows internals,
- apps that use .NET framework,
- x86 Metro apps,
- 16 or 64 bit Windows programs,
- buggy apps that require special workarounds.
The tool is currently on a beta stage, so don't expect much from it. It is far from being complete, but at least it runs something.
Current version: 0.061
Just a minor update. The project is not dead, I just had no time to continue the development.
Attached the fixed ntdll.nt.dll that works under Windows RT 8.1 (Microsoft removed some NTDLL exports, so I had to add more stubs). This fix is not needed on RT 8.0.
To install it: extract the attached 0.061-ntdll.nt.dll.zip to c:\x86node\windows\SystemNT\ overwriting the existing file.
Autostarting x86 programs does not work on RT 8.1 ("can't install CreateProcessInternal hook"). I'll look on this later.
Don't ask on jailbreaking the 8.1 beta in this thread - there is a good progress on it, more info would be on release (in october or when WZOR would leak the RTM).
Current version: 0.06
Seems that archive is too big to be attached, so I've uploaded it to google drive and here
Installation: extract the archive on your unlocked Windows RT device, run the MSI file and follow the instructions.
Note: Uninstall the previous version before installing a new one.
List of compatible apps is in this post: http://xdaforums.com/showthread.php?p=40924456
Trademarks
Windows is a registered trademark of Microsoft Corporation. ReactOS is a registered trademark or a trademark of ReactOS Foundation. All other trademarks are the property of their respective owners.
Disclaimer
This software is provided "as is". Use it on your own risk. I make no warranties as to performance, merchantability, fitness for a particular purpose, or any other warranties whether expressed or implied. No oral or electronic communication with me shall create a warranty of any kind. Under no circumstances should I be liable for direct, indirect, special, incidental, or consequential damages resulting from the use, misuse, or inability to use this software, even if I has been advised of the possibility of such damages.
I'm trying my best to make the software working, but I can't guarantee that it is free from defects.
All beta versions of this tool would be freeware. You may freely use it for your own, embed it into your tool, but you can't use it commercially without my confirmation. You can disassemble, analyze or modify this tool for yourself - later I'll provide SDK and document its internals. The only thing that is prohibited is changing embedded copyright notices. I reserve the right of making the project commercial, but this does not mean that this would ever happen.
This software contains unmodified binaries from the ReactOS project: a registry editor, cmd.exe, ole32.dll to name the few. Those binaries are left unmodified and are covered by LGPL license. Future versions may contain redistributable binaries provided by Microsoft and/or other companies.
Some more information may be found in my blog. If you want to support development - use the link or press the button on the left side of the post.11Changes:
15 may 2013: DInput and DInput8 changes for Fallout2 keyboard compatibility.
12 may 2013: A minor update. Fallout 2 now works, tested on Russian version from 1C.
01 may 2013: Added the ability to automatically launch x86 applications. Added the shell32 interfaces - so that installers now work (at least NSIS and InstallShield installers are known to be working).
05 apr 2013: Fixed a few bugs.
04 apr 2013: Uploaded a new build after a long delay. Now emulator supports 256-color modes. But due to a limitation on an updated Nvidia driver - 640x480 and 800x600 display modes are no longer supported on Windows RT. You'll see the black lines to the right and bottom of the screen if the program tries to set such mode.
25 feb 2013: The tool now outputs its version to log. Now one x86 program may launch another - so some of the installers and, for example, 7Z GUI frontend can now run under emulation. Added ~80 DLLs. Some of them are stubs (like D3D9.DLL), others are mostly untested. I have not done all that I've planned for this build, publishing it just as an update to show that the work is going on. Do not expect it to run much more than the previous build.
13 feb 2013: more informative errors from launcher. Emulator now supports program paths with spaces. EXE files with relocations are now processed correctly. Some bugfixes in kernel32 and advapi32.
11 feb 2013: fixed a typo in winmm.dll emulation, now pinball has sound. Also updated the launcher.
10 feb 2013: now the program reached the beta stage.
Known problems
No D3D and most of COM interfaces. Lots of programs would crash, don't run or have different issues.
Notes:
The program keeps its settings in the HKCU\Software\x86node\Settings registry key. Supported REG_SZ (string) values are:
DosboxCore: "dynamic", "simple" or "normal". Dynamic is the default as it is the fastest, but the most buggy core.
LogFile: path to the log file. If not present - log file is %temp%\win86emu.log
Supported REG_DWORD values:
LogLevel: 0=no log (default), 4=max logging
added 13 feb 2013: now default log level is 2: warnings+errors, so you don't need to edit registry
There are several compatibility hacks that may be useful. Compatibility settings are stored in HKCU\Software\x86node\Compatibility\[filename.exe] key. "filename.exe" - a name of the emulated EXE file without path. All values are DWORD:
SetProcessAffinityMask = bitmask. Specify which CPUs to use for running a program, read SetProcessAffinityMask description in MSDN. 0 or unset == run on all cores.
NoRaiseException = 1. RaiseException would just return. Now exceptions are emulated correctly, so this hack is no longer needed.
UseDirectRegistry = 1. Do not redirect emulated registry keys to HKCU\Software\x86node. Be careful when using it.
MaxProcessorFeaturePresent = max processor feature number that is "supported". See the IsProcessorFeaturePresent function in MSDN. All requests for the value above specified would return 0. Default: 0 (IsProcessorFeaturePresent always returns 0).
SimulateAdminRights = 1. Lie to installers that call OpenSCManager function to determine that it is running as administrator, allowing these programs to run without elevation. Redirect the "common start menu" and similar folders to the per-user folders.
You can fake the OS version to a running program. Default XP SP3:
OSVersionLo=dword:00000001
OSVersionHi=dword:00000005
OSVersionBuild=dword:00000a28
OSServicepackLo=dword:00000000
OSServicepackHi=dword:00000003
Some information on the project internals may appear in my blog: http://mamaich-eng.blogspot.ru, but this thread on XDA would be the main discussion place.3
It is better to port free download manager as is is opensource.
FDM may be compatible with some of the future versions, but there would be no internet explorer integration. I'm not going to support all those COM interfaces, at least in the foreseeable future.
I'll pause publishing updates for some time as I'm adding several dozens of new system DLLs to the emulator, and need to make them working before making them public. Ws2_32, msimg32 and other requested DLLs are among them. uTorrent 2.x is working now, though with some problems.
You may see it on the Screenshot
NSIS installers are working too (checked with 7Zip official installer), though they can't create shortcuts as I have not implemented that interface yet.3
Not exactly a port, it is a clean remake based on the old ideas.
The idea is very simple:
- a PE file loader (load files, process relocs, run TLS callbacks in an emulation mode). Support import loops (DLL A imports B while B imports A), ordinals, etc.
- a set of wrapper x86 DLLs (kernel32_stub.dll and so on) that "look like" the corresponding Win API functions for an emulated program:
- a 32-bit x86 emulation engine (currently 2 engines: from bochs and from dosbox, planning on adding my own) that intercepts the command "in eax,0xe5", determines which API is needed by a program and passes it to a handler.Code:#define DEFINE_FUNC1(name) \ static const ModuleDef str_##name={DLL_NAME,#name}; \ EXTERN_C DW STUB_EXPORT stub_##name(DW p1) \ { \ DW *p=&p1; \ __asm { mov eax,p } \ __asm { jmp f1 } \ __asm { mov eax,offset str_##name } \ f1: __asm { in eax,0xe5 } \ __asm { mov p,eax } \ return (DW)p; \ } ..... #define DEFINE_FUNC3(name) \ static const ModuleDef str_##name={DLL_NAME,#name}; \ EXTERN_C DW STUB_EXPORT stub_##name(DW p1,DW p2,DW p3) \ { \ DW *p=&p1; \ __asm { mov eax,p } \ __asm { jmp f1 } \ __asm { mov eax,offset str_##name } \ f1: __asm { in eax,0xe5 } \ __asm { mov p,eax } \ return (DW)p; \ } .... DEFINE_FUNC1(AddAtomA) DEFINE_FUNC1(AddAtomW) DEFINE_FUNC7(CreateFileA) -- number in macro == number of parameters to a __stdcall WinAPI function. Compiler automatically generates "ret N*4" at the end of such function. I've decided to use such c+asm approach instead of making a tiny assebler stub, as I can easily implement some of such functions in C directly in a stub DLL plus it simplifies debugging. And the functions have a usual C prologue/epilogue, so that the emulated program may even patch them in runtime, for example for hooks. ...
- native (arm) API handler DLLs (kernel32_yact.dll and so on). They are mostly autogenerated too:
Some of the functions require complex emulation due to their absence in ARM or due to the callbacks to x86 code:Code:#define DEFINE_FUNC1(name) \ EXTERN_C DW STUB_IMPORT name(DW); \ -- this behaves like a function prototype to compiler EXTERN_C DW STUB_EXPORT yact_##name(DW *R) \ -- R - pointer to the x86 stack { \ DW r=name(p1); \ // call the func passing it paramers from the emulated stack, p1==R[0], p2==R[1] and so on LEAVE(1); \ // empty macro, as the stack is unwinded in x86 stub DLL now return r; \ } ... #define DEFINE_FUNC3(name) \ EXTERN_C DW STUB_IMPORT name(DW,DW,DW); \ EXTERN_C DW STUB_EXPORT yact_##name(DW *R) \ { \ DW r=name(p1,p2,p3); \ LEAVE(3); \ return r; \ } ... DEFINE_FUNC1(AddAtomA) DEFINE_FUNC1(AddAtomW) DEFINE_FUNC7(CreateFileA) // as you see - implementation part is identical to an x86 stub, so I can use the same stub-generator tool
Some of the COM interfaces are already implemented, for example DirectDraw and DirectSound, though not heavily debugged. On a desktop emulator build I can already run "Heroes of might and magic 3" and old WinRAR, but there are several RT-specific OS limitations I need to bypass before making them run on ARM. Current work in progress is: overcoming the RT limitations, manually implementing the API functions that callback to a program code (like CreateThread, RegisterClassA and so on), adding stubs for other system DLLs/COM objects.Code:static DWORD WINAPI ThreadProc( LPVOID lpParameter // [0] == orig func, [1] == orig param ) { __EXCEPTION_REGISTRATION_RECORD R; DWORD *Parm=(DWORD*)lpParameter; DWORD *TEB=(DWORD*)PeLdrGetCurrentTeb(); R.Next=(__EXCEPTION_REGISTRATION_RECORD*)-1; R.Handler=(void*)CbReturnToHost(); TEB[0]=(DWORD)&R; // in case of unhandled exception - just return PeLdrNotifyNewThread(NULL,DLL_THREAD_ATTACH); DWORD Ret=EmuExecute(Parm[0],1,Parm[1]); // 1 == number of parameters to the emulated function delete Parm; return Ret; } EXTERN_C DW STUB_EXPORT yact_CreateThread(DW *R) { DWORD* Parm=new DWORD[2]; Parm[0]=p3; // TODO: no out-of-memory checking for now Parm[1]=p4; DWORD StackSize=p2; if(StackSize) StackSize+=1024*1024; // I reserve some space for my own needs (debugging) else StackSize=2*1024*1024; // TODO: I don't support autogrow stacks, so reserve 2 Mb DWORD t=(DWORD)CreateThread((LPSECURITY_ATTRIBUTES)p1,StackSize,ThreadProc,Parm,p5,(LPDWORD)p6); LEAVE(6); return t; }
Manually thrown SEH exceptions are fully supported, but access violation, int3 and similar OS-generated exceptions would cause program to crash. Some of the TEB fields (TLS and the fields required by the Borland compilers) are implemented too.
I don't make pointer translation in an emulated code nor make parameter checks passed to API. As a side-effect - the emulated program may trash the emulator in memory, but this greatly increases speed.
Most of the x86 EXE files don't contain relocations section and need to be loaded on the specific addresses (typically 0x400000). This is not a problem on a desktop, as I can rebase my emulator's EXE to any address I need, and free the corresponding RAM addrs for emulated program, but on ARM - this is a main problem. So currently only EXEs with relocs are supported for emulation, but there are ways to overcome this problem. And some EXEs produced by old Borland compilers contain "broken" relocs, this is a small problem too.
