Virtual Memory in Windows Mobile 6.5 and how it differs from 6.1 and 6.0, important when porting to a device that does not have a WM 6.5 Native Kernel
What is Virtual Memory?
Virtual Memory as it applies to the Windows CE 5.2 Kernel
There's a 32 bit address space available - a total of 4GB potential memory on WM. This total 4gb address space is split in half - 2gb to the Kernel, 2gb to the User. Kernel address space is only accessible by processes/threads with kernel level access, whereas user address space is accessible by all processes/threads. The user address space is from 0x00000000 to 0x7FFFFFFF, and the kernel address space is from 0x80000000 to 0xFFFFFFFF.
The user address space is split into 64 slots of 32mb a piece. (64x32=2048, or 2gb) - The first 33 slots (slot 0 and 1-32) are used for processes, and the remaining slots for object store, memory mapped files, and resources.
Each slot is easy to visualize in hex, you just increment a digit. Slot 0: 0x00000000 to 0x01FFFFFF, Slot 1: 0x02000000 to 0x03FFFFFF, Slot 2: 0x04000000 to 0x05FFFFFF, and so on. Each process gets assigned it's own slot, there are 33 total, minus the kernel process which is always running, so that's where you get the 32 process limit (which is always lower in practice, since there are always processes running in windows mobile to support the OS) - this is also the reason for a 32mb limit per process - you can see that's all the address space available in one slot.
How does that all apply to our situation with modules? When you create a module, you are telling Windows Mobile that you want that module to be memory-mapped, so that each time it loads, it's loading to the same, known area of ram - saving space in slot 0 - this is done on the computer-side, during 'cooking' and is the job of wmreloc, g'reloc, bepe's Platform Rebuilder, etc. Virtual allocations are aligned to a 64KB border, so if you memory map a .dll that's only 3Kb large, it's still going to eat up 64KB of memory space.
There are also pages that can be allocated to these slots, that are aligned to a 4KB boundry, and process/general allocations that take place during normal operation. The way the system handles this is that modules allocated on rom build-time (modules we allocate with g'reloc et. al.) are allocated from top-down (for slot 1 for example, starting at 0x03FFFFFF for the first module, taking up space to the nearest 64kb boundry, then the next module, in a line down to the 0x02000000 address, which is the beginning of the slot) - General allocations that take place during normal system operations are allocated in the remaining space, from the bottom-up (so again with slot 1 as an example, starting at 0x02000000 and ending at 0x03FFFFFF) - As you fill up these slots more and more with modules, that leaves less space for windows to dynamically allocate other, general allocations, which can, and does result in out of memory errors (even when the device has plenty of physical memory left, it cannot address this memory when virtual memory is full)
Here's where the kicker comes in for the difference between 6.0/6.1/6.5 - There are a total of 4 slots that can be used for module allocation, slot 0, 1, 60, and 61. 63 is also available to modules that contain no code (resource only modules, like .mui's)
could allocate slot 0, 1 - (in order of 1 first, 0 last) for a total of 64mb of vm space for modules/files (and the running process is always mapped to slot 0, so once you encroach on this memory space, you are removing available memory to each application running) - you can see how tight this is.
improved on this by opening up slot 60 and 61 (now the allocation order is 61, 60, 1, 0) - but modules could not be allocated here, only files. So for our relocation tools, this was essentially no change. (still only slot 1, 0 for modules) WM 6.1 also introduces the process initvmmap.exe in the MSXIPKernel, NK partition (xip.bin). This process draws a map of all virtual memory, and uses the following registry key from the boot hive (boot.hv):
This tells the system that the module dllname.dll is only required when filename.exe is running, or filename2.exe is running - and whenever these processes are not running, the virtual memory space is available, and can be dynamically allocated to by the system.
improves on this by opening up Slots 60 and 61 to Modules - yielding an extra 64mb of potential Virtual Memory space. (the allocation order is now 1, 61, 60, 0 for modules, 60, 61, 0 for files) - In order for the Kernel to recognize these new Slots as being mappable for Modules, it must be updated to the 6.5 codebase. This is where the 6.5 nk.exe comes in, and why it's so important.
Profiling Virtual Memory is an important job for an OEM - the less available in Slot 0, the sooner a device will kick back out of memory errors (even if it's not truly out of memory) - and the worse the user experience will be. Some ROM's I have seen have less than 20MB available in slot 0 (and the user experience is as bad as you might imagine) - There are many more intricacies to the whole process - like balancing the load between services.exe and device.exe to best utilize the 32mb VM space available to each, and storing all resource-only dlls as modules so they can be allocated to Slot 63, etc.
This is also why it's important that the re-alloc tools be updated to support the new slots - g'reloc will not ever try to allocate modules to slot 60/61 because as far as it's aware, this is not possible. For the moment I know of 2 tools that will realloc to slot 60/61, wmreloc 2.0, and bepe's Platform Rebuilder (used by ervius vk)
What's the take-home message about VM?
Keep Slot 0 as free as possible. WM 6.5 NK allows you to use more modules without taking up SLOT 0 space, so allows more flexibility to use modules (which are faster to load)