When we were back on NoDo there were quite a few homebrew apps that used native code to apply tweaks to WP7 devices. Most of those apps seized to work after the device is upgraded to Mango. There a several reasons for this behavior. I've done research on this, because I wanted to make WP7 Root Tools compatible with Mango. In this topic I'd like to explain how developers can fix their apps to work on Mango again. It has taken me quite some time to compile this guide, but I hope to give the Homebrew development on WP7.5 Mango a boost.
This guide is NOT about creating homebrew executables (exe-files) for WP7. This guide aims to utilize native code DLL's (C++ / ARM) from within your Silverlight app.
Note that with native code you get access to a lot of extra API's. But that does not mean you automatically get access to resources you normally won't have access to. For example, you can use the CopyFile() API. But if you try to copy a file to the \Windows folder, you will get errorcode 0x4ec (1260), which means "Blocked by policy". So you are still bound to the rules of the sandbox of your app. If you want Full Root Access for your app, you have to wait for a new version of WP7 Root Tools, which will allow you to give your app root-access. I'm also working on an SDK for that, which wraps all common task into a neat managed library. But don't hold your breath for that, because it's all taking a bit longer than I expected.
To understand everything in this guide you need basic knowledge of C++, COM-interop and Silverlight for Windows Phone. If you are new to all this, you might want to do some reading on these topics first. Currently there is no way to debug the native code. The only thing you can do is create test-functions which return formatted debug-info. This makes things pretty difficult. Read the guide carefully, because a little mistake can make your app crash easily!
Important note: If you have any long-running tasks, they may work fine while you are debugging. But you need to make sure that you start a new thread to run this code. Because, when you run without debugger the WatchDog will monitor your application and if the User Interface thread is blocked for more than 10 seconds the WatchDog will exit your app ungracefully!
It has been suggested that native homebrew DLL's need to be signed with approved code-signing keys. This is in fact not true! You can use native DLL's on Mango devices, which are not signed at all!
Basically there are two reasons why homebrew apps are not working anymore:
- Interop Lock
- DLL's were built against libraries, which are not supported anymore on Mango
Interop Lock is discussed in this thread. Interop Lock is a new protection mechanism in WP7.5 Mango. Basically it means you can't use apps with ID_CAP_INTEROPSERVICES, unless a device is Interop Unlocked. Without ID_CAP_INTEROPSERVICES an app can't call any drivers. And most homebrew apps call these drivers directly or indirectly. So if an app uses the Interop Capability, it can only run on devices that are Interop Unlocked. If you're going to build an app that uses this capability on Mango, you'll have to give your users instructions on how to apply Interop Unlock on their device.
Most of the native code libraries that were used on NoDo, were based on a hand full of projects. These projects were created and then extended for their own needs by other developers. The result was that most of these projects had the same project-types and library-references. In Mango, a lot of DLL's that were not used anymore by Microsoft, have been removed from the OS. Mostly in the ShellCore. The DLL's were meant for MFC-type functionality, which was never even supported on WP7. Actually, these DLL's are not even used by the homebrew apps either, but there are references to these DLL's in the homebrew libraries, which will cause the library to fail loading into memory. You can see this behavior when you try to run an app with non-Mango-compatible native code on an Interop Unlocked device from within the Visual Studio 2010 development environment. When the COM-class is instantiated it will throw an COMException: "COM object with CLSID '{...}' cannot be created due to the following error: The request is not supported." This is errorcode 0x80070032. This exception is actually caused due to the fact that the previous call to RegisterComDll() failed. If you get the returnvalue of that function you should have 0. In this case the return-value is probably 0x8007007E, which is "Module Not Found". This actually means that you directly or indirectly refer to a DLL, which cannot be found on the device. To fix this we need to create a clean project and add our new or existing native code to that project.
Here are the steps to setup your development environment and create a new, clean project for your native code. Please keep in mind that this guide is still work-in-progress. I may add more detailed instructions and examples later on, when people ask for it.
Update 2011/10/15: Some improvements in the guide, based on comments of rudelm and GoodDayToDie.
I hope this will help you port your homebrew apps to Mango or create some fresh new homebrew! If you created an app with native code, drop me a line here. Show me your Screen Recorders, Accent Changers and more!
Ciao,
Heathcliff74
This guide is NOT about creating homebrew executables (exe-files) for WP7. This guide aims to utilize native code DLL's (C++ / ARM) from within your Silverlight app.
Note that with native code you get access to a lot of extra API's. But that does not mean you automatically get access to resources you normally won't have access to. For example, you can use the CopyFile() API. But if you try to copy a file to the \Windows folder, you will get errorcode 0x4ec (1260), which means "Blocked by policy". So you are still bound to the rules of the sandbox of your app. If you want Full Root Access for your app, you have to wait for a new version of WP7 Root Tools, which will allow you to give your app root-access. I'm also working on an SDK for that, which wraps all common task into a neat managed library. But don't hold your breath for that, because it's all taking a bit longer than I expected.
To understand everything in this guide you need basic knowledge of C++, COM-interop and Silverlight for Windows Phone. If you are new to all this, you might want to do some reading on these topics first. Currently there is no way to debug the native code. The only thing you can do is create test-functions which return formatted debug-info. This makes things pretty difficult. Read the guide carefully, because a little mistake can make your app crash easily!
Important note: If you have any long-running tasks, they may work fine while you are debugging. But you need to make sure that you start a new thread to run this code. Because, when you run without debugger the WatchDog will monitor your application and if the User Interface thread is blocked for more than 10 seconds the WatchDog will exit your app ungracefully!
It has been suggested that native homebrew DLL's need to be signed with approved code-signing keys. This is in fact not true! You can use native DLL's on Mango devices, which are not signed at all!
Basically there are two reasons why homebrew apps are not working anymore:
- Interop Lock
- DLL's were built against libraries, which are not supported anymore on Mango
Interop Lock is discussed in this thread. Interop Lock is a new protection mechanism in WP7.5 Mango. Basically it means you can't use apps with ID_CAP_INTEROPSERVICES, unless a device is Interop Unlocked. Without ID_CAP_INTEROPSERVICES an app can't call any drivers. And most homebrew apps call these drivers directly or indirectly. So if an app uses the Interop Capability, it can only run on devices that are Interop Unlocked. If you're going to build an app that uses this capability on Mango, you'll have to give your users instructions on how to apply Interop Unlock on their device.
Most of the native code libraries that were used on NoDo, were based on a hand full of projects. These projects were created and then extended for their own needs by other developers. The result was that most of these projects had the same project-types and library-references. In Mango, a lot of DLL's that were not used anymore by Microsoft, have been removed from the OS. Mostly in the ShellCore. The DLL's were meant for MFC-type functionality, which was never even supported on WP7. Actually, these DLL's are not even used by the homebrew apps either, but there are references to these DLL's in the homebrew libraries, which will cause the library to fail loading into memory. You can see this behavior when you try to run an app with non-Mango-compatible native code on an Interop Unlocked device from within the Visual Studio 2010 development environment. When the COM-class is instantiated it will throw an COMException: "COM object with CLSID '{...}' cannot be created due to the following error: The request is not supported." This is errorcode 0x80070032. This exception is actually caused due to the fact that the previous call to RegisterComDll() failed. If you get the returnvalue of that function you should have 0. In this case the return-value is probably 0x8007007E, which is "Module Not Found". This actually means that you directly or indirectly refer to a DLL, which cannot be found on the device. To fix this we need to create a clean project and add our new or existing native code to that project.
Here are the steps to setup your development environment and create a new, clean project for your native code. Please keep in mind that this guide is still work-in-progress. I may add more detailed instructions and examples later on, when people ask for it.
Update 2011/10/15: Some improvements in the guide, based on comments of rudelm and GoodDayToDie.
- Install Visual Studio 2008 with latest service pack and hotfixes. Make sure you install C++. You need Visual Studio 2008, because the necessary SDK does not support Visual Studio 2010.
- Install Windows Mobile 6 Professional SDK Refresh.
- Install Visual Studio 2010 with latest service pack and hotfixes. You need this to create your Windows Phone Silverlight app.
- Install Windows Phone SDK 7.1.
- Download the attached Microsoft.Phone.InteropServices.zip. After you downloaded the zip-file, open the file-properties and make sure the file is "unblocked" (Windows will block downloaded files). Some unzippers, including the built-in unzipper from Windows will mark the unzipped files as "blocked", which would give problems later on if you don't unblock first.
- If your developmachine is 32-bit you go to "C:\Program Files\Reference Assemblies\Microsoft\Framework\Silverlight\v4.0\Profile\WindowsPhone71" or if you have a 64-bit machine you go to "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\Silverlight\v4.0\Profile\WindowsPhone71". Extract the DLL from the zip-file in this folder.
- Open the Visual Studio Commandprompt and change directory to the folder where you just extracted the DLL. Then enter this command:
Code:SN -Vr Microsoft.Phone.InteropServices.dll
- In the same folder there is a subfolder called "RedistList". Open that folder and open the file "FrameworkList.xml". Add this line to that file:
Code:<File AssemblyName="Microsoft.Phone.InteropServices" Version="7.0.0.0" Culture="neutral" ProcessorArchitecture="MSIL" InGac="false" />
- Install the latest version of Zune.
- Open Visual Studio 2008 and create a new project.
- Choose Visual C++ / Smart Device / ATL Smart Device Project and fill in a name and location for your native library. Do NOT choose MFC, or your library won't work on WP7! The name will be the name for the DLL. Later on you will create a COM-class. Choose a different name for your library and for your COM-class!
- In the new wizard click "Next".
- Remove the "Pocket PC 2003" from the Selected SDK list and add "Windows Mobile 6 Pro SDK" to the selected SDK's. Click "Next".
- In "Application Settings" keep everything default and click "Finish".
- Set your configuration to "Release", because you won't be able to debug anyway.
- Go to Project Properties / Configuration Properties / C/C++ / Preprocessor / Preprocessor Definitions and add this: _CE_ALLOW_SINGLE_THREADED_OBJECTS_IN_MTA
- Right-click the project and click "Add" / "Class" and choose "Simple ATL object".
- In the new dialog enter the "Short name" for your COM-class. All other names are filled in automatically. Keep those names default to avoid naming-conflicts. Also make sure the name of your COM-class is different from the name of the library. All other options can are default, so you can click "Finish" now.
- The basic layout for your native project is now ready. Note that you have these files: for your library you have a header-file (.h), a code-file (.cpp) and a COM-definition-file (.idl) and for your COM-class you have a header-file (.h) and a code-file (.cpp). I will refer to these files in the following steps, so make sure you can identify these files.
- The COM-class you have now is based on IDispatch. IDispatch is the COM-interface that supports reflection-like functionality. The COMBridge in WP7 does not support this interface. Instead we should use IUnknown, which is the base-interface for all COM-objects and supports reference-counting.
- In the header file of your COM-class you can see the public inheritance of IDispatchImpl. This is no problem and you can leave it as it is. But you can also see this COM-mapping:
Code:COM_INTERFACE_ENTRY(IDispatch)
- In the IDL file of your library you need to change the inheritance of the COM-class from IDispatch to IUnknown.
- Your native code layout is now ready to add your methods. A method in COM-class should always have HRESULT as return-type. This value should be 0 or positive in case of success (normally use constant S_OK for success). If you have an errorcode which should throw a COMException do a logical OR with 0x80070000 and return that value. If you want to return a variable, you'll to declare that as parameter of your method and decorate it as returnvalue in the IDL-file. The parameter-types are bound by the definition of COM. You can read about the supported COM-datatypes here and here. Study those parameter-types closely, because any mismatch in your managed and unmanaged declarations will make your app crash definitely. You need to add all your methods in 3 different places: in the COM-class code, in the COM-class interface and in the IDL-file. Later on you need to add an exactly matching interface to your managed code. All the declarations have their own specific format and decoration. I will give an example of two different functions for these 3 files. Note that in these examples, the COM-class was named "Native", so the class implementation is called "CNative" and the interface is called "INative". You have to change that if your class has a different name.
- In the COM-class implementation (.cpp-file) add this code:
Code:STDMETHODIMP CNative::TestMethod1() { BOOL result = ::CopyFile(L"\\Windows\\0000_System.Windows.xaml", L"\\Windows\\Test.xaml", TRUE); // This will fail due to insufficient privileges. This is expected behavior to show how errors can be handled. if (result) return S_OK; else return 0x80070000 | ::GetLastError(); } STDMETHODIMP CNative::TestMethod2(BSTR InputString, BSTR* OutputString) { size_t size = 1000; // in chars TCHAR* msg = new TCHAR[size]; wcscpy_s(msg, size, L"\0"); LPWSTR value = new WCHAR[20]; _itow((int)wcslen(InputString), value, 10); wcscat_s(msg, size, L"Length of string is: "); wcscat_s(msg, size, value); *OutputString = SysAllocString(msg); delete[] msg; delete[] value; return S_OK; }
- In the interface of the COM-class (.h-file) add this code immediately after END_COM_MAP():
Code:STDMETHOD(TestMethod1)(); STDMETHOD(TestMethod2)(BSTR InputString, BSTR* OutputString);
- Locate your interface in the IDL-file of the library. This may look a bit weird, because there are a lot of attributes that decorate the empty interface. Add these declarations to your interface (note the decoration of the parameters, read more here):
Code:HRESULT TestMethod1(); HRESULT TestMethod2(BSTR InputString, BSTR* OutputString);
- Now we need to locate two GUID's and copy them in a text-file, because we need these GUID's later on. These GUID's are in the IDL-file. We will call the first GUID "interface-GUID". It is the "uuid" in the tag RIGHT ABOVE the interface-declaration. We will call the second GUID "coclass-GUID". It is the "uuid" in the tag RIGHT ABOVE the coclass-declaration. There also a "uuid" in the tag above the library-declaration, but we don't need that one.
- Open Visual Studio 2010 and create a new project: Visual C# / Silverlight for Windows Phone and choose a project-type, name and location.
- Now go back to your native project in Visual Studio 2008. The compiled result DLL of this project will be used in your Windows Phone app. To make sure you always use the latest version of the native DLL in your Windows Phone app, you can add a Post Build Event to this project. This example assumes you will have a folder with a subfolder for the native solution and a subfolder for the Windows Phone solution. Go to Project Properties / Configuration Properties / Build Events / Post-build Events and add this (change the paths according to the soluton-foilder you will create for your Windows Phone app):
Code:copy "$(TargetPath)" "$(SolutionDir)..\MyApp
- Now build your native project! The compiled DLL should now also be copied to the folder of your Windows Phone app.
- Create a new file called "WPInteropManifest.xml" in the folder of your managed Windows Phone app. Copy this content in the file:
Code:<?xml version="1.0" encoding="UTF-8"?> <Interop> </Interop>
- Switch back to Visual Studio 2010. In the solution explorer click on "Show all files". Your native DLL and the "WPInteropManifest.xml" should be shown now.
- Select the "WPInteropManifest.xml" file and in the file-properties set "Build action" to "Content" and set "Copy" to "Always". You will always need this file in your project, regardless you will be calling drivers or not. If you don't have this file in your project, you won't be able to use your native DLL.
- Select your native DLL and in the file-properties set "Build action" to "Content" and set "Copy" to "Always".
- In the solution explorer, right-click on the project and choose "Add Reference". Then select "Microsoft.Phone.InteropServices".
- Open the "WMAppManifest.xml" file and add this line below the other capabilities:
Code:<Capability Name="ID_CAP_INTEROPSERVICES" />
- Now add a code-file to your project and copy this code into the file. You need the the coclass-GUID and interface-GUID you copied into a text-file earlier and you also need to replace the name of the class and interface to the names you used. Also note that the declaration must be an exact match (order and parameters) with the declaration in the IDL-file, although the IDL-file is differently formatted.
Code:using System.Runtime.InteropServices; [ComImport, ClassInterface(ClassInterfaceType.None), Guid("YOUR-COCLASS-GUID-GOES-HERE")] public class CNative { } [ComImport, Guid("YOUR-INTERFACE-GUID-GOES-HERE"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] public interface INative { void TestMethod1(); [return : MarshalAs(UnmanagedType.BStr)] string TestMethod2([MarshalAs(UnmanagedType.BStr)] string InputString); }
- Now you need to call the native code. You can add this code to the constructor of your Page or to the eventhandler of a button, or anywhere you like. Be sure to replace the DLL-name, interface-name and class-name and use your coclass-GUID. The exception is a well-known error-code and the exception will be casted to a UnauthorizedAccessException, instead of a COMException.
Code:uint retval = Microsoft.Phone.InteropServices.ComBridge.RegisterComDll("WP7Native.dll", new Guid("YOUR-COCLASS-GUID-GOES-HERE")); INative MyNativeCodeInstance = (INative)new CNative(); string result1 = "OK"; try { MyNativeCodeInstance.TestMethod1(); // UnauthorizedAccessException is thrown due to insufficient privileges. This is expected behavior to show how errors can be handled. } catch (Exception ex) { result1 = ex.Message; } string result2 = MyNativeCodeInstance.TestMethod2("Hello, Mango!"); MessageBox.Show(result1 + Environment.NewLine + result2);
- You can now run your project! Be sure that you deploy it to your device. The emulator won't work, because you project uses native ARM code. The emulator runs on x86, so your native DLL won't load in the emulator.
- When you go more advanced, you may need the Marshal-class. For example to copy a native memory-block to a managed byte-array. Be aware that there are actually two "Marshal" classes. There is "Microsoft.Phone.InteropServices.Marshal" and "System.Runtime.InteropServices.Marshal". They both look the same. But be sure you are using "Microsoft.Phone.InteropServices.Marshal", because it will allow you to do a lot more! Most methods in "System.Runtime.InteropServices.Marshal" will throw a MethodAccessException, because they are tagged [SecurityCritical], while the same methods in the other Marshal class will work.
I hope this will help you port your homebrew apps to Mango or create some fresh new homebrew! If you created an app with native code, drop me a line here. Show me your Screen Recorders, Accent Changers and more!
Ciao,
Heathcliff74
Attachments
Last edited: