OK, I decided to try attacking this problem from another angle.
Using the HtcUtility driver (HtcRoot project), I can elevate a process to SYSTEM. This essentially eliminates Error 1260 for that process, in my testing so far.
I can call CreateProcess() just fine, so that's my eqivalent of the Samsung driver function discussed earlier. On built-in apps, it works fine (Calc7, Settings3, etc.).
I created a self-signed certificate with the requisite characteristics using MakeCert (comes with the MS SDK, you can get to it easily using the Visual Studio Command Prompt). The command I used:
D:\Programming\Phone\Win32ExeTest>makecert -r -n "CN=GoodDayToDie" -pe -sky 0x86 -eku 18.104.22.168.22.214.171.124.3,126.96.36.199.4.1.3188.8.131.52 -sv GoodDayToDie.pvk GoodDayToDie.cer
I used this key and certificate to create a PFX file (private key + cert):
D:\Programming\Phone\Win32ExeTest>pvk2pfx -pvk GoodDayToDie.pvk -spc GoodDayToDie.cer -pfx GoodDayToDie.pfx
I used the PFX file to sign a very simple native EXE (Win32, not even ATL, does nothing but return a specific error code):
D:\Programming\Phone\Win32ExeTest>signtool sign /v /f GoodDayToDie.pfx Win32ExeTest.exe"
I used the HtcRoot Webserver to upload my signed executable and my certificate to \Windows (probably can put them anywhere, but KISS).
I used the Crypto API to install my cert. I'm not sure if I had to install it to both LOCAL_MACHINE and CURRENT_USER or only CURRENT_USER, but the error code for launching my homebrew EXE didn't change when I tried installing it to LOCAL_MACHINE only. This was a surprise. In retrospect, I could probably have just used registry access.
OK, at this point, here's what goes down:
Start my Silverlight ID_CAP_INTEROPSERVICES test app (custom build of HtcUtilityTest).
App uses HtcRoot project to elevate itself to SYSTEM.
App calls CreateProcess(L"\\Windows\\Win32ExeTest.exe", NULL, NULL, FALSE, 0x0, NULL, NULL, NULL, &psinfo) where psinfo is a PROCESS_INFORMATION struct.
CreateProcess fails, error code from GetLastError() is 1168.
COM function returns this error using return HRESULT_FROM_WIN32(GetLastError()).
Managed code interprets this as an InteropServices.COMException with the error code 0x80070490 and the message "Element not found".
OK, that's a message I haven't seen before (it's not in the MSDN CE7 list of error codes, either). It's probably because my binary isn't in the policy list. Now... how to get it there?
I can call PolicyLoader, but I'm not sure it would do any good if it doesn't have permissions (which seems to be the consesnus of this thread).
I can call PolicyLoader but specify the flag CREATE_SUSPENDED. That way I can use the HtcRoot project to elevate the process to SYSTEM, then "resume" it to let it run.
I can try tweaking the database directly.
However, I'm not sure any of these will work in a simple fashion, because the database files appear to be held exclusively by another process. Using the HtcRoot webserver to try and download one gives the following error:
CreateFile failed for /Windows/Security/accountdb.vol! GetLastError: 32
which is ERROR_SHARING_VIOLATION "The process cannot access the file because it is being used by another process."
OK, *somehow* Heathcliff74 has managed to modify his phone's policies, so I know it's possible. I'm pretty sure he can't have more access to his device than I have to mine, though the mechanism may be very different. Time to figure out how I can do this...
Modify the handle that is open on that file to allow shared access (probably possible with HtcUtility, but could damage things)?
Modify the memory of the process engine itself, either temporrily or such that it gets flushed back to the file (also probably possible, almost certainly a lot harder though)?
Fake an update, or else somehow load my own policy at a time where the database is unlocked (maybe I should look for the programs that are called while the phone is upgrading, see if I can fake out one of them to process a fake "update")?
Use HtcUtility and/or debug APIs to modify the behavior of the policy engine while it's running (lvmod.dll is a ROM Module, though, so maybe something else)?
Modify the policy XML files in \Windows\Security, and hope those changes at some point get merged back into the database?
Do something else that I haven't thought of yet (maybe somebody else has an idea)?
For what it's worth, here's my goal:
Create a persistent "root" app on the phone, one which is not dependent upon any driver DLL that may get patched (including HtcUtility) nor upon interop-unlock (obviously, if you're not already IUed, you'll need to roll back to NoDo to install this, but it would work for the HTC Arrive).
This root app would enable most if not all the advantages of a "fully unlocked" custom ROM. It would re-enable homebrew access that was lost in the recent updates, and keep working in the face of future updates. It would be able to patch itself, so that if it was ever blocked by an update the access could be restored without anything more painful that rolling back the update immediately after installing it (no data loss, no pain).
Obviously, I'd like to make this possible for all phones. For first-gen HTC, I can use HtcUtility via the current version of the HtcRoot project as a springboard to install this persistent app. For first-gen Samsung, Heathcliff74 probably could make this work for you too. For LG (first-gen, at least, don't know about later), Heathcliff74's Root Tools may enable this for you too. For second-gen phones, we'd need to find a springboard exploit to get the high permissions (ideally TCB/SYSTEM) needed for installing, but we'll keep looking. For other phones, the same applies (and there's still hope - go read this thread again, and you'll see that a few months ago we didn't think native code, even using COM, was possible on Mango at all).
So please, let's work together on this.