Make kaiser life easier: UC, SD-install and provisioning app and repository

[qmdm]

Development of User Customization/Provisioning App

In setting up my Kaiser, I want to use User Customization, to auto-install cabs and provisioning files for my personal settings. And to save precious device memory, non-essential apps are installed to the Storage Device, and only critical or continually running apps are stored in main memory. I believe this is a common setup.

Have other people found that getting this kind of setup is not trivial? Retargeting cabs for auto-installation to the storage card, and testing if this actually works for that application is a non-trivial task given the number of cabs available, and these efforts don't seem to be shared much between users. I've seen just a handful of SD-retargeted apps at xda-developers.

Keeping up with all the latest updates and tweaks also takes considerable effort and searching. The excellent chefs do a truly fantastic job with cooking ROMs with the latest apps built in, but burning a whole new ROM to get the latest update is inconvenient, particularly if your settings are not yet set up for easy restore via provisioning. The HTC cab collections by Dutty and others make this process easier by putting it all in one place, but you still have to download each cab install it and then try to keep track of which version you last installed, and of course this doesn’t address all the non-HTC apps that you might have on your device.

Building provisioning files for your persinalized settings is challenging to say the least, as the documentation for provisioning service configuration providers is sparse and doesn’t cover all the features. Making these provisioning files by hand can be involved (e.g. see the description for getting the Owner ID details into an XML file in the main UC thread.)

These are the main issues I’ve experienced trying to set up my Kaiser. Configuring a device with UC, apps installed to the SD and with provisonined settings and keeping the whole updated is a very time consuming task, and requires several tools to accomplish (WceCabManager, RapiConfig, SDConfig Tool etc..). Have others experienced this? If not, I’d like to hear what solutions people have used.

If this is a problem for others, then I would like to start collaboration on an application that can help simplify using UC and SD deployment and creating provisioning files as well as keeping installed applications up to date, and sharing knowledge about the applications.

The key features of the application would be:
• Runs on the PC. Maintains a local repository of CABs and provisioning files. User adds apps and settings to the repository either manually, or by downloading from a repository.
• Provisioning templates – similar to the XML file for KaiserTweak – describes provisioning characteristics that are available for tweaking or querying from the device. Querying settings from the device allows you to configure the device as normal (using the device UI) and then “suck” those settings out of the device into a provisioning file. You can then create a new provisioning file from the provisioning template for deployment to your device. (Anyone who's set up Wifi via hand-written provisioning xml knows what a time-saver this is!)
• metadata for CABs and provisioning templates in remote repositories: e.g. versions available, edition of CAB to install to SD, does installation require user input?, known issues for each version, etc.
• CABs and settings can be bundled into a Recipe for easier management. (E.g. all HTC apps.) Provisioning templates for the application can be bundled in the recipe, so users can install app and get a list of available registry tweaks for that app.
• Connection to a central repository for downloading updates to items in the local repository and discovering new CABs, templates etc.
• User builds their own local recipies containing cabs, provisioning files and other pre-cooked recipes to configure their device with apps and settings as desired.
• Deploy now: deploy selected items device now. (Given that a recipe can contain other recipes and cabs and provisioning files this can save a lot of work manually copying and running these on the device.)
• User customization: Build SDConfig.txt from selected items, and sync all files needed to SD.
• The features above are intended to be used by regular users. There will be tools for recipe concocters (advanced mode):
o Tool to retarget CAB to different installation path, or to indicate in metadata that this does not work.
o Build new Provisioning Templates
o Putting cabs and settings into a Recipe
o Updates to the repository

While there are already tools that can do bits and pieces of this, a turnkey solution would be much easier to use for novice users. The repository metadata allows valuable knowledge to be shared, e.g. versions available, SD-install, installer requires input, known issues etc. and I believe is an important feature that will make provisioning and keeping devices up to date with the latest CABs and registry tweaks much simpler.

In terms of development, I imagine the app will initially be a stringing together of existing tools, with tighter integration implemented as necessary in future revisions.

Does this seem like the right direction? Is it a problem worth solving? Let me know your thoughts. I'd like to kick this off on SourceForge if it resonates with others.

[met3ora]

I hope you've done a search, because there already is a program that builds SDConfig.txt's for you. It's actually linked to in Sleuth's thread about UC.

[qmdm]

I quick one before I take lunch!

I've done a search (in fact I've been reading and searching the forum/wiki for weeks, with great success in some areas and limited success in others.) SDConfig is a great tool, (I reference it in my post about one of the tools that we might bundle together in the app) but it only paints one part of the picture (Making SDConfig.txt). For a complete solution to SD-installed apps and convenient privisioning, we need more. Much of the solution (apart from the central repo) is available amongst many assorted apps, and that's the problem - users have to grab apps from here and there, figure out how to use them together, and then the work they do is just available on their device. There is no saving of that work for others to share and contribute.

So, when someone makes a version of a CAB that is customized for SD install, it would be great if that can be saved to a repository so other's don't have to replicate that work. Or if the user found the cab didn't work from SD the metadata in the repo indicates that this cab does not work from SD. When using a cab with SDConfig, we need to know if it requires user interaction. Again metadata in a repo can specify this to save each user having to make a note of this when building their UC config.

The repo also makes checking for updates an automatic process.

Finally, anyone that has worked with privisioing files knows what a pain they can be to create and manage. There are many useful tweaks already documented, so provisioning files for these would be useful, to reapply tweaks after flashing. And keeping track of all the tweaks made to a device during normal use and adding them to a provisioning file to making them available after flasing takes much committment. A orovisioning template can ease this task by documenting user-editable parts of the registry and fetching changes from the device.

In a nutshell, I'm looking at creating an app that brings the features from the other apps together, and adds time saving features, and collaboration via a repository. This is motivated by the amount of work it takes to get the common configuration of SD-installed apps and provisioning tailored to an individuals needs, and in keeping with updates to installed applications.

I hope this is clarifying!

[qmdm]

Very quiet here!

I'd really like a response, so let's take an example in detail.

How do you install GoogleMaps to your storage card?

Here's how I did it:
1. download googlemaps.cab to the PC
2. install googlemaps - when prompted for the install location, choose "Storage Device".
3. Let google maps install.
4. Disspointment that on finding that even after selecting install to storage device, googlemaps is installed to \Program Files and not \Storage Card\Program Files.
5. Download and run WinCeCabManager
6. Open the googlemaps cab in WinCeCabManager
7. Because the CAB does not use %InstallDir% to reference the default installation directory, we much change all file locations, shortcut targets and those registry values that refer to installed files.
8. Save the cab and deploy it to the device.

Step 7 - retarteting all the files, shortcuts and applicable registry values is a real pain. Changing the default installation folder is not enough, because this information is not passed on to the individual files and registry entries.

Does anyone have a quicker solution? Surely most people think it's inconvenient to retarget apps to the SD like this?

This is just one piece of the puzzle, but an app that retargets CABs to the storage device would save much time and energy. E.g. something like
1. right click on CAB on PC. Choose "Install to Device"
2. A dialog offers option to install to storage device, and other install details (remove after install, no UI etc...)
3. Hit OK, and the CAB is retargeted to the chosen location and installed to the device.

I can write this app, but I would like some confirmation that it's the right strategy - I've owned my kaiser for about a month and hope this isn't re-inventing the wheel or that there is already a simple solution available to all this inconvenience with cab installation to SD.

Your feedback appreciated!

[qmdm]

I had written some basic application to provide auto installation and extraction of provisioning settings. But progress is slower than ideal due to other commitments, so I will probably abandon this effort.

There's an auto installer here that is much along the lines of what I was thinking.

http://forum.xda-developers.com/showthread.php?t=392983

[l3v5y]

I hadn't seen this thread before now, but you seem to be going to a lot of effort to find ways of re targeting CAB installers. The easiest way of setting up cabs is install them manually to the Storage Card and then manually get any registry settings/links to files from the device and install those via UC. This saves remaking cabs, and saves time when UC is running as you only need copy the lnks to the device and install an XML file with registry tweaks.

[qmdm]

I hadn't seen this thread before now, but you seem to be going to a lot of effort to find ways of re targeting CAB installers. The easiest way of setting up cabs is install them manually to the Storage Card and then manually get any registry settings/links to files from the device and install those via UC. This saves remaking cabs, and saves time when UC is running as you only need copy the lnks to the device and install an XML file with registry tweaks.
Thanks for your comment. I'm puzzled why you think my approach takes a lot of effort - once the application is written, then it works for all cabs, and it's not really a complex app. (The retargeting part is done.)

My feeling is that manually doing anything with CABs is a PITA when new versions are being released almost continually. Why do anything manually when the device can do it for you?

My vision for the app is more than just retargeting cabs, it would manage applications, track new versions with updates, manage XML provisioning files (updating the XML files when you make changes on the device.) So you can set up the device, and have a zero-effort process to save all settings and apps, and restore those settings to a new ROM.

[l3v5y]

Thanks for your comment. I'm puzzled why you think my approach takes a lot of effort - once the application is written, then it works for all cabs, and it's not really a complex app. (The retargeting part is done.)

My feeling is that manually doing anything with CABs is a PITA when new versions are being released almost continually. Why do anything manually when the device can do it for you?

My vision for the app is more than just retargeting cabs, it would manage applications, track new versions with updates, manage XML provisioning files (updating the XML files when you make changes on the device.) So you can set up the device, and have a zero-effort process to save all settings and apps, and restore those settings to a new ROM.
I guess it's partly to do with the fact that what I install changes alot from ROM to ROM, so I don't like to have things already installed. My point is, though, that if you install the cab onto the storage card manually before hard reseting/flashing a ROM, you don't have to do that again until an update is released, this saves reinstalling apps to the storage card each time a new ROM is flashed.

[qmdm]

Ok, I see, that's certainly one way of saving time.

My initial impression was that manual tweaking for each app would be inconvenient and time-consuming. Obviously, you are happy with the manual approach, and that's the kind of feedback I was looking for!

If anyone else has thoughts on manually tweaking appsvs an automated solution, I'd be happy to hear them!

Cheers.