[webtop] webtop2sd 2.0.1

[Sogarth]

Now introducing: webtop2sd!

Please don't post in this thread if you happen to catch it when it's open - I'd like to keep it clean if possible. Discussion threads:

• Motorola Atrix/Olympus + general
• Motorola Droid Bionic/Targa
• Motorola Photon/Sunfire

Index:

• Main (this post!)
• Changelog
• Screenshots
• Instructions
• Partitioning Instructions
• Known Issues

Major features:

• As the name implies, you can now use your SD card as your Ubuntu disk. No more relying on a wimpy filesystem.
• Since this uses a separate partition, you can still export your filesystem to your desktop.*
• As a native** Android application, no more worrying about how to run a script. Just download the .apk, install, then run.
• Taking it a step further, you can now boot from external storage as well. If you put your phone into USB host mode (such as via the lapdock), you can boot from any external USB drive. As a nifty note, this means that you can actually have multiple Ubuntu partitions lying around for testing purposes. Particularly useful for me! (Not working yet!)
• Allows selection of various source and target partitions (so you can use this to create an Ubuntu partition, then use that new partition as a source for a testing partition if you so choose).
• Now has an uninstall feature (but why would you want to? ).
• Serves as the first of two applications. The other is a Python/Gtk program that will run within webtop to allow for configuration within the proper environment.

* Unfortunately, anything that can mount extfs (e.g. Linux) will still try to mount the Ubuntu filesystem, potentially causing double-mounting issues.
** Mostly native Android application, really, since it's performing several shell calls on the backend. But I'm sure you won't notice.

Other features:

• If you don't have the ability/desire to partition and format your SD card first, webtop2sd can take care of that for you.
• webtop2sd can also:
  • Delete an ubuntu.disk if you have one, freeing up space on your /data.
  • Disable TOMOYO Linux (mandatory access control).
  • Permanently fix /etc/mtab (which tends to become stale over time). Atrix and Photon only.
  • Grant sudo access.
  • Install a modified dock. Atrix and Photon only.

Tested hardware:

• Motorola Atrix
  • AT&T
    • 4.1.57
    • 4.1.83
    • 4.5.91 (1.1.0)
  • SmarTone-Vodafone
    • 4.5.2 (a.k.a. the HKTW build) (0.9.1)

• Motorola Droid Bionic
  • Verizon
    • 5.5.886 (2.0.0)

• Motorola Photon
  • Sprint
    • 4.5.1A (2.0.0)

End User License Agreement:
Due to the amount of time and effort I've put into this project (at least an order of magnitude more than the Ubuntu batch files, as I've been working on it for several months), I've implemented an end user license agreement (EULA) on webtop2sd, although I've tried to make it as sane and readable as possible. I know many of you won't care, and many of you will ignore it, but it still felt like the appropriate thing to do in this situation. So that you can take a look at it prior to downloading:

By using webtop2sd, you agree to the terms of this End User License Agreement (EULA). If you do not agree with this EULA, uninstall and remove all copies of it (you can reject it by hitting Home and killing the application).

You are free:

• to Use - execute this piece of software.

Under the following conditions:

• Noncommercial - You may not use this software for commercial purposes.
• Nondistributive - You may not distribute this software.
• No Derivative Works - You may not disassemble, distribute, alter, transform, or build upon this software.

With the understanding that:

• Waiver - Any of the above conditions can be waived if you get permission from the copyright holder.
• Ownership - Use of this software does not grant ownership of any kind.
• No Warranty/Liability - The author has done his best to write the software, but ultimately, you are responsible for anything that results from usage of the software.
• Nonseverable - If any part of this EULA is deemed invalid, the rest of it still applies.

In case you are wondering, yes, this is patterned after the Creative Commons licenses.​

As a result, xda-developers (specifically, this thread) is the only place you should be downloading this application. Which is conveniently attached below for your downloading pleasure.

Thanks:

• eval-, for pointing out that it's possible to create a device during bootup if the system hasn't done it yet (which allowed for this endeavor to even take place!).
• YellowGTO, for helping to debug the Atrix SmarTone-Vodafone build.
• Brandon15811, for pointing me at the Ubuntu archive.
• tallnerd1985, for lending me a Droid Bionic to port webtop2sd to.
• distaula, for providing some of the details for and testing the Motorola Photon.
• Lokifish Marz, for helping to test the Motorola Photon.

P.S. If you're wondering why there's an icon.zip attached, that's due to GPL requirements. You can safely ignore it, since it's just a svg of the icon.

 

[Sogarth]

Changelog:

• 2.0.1 (October 9, 2011):
  • Tweak webtop-configurator for the Photon's sources.list.

• 2.0.0 (October 8, 2011):
  • Multi-device support.

• 1.1.2 (October 2, 2011):
  • Fix busybox pathing issues.

• 1.1.1 (October 2, 2011): Long overdue bugfix:
  • Use busybox from /system if present.

• 1.1.0 (July 24, 2011): Gingerbread update:
  • If busybox is not present (*cough* Gingerbread *cough*), install it in /system/etc/webtop2sd so that the webtop2sd mount application can run properly.
  • Add a Diagnostics tab. At the moment, it shows this information:
    • webtop2sd version
    • Mount executable version (and corresponding source)
    • Whether or not the webtop2sd busybox is installed
    • The webtop configurator version
    • Currently mounted webtops
  • Add a "Reinstall mount executable" button.
  • Fix the "Install webtop configurator" button and rename it to "Reinstall mount executable".
  • With the increasing number of buttons, landscape mode on the Execution tab is becoming unusable. So, hide buttons when executing (except for one that makes it obvious that it's executing).
  • Expand the target tree in webtop configurator by default to make it more clear what options are available.
  • Swap over to using old-releases.ubuntu.com instead of jp.archive.ubuntu.com.
  • When rotating within webtop2sd, tab state is no longer lost.
  • When installing webtop configurator, don't choke if the /usr/local/share/pixmaps already exists.
• 1.0.0 (July 10, 2011): First "real" release:
  • webtop configurator now bundled.
  • Uninstallation implemented.
  • Workaround added for looping dock segfaults.
  • webtop2sd (the Android application) no longer has a reliance on busybox.
• 0.9.1 (June 29, 2011): Miscellaneous bug fixes:
  • Fixes for SmarTone-Vodafone 4.5.2.
  • Devices below 1 GB are no longer listed for partitioning.
  • If a target partition for synchronization isn't selected, synchronization won't be attempted.
  • The progress bar can now handle device rotation without losing state.
  • There is now a reminder to reboot after synchronization.
• 0.9.0 (June 26, 2011): Implementation of 95% of the final featureset. Partitioning and synchronization are no longer mutually exclusive operations.
  • Dropped features (deemed as a better fit for the companion program):
    • Fix Ubuntu packages
    • Fix APT source lists
    • Restrain APT dependencies
  • Known problems:
    • Boot from sd* not working.
    • Uninstall not working.
    • Companion program not written yet.
• 0.5.0 (June 11, 2011): UI prototype, although the UI should be fully functional (i.e. all menus should show accurate information). Of course, clicking on the "Go!" button doesn't actually do anything....

 

[Sogarth]

Screenshots:
The partitioning tab.
The partitioning tab: device selection. The program will create two partitions on the device: one FAT, one ext3 (for webtop). The FAT partition will be the first, the ext3 partition the second.
The partitioning tab: partition size. Here, you can select how large you want the webtop partition to be.
The locations tab, where you pick source and destination partitions for synchronization.
The locations tab: source selection. Source webtops include the (now deprecated) filesystem files. External storage partitions are also valid too. If the source is a filesystem file, it can be automatically deleted upon successful completion of synchronization.
The locations tab: target selection. Destination partitions can include partitions generated as a result of partitioning.

 

[Sogarth]

More Screenshots:
The advanced tab. This is reached via the "Toggle Advanced tab" option on the menu. The specific options are discussed in the instructions.
The diagnostics tab. This is reached via the "Toggle Diagnostics tab" option on the menu. Information shown here can be used to determine the state of webtop2sd.
The execution tab. Several different actions can be performed from here.
The execution tab: execution. A progress bar is shown for the synchronization task, to show an approximation of how much is left.

Requirements:

• A supported device with webtop (e.g. a Motorola Atrix/Olympus, a Photon/Sunfire, a Droid Bionic/Targa).
• Some type of external storage. At the moment, webtop2sd can only boot off a microSD card, though, so that's probably your best option.
• At least a partially functional webtop. The following executables are called:
  • /bin/umount (because it can unmount by device)
  • /sbin/mkfs.ext3
  • /sbin/blockdev
  • /usr/bin/dpkg-deb
  • /usr/bin/file

Instructions:
Note: webtop2sd scans devices and partitions upon startup, so if you change out SD cards and/or USB devices once it's launched, you'll need to forcibly close the application and re-launch it.

1. You will need a partitioned device, so that's the first tab shown, Partitioning. Performing partitioning via webtop2sd is destructive, so it will warn you appropriately. Non-destructive partitioning can be performed, and those instructions are below. If you still want to partition your device, follow these instructions:
   1. Enable partitioning external storage by turning on the checkbox for "Partition ext. storage".
   2. Select the target device by hitting "Device to partition". MicroSD cards are appropriately labelled as "SD card", and correspond to the mmcblk1 device. External storage is labelled as such, and correspond to sd* devices. The size of the device is marked in parentheses. Any device you have selected should be unmounted from Android first (although webtop2sd will detect if you've selected the microSD card, and send you to the appropriate Settings page - I haven't figured out the exact syntax to send users to the Blur device ejection dialog box).
   3. Once you've selected a device, hit the "Webtop partition size" option to determine how large the webtop partition should be. The minimim size is 1 GB (1,024 MB).
   4. Advanced options:
      • Format new partitions: If you turn this off, the partitions won't be automatically formatted after partitioning, but this will cause problems with synchronization if you have that enabled.
2. Once partitioning is set up, webtop synchronization can be configured in the Locations tab.
   1. The "Generate a webtop" box is checked by default.
   2. Select a source partition via the "Source webtop" option. It should always show "Internal flash" (mmcblk0p13) as an option unless something is horribly, horribly wrong. The "Filesystem file" (/data/ubuntu.disk) option will be shown if you installed a previous custom Ubuntu system. Any other extfs partitions labelled "osh" will be shown as valid source partitions as well, unless partitioning is active and the partition is on the device to be partitioned.
   3. Similarly, select a target partition. All extfs partitions are shown as valid target partitions, unless partitioning is active and the partition is on the device to be partitioned, or that partition has already been selected as the source partition. If partitioning is active, however, an entry is automatically entered.
   4. If the "Filesystem file" option is selected as a source partition, it can be automatically deleted on a successful synchronization, so that the space occupied on /data can be freed up.
   5. Advanced options:
      • Replace mount executable: An alternate mount executable is required if you want to boot a custom webtop. However, if you've tweaked the one you already have to work the way you want, you can uncheck this box.
      • Install busybox if needed: The alternate mount executable needs certain Unix commands to run properly. These were provided via busybox in 2.2/Froyo, but were removed in 2.3/Gingerbread, so webtop2sd is able to reinstall busybox in /system/etc/webtop2sd if needed.
      • Disable TOMOYO Linux: Highly recommended, as TOMOYO Linux mandatory access control can cause many problems down the road. In theory, it's not strictly required, as webtop2sd shouldn't trigger any changes that would cause TOMOYO to be unhappy, but it allows for the user to easily perform actions that would do so. So rather than making TOMOYO unhappy, it's preferable to effectively disable it (desu wa). But, if your TOMOYO configuration files (domain_policy.conf and exception_policy.conf) are already set up, you can uncheck thix box.
      • Fix /etc/mtab: This addresses a personal pet peeve of mine. /etc/mtab, which is supposed to keep track of the list of current mounts, doesn't get cleaned up properly, so can end up with a number of dead entries (seen when running "mount" to see a list of current mounts). It doesn't actually break anything per se, but it's ugly. Replacing it with a symlink to /proc/mounts fixes everything. Atrix and Photon only.
      • Missing dock workaround: If the AWN dock doesn't show up when you boot into your SD webtop, run webtop2sd with this enabled (it's disabled by default). Atrix and Photon only.
      • Grant sudo access: Moving webtop to external storage doesn't accomplish much if it can't be subsequently configured and improved upon. This can be difficult without some easy form of root access, and granting "sudo" access to the "%admin" group (which includes the webtop user, "adas") does exactly that. However, if your sudoers file is already set up exactly the way you want, you can uncheck this box.
      • Install modified dock: Similar to the above, moving webtop to external storage doesn't accomplish much if you have no additional options. So, if you're coming from the stock webtop installation, a modified dock can be installed to give you some more functionality to start with. Atrix and Photon only.
3. Once all options have been configured, you can head to the Execute tab and hit the "Go!" button. You can keep track of status via this tab. An "Executing..." item is added to the Ongoing Notifications section so that Android won't automatically kill webtop2sd.
   • If you're finding that the partitioning and formatting appear to be taking an inordinately long time, you can perform that on your computer (as described below) and then have your device only perform the synchronization.
4. After the program completes, reboot your device to use your new webtop.
5. Once you've rebooted, run the webtop configurator. Atrix and Photon only. It tends to be slow because it does a lot of network operations without revealing what it's doing.
   • When you first launch it, you'll probably see a screen where it says it needs to update the APT sources list. This can take a while.
   • At the moment, only two programs are supported (both under APT/Administration): LXTerminal and Synaptic.

 

[Sogarth]

Partition Instructions:
Depending upon your operating system, free software may be available so that you can partition and format your device without losing all the data on it. When you partition it, I recommend that you have your FAT32 partition first, and any extfs partitions later. I would also recommend using ext3 as your extfs filesystem type (the Olympus kernel doesn't support ext4, so any potential benefits it could offer are lost).

• Linux: gparted (you probably already have this installed)
• Mac OS X: ?
• Windows: MiniTool Partition Wizard

Warning: If you're worried about the number of write cycles on your SD card, you're probably best off copying all the data off, partitioning, then copying all the data back, as repartitioning tends to hit your SD card pretty heavily. That said, I personally am not worried, but it's your respective calls on this one.

Instructions: Windows

Those of you using Linux probably have a pretty good idea of how to use gparted, so I'll skip those for now. Here's a quick runthrough for Windows on how to partition your SD card. You'll need an SD card reader, of course.

• If you take a look at your SD card in Computer Management/Disk Management, you'll have doubtless noticed that you don't have the traditional partitioning options available there that are there for "proper" hard drives.
  
  So we'll need to use a different tool - in this case, the MiniTool Partition Wizard mentioned above.
• This is what my MiniTool Partition Wizard screen looks like when I have an 8 GB SD card inserted. As you can see, there's currently one single FAT32 partition present.
  
• When you right click on the partition, you'll see a Move/Resize option. That's the one you want to select for non-destructive partitioning.
  
• At that point, you can select how large you want your partition to be. You want to make sure that the "Unallocated Space Before" stays at 0.00 MB, so that the new partition will be the second one. In my case, I've opted to create a 4 GB partition as the second partition.
  
• Once that has been set up, right click on the Unallocated space, then select Create.
  
• We're not worried about the warning that Windows throws up, particularly since we know we'll be creating a partition that Windows can't read anyways. So just hit Yes.
  
• Create the second partition at the maximum size with the following settings:
  • Create As: Primary
  • File System: Ext3
  
• After that's done, click on the Apply Changes button.
  
• And now you're done! Put the card back into your Motorola Olympus.

 

[Sogarth]

Known webtop2sd Issues:
Major:

• On some devices, the AWN dock can go into an infinite segfault loop. (Partially fixed in 1.0.0, "proper" fix pending?)
• The companion Python/Gtk program doesn't exist yet. 1.0.0
• The "Install webtop configurator" button does nothing. 1.1.0
• webtop2sd is unable to boot from an SD card without busybox. 1.1.0

Moderate:

• External storage other than the microSD card can't be used as a boot device.
• Detection of root privileges doesn't work properly.
• busybox is only ever detected from /system/etc/webtop2sd. 1.1.1/1.1.2
• Uninstallation isn't implemented. 1.0.0
• It's not always easy to find out which webtop partitions are currently being used. 1.1.0

Minor:

• Localization? Should be easy, since everything is stored in appropriate resource files....
• You can selected devices to partition that are smaller than 1 GB. 0.9.1
• If no target partition is selected (i.e. the option is greyed out), you can still try to generate a new webtop partition. 0.9.1

Known webtop configurator Issues:
Major:

• What the hell is webtop configurator doing?! Status is a must.

Moderate:

• jp.archive.ubuntu.com isn't an ideal mirror to pull from. old-releases.ubuntu.com is a better choice. 1.1.0

Minor:

• Having the selection tree on the left fully expanded on boot would be nice, so that it's immediately obvious what options are available. 1.1.0