My work was done initially on a Find 7, but this should eventually be usable on many other devices (I have the Find 5 and N1 in mind for when I return from vacation). Also, this would not have been possible without the work Steven676 did years ago on the Nexus S, which has been used by all AOSP-derivative projects to support the Samsung Aries (Galaxy S) family for quite some time now.
The current state of things is that the patches are solid and work very well for the system side of things, but there is still a bit of work needed on the recovery side of things. This is due to TWRP having an architectural limitation I need to work on - Whether a device uses emulated storage or not is set at compile time, which is a problem if your design requires automatic detection of configuration at run time.
One of the key design goals here was to support both normal and LVM configurations automatically with a single build that detects which configuration is present on a device at run time.
A second key design goal was that the underlying partition table of the device is not touched in any way. Touching the partition table of a mobile device in the field is a fundamentally dangerous operation, as many partitions contain data that is device-unique or will render a device unbootable if altered. Recovery methods that involve DDing partition images to nonstandard partitions is asking for trouble due to typos... There's no protection against a user typoing the name of a critical partition.
Initially, I'm going to dump the contents of an email I wrote to someone giving them documentation on how to integrate LVM into their project. Over time I'll clean up and reorganize this post, including adding some more links. Also, since this email was written, I've added a LOT of comments to each patch explaining what is going on.
For additional documentation, especially a more user-oriented view of things (such as how to set this up if you want to use it with Omni nightlies) - see the Omni nightlies thread on XDA.
So here goes:
How it's implemented - the complete patch set is at:
https://gerrit.omnirom.org/#/q/topic:find7_lvm - Expect this to periodically change as work on this feature continues (Note: All patches required to support nightly builds of Omni have been merged. At this point, all remaining work that I expect is on polishing up TWRP.)
With the rest of this post, I'll talk about each individual patch and what it does.
https://gerrit.omnirom.org/#/c/9273/ - This is a patch against frameworks/base which adds an alternative to storage_list.xml called storage_list_lvm.xml - The frameworks will choose storage_list_lvm.xml instead of storage_list.xml if the property ro.lvm_storage is set to 1 - The device init scripts will set this property if they detect an LVM configuration.
https://gerrit.omnirom.org/#/c/9207/ - This is an Omni-specific patch. Omni builds for both the Find 7 and OnePlus One (also known as find7op) and both share a common device tree. The LVM patches do not apply to the find7op, so we move init.recovery.rc out of the msm8974-common tree - You likely don't have to worry about this unless you also have a -common tree for find7 and find7op
https://gerrit.omnirom.org/#/c/9276/ - Normal Android kernel ramdisks do not include busybox or any form of shell, making it impossible to run shell scripts without /system mounted. Since we need to run a shell script prior to mounting partitions, we need to add busybox to the ramdisk. This patch does that. For legal reasons you may wish to replace busybox with system/core/toolbox and system/core/sh - I have not tried doing so. If you choose to stay with busybox, you will have to provide the busybox source code in order to comply with the GPL.
https://gerrit.omnirom.org/#/c/9205/ - This adds the LVM binary and LVM configuration file to the ramdisks of both normal boot and recovery. This patch does not actually begin doing anything with the binaries, I separated it out from the other patches as a way to keep things organized so I could start working with the binaries when I began this project. The original source code and documentation for the binary is at https://github.com/steven676/android-lvm-mod
One change I made in lvm.conf that differs from the Samsung aries family (galaxysmtd, fascinatemtd, captivatemtd, etc.) is that I changed the filter line to only allow the userdata and sdcard partitions. This prevents LVM's vgscan from accidentally determining another partition is a physical volume, and also prevents users from accidentally running pvcreate on a critical partition.
https://gerrit.omnirom.org/#/c/9206/ - This is where all of the "heavy lifting" is done. I'm going to work on adding more comments to the init scripts and shell scripts to document them tonight and tomorrow, but I'll try to explain things here.
Android's init system is a bit limited in that it's very difficult to have conditional behavior defined in init.rc - which appears to be why Qualcomm loves to use shell scripts called from init. Similarly, much of the LVM magic happens in three shell scripts (which execute at three different phases within the boot sequence).
In the early-init phase, the two "wait" blocks ensure that the underlying block devices are ready before vgscan/vgchange are called. This will probably slow down booting by a few fractions of a second unfortunately.
vgscan will scan the volumes defined in lvm.conf (in this case, only the userdata and sdcard partitions) for LVM physical volumes. If LVM physical volumes are detected and form a proper volume group, vgscan will create appropriate device nodes. With the configuration I'm using, the device node will be /dev/lvpool/userdata - which consists of a single logical volume that merges the sdcard and physical userdata physical volumes (partitions). The configuration of lvm.conf prevents LVM commands (especially pvcreate) from altering partitions we don't want to alter. If someone accidentally tries to, for example, run pvcreate on the system partition, it will give an error indicating that the partition was not part of the filter.
vgchange will activate the physical volumes detected by vgscan
lvm_init.sh will check to see if /dev/lvpool/userdata exists, and copy fstab.qcom.lvm to fstab.qcom, init.fs.rc.lvm to init.fs.rc, and twrp.fstab.lvm to twrp.fstab if it does. If it does not, it selects fstab.qcom.std, etc.
In the "on init" section, the init script exports all environment variables from init.fs.rc, and creates all storage-related directories and symlinks needed for both configurations (except for when they conflict). lvm_symlinks.sh will create directories/symlinks that must be configuration-specific. Just like lvm_init.sh - it decides what to do based on whether /dev/lvpool/userdata exists
In the "on fs" section - we do an SELinux restorecon on /dev/mapper/lvpool-userdata (/dev/lvpool/userdata would probably work here too). If it doesn't exist, this will fail gracefully without causing any issues.
In "on early-boot" - lvm_setprop.sh uses /system/bin/setprop to set ro.lvm_storage to 0 or 1 depending on the detected configuration. The property service is not available until early-boot - so this cannot be in lvm_init.sh or lvm_symlinks.sh This propery is used by the frameworks/base patch above to determine which storage_list to choose.
At the end of the init.qcom.rc, the fuse daemon for emulated storage is added for all configurations. (I could not figure out a good way to make this conditional based on whether LVM was present or not). In a non-LVM configuration, it runs but is harmless - it maps /data/media (which is empty) to /mnt/shell/emulated (which nothing is looking at due to the environment variables and symlinks set in the "on init" section )
You will probably notice that Omni's standard storage configuration is fairly different from ColorOS - this is due to the way KitKat storage works, but it allowed us to get away without using Oppo's ext4 permissions hacks in our kernel (by remapping permissions instead, in a manner similar to how the emulated storage system works) The way we handle our /sdcard partition does interoperate without issues with the ColorOS approach.
https://gerrit.omnirom.org/#/c/9279/ is a patch specifically for TWRP. TWRP currently determines whether to use emulated storage (/sdcard on /data/media) at build time instead of at run time. Until I have time to fix this, the patch here operates as a workaround. It is similar to the behavior of the fuse sdcard daemon in the previous patch - it maps /data/media to /sdcard whether the configuration is actually emulated storage or not. If the device is not using emulated storage (LVM), mapping of /data/media to /sdcard is still mostly harmless. However it does result in undesirable changes to TWRP's user interface. DO NOT USE THIS APPROACH IN PRODUCTION RELEASES. It's a horrible hack. You'll need to figure out how to properly do /data/media handling depending on whether LVM is present or not based on how your own recovery architecture works.