Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,781,073 Members 45,202 Now Online
XDA Developers Android and Mobile Development Forum

[HOWTO] "Partitioning" your Nexus S using LVM

Tip us?
 
steven676
Old
(Last edited by steven676; 16th May 2012 at 10:50 AM.)
#1  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
Default [HOWTO] "Partitioning" your Nexus S using LVM

Always wanted a larger /data partition? Want to add a swap partition or an extra partition for your Debian/Ubuntu/Fedora chroot? Want your ROM to have the combined /data and /sdcard of Honeycomb and ICS devices? Then this guide is for you.

Table of contents
The Linux Logical Volume Manager (LVM) is a set of tools which allows you to create virtual disks ("logical volumes", "LVs") backed by a set of real storage devices ("physical volumes", "PVs") organized into "volume groups" ("VGs"). Using LVM, you can:
  • Create logical volumes which are larger than any of the individual physical volumes available on your device's real storage.
  • Create an arbitrary number of logical volumes of arbitrary size, regardless of the number of physical volumes or partitions available on your real storage devices. You could, for example, add a dedicated swap partition, or create a separate /system and /data for a dual-boot setup.
  • Take a snapshot of a logical volume -- a copy of the LV which only stores differences between it and the original (requires additional kernel support). You could, for example, take a snapshot of the external storage LV and export it via USB mass storage to a computer, without needing to unmount the LV from your device.
  • Grow or shrink logical volumes without unmounting the file system ("online resize", requires filesystem support).

LVM was developed to manage storage on Linux servers and workstations, but was also used by Palm/HP in WebOS devices. It uses the Linux device-mapper technology used in Android for apps installed on the SD card, so any Android kernel will work with LVM.

This guide was developed with the Nexus S in mind, but the technique described should be applicable to many other Android devices as well.
The Following 5 Users Say Thank You to steven676 For This Useful Post: [ Click to Expand ]
 
steven676
Old
(Last edited by steven676; 11th June 2012 at 10:05 AM.) Reason: update attached boot/recovery images with LVM 2.02.96
#2  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
Modifying your phone to use LVM

WARNING WARNING WARNING

While unlikely, you could brick your phone if you do this incorrectly!!!

Read this guide completely before attempting. If you don't understand the procedure, read it again until you do. If you still don't understand, wait for someone else to make this easier. If you're not sure you understand, you probably don't.

BACK UP EVERYTHING ON YOUR PHONE, INCLUDING THE CONTENTS OF /sdcard, BEFORE ATTEMPTING THIS PROCEDURE! This will erase everything on your phone.

The preassembled images attached to this post are for Nexus S GT-i9020T running stock Android 4.0.4 IMM76D ONLY. They may or may not work with other phones and ROMs. (See the next post for how to modify any ROM to use LVM.)

You will need:
  • A Nexus S with unlocked bootloader
  • A computer with fastboot and adb installed and working
  • A USB cable
  • Factory images for your phone, in case something goes wrong

Procedure:
  1. Take a nandroid backup of your phone and copy it to your computer. We'll use it to restore the ROM later. You MUST copy the backup to your computer! This procedure will erase EVERYTHING on your phone, including the contents of /sdcard.
  2. Download the boot and recovery images attached to this post. Reboot your phone into the bootloader, then flash the recovery to your phone using fastboot:
    Code:
    fastboot flash recovery recovery-lvm-clockwork-5.0.2.0-crespo.img
    (GPL information: this is the stock kernel image from ClockworkMod 5.0.2.0. For LVM source, see this post.)
  3. Boot into recovery.
  4. Enter the shell on the device:
    Code:
    adb shell
    In the shell on the device, do the following (# is your prompt):
    Code:
    # /lvm/sbin/lvm pvcreate /dev/block/mmcblk0p1 /dev/block/mmcblk0p2 /dev/block/mmcblk0p3
    # /lvm/sbin/lvm vgcreate lvpool /dev/block/mmcblk0p1 /dev/block/mmcblk0p2 /dev/block/mmcblk0p3
    This marks the partitions normally used for /system, /data, and /sdcard for use with LVM, and creates an LVM volume group named "lvpool" using those devices.
  5. In the shell on the device, set up your desired logical volumes (partitions).
    Code:
    # /lvm/sbin/lvm lvcreate -L 320M -n system lvpool
    # /lvm/sbin/lvm lvcreate -L 1G -n userdata lvpool
    # /lvm/sbin/lvm lvcreate -L 10G -n media lvpool
    [...]
    This creates the mandatory logical volumes named "system", "userdata", and "media", which will be mounted on /system, /data, and /sdcard, respectively. Replace the argument to -L with the desired size of each partition. (/system needs to be at least 300 MB to fit the stock ROM.) You can create additional volumes similarly:
    Code:
    # /lvm/sbin/lvm lvcreate -L [size] -n [name] lvpool
    replacing [size] with the desired size and [name] with the desired name. You have a little less than 15 GB space in the volume group to work with; depending on how many partitions you want to create, you can adjust the size of the "media" and "userdata" logical volumes as required.
  6. Reboot back into recovery. Mount the USB storage on your computer. On your computer, format your phone's USB storage as you would a USB flash drive.
  7. Copy your nandroid backup from your computer onto your phone.
  8. Reboot back into recovery. Restore the nandroid backup.
  9. Reboot into the bootloader. Flash the boot image to your phone via fastboot:
    Code:
    fastboot flash boot boot-lvm-crespo-IMM76D.img
    (GPL information: this is the stock kernel image for IMM76D. Source is available from AOSP git. For LVM source, see this post.)
  10. Reboot. You should have a working device running on LVM.

Additional tips and hints
  • To see the logical volumes you've created, use
    Code:
    # /lvm/sbin/lvm lvs
    and to see how much free space you have remaining in the volume group, use
    Code:
    # /lvm/sbin/lvm vgs
  • If you want to delete a logical volume (for example, if you screwed up), use
    Code:
    # /lvm/sbin/lvm lvremove /dev/lvpool/name
    (replacing "name" with the name of the LV you want to delete).
  • It's often useful to leave a bit of free space in the volume group (512 MB to 1 GB or so). This gives you a bit more flexibility to modify the layout or grow partitions in the future, and allows you to take snapshots of logical volumes.

Removing LVM from your phone
  1. Flash a standard recovery such as the regular ClockworkMod recovery to your phone.
  2. Do a full wipe/factory reset.
  3. Restore your phone from a backup.

If things go really wrong, flash the factory image to your phone via fastboot.
The Following User Says Thank You to steven676 For This Useful Post: [ Click to Expand ]
 
steven676
Old
(Last edited by steven676; 16th May 2012 at 11:05 AM.)
#3  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
Modifying an existing ROM to use LVM

(This is a condensed version of the full guide: https://raw.github.com/steven676/and...ster/HOWTO-MOD)

You will first want to grab a copy of the binaries and config files by cloning my git repository: https://github.com/steven676/android-lvm-mod.

Unpack the ramdisks from your boot and recovery images. Make the following changes:
  • Copy the contents of lvm-bin/ in the git repository to lvm/ in the ramdisks.
  • Copy devices/crespo/lvm.conf to lvm/etc/lvm.conf in the ramdisks.
  • For the boot image, integrate the changes in devices/crespo/boot/init.herring.rc-changes.diff into your init.herring.rc.
  • For the recovery image, integrate the changes in devices/crespo/recovery/init.rc-changes.diff into your init.rc.
  • For the recovery image, copy devices/crespo/recovery/recovery.fstab to etc/recovery.fstab.
Repack the ramdisks and create new boot and recovery images with them (the original kernels are fine to use).

If creating a flashable zip to install your ROM, you will also want to modify the installer script to set up the LVM volume group and logical volumes (see the previous post).

If you're building your own kernel, consider enabling CONFIG_DM_SNAPSHOT in your kernel's build configuration, to allow taking snapshots of logical volumes.

Remember that LVM is released under the terms of the GPL v2 (same as the Linux kernel), so you are required to comply with the GPL's source distribution requirements with respect to LVM.

More extensive modifications (such as changing the devices used to form the volume group) are possible; see the full HOWTO guide for details.
The Following 2 Users Say Thank You to steven676 For This Useful Post: [ Click to Expand ]
 
steven676
Old
(Last edited by steven676; 20th January 2013 at 11:27 AM.)
#4  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
Building LVM from source for Android

The LVM source was written for standard glibc-based GNU/Linux systems, and doesn't build agains Android libc. We could invest time and effort into porting it to Android, but it's easier to just statically link the LVM binary against glibc (we already have to use a statically linked binary, since the boot image doesn't ship shared libraries).

To build LVM with glibc, you will need either an actual GNU/Linux ARM system with a development environment (gcc, make, and glibc header files) set up, or a GNU/Linux ARM cross-compiler toolchain (more difficult to set up). The binaries in lvm-bin/ were compiled with an Emdebian squeeze/armel toolchain running on a Debian squeeze/amd64 machine; this procedure has also been tested with a Nexus S running a Debian squeeze chroot (though any recent Linux distribution and any device should do). The Linaro prebuilt GCC toolchain and the CodeSourcery CodeBench Lite toolchain for ARM GNU/Linux should work; the Android NDK toolchain will not.

Getting the source

Clone the lvm-mod git repository. Then, in the newly cloned repository, do
Code:
git submodule init
git submodule update
You will have a copy of the LVM source (as used to build the binaries in lvm-bin/) in lvm-src/.
(Alternately, clone https://github.com/steven676/lvm2 and checkout branch android-lvm-mod. You can also use upstream's git repository or source tarballs; see their website for details.)

Building the source

(This is a condensed version of the full guide: https://raw.github.com/steven676/and...er/HOWTO-BUILD)

These instructions are written for a native build (building in a Debian/Ubuntu/Fedora chroot on an actual ARM device).
  1. Configure the LVM source:
    Code:
    $ ./configure --prefix=/lvm --enable-static_link --disable-readline \
          --disable-selinux --with-pool=none --with-cluster=none \
          --with-confdir=/lvm/etc --with-default-run-dir=/data/lvm/run \
          --with-default-system-dir=/lvm/etc \
          --with-default-locking-dir=/data/lvm/lock \
          --with-optimisation="-Os -march=armv5te -mtune=cortex-a8 -mthumb"
    For a cross build, you need to set up the system to run ARM binaries using QEMU (see the full HOWTO for details). (For extra credit, fix the build so that it works with the configuration produced by running configure without qemu-arm enabled.) Then configure the source with:
    Code:
    $ export CC=arm-linux-gnueabi-gcc	# replace with your cross compiler
    $ ./configure --host=arm-linux-gnueabi \
          --prefix=/lvm --enable-static_link --disable-readline \
          --disable-selinux --with-pool=none --with-cluster=none \
          --with-confdir=/lvm/etc --with-default-run-dir=/data/lvm/run \
          --with-default-system-dir=/lvm/etc \
          --with-default-locking-dir=/data/lvm/lock \
          --with-optimisation="-Os -march=armv5te -mtune=cortex-a8 -mthumb"
  2. Build the source:
    Code:
    $ make
  3. Grab the pieces you need: the statically linked, unstripped lvm binary is located in tools/lvm.static, and a sample configuration file is in doc/example.conf.
 
steven676
Old
#5  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
(reserved)
 
solarispika
Old
#6  
Junior Member
Thanks Meter 0
Posts: 2
Join Date: Apr 2011
Hello steven676,
This post is helpful to me!
But I have some questions on LVM as I am new to it.

1. Is it possible to use LVM only on one pv, say /sdcard?
2. If 1. yes, will lv's on a pv be linear mapped, that is, lv's in a pv are placed one after one?
3. now that removing lvm is possible, is it also possible to backup lvm setting, and switching between non-lvm and lvm environment?

Thanks
 
steven676
Old
#7  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
Quote:
Originally Posted by solarispika View Post
1. Is it possible to use LVM only on one pv, say /sdcard?
Sure. Just modify the arguments to pvcreate and vgcreate accordingly:

Code:
# /lvm/sbin/lvm pvcreate /dev/block/mmcblk0p3
# /lvm/sbin/lvm vgcreate lvpool /dev/block/mmcblk0p3
Quote:
Originally Posted by solarispika View Post
2. If 1. yes, will lv's on a pv be linear mapped, that is, lv's in a pv are placed one after one?
Probably, but in general it is not safe to assume this. Imagine the following sequence of commands:

Code:
# /lvm/sbin/lvm lvcreate -L 1G -n vola lvpool
# /lvm/sbin/lvm lvcreate -L 1G -n volb lvpool
# /lvm/sbin/lvm lvresize -L +1G /dev/lvpool/vola
What's likely to happen here is that vola is going to be split into two segments on disk, like this:
Code:
[ 1GB vola ][ 1 GB volb ][ 1 GB vola ][ remaining free space ]
There are ways in which you can give LVM hints as to what on-disk layout you want, and the LVM tools will generate information about the current layout in /data/lvm/backup if that directory's available (and mounted read-write) at the time, but in general, it's best to treat the on-disk layout of a logical volume as unknown. In practice, unless you have physical volumes with widely varying characteristics in the same volume group (say, an SSD and some spinning-platter hard disks), you shouldn't need to worry about the actual layout.


Quote:
Originally Posted by solarispika View Post
3. now that removing lvm is possible, is it also possible to backup lvm setting, and switching between non-lvm and lvm environment?
If /data is mounted read-write, running

Code:
# /lvm/sbin/lvm vgcfgbackup
will create a human-readable backup of the current LVM volume group metadata in /data/lvm/backup (if that directory is available and mounted read-write).

I'm not sure what you mean by "switching" between non-LVM and LVM environments. If you mean going between an unmodified stock boot image and an LVM boot image, the answer is yes, but you won't have access to any of the data in your LVM logical volumes while running an image not configured for LVM, and you need to be careful not to accidentally format the LVM physical volume.

If you're asking about having two or more ROMs on the phone, one installed to the normal /data and /system eMMC partitions, and another on LVM logical volumes, the answer is also yes, but both ROMs will need modified ramdisks. The one for the eMMC-installed ROM should only mount /sdcard from LVM, while the others should be set up to mount /data and /system from LVM as well.
The Following User Says Thank You to steven676 For This Useful Post: [ Click to Expand ]
 
wizardslovak
Old
#8  
Senior Member
Thanks Meter 14
Posts: 390
Join Date: Mar 2011
Good write up;)
wonder if lvm swap could be connected as extra memory .

Sent from my Nexus S using XDA
Galaxy Nexus Gsm

Galaxy S5 --- Blocked Bootloader !! REally ATT Really??
Iphone 5 -- Finally got rid of it ,
Galaxy nexus -backup phone
Nexus S (somewhere in boxes)
Iphone 4 traded for nexus s
Galaxy S 4G ---traded for iphone 4
 
steven676
Old
#9  
Senior Member - OP
Thanks Meter 2,372
Posts: 582
Join Date: Nov 2011
Quote:
Originally Posted by wizardslovak View Post
wonder if lvm swap could be connected as extra memory .
Sure, of course. From my phone:

Code:
$ cat /proc/swaps
Filename				Type		Size	Used	Priority
/dev/mapper/lvpool-swap                 partition	65532	28196	-1
It works just fine, even if it's not a setup I'd use on a mission-critical server.
 
mandaman2k
Old
#10  
Senior Member
Thanks Meter 163
Posts: 165
Join Date: Sep 2008

 
DONATE TO ME
Hey Guys,

Thanks for this awesome guide.

I just have a quick question, what are the advantages of using LVM instead of the normal partitioning?

I just want to know cause i might added to my rom.

Thanks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes