[MOD] Xoom: symlink /system/apps to /data for GApps on CM10-based ROMs (11/28 update)

[oscillot]

Disclaimer: This mod is provided as is. I am not responsible for you or your actions, I can barely handle myself most days. This will probably kill your dog and frame your spouse, so if your not feeling up for it, maybe sit this out.

What is this?
This addresses the lack of space on the /system partition that has been forcing people to compromise on GApps packages, and generally be subjected to wonky behavior.

How do I install it?
First you must make a nandroid. This is a flashable zip that you apply in recovery *in between* flashing cm10 and GApps. *NOTE* There are -NO- GApps in this download. This is just a shim to make room for them!

How does this work?
It's mostly edify, with a shell script that shouldn't have been necessary, but I got impatient, and hell, it works.

• What it does is it will create folder on your /data partition called /data/system.app.
• It makes sure that the permissions match the original.
• Then it will migrate the *entire* contents of the traditional /system/app folder into the new folder, also preserving the owners, permissions and attributes of files
• Finally, it deletes the original, and creates a symlink in its place, so that anything that tries to read, or write, to that folder, will end up on the /data partition.
• You can now install a full GApps package, and you'll have a lofty 25MB or so to spare!

Is it safe?
I have tested this on my 4G Stingray using CM10 nightlies and Rogue Recovery as both a clean flash and an upgrade. It works under that configuration. See "Before you download" below for currently known issues.

Is this only for the Stingray?
Yes and no. Yes, I only have that one device that suffers the partition issue, and on which to test this. No, I don't see why it couldn't work for at least one other device. But still probably don't try it unless you really know what you're doing.

Why did you make this?
I wanted to solved this problem, and now it's my gift to you all! Enjoy!

Can I incorporate this in my ROM or on my website or at my grandmother's bingo party?
Knock yourselves out. I hope this ends up being super helpful for everybody.

Something broke and it's all your fault!


Before you download!

• Make a nandroid! This is twice that I'm telling you, so I mean it!
• As of right now, if you were to factory reset your device, it would be necessary to restore or reflash from recovery. Do not factory reset if you apply this mod! (Thanks runandhide05!)
• This mod does not work with TWRP recovery at the moment (But I'm working on it). (Thanks dodgefan67!)
• Face unlock currently fails to install. I am working to address this.
• Some future version will use addon.d so that you don't have to flash this every time. This will be implemented once the above issues have been dealt with.

The zip file was updated on 11/28 to fix an issue where the shell script had windows-style line endings, causing it not to run in TWRP and possibly some other recoveries.

The download:
http://www.tangerinepulsar.com/files/cm10-xoom-system-symlink-mod-signed.zip

 

[runandhide05]

the only problem i see here is that you have no way to backup you rom, well, if you run a backup it will say its successful, but you have nothing to get /data/system.app backed up and re symlinked when you restore, and if you preform a factory reset you will loose all of your system apps,
you should look at how EOS symlinks there gapps data at /data/ you have to include addon.d scripts so that if you run a backup it will backup your new folder too, otherwise problems may/will happen.

edit:
for example for the /usr/srec stuff ( google now hotword and talk back files) you have to do something like this

. /tmp/backuptool.functions

list_files() {
cat <<EOF
EOF
}

case "$1" in
  backup)
    list_files | while read FILE DUMMY; do
      backup_file $S/$FILE
    done
  ;;
  restore)
    list_files | while read FILE REPLACEMENT; do
      R=""
      [ -n "$REPLACEMENT" ] && R="$S/$REPLACEMENT"
      [ -f "$C/$S/$FILE" ] && restore_file $S/$FILE $R
    done
    ln -s /data/eos-srec /system/usr/srec
  ;;
  pre-backup)
    # Stub
  ;;
  post-backup)
    # Stub
  ;;
  pre-restore)
    # Stub
  ;;
  post-restore)
    # Stub
  ;;
esac

 

[oscillot]

the only problem i see here is that you have no way to backup you rom, well, if you run a backup it will say its successful, but you have nothing to get /data/system.app backed up and re symlinked when you restore, and if you preform a factory reset you will loose all of your system apps,
you should look at how EOS symlinks there gapps data at /data/ you have to include addon.d scripts so that if you run a backup it will backup your new folder too, otherwise problems may/will happen.

I'll definitely look at incorporating this approach, thanks!

At the time I didn't think that backups were that big of a deal so long as you backed up both system and data, all of the symlinks would still be intact. I did write the edify script in such a way that it allows for an existing /system.app folder to already be there and will re-symlink it. But there are always times when you want to restore one and not the other, or switch ROMs without losing data, and this doesn't handle that at all. I definitely agree that a factory reset would trash you pretty badly, it's not a scenario that I accounted for in any way.

Thanks for the nudge in the right direction, I'll keep working on this.

 

[urushiol]

Thanks for workings on this, very exciting news.

 

[bigrushdog]

I'll start cranking out eos test builds soon. Then I'll take a look at what kind of gapps solutions are available. One thing for sure is it will be a bare bones package. That means anything that can be acquired from the Store will be.

 

[DsturbD]

this is good news indeed. i will try this out on my wifi xoom in the next day or two

i understand about doing a factory reset, but if i just update from one nightly to another (dirty flash) i would have to run this each time as well, correct?

 

[runandhide05]

this is good news indeed. i will try this out on my wifi xoom in the next day or two

i understand about doing a factory reset, but if i just update from one nightly to another (dirty flash) i would have to run this each time as well, correct?

You should not have to.

Sent from my Galaxy Nexus using Tapatalk 2

 

[DsturbD]

You should not have to.

Sent from my Galaxy Nexus using Tapatalk 2

doesn't the script actually move the contents of /system/apps though? installing a new version would put apps there right? and overwrite the symlinks?

 

[runandhide05]

doesn't the script actually move the contents of /system/apps though? installing a new version would put apps there right? and overwrite the symlinks?

Well it depends on how/if he updated it.

But if its the same as earlier then ya you will need to reflash.

Sent from my Galaxy Nexus using Tapatalk 2

 

[oscillot]

You should not have to.

Sent from my Galaxy Nexus using Tapatalk 2

this is good news indeed. i will try this out on my wifi xoom in the next day or two

i understand about doing a factory reset, but if i just update from one nightly to another (dirty flash) i would have to run this each time as well, correct?

Well it depends on how/if he updated it.

But if its the same as earlier then ya you will need to reflash.

Sent from my Galaxy Nexus using Tapatalk 2

I haven't released an update yet. Please reflash per the current instructions. I'll make it obvious when the script has changed.

 

[DsturbD]

ok, i used CM10 stable from 11/13/2012 and the official GApps JB package from 10/11/2012

flashed CM10
flashed this mod
flashed gapps

on first boot, i did not get any google account sign on screens, went straight to homescreen, no play store or any google apps installed
reflashed gapps and rebooted and this time i got the play store and google search, but that's it. is there more to gapps? i think they took out gmail, but would expect more for a 90meg file. never really looked at this, i have been using the EOS version of gapps with Zigackly's rom

by the way, according to TB i have 29.4mb left on /system and that is before i remove the other stuff i dont want. so it seems to have worked on a MZ604 wifi device

i will use it this way for a couple of days, restoring some apps and installing things and changing settings and see how it goes. then i will update to a nightly and see how that goes

so far so good

 

[oscillot]

ok, i used CM10 stable from 11/13/2012 and the official GApps JB package from 10/11/2012

flashed CM10
flashed this mod
flashed gapps

on first boot, i did not get any google account sign on screens, went straight to homescreen, no play store or any google apps installed
reflashed gapps and rebooted and this time i got the play store and google search, but that's it. is there more to gapps? i think they took out gmail, but would expect more for a 90meg file. never really looked at this, i have been using the EOS version of gapps with Zigackly's rom

by the way, according to TB i have 29.4mb left on /system and that is before i remove the other stuff i dont want. so it seems to have worked on a MZ604 wifi device

i will use it this way for a couple of days, restoring some apps and installing things and changing settings and see how it goes. then i will update to a nightly and see how that goes

so far so good

I'm a little perturbed about the way that first flash went. If you experience that again, will you please post the log from /cache/recovery prior to the re-flash? I want to make sure that there's not some scenario that I'm not handling properly with just one flash.

BTW, Did you flash it all from recovery or did you use something like ROM Manager or ROM Toolbox to script the recovery actions? Just so I cover my bases.

 

[DsturbD]

I'm a little perturbed about the way that first flash went. If you experience that again, will you please post the log from /cache/recovery prior to the re-flash? I want to make sure that there's not some scenario that I'm not handling properly with just one flash.

BTW, Did you flash it all from recovery or did you use something like ROM Manager or ROM Toolbox to script the recovery actions? Just so I cover my bases.

i will do it again and get you the log file. i'm using twrp and did it all from there. i even did each zip one at a time even though twrp lets you do multiple ones in one go

 

[DsturbD]

good news and bad, i got the log file but i had to dump it to a plain text file so it is hard to read, but if you search for symlink-mod you will find the section where your mod starts and right after that is where i flashed gapps ***edit***never mind about the formatting, i clicked the attachment below using Chrome and it came up fine

i was able to duplicate my issue. i wiped everything three times and installed a different rom and different gapps package and everything was fine. then i wiped three more times and installed cm10 stable, your mod, then the official jb gapps, rebooted and same thing. no google apps installed. this log file is from right after the first reboot

then i went back and flashed gapps again, rebooted and have the play store

i can put CWM on it and try it with that recovery if you want

 

[runandhide05]

Here is the problem, u can not have anything in the /system/app folder.. Nothing at all, so the problem lays that if you try to flash a ROM, then the sy link it will copy everything in system/app to data, remove the directory (system/app) then link it to the new directory in /data. Then you are trying to flash gapps which tries to put stuff in /system/app which that directory does not exist so its going to create it. Which will undo the symlink or will not be able to flash anything to /app.
So for the sy link to work u must flash both rom and gapps, which is where the problem of limited space cause now u won't get all your files over to the /system because not enough room. So to fix this you can do one of several things, one of which would be...
Change the gapps package to where the system apps flash to the new /data directory created by the sysmlink.
So if the new directory is /data/sysapp/ open the gapp package and take all the system/app/ and move it to a new folder on the root of the zip called data then in there create a folder called sysapp and put all the /system/app/ content in there. And remember to update the updater script for the resource "data" "/data" and that "should" fix it
Sent from my Galaxy Nexus using Tapatalk 2

 

[oscillot]

good news and bad, i got the log file but i had to dump it to a plain text file so it is hard to read, but if you search for symlink-mod you will find the section where your mod starts and right after that is where i flashed gapps ***edit***never mind about the formatting, i clicked the attachment below using Chrome and it came up fine

i was able to duplicate my issue. i wiped everything three times and installed a different rom and different gapps package and everything was fine. then i wiped three more times and installed cm10 stable, your mod, then the official jb gapps, rebooted and same thing. no google apps installed. this log file is from right after the first reboot

then i went back and flashed gapps again, rebooted and have the play store

i can put CWM on it and try it with that recovery if you want

Okay, I see a couple of things that are downright strange. To start off, the reason it looks like a flat text file is because the line-breaks are unix-style ('\n') and windows apps generally don't show linebreaks unless they have a carriage return preceding a newline character ('\r\n'). Open files from android in Notepad++ and they'll render much more nicely. Turn on viewing end of line characters and you'll see the difference between the Windows and Unix styles of newlines.

In practice, these are invisible control characters that don't visibly render, except in so far as their introduction of a line-break. '\r' and '\n' are aliases of them used in programming. They are actually the hex values 0A and 0D. But, since they are also characters, unexpected ones will have some kind of effect.

On lines 4066-4067 I see an error in the copy.sh: "/tmp/copy.sh: line 2: '\r' : not found"
There is a stray carriage return in the middle of this log file. There should be none. Line 2 in that file is BLANK. The source *should* have a single unix-newline '\n' and nothing else. I would bet that busybox is trying to interpret it and as a result, the shell script never runs. Nothing is getting copied out of /system/app.

I checked the zip I uploaded, and sure enough, the shell script has windows-style line endings. Oops. I've replaced it with a version that does not, but is identical other than that. So that part is fixed. Not sure why it works in CWM and not in TWRP, as far as I'm concerned, it shouldn't have worked in either. They must have slightly different busybox binaries is all that I can think.

The other issue is that both the deletion of /system/app and the subsequent symlinking both claim to have worked. However it's clear that /system/app was not deleted because if it were, you wouldn't have booted correctly. The symlink couldn't have worked either because you can't symlink a location that has files in it. So what in the world happened here? I'm kind of not sure, especially since those steps each returned 0, or else we wouldn't have gotten the success messages.

The wrinkle that you got *no* gapps after a flash is even stranger. But There is one more bit of interest here. I see an error message on line 4135 "cp: target '/system/app' is not a directory". At this point, this is the gapps trying to install face-unlock. I went ahead and looked in my own log files and here's what I found: "cp: target '/system/app' is not a directory." I never used it, but further inspection confirms, no portion of the face-unlock is installed using my mod, and the reason is the syntax of the gapps install-optional.sh: "cp -a /tmp/face/* /system/". Because /system/app is a symlink, you cannot copy the folder /tmp/face/app over it successfully.

This begs the question, how is it displaying the properties of being both a symlink and NOT a symlink?

I'm going to work on these issues, and I'll probably flash TWRP and test against it as well and try to come up with some answers and a one-size-fits-all solution. In the mean time, I would appreciate it if you could confirm whether you have a /data/system.app directory, and if so, what the contents are. I have a hunch that you'll find the gapp apks (minus face-unlock) in that directory, but none of the system apps. I'm not sure how that would be possible, or what it means, but it makes the most sense from where I'm standing. In the meantime, I'm going to add some notes to the main post.

I'll hopefully get some of this figured out by this weekend.

 

[rmkilc]

Okay, I see a couple of things that are downright strange. To start off, the reason it looks like a flat text file is because the line-breaks are unix-style ('\n') and windows apps generally don't show linebreaks unless they have a carriage return preceding a newline character ('\r\n'). Open files from android in Notepad++ and they'll render much more nicely. Turn on viewing end of line characters and you'll see the difference between the Windows and Unix styles of newlines.

In practice, these are invisible control characters that don't visibly render, except in so far as their introduction of a line-break. '\r' and '\n' are aliases of them used in programming. They are actually the hex values 0A and 0D. But, since they are also characters, unexpected ones will have some kind of effect.

On lines 4066-4067 I see an error in the copy.sh: "/tmp/copy.sh: line 2: '\r' : not found"
There is a stray carriage return in the middle of this log file. There should be none. Line 2 in that file is BLANK. The source *should* have a single unix-newline '\n' and nothing else. I would bet that busybox is trying to interpret it and as a result, the shell script never runs. Nothing is getting copied out of /system/app.

I checked the zip I uploaded, and sure enough, the shell script has windows-style line endings. Oops. I've replaced it with a version that does not, but is identical other than that. So that part is fixed. Not sure why it works in CWM and not in TWRP, as far as I'm concerned, it shouldn't have worked in either. They must have slightly different busybox binaries is all that I can think.

The other issue is that both the deletion of /system/app and the subsequent symlinking both claim to have worked. However it's clear that /system/app was not deleted because if it were, you wouldn't have booted correctly. The symlink couldn't have worked either because you can't symlink a location that has files in it. So what in the world happened here? I'm kind of not sure, especially since those steps each returned 0, or else we wouldn't have gotten the success messages.

The wrinkle that you got *no* gapps after a flash is even stranger. But There is one more bit of interest here. I see an error message on line 4135 "cp: target '/system/app' is not a directory". At this point, this is the gapps trying to install face-unlock. I went ahead and looked in my own log files and here's what I found: "cp: target '/system/app' is not a directory." I never used it, but further inspection confirms, no portion of the face-unlock is installed using my mod, and the reason is the syntax of the gapps install-optional.sh: "cp -a /tmp/face/* /system/". Because /system/app is a symlink, you cannot copy the folder /tmp/face/app over it successfully.

This begs the question, how is it displaying the properties of being both a symlink and NOT a symlink?

I'm going to work on these issues, and I'll probably flash TWRP and test against it as well and try to come up with some answers and a one-size-fits-all solution. In the mean time, I would appreciate it if you could confirm whether you have a /data/system.app directory, and if so, what the contents are. I have a hunch that you'll find the gapp apks (minus face-unlock) in that directory, but none of the system apps. I'm not sure how that would be possible, or what it means, but it makes the most sense from where I'm standing. In the meantime, I'm going to add some notes to the main post.

I'll hopefully get some of this figured out by this weekend.

Any new news on this?

 

[nabeelimran]

A little update on what I tried on my MZ-601 Everest. Any feedback will be welcome

Basically what I did was:
1. Trimmed down the CM10 Stable a little (132MB vs the original 141MB) by removing mainly the wallpapers.
2. Flashed the modified CM10 Stable zip (after a full wipe - factory plus cache).
3. Modified the Mod Script to point to "/data/system/app" (just to satisfy my OCD ) then flashed the zip. On first try it aborted with Status 7 on "Fixing permissions" so I had to manually create the directory "/data/system/app" after which the flash was successful.
4. Modified the GApps package (GApps from 11th Oct) by:

a. Removing the "optional" directory, install script and removing the relevant lines from "updater-script",
b. Moving the "/system/app" directory to "/data/system" and
c. Modifying the "updater-script" to reflect the change.

5. Flashed the GApps zip package which went uneventfully.
6. I now have 51.4MB free in "/system" according to Titanium Backup

PS: I am including the modified "Mod Script" and "updater-script" from GApps package if anyone else wants to try it. Of course you mileage may vary and I cannot claim any responsibility for any loss of data that may occur

UPDATE: Tried it with the CM10.1 nightlies and it works flawlessly. Flashed the nightly over the last nightly, flashed this script then the official gpaps and rebooted. All the settings and customizations retained! Still have some 56MB left in /system