[REQUEST] Move App-Data to/back from /data to /sd-ext

Search This thread

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
Are there any plans to implement a Button into the App-Manager, to move the Data-Part of an app to the /ext2 Partition ?

Especially on low memory devices (like the G1) this will give us the chance to install a lot of apps without runnung to "low space on device".

I have written a very small bash-script to manually do this job, but it would be more comfortable with a Button.

move.sh
Code:
#!/system/bin/sh
#
#data_to_move=com.alk.copilot
#data_to_move=com.camelgames.blowup
#data_to_move=com.drodin.tuxrider
#data_to_move=com.gameloft.android.AMEU.GloftAsphalt5.asphalt5
#data_to_move=com.navigon.navigator
#data_to_move=com.polarbit.ironsightlite
#data_to_move=com.polarbit.ragingthunder
#data_to_move=com.polarbit.rthunder2
#data_to_move=com.polarbit.waveblazerlite
#data_to_move=com.estrongs.android.pop

su
mount -o remount,rw /data
mkdir /sd-ext/data
cd /data/data

echo About to Move ${data_to_move}

cp -r -p ${data_to_move} /sd-ext/data
rm -r -f ${data_to_move}
ln -s /sd-ext/data/${data_to_move} ${data_to_move}

unmove.sh
Code:
#!/system/bin/sh
#
#data_to_move=com.alk.copilot
#data_to_move=com.camelgames.blowup
#data_to_move=com.drodin.tuxrider
#data_to_move=com.gameloft.android.AMEU.GloftAsphalt5.asphalt5
#data_to_move=com.navigon.navigator
#data_to_move=com.polarbit.ironsightlite
#data_to_move=com.polarbit.ragingthunder
#data_to_move=com.polarbit.rthunder2
#data_to_move=com.polarbit.waveblazerlite
#data_to_move=com.estrongs.android.pop

su
mount -o remount,rw /data
mkdir /sd-ext/data
cd /data/data

echo About to unMove ${data_to_move}

rm ${data_to_move}
mkdir ${data_to_move}
cp -r -p /sd-ext/data/${data_to_move} ${data_to_move}
 
Last edited:

kernelzilla

Member
Apr 30, 2009
41
0
Apps that need lots of space should use the fat32 partition gracefully, using this hack would considerably slow down your system(even class6). Why double the reads and writes to the slow mmc?
 

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
Because not all apps using the fat32-Partition.
Navigon for example uses 14 MB of internal storage.
If you install such apps on a G1, you can quickly run out of space.
Low on cache space will considerably slow down the overall system performance.
I would not suggest to move the data of any installed apps, but from the big ones.
Apps like Navigation and 3D-Games are writing not frequently their data and if you own a G1 you are already swapping when using an Eclaid based ROM. So this wouldn't make a big difference in the livetime of your sdcard.
Perhaps someone could implement a partial move (e.g. only libraries) to avoid massive writes to the card ... and/or setting a threshold value, so apps with small data could not be moved.

For me, moving the apps in the Script above has given me additional 35 MB internal space and everything is still runnung "fast".
 
Last edited:

LucidREM

Senior Member
Jan 18, 2009
1,571
5
Baltimore
www.lucidrem.us
/data/data

Because not all apps using the fat32-Partition.
Navigon for example uses 14 MB of internal storage.
If you install such apps on a G1, you can quickly run out of space.
Low on cache space will considerably slow down the overall system performance.
I would not suggest to move the data of any installed apps, but from the big ones.
Apps like Navigation and 3D-Games are writing not frequently their data and if you own a G1 you are already swapping when using an Eclaid based ROM. So this wouldn't make a big difference in the livetime of your sdcard.
Perhaps someone could implement a partial move (e.g. only libraries) to avoid massive writes to the card ... and/or setting a threshold value, so apps with small data could not be moved.

For me, moving the apps in the Script above has given me additional 35 MB internal space and everything is still runnung "fast".

there is already a script in existance to move things like that:

# lucid -d -sd

would move app data to the sdcard and symlink .. this (however) does not move individual pieces .. i would be interested to know the speed difference on these apps that you moved .. also .. this will create extra difficulty when attempting any nandroid backup/restore .. i have seen people putting up comments because their phone crashed due to the excessive number of symlinks across the phone .. if you are not careful with them you could lose all your data
 

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
The Space allocated by the "big apps" is mainly used by their ./lib dir. Because of this, it would be enough to move and symlink only those dirs.
With Games the "rest" goes to settings and scores.
On a G1 there is absolutely no difference in speed when moving data to sd.
Perhaps its because the G1 is not the top performer at all :)

I have played those games in the list with data in internal and on sd-ext and there is no difference in speed ... loading time is also the same.

I'm satisfied with the results, but these scripts didn't remove the data when apps where uninstalled or re-installed and thats the reason of my request.

Take a look on your storage and see how much space (libraries) should be on the sdcard.

Code:
du -sk /data/data/* | sort -rn | head

btw ... did you ever enabled JIT on a G1 and played ExZeus or Armageddon Squadron ?
Its amazing what is possible on this "outdated" Hardware!

P.S. Nandroid Backup runs perfectly with this symlinks (no recursive/double Backups)
 
Last edited:

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
Update:

Nandroid Backup only saves app and app-private ... could you edit this to save everything excluding "crap-dirs" ?

If nandbackup uses standard tar calls, you can use the following command:

Code:
tar pcvf /sdcard/nandroid/sd-ext.tar . -C /sd-ext --exclude dalvik-cache --exclude lost+found

... it would save everything the user place on the partition including userinit.sh

Restoring such a tarball works perfectly with nandbackup.

Thx in advance
Thom
 

zelipukin

Senior Member
Aug 24, 2009
159
40
Jackson
Int. mem for SWAP ?

well, after the moving some heavy apps data
I have 54mb free Int. mem (out of 90)
is that possible to use 24-32mb of this (fast?)memory
for SWAP ? instead of linux swap
and does it make any sense?
...just tried to enable /swapfile.swp via Swapper2
though it says
-creating swap - ok
-changing permission - ok
-formatting swap - ok
but
- enabling swap(file) - FAIL

sorry if it's just another stupid question

G1, stock cm5.0.8 test4, 32mb linux swap
 

dumfuq

Retired Recognized Developer
May 23, 2009
542
42
Update:

Nandroid Backup only saves app and app-private ... could you edit this to save everything excluding "crap-dirs" ?

If nandbackup uses standard tar calls, you can use the following command:

Code:
tar pcvf /sdcard/nandroid/sd-ext.tar . -C /sd-ext --exclude dalvik-cache --exclude lost+found
... it would save everything the user place on the partition including userinit.sh

Restoring such a tarball works perfectly with nandbackup.

Thx in advance
Thom

Have you tried BART? http://xdaforums.com/showthread.php?t=562292

It's included in the recovery, and lets you backup/not_backup whatever you want.
 

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
well, after the moving some heavy apps data
I have 54mb free Int. mem (out of 90)
is that possible to use 24-32mb of this (fast?)memory
for SWAP ? instead of linux swap
and does it make any sense?
...just tried to enable /swapfile.swp via Swapper2
though it says
-creating swap - ok
-changing permission - ok
-formatting swap - ok
but
- enabling swap(file) - FAIL

sorry if it's just another stupid question

G1, stock cm5.0.8 test4, 32mb linux swap

It's not a stupid question ... I have had the same Idea yesterday ...
I have googled about life spawn of the internal flash memory, but I haven't found any satisfactory answer yet.

Anywhere here who know how fast the internal flash is ?
What about write cycles and wear levelling ?

If it has no integrated wear levelling, swapping will kill the phone in a few days.

I think your enable swap has failed due to wrong permissions ... try to enable with a defered call in your userinit.

Update: I have checked my filesystem ... /cache has actually 29 MB free ... is it correct, that /cache is only used by OTA updates ? Probably we can create a priorized swap there in addition to ext.
 
Last edited:

kernelzilla

Member
Apr 30, 2009
41
0
Navigon for example uses 14 MB of internal storage.
If you install such apps on a G1, you can quickly run out of space.
Low on cache space will considerably slow down the overall system performance.
I would not suggest to move the data of any installed apps, but from the big ones.
Apps like Navigation and 3D-Games are writing not frequently their data and if you own a G1 you are already swapping when using an Eclaid based ROM. So this wouldn't make a big difference in the livetime of your sdcard.
Perhaps someone could implement a partial move (e.g. only libraries) to avoid massive writes to the card ... and/or setting a threshold value, so apps with small data could not be moved.

For me, moving the apps in the Script above has given me additional 35 MB internal space and everything is still runnung "fast".

If navigon is using that much internal storage it's a very poorly written application. Low cache doesn't slow the system down, just apps that require such huge amounts of it.
I'm already swapping when using Eclair? Wrong, using swap on the sdcard is horrible, I'd never recommend it to anyone and I personally don't use it. I have 114 apps installed and my cache is 9% used, it seems to me like your apps aren't clearing their cache correctly, or they're just poorly written. The argument that 'you are already swapping so this wouldn't make a big difference in the livetime of your sdcard' is untrue, you are effectively doubling the amount of read/writes to the mmc, if not more, so the lifetime could potentially be cut in half, of course depending on use.

I'd say nice try, but this really just working around crap apps.
 

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
I use a 16 GB class 6 SD-Card with static wear-levelling.
Assuming that a standard-flash-nand-cell lasts about 10.000 write cycles, and my swap-write-turnover is currently about 1,6 GB per day, my SD-Card will last about 273 years (minus regular writes).
So I don't care about livetime.

Besides Navigon, there are many apps, that store their huge libraries to /data .... Games in most cases ... If you aren't using "bad written apps" its fine for YOU ... everyone else has to do some tweaks when installing some of them to the limited internal storage.

I have 278 apps installed and the only limit for me, is currently the free space on my sdcard.

If you haven't enabled swap since you have flashed your first Eclair ROM, you have probably never felt what is "speed" or you never need more than 1 app running simultaniously .... or you are using a different phone instead of a G1 :D

You say "swapping to sd is horrible" ... I think you have used the wrong parameters ... when I diable swapping my system is lagging ... even when I work with one app the same time.

Did you enabled compcache while swapping ? Did you use a swapfile on FAT32 ? Is your swappinness levor 50 or above 60 ? Are you using al class 4 or slower sd-card ? Are you running heavy memory consuming apps without killing them from time to time ?

All these can turn a fast swapping system into an unusable phone.

You cannot enable swap and use the system like before.
 
Last edited:

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
Update:

I have copied 18 MB from cache to data and it tooks round about 18 seconds.

Same file from sdcard (FAT32) to sdcard tooks 6 seconds ...

I will use the sdcard ;)
 

kernelzilla

Member
Apr 30, 2009
41
0
I should have listened to internal voice telling me not to argue with a fool cause people might not know the difference..
 

zelipukin

Senior Member
Aug 24, 2009
159
40
Jackson
Something strange

I did some stupid (owing to absence of linux knowledge)
experiments regarding to swap_2_/
I believe if it possible it should be done through userinit/config
or smth during boot to enable r/w. give necc permissions etc.
I just used Swapper and RootManager
If I create .swp (it creates but does not work) in any place but /cache
it (just existence of this file) does system unstable, slow and unresponsive
in /cache or any existing or newly created folders inside /cache/ it's OK
before reboot when those new folders/files disappear
=
After a wile something happened with my phone (not a first or last time)
many apps caused FC, settings were lost etc
tried "fix uid missmatches" - dots filled out numerous screens
and after ~20min I decided to reboot
tried nandroid - same endless ....................................................
after the reboot I found no FCs but still missing settings for some apps
(sim_linked_data apps like CoPilot were OK) so I Titaniumed non-working apps
data (5-6 apps) and evrthng seems fine
=
BUT when I look at internal memory available
I find 73Mb (out of ~90) FREE
there was 53Mb free before the accident
Is that normal? And whats the limit?
I have my laps and brain scratched to find some application of this
=
just my experience
-compcache always gives me horrible slow phone - not using
-linux swap - best results compare to no_swap - allways use
 

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
Some apps are storing many data to /data and sometimes to /cache.
I think your restores have cleared some of them.
Try CacheMate instead of such manouvers ;)
 

TheGenesis

Senior Member
Jan 5, 2010
53
0
Cologne
I have checked the write throughput using dd:

Internal Storage: 3,5 MB/s
Class 6 SD Card: 7 MB/s

... annoying ... USB to SD-Card is 3,5 MB/s and SD-Card via Card-Reader (PC) is about 9 MB/s.
 

gilibs

Member
Nov 7, 2010
34
2
Haifa
Hey man, i had the same problem and decided to go a head and write a small tool that does exactly this, this is a UI tool that shows all the folders in /data/data (excluding system folders) and let you move your apps to your sd, you should have an APP2SD ROM installed with root (of-course) and sd card partitioned to EXT and FAT32.
Contact me if you want to check it out, i never found the time to publish it (gbenshim@gmail.com)
 

korkin25

Member
Aug 6, 2010
13
2
hi,

I have tried that and it works, but...it works until reboot...
After reboot I don't see directory /sd-ext/data....
I don't know why it always been deleted....