Data2ext CyanogenMOD - UPDATE 26/01/11

Search This thread

TheGrammarFreak

Senior Member
Jul 29, 2010
2,820
678
twitter.com
I am not responsible for anything you do to your phone. If this goes wrong, it's not my fault. Read the whole post, and the last few pages of the thread. This will get you up to speed. Backup everything! Twice. No, I'm not joking. Do a nandroid, then do a titanium backup.

Important note! Ext4 does NOT work with ClockworkMOD 2.5.x. Nandroid backups will not work unless you upgrade to CWM 3! (data2ext4)



Greetings, earthlings! I have got a new version for you! Almost. Read this: http://forum.xda-developers.com/showpost.php?p=11931210&postcount=341

Update 26/01/2011
Ok, so someone asked about continuing support for ext2. So I will. Ain't I nice. ;) Both scripts are updated, and should have improved performance, as well as a bugfix ;) With both you don't need to start over, just flash the zip and you're done. Backup anyway (of course).


EXT4 version:
Yes, ext4 is slower in benchmarks. No, it doesn't have a real world impact. Not that I can see anyway. As always feedback is welcomed. I really mean that, if your phone genuinely becomes slower, let me know. :D

The zip is named data2ext4-V4.zip

Instructions (new users):
  • Ensure that mmcblk0p2 isn't mounted anywhere. You can do this by running "mount|egrep mmcblk0p2". If that returns anything, the ext partition is mounted. Unmount it using "umount directory/that/mmcblk0p2/is/mounted/to". Most often it's mounted to /sd-ext; in this case run "umount /sd-ext"
  • Format the ext partition to ext4:
    Code:
    mke2fs -m0 -b4096 /dev/block/mmcblk0p2
    tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p2
    e2fsck -yf /dev/block/mmcblk0p2
    tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
    tune2fs -O ^has_journal /dev/block/mmcblk0p2
  • Flash the attached zip and reboot to normal mode
  • Open up a terminal or use ADB to execute "data2ext --enable"
  • Reboot


Intructions (upgrading from ext2):
This should keep all your existing data. Back it up anyway
  • Disable data2ext
  • Upgrade to ext4 thusly:
    Code:
    tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p2
    e2fsck -yf /dev/block/mmcblk0p2
    tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
    tune2fs -O ^has_journal /dev/block/mmcblk0p2
  • Flash the attached zip
  • Enable data2ext



EXT2 version:
This is rather simpler to set up, but is not as safe. It is slightly faster though ;)
The zip is named data2ext2-v7.zip
  • Flash the zip
  • ensure /dev/block/mmcblk0p2 isn't mounted anywhere
  • Format the ext partition to ext2 with a 4KB block size
    Code:
    mke2fs -m0 -b4096 /dev/block/mmcblk0p2
  • Run this:
    Code:
    data2ext --enable
  • Reboot and enjoy




1st boot will take a while, it's copying stuff across.
Java on my PC sucks (it won't work), and I still can't sign zips. You'll live, just toggle signature verification in CWM

Semi-disclaimer:
This is my first real foray into scripting. I am still learning, so this is not as good as it could be. I release this to you to give me an incentive to make it better. I still need to look into mount options and the like, I'm sure this can go faster. Once again, I would class this as beta quality.




What is data2ext?
The phones internal memory is divided into several partitions. The 3 key ones are /system, /data and /cache. The rest we don't need to worry about too much. /system is where the ROM lives. By default it's read only (regardless of S-ON or S-OFF). All the data on our phones (apps, app data, game progress and a few other bits and bobs) is stored on /data. We have a pitiful amount of storage on /data (about 190MB), so we quickly run out when installing apps. 1 solution is to move /data/app (this is where most of the apps are stored) to a partition on your SD card (formatted to ext). But the internal memory still fills up, as anything not in /data/app stays on internal (caches, data and shizz). Other solutions have arisen, like moving dalvik-cache there to the ext partition too. But with the advent of fast SD cards it became possible to move the whole data partition to the ext partition. So people did; this means that the internal memory is barely used (aside from a few bits here and there, they're symlinked back to their original location).

It ain't workin'!

Ok. Get a logcat, and use pastebin to paste it here. Then we might be able to work it out, ye?

To use logcat:

Download this and extract it to a known location. Navigate to that folder, and press SHIFT while right-clicking. In the menu there should be a "Open command window here" option. Click it. Now type "adb logcat". Press CTRL+C to end the output. Right click and press mark, then click and drag over all the text and hit ENTER. It's now copied to the clipboard. CTRL+V to paste it somewhere... To debug issues with this script I need the first 5-10 seconds of the boot process.

To make life easier, I need the bit between "--START OF SCRIPT, PASTE LOGCAT FROM HERE ON!--" and "I DON'T NEED ANY MORE OF THE LOGCAT!" ;)

Changelog
Version 1
  • initial release
Versions 2 through 4 were internal testing ones

Version 5
  • Fixed dalvik-cache rebuild
  • Made more likely to work
EXT4(v3) version initial release

EXT4 V4
  • Improved performance
  • Bugfix (unmount /sd-ext if it exists)
EXT2 V7
  • Improved performance
  • Bugfix (unmount /sd-ext if it exists)

Hereafter I shall rename these to make managing this changelog easier ;)
Credits:
BlaY0
Anyone who's ever made data2ext scripts. I looked at a few ;)
 

Attachments

  • data2ext2-v7.zip
    152.4 KB · Views: 8,336
  • data2ext4-V4.zip
    152.5 KB · Views: 7,505
Last edited:

gyroing

Senior Member
Sep 9, 2010
226
8
nooob.
but could you also make just ap2sd for ext3 ?

Sent from my Legend using Tapatalk
 

fabi280

Senior Member
Oct 20, 2010
347
365
Karlsruhe
I don't like to ask questions, so I try to introduce myself into those new things. But I need a confirmation if my theories are correct

1. Google's app2sd is kinda stupid. It uses the normal fat partition on your SD, correct?

2. The a2sd linked here copies all the stuff from /data/app to /sd-ext/app and remounts /data/app to /sd-ext/app
So the Phone means the App is stored on the Phone Memory but in real its stored on the sd.
Do I have to wipe everything after flashing the script. What happens to apps that are currently moved to the SD (by Google Apps2SD). Can I just copy them "to phone" again and they are inside the sd-ext then?

3. Data2ext => Speedup because everything from /data is now on the extfs on SD. Apps are also included, as they are located in /data/app so its has a2sd integrated, right?

EDIT: Damn, now I have to ask a question...
What's stored inside the dalvik-cache? I don't know much about dalvik, thats my biggest problem with android :D...
Is it good to move it also to sd-ext with the a2sd script?
 
Last edited:

TheGrammarFreak

Senior Member
Jul 29, 2010
2,820
678
twitter.com
I don't like to ask questions, so I try to introduce myself into those new things. But I need a confirmation if my theories are correct

1. Google's app2sd is kinda stupid. It uses the normal fat partition on your SD, correct?

2. The a2sd linked here copies all the stuff from /data/app to /sd-ext/app and remounts /data/app to /sd-ext/app
So the Phone means the App is stored on the Phone Memory but in real its stored on the sd.
Do I have to wipe everything after flashing the script. What happens to apps that are currently moved to the SD (by Google Apps2SD). Can I just copy them "to phone" again and they are inside the sd-ext then?

3. Data2ext => Speedup because everything from /data is now on the extfs on SD. Apps are also included, as they are located in /data/app so its has a2sd integrated, right?

EDIT: Damn, now I have to ask a question...
What's stored inside the dalvik-cache? I don't know much about dalvik, thats my biggest problem with android :D...
Is it good to move it also to sd-ext with the a2sd script?

1. Correct
2. No wipe needed for a2sd or this data2ext. Any app on what the phone believes to be internal memory is in fact sd-ext
3. Correct

Dalvik-cache is a shedload of dex files, quite important. The bug with re-creating it on every boot isn't major enough for me to worry about for now, as I don't think it affects anything other than the startup time. I need to investigate this, so be careful if you use this mod. My worry is that it's due to data being lost somewhere

When I used a2sd I had dalvik-cache on the sd card, but then I have a fast SD card. It can slow things down.

Anyhoo, I would like this thread to be for testing data2ext, not asking questions about a2sd. I have made this to help me learn how these things work, so feedback is welcomed. If anyone has tested this on CM6 then please let me know. And can you also post feedback about the dalvik-cache issue. ;)


i tried that one but doesn't work on gingerbread cm7 :(
OlegKrikun :
"Now cm7 exp. builds not support run start up scripts from /etc/init.d =( when is fix, I publish app2sd for cm7"

noob=== my noob wish.
/system/etc/init.d is where this is run from. It works on CM7
 
Last edited:
  • Like
Reactions: SRD720 and fabi280

lwiss

Senior Member
Jul 31, 2010
195
53
@Grammar,

thx for your work!!! it works very good on my phone. ext2(1 gb) partition as ext3 formatted. i have the "signed" zip from exicar(thx) installs....

lwiss

quadrant: 1686
card: 8gb class 6
rom-version: CM 7 | Gingerbread, 01/03
 
Last edited:

fabi280

Senior Member
Oct 20, 2010
347
365
Karlsruhe
Anyhoo, I would like this thread to be for testing data2ext, not asking questions about a2sd. I have made this to help me learn how these things work, so feedback is welcomed. If anyone has tested this on CM6 then please let me know. And can you also post feedback about the dalvik-cache issue. ;)

Sorry for my Off-Topic-Questions.
I'll try it for you on CM6.
How fast has it to be? I've got a 8GB Class 6 Card, is it enough?

EDIT: Note to myself, never read an article description on amazon on a second screen while writing a reply.
It's a class 4, I looked for a Class 6 on Amazon -_-
But I'll try it anyway..
 
Last edited:

TheGrammarFreak

Senior Member
Jul 29, 2010
2,820
678
twitter.com
@Grammar,
thx for your work!!! it works very good on my phone. ext2(1 gb) partition as ext3 formatted. i have the "signed" zip from exicar(thx) installs....
lwiss

What version of CM? And it's not my work, it's BlaY0's. I just made his script work on CM :D

Sorry for my Off-Topic-Questions.
I'll try it for you on CM6.
How fast has it to be? I've got a 8GB Class 6 Card, is it enough?

It's not the most off-topic I've seen. Look at the CM experimental build thread ;). I also have 8GB class 6. I get 1600+
 

TheGrammarFreak

Senior Member
Jul 29, 2010
2,820
678
twitter.com
Stupid question. Where to enable data2ext in cyanogenmod settings?

If you don't even read the first post I'm not gonna help you. If I haven't made the point clear enough:

This is in testing. There is a major bug where dalvik-cache is re-built on every reboot. This is not a good thing. I would advise that you not use this unless you know what you're doing
 

fabi280

Senior Member
Oct 20, 2010
347
365
Karlsruhe
Sorry for my Off-Topic-Questions.
I'll try it for you on CM6.
How fast has it to be? I've got a 8GB Class 6 Card, is it enough?

EDIT: Note to myself, never read an article description on amazon on a second screen while writing a reply.
It's a class 4, I looked for a Class 6 on Amazon -_-
But I'll try it anyway..

Class 4 is definitely too slow, at least on CM6.
I will not provide a Quadrant Benchmark because my Phone is waaay too slow now. It really lags everywhere.

First Start:
Code:
I//system/xbin/run-parts(   59): Initiating Data2EXT...
I//system/xbin/run-parts(   59): Waiting for SD card to settle...
I//system/xbin/run-parts(   59): Executing file system check...
I//system/xbin/run-parts(   59): /dev/block/mmcblk0p2: 11/65664 files (0.0% non-contiguous), 4451/262144 blocks
I//system/xbin/run-parts(   59): Mounting /dev/block/mmcblk0p2 to /data...
I//system/xbin/run-parts(   59): EXT partition /dev/block/mmcblk0p2 mounted on /data...
I//system/xbin/run-parts(   59): First time initiation, copying files to EXT partition...
I//system/xbin/run-parts(   59): Data2EXT enabled...
I//system/xbin/run-parts(   59): Setting read-write /system overlay...
I//system/xbin/run-parts(   59): Checking for kernel aufs support...
I//system/xbin/run-parts(   59): No aufs driver present in kernel, trying to load aufs module...
I//system/xbin/run-parts(   59): Can not load aufs module, aborting...

Second start:
Code:
I//system/xbin/run-parts(   59): Initiating Data2EXT...
I//system/xbin/run-parts(   59): Waiting for SD card to settle...
I//system/xbin/run-parts(   59): Executing file system check...
I//system/xbin/run-parts(   59): /dev/block/mmcblk0p2: Entry 'CheckinService.xml' in /data/com.google.android.gsf/shared_prefs (25081) has deleted/unused inode 25583.  CLEARED.
I//system/xbin/run-parts(   59): /dev/block/mmcblk0p2: Entry 'EventLogService.xml' in /data/com.google.android.gsf/shared_prefs (25081) has deleted/unused inode 25074.  CLEARED.
I//system/xbin/run-parts(   59): /dev/block/mmcblk0p2: Entry 'googlesettings.db-journal' in /data/com.google.android.gsf/databases (25087) has deleted/unused inode 25564.  CLEARED.
I//system/xbin/run-parts(   59): /dev/block/mmcblk0p2: 1834/65664 files (0.4% non-contiguous), 46535/262144 blocks
I//system/xbin/run-parts(   59): /dev/block/mmcblk0p2: 1834/65664 files (0.4% non-contiguous), 46535/262144 blocks
I//system/xbin/run-parts(   59): Mounting /dev/block/mmcblk0p2 to /data...
I//system/xbin/run-parts(   59): EXT partition /dev/block/mmcblk0p2 mounted on /data...
I//system/xbin/run-parts(   59): Data2EXT enabled...
I//system/xbin/run-parts(   59): Setting read-write /system overlay...
I//system/xbin/run-parts(   59): Checking for kernel aufs support...
I//system/xbin/run-parts(   59): No aufs driver present in kernel, trying to load aufs module...
I//system/xbin/run-parts(   59): Can not load aufs module, aborting...

I can send you the complete outputs for both starts (got em till the UI was visible)

EDIT: But is the Class4 Card fast enough to handle a2sd (maybe with Dalvik Cache)?
 
Last edited:

BlaY0

Retired Recognized Developer
Sep 15, 2007
1,553
566
Medvode
There are several factors affecting read/write speed to/from SD card not just class:
  • partition alignment
  • partition type
  • format options
  • mount options
  • readahead settings
  • etc.
 
  • Like
Reactions: jinxul and fabi280

m1schi

Senior Member
Nov 3, 2010
106
5
Not going to try and use it yet, but I'm curious now. When using this, you're automatically having all of your apps on your sd card and the app2sd feature isn't needed anymore, right? And your phone gets a speed boost when using it?
 

HpX

Senior Member
Oct 1, 2010
183
12
Not going to try and use it yet, but I'm curious now. When using this, you're automatically having all of your apps on your sd card and the app2sd feature isn't needed anymore, right? And your phone gets a speed boost when using it?
Apps on sd-ext = yes
Speed Boost, sometimes yes, sometimes its laggy as hell



Sent from my HTC Legend using XDA App
 
  • Like
Reactions: m1schi

Rapier

Recognized Contributor
Stupid question. How do you guys get 1600+ in Quadrant on CM7?
For me Quadrant is not finishing tests due to the 3D graphics not working. But even if it'll work, I don't think I'd get so much. On CM 6.1 oc'ed to 787, I'd hardly get 1150. Is it just due to overclocking or this data2ext is speeding it up?

Sent from my Legend using XDA App
 

Top Liked Posts

  • There are no posts matching your filters.
  • 22
    I am not responsible for anything you do to your phone. If this goes wrong, it's not my fault. Read the whole post, and the last few pages of the thread. This will get you up to speed. Backup everything! Twice. No, I'm not joking. Do a nandroid, then do a titanium backup.

    Important note! Ext4 does NOT work with ClockworkMOD 2.5.x. Nandroid backups will not work unless you upgrade to CWM 3! (data2ext4)



    Greetings, earthlings! I have got a new version for you! Almost. Read this: http://forum.xda-developers.com/showpost.php?p=11931210&postcount=341

    Update 26/01/2011
    Ok, so someone asked about continuing support for ext2. So I will. Ain't I nice. ;) Both scripts are updated, and should have improved performance, as well as a bugfix ;) With both you don't need to start over, just flash the zip and you're done. Backup anyway (of course).


    EXT4 version:
    Yes, ext4 is slower in benchmarks. No, it doesn't have a real world impact. Not that I can see anyway. As always feedback is welcomed. I really mean that, if your phone genuinely becomes slower, let me know. :D

    The zip is named data2ext4-V4.zip

    Instructions (new users):
    • Ensure that mmcblk0p2 isn't mounted anywhere. You can do this by running "mount|egrep mmcblk0p2". If that returns anything, the ext partition is mounted. Unmount it using "umount directory/that/mmcblk0p2/is/mounted/to". Most often it's mounted to /sd-ext; in this case run "umount /sd-ext"
    • Format the ext partition to ext4:
      Code:
      mke2fs -m0 -b4096 /dev/block/mmcblk0p2
      tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p2
      e2fsck -yf /dev/block/mmcblk0p2
      tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
      tune2fs -O ^has_journal /dev/block/mmcblk0p2
    • Flash the attached zip and reboot to normal mode
    • Open up a terminal or use ADB to execute "data2ext --enable"
    • Reboot


    Intructions (upgrading from ext2):
    This should keep all your existing data. Back it up anyway
    • Disable data2ext
    • Upgrade to ext4 thusly:
      Code:
      tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p2
      e2fsck -yf /dev/block/mmcblk0p2
      tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
      tune2fs -O ^has_journal /dev/block/mmcblk0p2
    • Flash the attached zip
    • Enable data2ext



    EXT2 version:
    This is rather simpler to set up, but is not as safe. It is slightly faster though ;)
    The zip is named data2ext2-v7.zip
    • Flash the zip
    • ensure /dev/block/mmcblk0p2 isn't mounted anywhere
    • Format the ext partition to ext2 with a 4KB block size
      Code:
      mke2fs -m0 -b4096 /dev/block/mmcblk0p2
    • Run this:
      Code:
      data2ext --enable
    • Reboot and enjoy




    1st boot will take a while, it's copying stuff across.
    Java on my PC sucks (it won't work), and I still can't sign zips. You'll live, just toggle signature verification in CWM

    Semi-disclaimer:
    This is my first real foray into scripting. I am still learning, so this is not as good as it could be. I release this to you to give me an incentive to make it better. I still need to look into mount options and the like, I'm sure this can go faster. Once again, I would class this as beta quality.




    What is data2ext?
    The phones internal memory is divided into several partitions. The 3 key ones are /system, /data and /cache. The rest we don't need to worry about too much. /system is where the ROM lives. By default it's read only (regardless of S-ON or S-OFF). All the data on our phones (apps, app data, game progress and a few other bits and bobs) is stored on /data. We have a pitiful amount of storage on /data (about 190MB), so we quickly run out when installing apps. 1 solution is to move /data/app (this is where most of the apps are stored) to a partition on your SD card (formatted to ext). But the internal memory still fills up, as anything not in /data/app stays on internal (caches, data and shizz). Other solutions have arisen, like moving dalvik-cache there to the ext partition too. But with the advent of fast SD cards it became possible to move the whole data partition to the ext partition. So people did; this means that the internal memory is barely used (aside from a few bits here and there, they're symlinked back to their original location).

    It ain't workin'!

    Ok. Get a logcat, and use pastebin to paste it here. Then we might be able to work it out, ye?

    To use logcat:

    Download this and extract it to a known location. Navigate to that folder, and press SHIFT while right-clicking. In the menu there should be a "Open command window here" option. Click it. Now type "adb logcat". Press CTRL+C to end the output. Right click and press mark, then click and drag over all the text and hit ENTER. It's now copied to the clipboard. CTRL+V to paste it somewhere... To debug issues with this script I need the first 5-10 seconds of the boot process.

    To make life easier, I need the bit between "--START OF SCRIPT, PASTE LOGCAT FROM HERE ON!--" and "I DON'T NEED ANY MORE OF THE LOGCAT!" ;)

    Changelog
    Version 1
    • initial release
    Versions 2 through 4 were internal testing ones

    Version 5
    • Fixed dalvik-cache rebuild
    • Made more likely to work
    EXT4(v3) version initial release

    EXT4 V4
    • Improved performance
    • Bugfix (unmount /sd-ext if it exists)
    EXT2 V7
    • Improved performance
    • Bugfix (unmount /sd-ext if it exists)

    Hereafter I shall rename these to make managing this changelog easier ;)
    Credits:
    BlaY0
    Anyone who's ever made data2ext scripts. I looked at a few ;)
    5
    The script itself is now rock solid but solution is still unstable. EXT2 is fast but may cause you some problems if you don't unmount it properly. Even auto check (e2fsck) may fail if your phone experienced power failure. Power failure here means anything else than shutdown using power button!

    I'm working on a more robust solution using EXT4 but it is still in test phase.
    3
    signd version attached

    edit: removed
    2
    I don't like to ask questions, so I try to introduce myself into those new things. But I need a confirmation if my theories are correct

    1. Google's app2sd is kinda stupid. It uses the normal fat partition on your SD, correct?

    2. The a2sd linked here copies all the stuff from /data/app to /sd-ext/app and remounts /data/app to /sd-ext/app
    So the Phone means the App is stored on the Phone Memory but in real its stored on the sd.
    Do I have to wipe everything after flashing the script. What happens to apps that are currently moved to the SD (by Google Apps2SD). Can I just copy them "to phone" again and they are inside the sd-ext then?

    3. Data2ext => Speedup because everything from /data is now on the extfs on SD. Apps are also included, as they are located in /data/app so its has a2sd integrated, right?

    EDIT: Damn, now I have to ask a question...
    What's stored inside the dalvik-cache? I don't know much about dalvik, thats my biggest problem with android :D...
    Is it good to move it also to sd-ext with the a2sd script?

    1. Correct
    2. No wipe needed for a2sd or this data2ext. Any app on what the phone believes to be internal memory is in fact sd-ext
    3. Correct

    Dalvik-cache is a shedload of dex files, quite important. The bug with re-creating it on every boot isn't major enough for me to worry about for now, as I don't think it affects anything other than the startup time. I need to investigate this, so be careful if you use this mod. My worry is that it's due to data being lost somewhere

    When I used a2sd I had dalvik-cache on the sd card, but then I have a fast SD card. It can slow things down.

    Anyhoo, I would like this thread to be for testing data2ext, not asking questions about a2sd. I have made this to help me learn how these things work, so feedback is welcomed. If anyone has tested this on CM6 then please let me know. And can you also post feedback about the dalvik-cache issue. ;)


    i tried that one but doesn't work on gingerbread cm7 :(
    OlegKrikun :
    "Now cm7 exp. builds not support run start up scripts from /etc/init.d =( when is fix, I publish app2sd for cm7"

    noob=== my noob wish.
    /system/etc/init.d is where this is run from. It works on CM7
    2
    There are several factors affecting read/write speed to/from SD card not just class:
    • partition alignment
    • partition type
    • format options
    • mount options
    • readahead settings
    • etc.