[MOD][ANY][Mitsuyoshi EXT4 Mod][15/05/13]

Search This thread

jubei_mitsuyoshi

Senior Member
Mar 23, 2013
50
92
Background..

With any linux system i build first thing i do is remove the journal from all my EXT4 partitions, to make a long story short ( i wont bore you with details of just how journal writes totally slow disk i/o , google it ) this gave me on average a 25% increase in system speed ( obviously due to the increased i/o rate )

I usually did it with

Code:
mke2fs.ext4 -O ^has_journal ...device...

before installing the distro, now i thought hey maybe this would work on android?

Now mke2fs is not well supported on arm but tune2fs is! in fact its included in TWRP 2.5 so

Code:
tune2fs -O ^has_journal /dev/block/platform/sdhci-tegra.3/by-name/APP

( for the system partition) should work right , and in fact yes it does !!!, tried it from TWRP command line and hey presto %^$%^$^%$ FAST system

This is the code for a recovery script version

Code:
ui_print("test script");
ui_print("");
ui_print("");
ifelse(is_mounted("/system") == "/system", unmount("/system"));
ifelse(is_mounted("/cache") == "/cache", unmount("/cache"));
ifelse(is_mounted("/data") == "/data", unmount("/data"));
ui_print("converting partitions to ext4 minus journal");
ui_print(" ");
set_progress(1.000000);
ui_print("e2fsck system partition");
ui_print(" ");
run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/APP");
ui_print("e2fsck data partition");
ui_print(" ");
run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/UDA");
ui_print("e2fsck cache partition");
ui_print(" ");
run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/CAC");
ui_print("now lets rub out those nasty journals");
ui_print(" ");
run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/APP");
run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/UDA");
run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/CAC");
ui_print("and re-fsck to clean any set bits");
ui_print(" ");
ui_print("e2fsck system partition");
ui_print(" ");
run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/APP");
ui_print("e2fsck data partition");
ui_print(" ");
run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/UDA");
ui_print("e2fsck cache partition");
ui_print(" ");
run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/CAC");

And i include a zip for those who like to test

Ps have not checked the function of this line

Code:
run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/APP");

the bash equivalent is

Code:
tune2fs -O ^has_journal /dev/block/platform/sdhci-tegra.3/by-name/APP

where ^has_journal is a parameter of -O not sure the recovery script is wrapping that correctly, if anyone flashes this script could you please check by typing

Code:
tune2fs -l /dev/block/platform/sdhci-tegra.3/by-name/APP

in the TWRP terminal, scroll upward with an up finger swipe and check has_journal is not in the filesystem properties ( going to do it myself on next system wipe, but just enjoying the speed at the mo !! , am testing it on PRIME 04-20 mildly tweaked by me )
Thanks


Edit ... Since there has been more than passing interest in this obscure piece of code i have decided to tidy this post up

Hence now there are 2 scripts, one to remove the journal from your ext4 partitions, and one to re-instate the journal, this should allow people to test their setup with each without any wiping or reformatting

Instructions.......
( tested with TWRP 2.5+ only )
Flash the bugger :)
DO NOT WIPE ANYTHING, after flashing, doing so will only reformat the wiped partition with a journal
 

Attachments

  • jubei_test_script_add_journal.zip
    139.7 KB · Views: 20,429
  • jubei_test_script_remove_journal.zip
    139.7 KB · Views: 18,200
Last edited:

JetOhmNet

Member
Apr 24, 2013
44
13
For us who have no background in Linux. What is this for? Does it just increase system speed? I'm running prime 4/20 as well and would definitely like to test it once I know exactly what it's supposed to do.
 
  • Like
Reactions: NabhanKhan100

psyren

Senior Member
Dec 2, 2010
272
78
Perth
For us who have no background in Linux. What is this for? Does it just increase system speed? I'm running prime 4/20 as well and would definitely like to test it once I know exactly what it's supposed to do.

It forgoes recording disk writing (?) operations to a journal and instead writes things directly to disk (or a buffer, if you want to be pedantic).
Instead of going:
Write to journal (I'm doing this write) -> Write to journal (I'm currently doing this write) -> execute operation -> erase entry from journal
It goes:
Execute operation

The end result is greater write speeds, but greater risk of data loss if there is power loss.
If you were in the middle of a write operation when the power goes dead, the journal entry saying that you were doing something will still be there, so the OS can either resume/restart the operation (since it knows it was incomplete) or do some sort of error handling since it knows something went wrong.

For more/related info, see this thread.
 

JetOhmNet

Member
Apr 24, 2013
44
13
No linux was just for explanation.
But it also work for Android.
So that's a good thing

The script he has posted is for twrp.

I got that much, just wasn't convinced on what exactly this script does.

It forgoes recording disk writing (?) operations to a journal and instead writes things directly to disk (or a buffer, if you want to be pedantic).
Instead of going:
Write to journal (I'm doing this write) -> Write to journal (I'm currently doing this write) -> execute operation -> erase entry from journal
It goes:
Execute operation

The end result is greater write speeds, but greater risk of data loss if there is power loss.
If you were in the middle of a write operation when the power goes dead, the journal entry saying that you were doing something will still be there, so the OS can either resume/restart the operation (since it knows it was incomplete) or do some sort of error handling since it knows something went wrong.

For more/related info, see this thread.

And thank you on the specifics and pros and cons. I'll definitely give it a try now and let you know what I think/if i notice any bugs!
 

Jack'O

Senior Member
Aug 13, 2011
1,873
815
One of many.
Sorry, wrong post!
I was tring to send a message to a Razr owner but i bungled with the XDA app.

Thanks anyway for your reply.
 
Last edited:

jubei_mitsuyoshi

Senior Member
Mar 23, 2013
50
92
yes will work on any android/linux system using ext4 partitions, in the case of n7 thats only system,data and cache.
But the actual TWRP script needs checking for syntax for every device. ( obviously checked it on the n7 )

I can confirm a MASSIVE speed increase, everything in noticeably smoother and faster, which is to be expected since you have increased disk i/o by at least 25%, especially important in the data and cache partitions.

To really understand whats going on you really must google EXT4 and the journal folk's, a journaling filesystem is a legacy of the linux server heritage, its very useful in a server context, if you have a crash and are running 100+ TB of data, the journal allows you to reboot and instantly be back online ( which is massively important is the server world where seconds offline means millions of dollars ) rather than wait many hours for a fsck to complete. For android it is completely inappropriate! it should be obvious why :)

Psyren's post is a good simple explanation of the write structure :) and how removing the journal speeds things up.

If this code goes into anyone's rom please give me some love in the credits :)

Ps as mentioned in the post there is a line to double check, if your doing it by hand from the TWRP terminal then i can confirm it def works, i have not double checked the script ( except to prove it flashes etc ) and wont for about a week, i am getting ready my own rom that concentrates on improving the linux sub-structure, and that's taking my time
 

frost_nz

Senior Member
May 14, 2012
506
238
Christchurch
Sorry if this is a stupid question, but does this wipe any partitions or will there be any data loss flashing the zip?

Or does it need to be flashed before flashing Rom?


Sent from my MB865 using xda app-developers app
 
Last edited:

JetOhmNet

Member
Apr 24, 2013
44
13
Sorry if this is a stupid question, but does this wipe any partitions or will there be any data loss flashing the zip?

Or does it need to be flashed before flashing Rom?


Sent from my MB865 using xda app-developers app

Flashing does not wipe anything, but I also haven't noticed any i/o improvements.
 

bcvictory

Senior Member
Oct 6, 2011
566
482
Bendigo
The script stuffed up my nexus. I had a backup but just letting you know. It booted to the animation but went no further. Any benchmarks?

Sent from my Nexus 7 using xda premium
 

jubei_mitsuyoshi

Senior Member
Mar 23, 2013
50
92
The script stuffed up my nexus. I had a backup but just letting you know. It booted to the animation but went no further. Any benchmarks?

Sent from my Nexus 7 using xda premium

Hmm thats interesting, which rom you running ? You using TWRP ( im on 2.5 ) I tried it on my nexus and no problems, even with multi-rom in.
Can anyone suggest an appropriate benchmark prog to use please, ( something that tests io as-well as ui responsiveness ) and i will post results on the next wipe. Like i said i am def claiming a noticeable improvement on Prime 02-04 as far as ui is concerned, eg when i go to playstore and scroll my apps there is no longer any pause in the scroll, lots of other things like that, and all the "pages" of various apps come up faster. But really need a quantitative test.
Thanks
 

bcvictory

Senior Member
Oct 6, 2011
566
482
Bendigo
Hmm thats interesting, which rom you running ? You using TWRP ( im on 2.5 ) I tried it on my nexus and no problems, even with multi-rom in.
Can anyone suggest an appropriate benchmark prog to use please, ( something that tests io as-well as ui responsiveness ) and i will post results on the next wipe. Like i said i am def claiming a noticeable improvement on Prime 02-04 as far as ui is concerned, eg when i go to playstore and scroll my apps there is no longer any pause in the scroll, lots of other things like that, and all the "pages" of various apps come up faster. But really need a quantitative test.
Thanks

Well can you write a how-to install this? I flashed the zip using Twrp 2.5 over the top of my PA install. Is that what we are meant to do?
If you are claiming io speed then maybe and io tester? (Can't think of one off the top of my head)
And an antutu just for kicks
Maybe a before and after?

Sent from my Nexus 7 using xda premium
 

JetOhmNet

Member
Apr 24, 2013
44
13
So i forgot to run a benchmark prior to flashing this. Silly me. But here are my speeds after flashing. I'm running Prime (4–20) with faux ultimate kernel clocked at 1.7Ghz.
 

Attachments

  • uploadfromtaptalk1368745654324.jpg
    uploadfromtaptalk1368745654324.jpg
    60.7 KB · Views: 1,475

JetOhmNet

Member
Apr 24, 2013
44
13
Also here are my Antutu and Quadrant marks:
 

Attachments

  • uploadfromtaptalk1368745700715.jpg
    uploadfromtaptalk1368745700715.jpg
    41.2 KB · Views: 1,397
  • uploadfromtaptalk1368745709696.jpg
    uploadfromtaptalk1368745709696.jpg
    104.1 KB · Views: 1,490

jubei_mitsuyoshi

Senior Member
Mar 23, 2013
50
92
So i forgot to run a benchmark prior to flashing this. Silly me. But here are my speeds after flashing. I'm running Prime (4–20) with faux ultimate kernel clocked at 1.7Ghz.

cool thanks for that Androbench def best in this category but readings can vary 50% or more per run, eg did 2 runs separated by seconds

run1 sqlite benchmark
insert 1052.63 TPS
Update 789.47 TPS
Delete 348.02 TPS

run2 sqlite benchmark
insert 617.28 TPS
Update 274.97 TPS
Delete 910.87 TPS

Maybe we should ignore " small differences " LOL

On the up side the RD/WRT stats only vary buy about 30% so should be a better guide.
Will have some before/after comparisons for you tomorrow,

On the subject tho should not a benchmark program have exclusive access to the system? ( which is impossible in android ) otherwise isnt it just reading and writing along side all the other system RW calls

PS.. how do you get the screenshots, would be useful for posting benchmarks
 
Last edited:

jubei_mitsuyoshi

Senior Member
Mar 23, 2013
50
92
Well can you write a how-to install this? I flashed the zip using Twrp 2.5 over the top of my PA install. Is that what we are meant to do?
If you are claiming io speed then maybe and io tester? (Can't think of one off the top of my head)
And an antutu just for kicks
Maybe a before and after?

Sent from my Nexus 7 using xda premium

Well seems good to me, yes just flash it, do not wipe anything after since TWRP will reformat with a journal. To be honest i have no idea why it failed in your case, thought maybe it is a CM vs AOKP thing but just had a pm from a bloke on a CM clone that flashed great and having good results, so hey cant be that :)

THe only thing i can think of is the kernel, could be something tweaked in that causing a confusion between ext4-minus-juornal and ext2, that happened on very old linux kernels, but not seen that for years, so doubt that to.

If you can be bothered try again, post if you get the same result and i will give it a go with a PA when finish my little rom :)
 

bcvictory

Senior Member
Oct 6, 2011
566
482
Bendigo
Well seems good to me, yes just flash it, do not wipe anything after since TWRP will reformat with a journal. To be honest i have no idea why it failed in your case, thought maybe it is a CM vs AOKP thing but just had a pm from a bloke on a CM clone that flashed great and having good results, so hey cant be that :)

THe only thing i can think of is the kernel, could be something tweaked in that causing a confusion between ext4-minus-juornal and ext2, that happened on very old linux kernels, but not seen that for years, so doubt that to.

If you can be bothered try again, post if you get the same result and i will give it a go with a PA when finish my little rom :)

I use M-Kernel. Don't wipe cache and dalvik?

Sent from my Nexus 7 using xda premium
 

Top Liked Posts

  • There are no posts matching your filters.
  • 53
    Background..

    With any linux system i build first thing i do is remove the journal from all my EXT4 partitions, to make a long story short ( i wont bore you with details of just how journal writes totally slow disk i/o , google it ) this gave me on average a 25% increase in system speed ( obviously due to the increased i/o rate )

    I usually did it with

    Code:
    mke2fs.ext4 -O ^has_journal ...device...

    before installing the distro, now i thought hey maybe this would work on android?

    Now mke2fs is not well supported on arm but tune2fs is! in fact its included in TWRP 2.5 so

    Code:
    tune2fs -O ^has_journal /dev/block/platform/sdhci-tegra.3/by-name/APP

    ( for the system partition) should work right , and in fact yes it does !!!, tried it from TWRP command line and hey presto %^$%^$^%$ FAST system

    This is the code for a recovery script version

    Code:
    ui_print("test script");
    ui_print("");
    ui_print("");
    ifelse(is_mounted("/system") == "/system", unmount("/system"));
    ifelse(is_mounted("/cache") == "/cache", unmount("/cache"));
    ifelse(is_mounted("/data") == "/data", unmount("/data"));
    ui_print("converting partitions to ext4 minus journal");
    ui_print(" ");
    set_progress(1.000000);
    ui_print("e2fsck system partition");
    ui_print(" ");
    run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/APP");
    ui_print("e2fsck data partition");
    ui_print(" ");
    run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/UDA");
    ui_print("e2fsck cache partition");
    ui_print(" ");
    run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/CAC");
    ui_print("now lets rub out those nasty journals");
    ui_print(" ");
    run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/APP");
    run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/UDA");
    run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/CAC");
    ui_print("and re-fsck to clean any set bits");
    ui_print(" ");
    ui_print("e2fsck system partition");
    ui_print(" ");
    run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/APP");
    ui_print("e2fsck data partition");
    ui_print(" ");
    run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/UDA");
    ui_print("e2fsck cache partition");
    ui_print(" ");
    run_program("/sbin/e2fsck", "-p", "/dev/block/platform/sdhci-tegra.3/by-name/CAC");

    And i include a zip for those who like to test

    Ps have not checked the function of this line

    Code:
    run_program("/sbin/tune2fs", "-O", "^has_journal", "/dev/block/platform/sdhci-tegra.3/by-name/APP");

    the bash equivalent is

    Code:
    tune2fs -O ^has_journal /dev/block/platform/sdhci-tegra.3/by-name/APP

    where ^has_journal is a parameter of -O not sure the recovery script is wrapping that correctly, if anyone flashes this script could you please check by typing

    Code:
    tune2fs -l /dev/block/platform/sdhci-tegra.3/by-name/APP

    in the TWRP terminal, scroll upward with an up finger swipe and check has_journal is not in the filesystem properties ( going to do it myself on next system wipe, but just enjoying the speed at the mo !! , am testing it on PRIME 04-20 mildly tweaked by me )
    Thanks


    Edit ... Since there has been more than passing interest in this obscure piece of code i have decided to tidy this post up

    Hence now there are 2 scripts, one to remove the journal from your ext4 partitions, and one to re-instate the journal, this should allow people to test their setup with each without any wiping or reformatting

    Instructions.......
    ( tested with TWRP 2.5+ only )
    Flash the bugger :)
    DO NOT WIPE ANYTHING, after flashing, doing so will only reformat the wiped partition with a journal
    11
    For us who have no background in Linux. What is this for? Does it just increase system speed? I'm running prime 4/20 as well and would definitely like to test it once I know exactly what it's supposed to do.

    It forgoes recording disk writing (?) operations to a journal and instead writes things directly to disk (or a buffer, if you want to be pedantic).
    Instead of going:
    Write to journal (I'm doing this write) -> Write to journal (I'm currently doing this write) -> execute operation -> erase entry from journal
    It goes:
    Execute operation

    The end result is greater write speeds, but greater risk of data loss if there is power loss.
    If you were in the middle of a write operation when the power goes dead, the journal entry saying that you were doing something will still be there, so the OS can either resume/restart the operation (since it knows it was incomplete) or do some sort of error handling since it knows something went wrong.

    For more/related info, see this thread.
    5
    To really understand whats going on you really must google EXT4 and the journal folk's, a journaling filesystem is a legacy of the linux server heritage, its very useful in a server context, if you have a crash and are running 100+ TB of data, the journal allows you to reboot and instantly be back online ( which is massively important is the server world where seconds offline means millions of dollars ) rather than wait many hours for a fsck to complete. For android it is completely inappropriate! it should be obvious why :)
    nearly everything in this block of text is wrong.

    the journaling is is not a legacy of the linux server heritage nor is it primary for servers. first of all journaling as a quite new feature of all modern filesystems. sedond it is not designed for servers only. it yields a nice benefit for non professional servers as it allows to skip some depper checks but in the professional enviroment if such dramatic event occurs like all UPS going out of power no enterprise would risk getting their servers up wihtout a full integrity check.

    journaling is of great interest for desktop system and... now sit down... for EMBEDED DEVICES! especially on tha latter ones they usually do NOT have much IO hence the overhead usually DOES NOT impact their overall performance but it improves GREATLY overal system integrity in case of sudden power loss which happens on no devices as often as on ... guess what... embeded devices or mobile systems. the number one devices with sudden power loses.


    and to end this post. i very doubt any of you really have any speed improvements by disabling this feature (a mdoern and not a legacy feature!!!). this so called "trick" is already a few years old on android (as it also works on ext3 btw.) and so far i've seen many claiming a feelable speed improvement but none of these claims persisted till now which would contradict the whole knwoledge about the features implementation btw.

    p.s. though, just in case, if this here woudl be the exception where you indeed see a notable speed improvement during normal usage, than the device most probalby would have totally different issues as to why it even has so high io loads that the journaling overhead becomes noteable.

    one may chose to disable it in such a case but at least don't tell fud about the facts.


    EDIT: forgot to mention... disabling journaling can at most speed up WRITE operations but never read operations. and on a mobile device the very very most of your io operations that have impact on your "speed" feeling are read oeprations. there are nearly no situations where a mobile device has to suffer from a high write load.
    2
    I don't think it will because the updater-script included in the package seems like it is for the Nexus 7 only. You can update it for your device I'm sure of it, but it might be difficult if you do not know what you are doing.


    So, I know this is in the Nexus 7 forum, but will this work on other Android devices, like the Sprint Galaxy S3?

    Its only for n7 devices due to the partitions stated in the updater script, just adb shell your device on linux and type mount, gives you the list of partitions for device. Just look for cache, system and data. Usually it is listen by name
    2
    yes will work on any android/linux system using ext4 partitions, in the case of n7 thats only system,data and cache.
    But the actual TWRP script needs checking for syntax for every device. ( obviously checked it on the n7 )

    I can confirm a MASSIVE speed increase, everything in noticeably smoother and faster, which is to be expected since you have increased disk i/o by at least 25%, especially important in the data and cache partitions.

    To really understand whats going on you really must google EXT4 and the journal folk's, a journaling filesystem is a legacy of the linux server heritage, its very useful in a server context, if you have a crash and are running 100+ TB of data, the journal allows you to reboot and instantly be back online ( which is massively important is the server world where seconds offline means millions of dollars ) rather than wait many hours for a fsck to complete. For android it is completely inappropriate! it should be obvious why :)

    Psyren's post is a good simple explanation of the write structure :) and how removing the journal speeds things up.

    If this code goes into anyone's rom please give me some love in the credits :)

    Ps as mentioned in the post there is a line to double check, if your doing it by hand from the TWRP terminal then i can confirm it def works, i have not double checked the script ( except to prove it flashes etc ) and wont for about a week, i am getting ready my own rom that concentrates on improving the linux sub-structure, and that's taking my time