App Review: Calculating Your Android Calculator Options

Sure, everyone has a default Calculator and its something that you dont think about … more

MultiROM Once Again Available for the Galaxy S4

Multiboot, or the ability to select betweenmultiple operating systems on a single device at … more

Build an Impractical But Awesome Lego Mindstorm Dock

The vast majority of what we cover here on the XDA-Developers News Portal relates to … more

MultiROM Makes its Way Over to the HTC One (M8)

While browsing our forum, you will undoubtedly find more than a few enticing ROMs available … more
Post Reply

EXT4, journaling, data integrity, reboot, space aliens, and robots

30th March 2011, 04:16 AM   |  #1  
OP Senior Member
Thanks Meter: 61
 
493 posts
Join Date:Joined: Dec 2007
Well, okay, nothing really about space aliens and robots, but you're here reading, aren't you?

Lots of discussion about pro/con of the EXT4 filesystem, risks to data integrity without journaling (analogous to a transaction log for a database), the disappearance of the reboot option on Bonsai (and maybe other ROMs) because of this, blah blah blah this, blah blah that, and on and on and on.

There is another solution. You can still have the performance of EXT4, without journaling. With much lower risk to your data in case of the battery leaping out of the phone on a whim. And good ol' reboot can make a comeback too!

The nirvana is a linux command called 'sync'. Crusty old unix hacks like me will get a twinkle in their eye at the mention of this command.

sync does somesthing very straightforward and simple: It syncs the filesystems. Put another way, it flushes the in-ram buffers out to "disk", syncronizing the actual state of the filesytem with what is stored -- and stale -- in secondary storage (hard discs on the big boys, NVRAM/SD/whatever on phones).



I've been experimenting with this to see if I can improve data integrity while minimally impacting performance. Here's what I've done:
  • Using GScript Lite, created a simple Superuser script for rebooting that looks like,

    sync
    sync
    sync
    /sbin/reboot #Bonsai ROM

    Why 3 syncs? Paranoid. Nothing more.

    Optionally create a shortcut on the homescreen to invoke this script to easy one-button reboot. I did this a week ago, have been using this to reboot Bonsai 4.0.0 a gazillion (actually, a bazillion, but I'm rounding) times, and have had absolutely no problems at all. Seems to work.
  • Created a shell script that launches at boot, as superuser, that runs in an infinite loop waking up every 10 seconds to do a sync. No detectable impact on performance that I can see. This is what I'd expect, as there is never more than 10 seconds of filesystem activity sitting "dirty" in the cache, so the sync doesn't usually have much to do (most of the time, nothing).

    What does all this mean? Well, it's sort of a "lazy" journaling, and much more efficient. There's still a higher risk of data corruption under uncontrolled loss of power than with journaling, but in my considered opinion its negligible with the usage model/patterns for this particular situation (a smartphone).
FWIW, before I implemented this method for reboot, I like others got FCs on apps randomly after using the 3-finger reboot, simply running reboot directly from a shell prompt, or back on Bonsai 3, using the power-button menu reboot command. With this sync approach, I have not had a single problem -- and I can reboot the phone again easily!

What this means for you

If you're enough of a hack to understand how to implement this stuff yourself, give it a shot (at your own risk!), and let us know how it works out.

For the rest of you, be patient... I'm putting together a package to make it simple to install all this (initially just for the Bonsai ROM, others to follow, maybe), and should have something to test in a day or two. If you're interested in being a tester, PM me. Looking for 10 people.
Last edited by dwallersv; 30th March 2011 at 04:19 AM.
The Following 4 Users Say Thank You to dwallersv For This Useful Post: [ View ]
30th March 2011, 06:38 AM   |  #2  
Top Nurse's Avatar
Senior Member
Flag Cocoa Beach, FL
Thanks Meter: 58
 
1,048 posts
Join Date:Joined: Sep 2010
More
I'm interested in this as I have been wondering about no_journaling for some time. I think it would help to prove or disprove the theory that no_journaling is causing data corruption. PM sent.
30th March 2011, 07:31 AM   |  #3  
k0nane's Avatar
Recognized Developer
Flag 127.0.0.1
Thanks Meter: 3,767
 
3,981 posts
Join Date:Joined: Feb 2008
More
This is very interesting. I'll be watching this - depending on the results, it may be a good option for future versions of SRF.
30th March 2011, 11:38 AM   |  #4  
Top Nurse's Avatar
Senior Member
Flag Cocoa Beach, FL
Thanks Meter: 58
 
1,048 posts
Join Date:Joined: Sep 2010
More
I think a mod should move this thread over into the Development forum, please?
30th March 2011, 12:08 PM   |  #5  
Senior Member
Thanks Meter: 102
 
647 posts
Join Date:Joined: Nov 2010
This sounds interesting. So will this "lazy" journaling put the same or less wear on the nand chip vs journaling being enabled?

If it has the same or similar wear factor as no journaling, as well as no impact on performance. Then this mod is a no brainer IMO.
30th March 2011, 12:26 PM   |  #6  
Senior Member
Thanks Meter: 1,253
 
2,176 posts
Join Date:Joined: Nov 2010
Does sync only cause a write operation for data that has changed, or simply rewrite the entire buffer to disk each time? I'm sure you can see where I'm going with this...


Sent from my SPH-D700 using XDA App
30th March 2011, 03:23 PM   |  #7  
poit's Avatar
Senior Member
Flag Colorado
Thanks Meter: 491
 
1,238 posts
Join Date:Joined: May 2009
More
It's better than nothing, but it still doesn't address battery pulls or phone freezes. It just provides a more graceful shutdown IF you do a clean shutdown.
30th March 2011, 03:46 PM   |  #8  
Senior Member
Thanks Meter: 26
 
159 posts
Join Date:Joined: Aug 2007
Quote:
Originally Posted by dwallersv

  • Using GScript Lite, created a simple Superuser script for rebooting that looks like,

    sync
    sync
    sync
    /sbin/reboot #Bonsai ROM

I tried this on my phone and so far, so good. I am using ACS Frozen Rom 1.1.0 with Twilight 1.1.0. When I downloaded the GScript lite, there was already a script for reboot, I just edited it to add the sync lines. I assume that the #Bonsai ROM is a comment and not needed, so that was omitted. I have rebooted about 10 times (I know...many, many less than a Gazillion) and have not had any issues.

Thank you!
30th March 2011, 07:57 PM   |  #9  
OP Senior Member
Thanks Meter: 61
 
493 posts
Join Date:Joined: Dec 2007
Quote:
Originally Posted by epic4GEE

This sounds interesting. So will this "lazy" journaling put the same or less wear on the nand chip vs journaling being enabled?

If it has the same or similar wear factor as no journaling, as well as no impact on performance. Then this mod is a no brainer IMO.

In terms of the precise impact on storage, it is indistinguishable from not doing it at all. All this does if change the timing of the writes -- you control it, rather than waiting for the OS to decide to flush the filesystem buffers.

In the case of a reboot, this makes a ginormous difference, because anything in cache that hasn't been flushed is lost if the system doesn't sync before quitting back into the bootloader.



Sent from my mind using telepathitalk
30th March 2011, 08:01 PM   |  #10  
OP Senior Member
Thanks Meter: 61
 
493 posts
Join Date:Joined: Dec 2007
Quote:
Originally Posted by styles420

Does sync only cause a write operation for data that has changed, or simply rewrite the entire buffer to disk each time? I'm sure you can see where I'm going with this...

only dirty blocks.

sync should not cause anything to be written that wouldn't be eventually written anyway, when the kernel decides it is either idle enough to perform the deferred task, or cache "fullness" requires it to make room for newer data by flushing older stuff that hadn't been written yet.



Sent from my mind using telepathitalk

Post Reply Subscribe to Thread
Previous Thread Next Thread
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes