Attend XDA's Second Annual Developer Conference, XDA:DevCon 2014!
5,813,477 Members 41,193 Now Online
XDA Developers Android and Mobile Development Forum

EXT4, journaling, data integrity, reboot, space aliens, and robots

Tip us?
 
dwallersv
Old
(Last edited by dwallersv; 30th March 2011 at 04:19 AM.)
#1  
Senior Member - OP
Thanks Meter 61
Posts: 493
Join Date: Dec 2007
Default EXT4, journaling, data integrity, reboot, space aliens, and robots

Well, okay, nothing really about space aliens and robots, but you're here reading, aren't you?

Lots of discussion about pro/con of the EXT4 filesystem, risks to data integrity without journaling (analogous to a transaction log for a database), the disappearance of the reboot option on Bonsai (and maybe other ROMs) because of this, blah blah blah this, blah blah that, and on and on and on.

There is another solution. You can still have the performance of EXT4, without journaling. With much lower risk to your data in case of the battery leaping out of the phone on a whim. And good ol' reboot can make a comeback too!

The nirvana is a linux command called 'sync'. Crusty old unix hacks like me will get a twinkle in their eye at the mention of this command.

sync does somesthing very straightforward and simple: It syncs the filesystems. Put another way, it flushes the in-ram buffers out to "disk", syncronizing the actual state of the filesytem with what is stored -- and stale -- in secondary storage (hard discs on the big boys, NVRAM/SD/whatever on phones).



I've been experimenting with this to see if I can improve data integrity while minimally impacting performance. Here's what I've done:
  • Using GScript Lite, created a simple Superuser script for rebooting that looks like,

    sync
    sync
    sync
    /sbin/reboot #Bonsai ROM

    Why 3 syncs? Paranoid. Nothing more.

    Optionally create a shortcut on the homescreen to invoke this script to easy one-button reboot. I did this a week ago, have been using this to reboot Bonsai 4.0.0 a gazillion (actually, a bazillion, but I'm rounding) times, and have had absolutely no problems at all. Seems to work.
  • Created a shell script that launches at boot, as superuser, that runs in an infinite loop waking up every 10 seconds to do a sync. No detectable impact on performance that I can see. This is what I'd expect, as there is never more than 10 seconds of filesystem activity sitting "dirty" in the cache, so the sync doesn't usually have much to do (most of the time, nothing).

    What does all this mean? Well, it's sort of a "lazy" journaling, and much more efficient. There's still a higher risk of data corruption under uncontrolled loss of power than with journaling, but in my considered opinion its negligible with the usage model/patterns for this particular situation (a smartphone).
FWIW, before I implemented this method for reboot, I like others got FCs on apps randomly after using the 3-finger reboot, simply running reboot directly from a shell prompt, or back on Bonsai 3, using the power-button menu reboot command. With this sync approach, I have not had a single problem -- and I can reboot the phone again easily!

What this means for you

If you're enough of a hack to understand how to implement this stuff yourself, give it a shot (at your own risk!), and let us know how it works out.

For the rest of you, be patient... I'm putting together a package to make it simple to install all this (initially just for the Bonsai ROM, others to follow, maybe), and should have something to test in a day or two. If you're interested in being a tester, PM me. Looking for 10 people.
The Following 4 Users Say Thank You to dwallersv For This Useful Post: [ Click to Expand ]
 
Top Nurse
Old
#2  
Top Nurse's Avatar
Senior Member
Thanks Meter 58
Posts: 1,048
Join Date: Sep 2010
Location: Cocoa Beach, FL
I'm interested in this as I have been wondering about no_journaling for some time. I think it would help to prove or disprove the theory that no_journaling is causing data corruption. PM sent.


 
k0nane
Old
#3  
k0nane's Avatar
Recognized Developer
Thanks Meter 3,766
Posts: 3,981
Join Date: Feb 2008
Location: 127.0.0.1
This is very interesting. I'll be watching this - depending on the results, it may be a good option for future versions of SRF.





Have I helped? Enjoy my network's fast downloads? Hit the Thanks button!



Follow me on Twitter @k0nane / @publik0! Chat with the OUDHS: irc.freenode.net #oudhitsquad (webchat)
 
Top Nurse
Old
#4  
Top Nurse's Avatar
Senior Member
Thanks Meter 58
Posts: 1,048
Join Date: Sep 2010
Location: Cocoa Beach, FL
I think a mod should move this thread over into the Development forum, please?


 
epic4GEE
Old
#5  
Senior Member
Thanks Meter 102
Posts: 647
Join Date: Nov 2010
This sounds interesting. So will this "lazy" journaling put the same or less wear on the nand chip vs journaling being enabled?

If it has the same or similar wear factor as no journaling, as well as no impact on performance. Then this mod is a no brainer IMO.
 
styles420
Old
#6  
Senior Member
Thanks Meter 1,252
Posts: 2,171
Join Date: Nov 2010
Does sync only cause a write operation for data that has changed, or simply rewrite the entire buffer to disk each time? I'm sure you can see where I'm going with this...


Sent from my SPH-D700 using XDA App
"What is this?! When does this happen in the movie?!"

"Now, sir. You're looking at now - everything that's happening now, is happening now."
 
poit
Old
#7  
poit's Avatar
Senior Member
Thanks Meter 491
Posts: 1,238
Join Date: May 2009
Location: Colorado
It's better than nothing, but it still doesn't address battery pulls or phone freezes. It just provides a more graceful shutdown IF you do a clean shutdown.
Novatel Bagphone, Motorola Microtac, Qualcom Q-Phone, Motorola Startac, Sanyo MM-8100, Sanyo MM7400, Samsung A900 Blade, HTC Touch Pro 2, Sammy Epic 4G,

Currently Sammy Galaxy S III running Stock Rooted KitKat 4.4.2 because it's just that good. Knox don't scare me
Sprint SERO Premium 500
 
hotwired34
Old
#8  
Senior Member
Thanks Meter 26
Posts: 159
Join Date: Aug 2007
Quote:
Originally Posted by dwallersv View Post
  • Using GScript Lite, created a simple Superuser script for rebooting that looks like,

    sync
    sync
    sync
    /sbin/reboot #Bonsai ROM

I tried this on my phone and so far, so good. I am using ACS Frozen Rom 1.1.0 with Twilight 1.1.0. When I downloaded the GScript lite, there was already a script for reboot, I just edited it to add the sync lines. I assume that the #Bonsai ROM is a comment and not needed, so that was omitted. I have rebooted about 10 times (I know...many, many less than a Gazillion) and have not had any issues.

Thank you!
 
dwallersv
Old
#9  
Senior Member - OP
Thanks Meter 61
Posts: 493
Join Date: Dec 2007
Quote:
Originally Posted by epic4GEE View Post
This sounds interesting. So will this "lazy" journaling put the same or less wear on the nand chip vs journaling being enabled?

If it has the same or similar wear factor as no journaling, as well as no impact on performance. Then this mod is a no brainer IMO.
In terms of the precise impact on storage, it is indistinguishable from not doing it at all. All this does if change the timing of the writes -- you control it, rather than waiting for the OS to decide to flush the filesystem buffers.

In the case of a reboot, this makes a ginormous difference, because anything in cache that hasn't been flushed is lost if the system doesn't sync before quitting back into the bootloader.



Sent from my mind using telepathitalk
 
dwallersv
Old
#10  
Senior Member - OP
Thanks Meter 61
Posts: 493
Join Date: Dec 2007
Quote:
Originally Posted by styles420 View Post
Does sync only cause a write operation for data that has changed, or simply rewrite the entire buffer to disk each time? I'm sure you can see where I'm going with this...
only dirty blocks.

sync should not cause anything to be written that wouldn't be eventually written anyway, when the kernel decides it is either idle enough to perform the deferred task, or cache "fullness" requires it to make room for newer data by flushing older stuff that hadn't been written yet.



Sent from my mind using telepathitalk

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Create a Simple Custom Launcher from Scratch

When you are starting your journey intoany sort of development, it’s never easy at the … more

MediaTek Continues Steps Towards Being Developer-Friendly

Not too long ago MediaTek was very closed off towards the mobile developer … more

Sony PS4 Remote Play Ported to All Android Devices

Im pretty sure that every informed gamer out there perked their ears up when Sony came out … more