EXT4, journaling, data integrity, reboot, space aliens, and robots

Well, okay, nothing really about space aliens and robots, but you're here reading, aren't you?

Lots of discussion about pro/con of the EXT4 filesystem, risks to data integrity without journaling (analogous to a transaction log for a database), the disappearance of the reboot option on Bonsai (and maybe other ROMs) because of this, blah blah blah this, blah blah that, and on and on and on.

There is another solution. You can still have the performance of EXT4, without journaling. With much lower risk to your data in case of the battery leaping out of the phone on a whim. And good ol' reboot can make a comeback too!

The nirvana is a linux command called 'sync'. Crusty old unix hacks like me will get a twinkle in their eye at the mention of this command.

sync does somesthing very straightforward and simple: It syncs the filesystems. Put another way, it flushes the in-ram buffers out to "disk", syncronizing the actual state of the filesytem with what is stored -- and stale -- in secondary storage (hard discs on the big boys, NVRAM/SD/whatever on phones).



I've been experimenting with this to see if I can improve data integrity while minimally impacting performance. Here's what I've done:

• Using GScript Lite, created a simple Superuser script for rebooting that looks like,
  
  sync
  sync
  sync
  /sbin/reboot #Bonsai ROM
  
  Why 3 syncs? Paranoid. Nothing more.
  
  Optionally create a shortcut on the homescreen to invoke this script to easy one-button reboot. I did this a week ago, have been using this to reboot Bonsai 4.0.0 a gazillion (actually, a bazillion, but I'm rounding) times, and have had absolutely no problems at all. Seems to work.
• Created a shell script that launches at boot, as superuser, that runs in an infinite loop waking up every 10 seconds to do a sync. No detectable impact on performance that I can see. This is what I'd expect, as there is never more than 10 seconds of filesystem activity sitting "dirty" in the cache, so the sync doesn't usually have much to do (most of the time, nothing).
  
  What does all this mean? Well, it's sort of a "lazy" journaling, and much more efficient. There's still a higher risk of data corruption under uncontrolled loss of power than with journaling, but in my considered opinion its negligible with the usage model/patterns for this particular situation (a smartphone).

FWIW, before I implemented this method for reboot, I like others got FCs on apps randomly after using the 3-finger reboot, simply running reboot directly from a shell prompt, or back on Bonsai 3, using the power-button menu reboot command. With this sync approach, I have not had a single problem -- and I can reboot the phone again easily!

What this means for you

If you're enough of a hack to understand how to implement this stuff yourself, give it a shot (at your own risk!), and let us know how it works out.

For the rest of you, be patient... I'm putting together a package to make it simple to install all this (initially just for the Bonsai ROM, others to follow, maybe), and should have something to test in a day or two. If you're interested in being a tester, PM me. Looking for 10 people.

epic4GEE said:

This sounds interesting. So will this "lazy" journaling put the same or less wear on the nand chip vs journaling being enabled?

If it has the same or similar wear factor as no journaling, as well as no impact on performance. Then this mod is a no brainer IMO.

Click to expand...

Click to collapse

In terms of the precise impact on storage, it is indistinguishable from not doing it at all. All this does if change the timing of the writes -- you control it, rather than waiting for the OS to decide to flush the filesystem buffers.

In the case of a reboot, this makes a ginormous difference, because anything in cache that hasn't been flushed is lost if the system doesn't sync before quitting back into the bootloader.



Sent from my mind using telepathitalk

styles420 said:

Does sync only cause a write operation for data that has changed, or simply rewrite the entire buffer to disk each time? I'm sure you can see where I'm going with this...

Click to expand...

Click to collapse

only dirty blocks.

sync should not cause anything to be written that wouldn't be eventually written anyway, when the kernel decides it is either idle enough to perform the deferred task, or cache "fullness" requires it to make room for newer data by flushing older stuff that hadn't been written yet.



Sent from my mind using telepathitalk

poit said:

It's better than nothing, but it still doesn't address battery pulls or phone freezes. It just provides a more graceful shutdown IF you do a clean shutdown.

Click to expand...

Click to collapse

True on the reboot mod.

On the monitor, though, it can make a big difference in the case of catastrophe. The FS cache is never more than 10 seconds out of sync with the underlying NV secondary storage. Depending on the usage model, this may be enough to reduce risk significantly.

The interval, of course, is configurable. The monitor could sync every second, reducing risk further. Given the speed of the processor in the 4G, and the low overhead hitting sync when there's nothing to flush, the overhead at a one sec interval may be trivial.

I haven't progressed far enough with this nascent idea to have characterized such questions. It's on the work order, though.

Also, my rough experimentation with this is all via shell scripting, which has a lot of unnecessary overhead. Coding this into an Android service, calling the linux sync(2) system call directly will be much more efficient.



Sent from my mind using telepathitalk

hotwired34 said:

I assume that the #Bonsai ROM is a comment and not needed, so that was omitted.

Click to expand...

Click to collapse

Yes, just a comment... across different ROMs I've found that devs mess around with the location, and linking, of the reboot command.

I have rebooted about 10 times (I know...many, many less than a Gazillion) and have not had any issues.

Thank you!

Click to expand...

Click to collapse

let us know when you get to a gazillion... our at least to a bazillion.

Oh, and anything with aliens or robots that comes up as well



Sent from my mind using telepathitalk