LOST.DIR & sdcard data corruption

rsage · May 12, 2011

I've been experiencing some maddening issues with corruption of sdcard data. After much googling and finding some other reported occurrences of this, but not finding much info that's been helpful in solving the problem, I thought I'd compile and post some data in the hopes that others could add to it - hopefully leading to some kind of conclusion and maybe even a solution. I apologize because this is going to be a long post - hopefully it will be worth it.

Issue description:
There's a directory called LOST.DIR on sdcards in android phones. From what I've gathered, it contains lost/corrupted/orphaned files from the sdcard. When these files are moved to LOST.DIR, they're renamed with seemingly random strings of numbers and any file suffixes are stripped. Other OS's (Mac, Windows, etc.) have similar directories that serve similar purposes. The problem that I and others are experiencing is an alarming rate of sdcard data corruption, causing the LOST.DIR directory to fill up rather quickly, sometimes reaching multiple GB's in size. I’ve seen up to a dozen new files per day appearing in this directory. It's gotten to the point where I no longer trust that any data I put on my sdcard will remain intact.

I've read a few theories on how/why this happens. I've read that this is a Mac-related issue and only happens when mounting the sdcard on a Mac. This is not the case, as I've had this issue occur regardless if I'm mounting the card on a Mac, on a Windows PC (XP or 7), or even when mounting the card in a card reader and not from the phone directly (though I don't think the data corruption is occurring during mounting the card, but rather when the card is unmounted from the phone either when shutting the phone down or when unmounting the card from the phone during a request to remount the card via USB). I've even had a few (rare) occurrences of new files appearing in LOST.DIR without mounting the card on any device except the phone itself. I've also read that it's ok to just delete the files in this directory to reclaim space on the sdcard, but I've found that the files in LOST.DIR are often files I need, or files that various applications need to function. For the most part, when files are found in LOST.DIR, they've actually been removed from their original location. Deleting the files in LOST.DIR means you'll never be able to recover those files.

Once files end up in LOST.DIR, it's painstaking to recover them. Since they no longer have file types/suffixes, you may have to try opening the files in a utility to be able to tell what they are. Or by trial and error try appending different suffixes until you can open the files in an appropriate application.

Examples of files from my sdcard that have ended up in LOST.DIR:

whole mp3's
partial mp3's (more on this below)
photos & videos taken with the Epic's camera
xml config files
cached browser files
sql lite db's stored by various apps on the sdcard (causing one or two apps to FC since the sql lite files are no longer where the app expects them to be)
flashable zip files
etc.

Basically, any file I have on my sdcard can become corrupted or orphaned and end up in LOST.DIR in a near-unidentifiable state. To add to the pain, sometimes the files in LOST.DIR are not complete files, and sometimes they're not moved there intact. I was playing an album stored on the sdcard, and one track cut off in the middle. I found the other half of the mp3 file in LOST.DIR. I've lost entire photos and videos to LOST.DIR, but for the most part the files that are moved there are munged together with other files. If you view some files in LOST.DIR with a utility that can open any file as text, you'll see that a single file in LOST.DIR can really be multiple files or parts of files munged together into one. I've seen parts of text files mixed in with jpg's from browser cache mixed in with html mixed in with xml mixed in with other unidentifiable data. Obviously if this happens the data is not recoverable.

Some theories which I've ruled out or partially ruled out:

Mounting/unmounting method: I always shut down, mount, unmount, etc. the phone and the sdcard properly. If I have the card mounted on a Mac or PC, I always use the appropriate OS-specific method of unmounting the card before turning off USB mounting on the Epic. I ran tests where I kept track of every step I took in booting, unmounting, remounting, etc. the sdcard to confirm this. Other users have also ruled this out as a cause (though I'm sure improperly unmounting a card can cause data problems)
Journaling: I confirmed this happens regardless of whether journaling is turned off or on
Ext4 file system: don't think this has anything to do with it, as I've seen this happen on Ext4 and RFS
ROM's: I confirmed that this seems to be happening on multiple ROMs: midNIGHT/Bonsai, Baked Snack, stock deodexed, etc.
Kernels: Seems to be happening on multiple kernels: Bonsai kernel, Baked Snack kernel, maybe one or two others I've tried
Card file system corruption: I've reformatted my sdcard more times that I can count in trying to get rid of this problem, doesn't seem to have any effect
Epic4g itself: this seems to be happening on many different android phones, so I'm ruling out a design flaw with the Epic4g

Theories I haven't ruled out:

android (froyo?) bug related to data integrity when mounting/unmounting the sdcard (let's hope not)
damaged or defective sdcard

I've read that some people have had success in banishing this problem by moving to a higher-class sdcard (doesn't really make sense to me, but whatever). I'm currently using the OEM 16gb (I'm assuming class 2 or 4) card that came with the Epic. In researching class 6 cards, it seems that the 8gb cards are generally more reliable than the 16gb cards, so I ordered a highly-rated 8gb class 6 card (still waiting for its arrival). I'll gladly give up storage space if I can count on data integrity. If it's a low-level android bug, then who knows if/when it will ever get dealt with.

Here are two links that have more info and user experiences with this issue:
- Google Code issue:
http://code.google.com/p/android/issues/detail?id=2500

- ZDNet article:
http://www.zdnet.com/blog/sybase/could-your-sd-card-cause-your-android-smartphone-to-crash-or-lose-data/1128

To be honest, I don't know how long I've been experiencing this problem. I don't know if it happened on Eclair, but it definitely happens on Froyo.

I apologize that this is so long, but hopefully it will help someone else and possibly even lead to some helpful information in dealing with this problem. I'll post my findings once I receive the new class 6 sdcard.

Peace.

p3dr0maz · May 12, 2011

Good info.

Do the original files get messed up anyway? I'm guessing the original files stay intact yeah? Or do they end up being messed up and unusable?

So for example, an mp3 file ends up in LOST.DIR but really it's still on your sdcard and you can play it with Media Player?

Let us know if a higher class card == less LOST.DIR files once you get that.

rsage said:
I've been experiencing some maddening issues with corruption of sdcard data. After much googling and finding some other reported occurrences of this, but not finding much info that's been helpful in solving the problem, I thought I'd compile and post some data in the hopes that others could add to it - hopefully leading to some kind of conclusion and maybe even a solution. I apologize because this is going to be a long post - hopefully it will be worth it.

Issue description:
There's a directory called LOST.DIR on sdcards in android phones. From what I've gathered, it contains lost/corrupted/orphaned files from the sdcard. When these files are moved to LOST.DIR, they're renamed with seemingly random strings of numbers and any file suffixes are stripped. Other OS's (Mac, Windows, etc.) have similar directories that serve similar purposes. The problem that I and others are experiencing is an alarming rate of sdcard data corruption, causing the LOST.DIR directory to fill up rather quickly, sometimes reaching multiple GB's in size. I’ve seen up to a dozen new files per day appearing in this directory. It's gotten to the point where I no longer trust that any data I put on my sdcard will remain intact.

I've read a few theories on how/why this happens. I've read that this is a Mac-related issue and only happens when mounting the sdcard on a Mac. This is not the case, as I've had this issue occur regardless if I'm mounting the card on a Mac, on a Windows PC (XP or 7), or even when mounting the card in a card reader and not from the phone directly (though I don't think the data corruption is occurring during mounting the card, but rather when the card is unmounted from the phone either when shutting the phone down or when unmounting the card from the phone during a request to remount the card via USB). I've even had a few (rare) occurrences of new files appearing in LOST.DIR without mounting the card on any device except the phone itself. I've also read that it's ok to just delete the files in this directory to reclaim space on the sdcard, but I've found that the files in LOST.DIR are often files I need, or files that various applications need to function. For the most part, when files are found in LOST.DIR, they've actually been removed from their original location. Deleting the files in LOST.DIR means you'll never be able to recover those files.

Once files end up in LOST.DIR, it's painstaking to recover them. Since they no longer have file types/suffixes, you may have to try opening the files in a utility to be able to tell what they are. Or by trial and error try appending different suffixes until you can open the files in an appropriate application.

Examples of files from my sdcard that have ended up in LOST.DIR:

whole mp3's

partial mp3's (more on this below)

photos & videos taken with the Epic's camera

xml config files

cached browser files

sql lite db's stored by various apps on the sdcard (causing one or two apps to FC since the sql lite files are no longer where the app expects them to be)

flashable zip files

etc.

Basically, any file I have on my sdcard can become corrupted or orphaned and end up in LOST.DIR in a near-unidentifiable state. To add to the pain, sometimes the files in LOST.DIR are not complete files, and sometimes they're not moved there intact. I was playing an album stored on the sdcard, and one track cut off in the middle. I found the other half of the mp3 file in LOST.DIR. I've lost entire photos and videos to LOST.DIR, but for the most part the files that are moved there are munged together with other files. If you view some files in LOST.DIR with a utility that can open any file as text, you'll see that a single file in LOST.DIR can really be multiple files or parts of files munged together into one. I've seen parts of text files mixed in with jpg's from browser cache mixed in with html mixed in with xml mixed in with other unidentifiable data. Obviously if this happens the data is not recoverable.

Some theories which I've ruled out or partially ruled out:

Mounting/unmounting method: I always shut down, mount, unmount, etc. the phone and the sdcard properly. If I have the card mounted on a Mac or PC, I always use the appropriate OS-specific method of unmounting the card before turning off USB mounting on the Epic. I ran tests where I kept track of every step I took in booting, unmounting, remounting, etc. the sdcard to confirm this. Other users have also ruled this out as a cause (though I'm sure improperly unmounting a card can cause data problems)

Journaling: I confirmed this happens regardless of whether journaling is turned off or on

Ext4 file system: don't think this has anything to do with it, as I've seen this happen on Ext4 and RFS

ROM's: I confirmed that this seems to be happening on multiple ROMs: midNIGHT/Bonsai, Baked Snack, stock deodexed, etc.

Kernels: Seems to be happening on multiple kernels: Bonsai kernel, Baked Snack kernel, maybe one or two others I've tried

Card file system corruption: I've reformatted my sdcard more times that I can count in trying to get rid of this problem, doesn't seem to have any effect

Epic4g itself: this seems to be happening on many different android phones, so I'm ruling out a design flaw with the Epic4g

Theories I haven't ruled out:

android (froyo?) bug related to data integrity when mounting/unmounting the sdcard (let's hope not)

damaged or defective sdcard

I've read that some people have had success in banishing this problem by moving to a higher-class sdcard (doesn't really make sense to me, but whatever). I'm currently using the OEM 16gb (I'm assuming class 2 or 4) card that came with the Epic. In researching class 6 cards, it seems that the 8gb cards are generally more reliable than the 16gb cards, so I ordered a highly-rated 8gb class 6 card (still waiting for its arrival). I'll gladly give up storage space if I can count on data integrity. If it's a low-level android bug, then who knows if/when it will ever get dealt with.

Here are two links that have more info and user experiences with this issue:
- Google Code issue:
http://code.google.com/p/android/issues/detail?id=2500

- ZDNet article:
http://www.zdnet.com/blog/sybase/could-your-sd-card-cause-your-android-smartphone-to-crash-or-lose-data/1128

To be honest, I don't know how long I've been experiencing this problem. I don't know if it happened on Eclair, but it definitely happens on Froyo.

I apologize that this is so long, but hopefully it will help someone else and possibly even lead to some helpful information in dealing with this problem. I'll post my findings once I receive the new class 6 sdcard.

Peace.

mkasick · May 12, 2011

If I had to guess, it sounds like silent write corruption on the SD card, whereby writes to the card are reported successful, but the written blocks are (ocasionally) corrupted, which don't show up until they're later read. In which case I'd suspect the controller on the SD card itself is to blame.

If it wasn't silent corruption, i.e., writes were returning errors, the SD card should remount read-only automatically and there would be some OS-level notification. Although I suppose any such notification could be ignored.

The mounting issue is possibly a red herring, for two reasons. First, data blocks written to the SD card are also cached in RAM. Thus, once mounted, most of the read operations on critical file system metadata will come out of the file system buffer cache and not actually from the SD card. So even if the write corruption is an on-going thing, it could easily go masked until a device reboot or SD card unmount/remount, at which point the cached blocks are purged and (the corrupted versions) read from the card.

Second, fsck_msdos, which is responsible for populating LOST.DIR, runs at the time the SD card is mounted which is why mount events often corresponds to lost files showing up there.

rsage said:
Here are two links that have more info and user experiences with this issue:
- Google Code issue:
- ZDNet article:

Not to outright dismiss the existence of a kernel-level software bug, but the fact that relatively few folks for a given device are frequently reporting the issue leads me to think it's often hardware (likely SD card) related, or a result of a very unusual set of circumstances. At least, for the many bugs folks report having on the Epic, rampant SD card data corruption is not one I'm aware of.

See if the new SD card helps, if it does, I'd say you just have a bum card and wouldn't even blame it on a particular brand or class.

If you continue to have issues even with the new card, it would be nice to rule out a hardware defect with your particular Epic, but short of you getting a new phone or other people reporting the same issue, that would be difficult to do.

Otherwise the two best things to do are to: (i) figure out a set of files/operations/whatever that reliably repeat the corruption issue, if possible, and (ii) narrow the sources of corruption down by avoiding mounting the SD card on other machines. You may have to occasoinally "Unmount SD card" and remount it from the Settings -> Sd card and phone storage for the corruption to show up though.

rsage · May 12, 2011

p3dr0maz said:
Good info.

Do the original files get messed up anyway? I'm guessing the original files stay intact yeah? Or do they end up being messed up and unusable?

So for example, an mp3 file ends up in LOST.DIR but really it's still on your sdcard and you can play it with Media Player?

Let us know if a higher class card == less LOST.DIR files once you get that.

Very good question - the answer is "all of the above"

Some files (I'd say ~25%) have been recoverable once I've been able to figure out what they are. The majority of the files have been unrecoverable though - either because part of the file remained in its original location and part of the file got moved to LOST.DIR, or because several files or pieces of several different files got moved from their original locations to LOST.DIR and got "munged" together in one file.

rsage · May 12, 2011

mkasick said:
If I had to guess, it sounds like silent write corruption on the SD card, whereby writes to the card are reported successful, but the written blocks are (ocasionally) corrupted, which don't show up until they're later read. In which case I'd suspect the controller on the SD card itself is to blame.

Very good theory, though I and others have noticed one thing that goes against it - some of the files were living and being used on the sdcard for quite a while before they got corrupted and moved to LOST.DIR. A couple examples: 1) I had listened to the affected mp3 several times before one day it suddenly cut off near the 1/2 way mark, after which I discovered the other half of the file was in LOST.DIR, and 2) I had used/read several other read-only files and sql lite db's numerous times before they suddenly no longer existed and were found in LOST.DIR (sometimes recoverable, sometimes not). When the latter happened, a zero byte file with the same name existed in the original file location, but the real file had been renamed as something like "567801" and moved to LOST.DIR. So it seems that when they were originally written to disk, the writes were successful.

The mounting issue is possibly a red herring, for two reasons. First, data blocks written to the SD card are also cached in RAM. Thus, once mounted, most of the read operations on critical file system metadata will come out of the file system buffer cache and not actually from the SD card. So even if the write corruption is an on-going thing, it could easily go masked until a device reboot or SD card unmount/remount, at which point the cached blocks are purged and (the corrupted versions) read from the card.

Second, fsck_msdos, which is responsible for populating LOST.DIR runs at the time the SD card is mounted, which is why mount events often corresponds to lost files showing up there.

Great information, thanks!

Not to outright dismiss the existence of a kernel-level software bug, but the fact that relatively few folks for a given device are frequently reporting the issue leads me to think it's often hardware (likely SD card) related, or a result of a very unusual set of circumstances. At least, for the many bugs folks report having on the Epic, rampant SD card data corruption is not one I'm aware of.

Possibly (and hopefully) you're right. Another possibility is that it's a more widespread issue, and users aren't realizing it for various reasons. Most of the corrupt/orphaned files I'm seeing in LOST.DIR wouldn't necessarily cause any overt symptom that anyone would notice - files like browser cache files, etc. It might not be noticeable to the user until a more important file is affected, and the issue may be dismissed out-of-hand or chalked up to some other more common issue usually fixed by clearing the data for a problematic app. One of the most common things I see on these boards is sudden and unexplained FC's of various apps, sometimes remedied by clearing data for the app, sometimes by reinstalling the app, and often users simply wipe all data and start over - never sure what the cause of the problem was. Every phone that's mentioned in various threads on this issue, or other android phones I've personally checked, have had LOST.DIR and LOST.DIR has never been empty. This seems to be true even if a specific phone wasn't having any issues that the user noticed. Obviously my sample size is very small, so I'm not saying this is universally true. But I think it's at least a possibility that the issue is more widespread and users may not be noticing it due to the nature of the affected files or maybe they're thinking it's some other problem.

I've also read a fair number of comments on this issue like "dude, wow I didn't know I had this directory but I do and there's 2gb of stuff in there!" Well, that 2gb of "stuff" had to come from somewhere

See if the new SD card helps, if it does, I'd say you just have a bum card and wouldn't even blame it on a particular brand or class.

If you continue to have issues even with the new card, it would be nice to rule out a hardware defect with your particular Epic, but short of you getting a new phone or other people reporting the same issue, that would be difficult to do.

Agreed, though this problem does seem to span many different brands/models of android devices (for those that have noticed the problem), so I'm thinking it's probably not my Epic. I'm hoping it's the card like you said. MicroSD cards aren't the most reliable storage media on the planet, so I'm really hoping it's just the card.

Otherwise the two best things to do are to: (i) figure out a set of files/operations/whatever that reliably repeat the corruption issue, if possible, and (ii) narrow the sources of corruption down by avoiding mounting the SD card on other machines. You may have to occasoinally "Unmount SD card" and remount it from the Settings -> Sd card and phone storage for the corruption to show up though.

Agreed. Though I've had occasions when I've been testing and the files in LOST.DIR will increase without me really doing anything. In other words, I'll do a fresh boot of the phone, look in LOST.DIR with Root Explorer or ES and check the file count, then shut down the phone normally, wait a while, reboot normally, and there may be a few additional files in there. Of course there are always background processes running and probably accessing the card even if I'm not doing anything on the phone, so who knows. I've also rebooted the phone, checked the file count, properly mounted and unmounted the phone via USB on a Mac or PC, then checked the file count again just to see that there were a dozen new files in there. Good times...

mkasick - you seem very knowledgeable about the mechanics of reads/writes and internal android processes - do you think it's possible that the characteristics of certain ROM's, kernels, CW versions, or simply the act of flashing things all the time could have anything to do with this? I'm thinking not...

mkasick · May 13, 2011

rsage said:
Very good theory, though I and others have noticed one thing that goes against it - some of the files were living and being used on the sdcard for quite a while before they got corrupted and moved to LOST.DIR.

The Wikipedia page on the FAT file system explains how it works a bit, although I imagine there's more illustrative examples out there. But basically, file data blocks are stored in clusters, and there's a table that contains entires for each cluster in the file system, where each table entry contains the index (location) of the next cluster, if that cluster is part of a file.

So it's quite possible that file data blocks for a given file are written out fine, but when the FAT is modified while writing/modifying a file that happens to be adjacent to it, the cluster indexes for the existing file are corrupted.

rsage said:
1) I had listened to the affected mp3 several times before one day it suddenly cut off near the 1/2 way mark, after which I discovered the other half of the file was in LOST.DIR

This is likely the result of a single corrupt cluster index. The corrupt index points to a cluster in the middle of the mp3, and if it were accidentally changed to "0" then (roughly) half the file is lopped off. Later, when fsck_msdos runs and scans the FAT and directory entries, it sees a chain of valid clusters that's not attached to any file. So it gives it a new file name and stuffs it in LOST.DIR.

rsage said:
When the latter happened, a zero byte file with the same name existed in the original file location, but the real file had been renamed as something like "567801" and moved to LOST.DIR.

If the entire file was in LOST.DIR, it sounds like a corrupt directory entry. That's a bit more suspicious, particularly if other part of the directory entry (name, time, etc.) were OK.

rsage said:
Most of the corrupt/orphaned files I'm seeing in LOST.DIR wouldn't necessarily cause any overt symptom that anyone would notice - files like browser cache files, etc.

One way orphaned files happen is when a file is deleted, and it's corresponding directory entry is removed, but the updated FAT hasn't been written out yet. The phone shouldn't be doing this too often, as long as the SD card is always properly unmounted (e.g., phone is properly shutdown). I could see computers doing this rather frequently, particular if the USB device isn't "safely removed".

Afterall, orphaned files in that instance aren't a safety/corruption issue. There's just less free space in the file system until a fsck or other consistency check tool is run.

In any event, I don't find the existence of these alone to be terribly surprising, especially in absence of missing files or other more significant corruption.

rsage said:
One of the most common things I see on these boards is sudden and unexplained FC's of various apps, sometimes remedied by clearing data for the app, sometimes by reinstalling the app, and often users simply wipe all data and start over - never sure what the cause of the problem was.

In almost all instances I expect those are due to data corruption on the /data file system, which, with ext4 can often happen on a phone crash or battery pull. Even with a journal, ext4 uses a delayed allocation scheme whereby newly allocated data may not be committed up to a few minutes after being "written", a window during which if the phone crashes, those files will experience data loss.

rsage said:
Every phone that's mentioned in various threads on this issue, or other android phones I've personally checked, have had LOST.DIR and LOST.DIR has never been empty.

As an anecdote: I usually end up with 2-3 orphaned files in LOST.DIR everytime I do a kernel upgrade, which I issue from within Android using redbend_ua without a clean shutdown. But I've never seen more obvious corruption. And I never see files in LOST.DIR after preforming a clean shutdown and clean unmounts.

rsage said:
But I think it's at least a possibility that the issue is more widespread and users may not be noticing it due to the nature of the affected files or maybe they're thinking it's some other problem.

You may be right about that, particularly if the affected files are thumbnails or related to the media cache.

Still, I suspect if the problem were somewhat-spread among Epic owners here, we'd see a lot more posts about SD card corruption.

rsage said:
do you think it's possible that the characteristics of certain ROM's, kernels, CW versions, or simply the act of flashing things all the time could have anything to do with this?

Since most of those activities don't result in SD card mutations, I suspect not. Also, it doesn't look like "the problem" is confined to users to root and custom ROM.

A few more notes:

If it always happens after a phone crash, random reboot, etc, it's just file system corruption and there's little that can be done. Amusingly, RFS is essentially a "journaled FAT" and would solve that problem, but it's not
used on SD cards on Samsung devices, which otherwise seems like a perfectly appropriate use for it.

I'm suspicious of it being a bug in the file system driver. If it was, corruption would be discovered at random times instead of specifically after a reboot or remount event, which is when folks seem to report it happening.

It could be a bug in fsck_msdos. I'd be a bit surprised if it was, but I don't think fsck_msdos is used particularly often outside of Android or other embedded devices, so the code may not be as well vetted as, say, fsck.ext3. One way to tell would be to disable fsck_msdos entirely and see if data still goes missing.

k0nane · May 13, 2011

EDIT 10char

stackz! · May 13, 2011

k0nane said:
Just to jump in with my opinions - I'd format that SD card. If you continue to have issues, something else is at fault - but at this point, it sounds like you've just got a corrupted/out-of-sync FAT, and the easiest way to guarantee a fix is a format.

Someone didn't bother to read the thread...

k0nane · May 13, 2011

stackz! said:
Someone didn't bother to read the thread...

I did read the thread. I missed the part where he stated he'd formatted his SD card. Piss off.

rsage · May 13, 2011

mkasick

Thanks for the great post, that really clarifies what could potentially be happening with the data. I'll report back when I have more info.

Sent from my SPH-D700 using XDA App

ecooce · May 13, 2011

You know Ive noticed this also,photos will just up and dissapear or get corrupted and wont open and really lag the gallery app,same thing happens to songs; Ive never worried about it,I just chalked it up to a bad sdcard,and Ive always got 3 backups of all data I plan on keeping. I would also like to know what the real cause is,keep us informed...

Sent from my MyFrankenstein EC05OCE using XDAPA...

zman519 · May 13, 2011

I have also had this same problem. Its really disgusting. This is about all I can take from this phone. I'm about to smash it and go back to a dumb phone. What's the point of a smart phone is the data is not relable.

OK sorry about the ranting.

I started a thread about this in Q&A. I refomatted the card yesterday again I think that's the 5th time. And so far its good AFAIK 21 hours in. O all the things/Devices I had moded hacked I have never had data ****ing up like this.

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.

rsage · May 13, 2011

zman519 said:
I have also had this same problem. Its really disgusting. This is about all I can take from this phone. I'm about to smash it and go back to a dumb phone. What's the point of a smart phone is the data is not relable.

OK sorry about the ranting.

I started a thread about this in Q&A. I refomatted the card yesterday again I think that's the 5th time. And so far its good AFAIK 21 hours in. O all the things/Devices I had moded hacked I have never had data ****ing up like this.

I feel your pain. Found your thread in Q&A, I missed it when I was searching the forum before I started this thread, probably because there's no reference to LOST.DIR. A lot of similarities in our setups and the issues we're seeing. Keep us posted.

By the way, you mentioned that you've had files go down to zero bytes but then at some point go back to normal. I've never had a file come back from this kind of corruption. Do you have any idea how this happened - did you run any kind of utility on the sdcard from windows or android OS for example?

mkasick · May 13, 2011

How repeatable is the corruption?

If, for example, you reformat the SD card, restore its contents from backups, is it the same files that are corrupted after some time? Roughly how long after a reformat is corruption observable?

rsage · May 13, 2011

mkasick said:
How repeatable is the corruption?

If, for example, you reformat the SD card, restore its contents from backups, is it the same files that are corrupted after some time? Roughly how long after a reformat is corruption observable?

Only answering for myself - never been the same files for me. Seems that it could be any file at any time, I've never actually seen the same file get corrupted twice before or after a reformat (assuming I was able to restore it after the first corruption). I can see new corruption after a reformat as soon as I start unmounting/mounting the sdcard. If I reformat then never unmount/mount the card via USB, occurrences are much rarer (though I have seen a few occurrences after just rebooting the phone, which of course unmounts & mounts the card as well).

mkasick · May 13, 2011

rsage said:
I can see new corruption after a reformat as soon as I start unmounting/mounting the sdcard.

And these are confirmed missing files that are no longer in their original location, or are zero-length or something?

When you say "unmounting/mounting", do you mean just on the phone, or only after mounting/unmounting it on a PC? For example, if you were to reformat, copy a bunch of data over, verify (on the phone) that it's correct, then click "Unmount SD card" on the phone a bunch of times (remounting inbetween), does that pretty much guarantee corrupt/lost files?

Also, do you store any apps on the SD card? If so, do those appear to be OK, or can they get corrupted as well?

Edit: The more times you mount/unmount the SD card, does corruption appear to worsen? Is this independent of how much you otherwise actually use the card?

rsage · May 13, 2011

mkasick said:
And these are confirmed missing files that are no longer in their original location, or are zero-length or something?

Well, I've seen both. I've had files go completely missing (photos taken with the camera, an mp3 or two, etc.), I've had files go to zero bytes (zip's, sql lite db's), and I've had files stay in place but get partially truncated (other mp3's, maybe some others).

When you say "unmounting/mounting", do you mean just on the phone, or only after mounting/unmounting it on a PC? For example, if you were to reformat, copy a bunch of data over, verify (on the phone) that it's correct, then click "Unmount SD card" on the phone a bunch of times (remounting inbetween), does that pretty much guarantee corrupt/lost files?

I'd say 90% of the issues I've had have been noticed after mounting the card, while in the phone, on a Mac or Windows PC. I notice the issue after I then unmount the card from the Mac/PC, at which point the Epic mounts the card. However, I've also seen it happen when shutting down the Epic, removing the card, and mounting it on a Mac or PC with a card reader (I see the evidence after reinserting the card back in the Epic and booting back up). I'm not sure if it's happening when the card is unmounted from the Epic, or when it's mounted on some other device, I could try to track that down.

I have less often seen it happen when shutting down and restarting the phone. I haven't tried repeatedly hitting unmount/mount in the Epic settings.

Also, do you store any apps on the SD card? If so, do those appear to be OK, or can they get corrupted as well?

Good question - until a few days ago, I had no apps on the SD card. I've been noticing the data corruption well before that - weeks, if not a couple months. Really don't know how long it's been going on. Within the last few days I have had to move some apps to the SD card. I haven't had an actual app/apk get corrupted (that I'm aware of), but I have had data get corrupted for apps that store data on the SD card.

Edit:

Edit: The more times you mount/unmount the SD card, does corruption appear to worsen? Is this independent of how much you otherwise actually use the card?

Yes, and yes

zman519 · May 13, 2011

rsage said:
I feel your pain. Found your thread in Q&A, I missed it when I was searching the forum before I started this thread, probably because there's no reference to LOST.DIR. A lot of similarities in our setups and the issues we're seeing. Keep us posted.

By the way, you mentioned that you've had files go down to zero bytes but then at some point go back to normal. I've never had a file come back from this kind of corruption. Do you have any idea how this happened - did you run any kind of utility on the sdcard from windows or android OS for example?

I did not run any thing on the sdcard with the data on it, win or android. At frist I saw this happening to videos I hid with "pandora" (its a media hider) so I thought that was messing up the data. But then my nandroid imgs where not the right hash & some times 0 bytes so at that point I ruled out pandora.

This is just an idea I have know testing to conferm it. BUT what if the problem is simpley a junk samsuck sdcard? I mean look at it, the usb cable is junk that came with the phone, froyo from them is junk, bluetooth, gps on 2.1 (if it where not for the devs hear the phone would be totaly junk to me and I would have got a new phone)

Sent from the Drivers Seat of my Suby txting and Driving doing 100MPH+ in a school zone! Ha.

mkasick · May 13, 2011

rsage said:
I'd say 90% of the issues I've had have been noticed after mounting the card, while in the phone, on a Mac or Windows PC. I notice the issue after I then unmount the card from the Mac/PC, at which point the Epic mounts the card.

I think what we're establishing here, and correct me if I'm wrong, but data corruption appears to be correlated with mount events on the Epic. The more times you mount, the higher probability of encountering data corruption.

Furthermore, there's instances in which the corrupt data was observed to be OK on a PC prior to the remount. And it's independent of how much the card is actually used on the Epic between mounts.

If that's all the case, then there's definitely something suspicious about the mounting process. It's certainly not spurious corruption. It could still be "bad cards," in the sense that the mounting process is correct but tickles something in the controller. But I'm inclined to believe it may be a software-level issue afterall.

I'm still curious if the problem persists across multiple SD cards in the same device.

rsage · May 13, 2011

zman519 said:
This is just an idea I have know testing to conferm it. BUT what if the problem is simpley a junk samsuck sdcard? I mean look at it, the usb cable is junk that came with the phone, froyo from them is junk, bluetooth, gps on 2.1 (if it where not for the devs hear the phone would be totaly junk to me and I would have got a new phone)

That's possible, new card is coming today so hopefully I'll have new info soon

LOST.DIR & sdcard data corruption

Senior Member

Senior Member

Retired Recognized Developer

Senior Member

Senior Member

Retired Recognized Developer

Inactive Recognized Developer

Member

Inactive Recognized Developer

Senior Member

Senior Member

Senior Member

Senior Member

Retired Recognized Developer

Senior Member

Retired Recognized Developer

Senior Member

Senior Member

Retired Recognized Developer

Senior Member

Similar threads

Top Liked Posts