[math-fun] Flakey flash memory cards
(This question is also relevant to certain robots on Mars.) I bought a number of questionable flash memory cards (formatted as FAT32), and I wasn't disappointed with their questionableness -- they're failing, so I've been trying to figure out their failure modes w/o having access to their internal management software. They are showing bizarre failure modes. The files don't seem to be failing in a single block, but the failures seem to be more global. For example, a folder will cease being a folder, and look like a regular file, while the files in the folder can be recovered by file checking software. I see almost no failures (= modifications) on files less than 100Mbytes, but nearly 100% failures on files >200Mbytes. Has anyone else here studied flash memory card (uSD card) failures? Is there any software on the internet that can pinpoint failures so that I can simply remove these blocks from consideration? (I realize that it's cheaper to buy new uSD cards, but I thought it would be fun to understand the failures.)
Sounds like filesystem (metadata) corruption. Fist make sure that the device is working at all, dd_rescue is your friend: Overwrite the _whole_ device with zeros. dd_rescue /dev/zero /dev/sdx (find out what sdx should really be, see the output of the command dmesg after attachment of device). If any error occurs, bin the thing. Then read the whole device. dd_rescue /dev/sdx /dev/null If any error occurs, bin the thing. Partition (optional), and create a new filesystem. Enjoy. I do this for every storage device I use. Helps to sort out DOAs (which otherwise _appear_ to be OK until you use the majority of space; i.e., fail when it hurts most). Also sorts out fakes (more advertised space than actually there) and broken interfaces (sadly more common than one may expect). With flashy storage learn about MTBF, max. transfer per time, expected lifetime etc. beforehand 8^) Best, jj P.S.: hope you got a backup. * Henry Baker <hbaker1@pipeline.com> [Jan 23. 2015 08:50]:
(This question is also relevant to certain robots on Mars.)
I bought a number of questionable flash memory cards (formatted as FAT32), and I wasn't disappointed with their questionableness -- they're failing, so I've been trying to figure out their failure modes w/o having access to their internal management software.
They are showing bizarre failure modes.
The files don't seem to be failing in a single block, but the failures seem to be more global.
For example, a folder will cease being a folder, and look like a regular file, while the files in the folder can be recovered by file checking software.
I see almost no failures (= modifications) on files less than 100Mbytes, but nearly 100% failures on files >200Mbytes.
Has anyone else here studied flash memory card (uSD card) failures?
Is there any software on the internet that can pinpoint failures so that I can simply remove these blocks from consideration?
(I realize that it's cheaper to buy new uSD cards, but I thought it would be fun to understand the failures.)
_______________________________________________ math-fun mailing list math-fun@mailman.xmission.com https://mailman.xmission.com/cgi-bin/mailman/listinfo/math-fun
I've never trusted these particular devices, so I haven't lost any data. However, I'd love to figure out how to deal with errors on these types of devices, because sooner or later I will have to try to recover data. These are the new multi-level-cell flash devices, in which each charge well stores 3? (4?) bits. https://en.wikipedia.org/wiki/Flash_memory https://en.wikipedia.org/wiki/Multi-level_cell The problem with these multi-level cell devices is that you can write the first 1/8'th (say) of the cells on the device before you start writing over some of the first cells again (using additional levels). So even if you check every byte you just wrote, you can write 4 GB (say) before starting to put additional bits into the first cells. If your bits start interfering, the stuff you wrote minutes (or weeks) ago then starts to get clobbered. This effect is further masked by the "wear-leveling" "journaling file system". https://en.wikipedia.org/wiki/Flash_Translation_Layer It would be much easier to deal with errors if one could see the "raw" flash device, and utilize one of the newer file systems designed specifically for flash memory devices, rather than FAT32. However, these uSD cards have their own little processor which implements the journaling file system itself. I don't know if there is any way to force this little processor to do "fscheck", though. One good thing about FAT32 is that the file chains are all stored in one place, so as long as the file chain area itself is reliable, the file chains can be recovered. However, the directory structure may be lost, since the directories are stored like other files in file chains. It appears that these flakey flash cards are tested only on the file chain area, so the file chains appear to be reliable. However, the file data themselves seems to get corrupted relatively easily due to the multi-level phenomena discussed above. At 12:25 AM 1/23/2015, Joerg Arndt wrote:
Sounds like filesystem (metadata) corruption.
Fist make sure that the device is working at all, dd_rescue is your friend:
Overwrite the _whole_ device with zeros. dd_rescue /dev/zero /dev/sdx (find out what sdx should really be, see the output of the command dmesg after attachment of device). If any error occurs, bin the thing.
Then read the whole device. dd_rescue /dev/sdx /dev/null If any error occurs, bin the thing.
Partition (optional), and create a new filesystem.
Enjoy.
I do this for every storage device I use. Helps to sort out DOAs (which otherwise _appear_ to be OK until you use the majority of space; i.e., fail when it hurts most). Also sorts out fakes (more advertised space than actually there) and broken interfaces (sadly more common than one may expect).
With flashy storage learn about MTBF, max. transfer per time, expected lifetime etc. beforehand 8^)
Best, jj
P.S.: hope you got a backup.
participants (2)
-
Henry Baker -
Joerg Arndt