A Guaranteed Way to Corrupt SD Card

Question

I am working on an embedded Linux project that it is important to be sure that the file system won't be destroyed when some unexpected thing occurs (e.g., power failure during a write operation on SD Card). For this purpose, I am trying to implement a possible failure scenario.

I have read that there is sdctl for this purpose, but in the article, it says that it may take approximately 4 weeks, which is very long time for a single test.

Do you know, or think of, any good scenarios or special combinations (writing data to a certain place etc.) to speed up this fail process?

By the way, I also appreciate the "corruption-immune combination for SD card" ideas (e.g., using a journaling file system). Thanks in advance.

Edit: Preventing (minor) data loss is not the priority for me. When corruption occurs, file system gets destroyed. So, I have to set the file system on SD card all over again, manually. This is what I want to prevent at least. And the question above is about setting a good test environment to shorten the test time.

Answer 1

If the data is really important, you should use a modern checksumming CoW (copy-on-write) file system like btrfs or ZFS. A recent kernel+btrfs can do "raid" on a single device, storing the data twice with checksumming to determine if one block is damaged, and repair from the (hopefully correct) copy if it happens. Copy-on-write guarantees that data is never overwritten and any eventual changeover happens atomically. Combined with features like snapshots for reliable factory reset, and automatic compression makes it a good choice for space-constrained storage in embedded systems. The main drawback is that it hasn't seen nearly as much testing as "less reliable" file systems, which can be a huge turn-off.

To answer your first question about data corruption, you need to add more details. It is unclear what you expect to happen. The failure mode for Flash storage is typically that you can no longer change a bit from 1 to 0; they are permanently 1. You also haven't told us which file system you are currently using, so I can't advice on sectors and places where you could write random data. Liberal sprinkling of 0xFF to random bytes in the card should do it, perhaps even 0xFF..FF on whole sectors.

Answer 2

Having had a similar problem with worse constraints - the requirement to use FAT filesystem - what we did for test purposes was build a software-controlled relay box to repeatedly power-cycle the device at intervals of a couple of minutes (it took >1m to boot up and resume testing). Systems usually failed overnight.

Answer 3

If you need to fail one or more bits/sectors on an SD card, I would suggest find an old/cheap one without wear leveling. That wat you can bit flip one bit/sector over and over until it fails. Assuming it’s an MCL flash (most common) inside the SD card, 10 000 writes will do the job.

If the SD card has wear leveling, your effort will increase by several orders of magnitude and you need to write much more data. Get one as small as possible to minimize the effort and fill everything but a few bytes. Write those bytes over and over until failure.

Answer 4

I've achieved my goal with this combination:

• A relay to simulate hard reset (power off) for the embedded Linux board. It is controlled by another mCU, creates periodical-ish power on/off cycles (ON time is between 10-15 minutes randomly, OFF time is 5 seconds).

On the Target Board:

• A script that copies /etc/ and /lib/ directories to /home/root/. This one starts initially and loops all the time that the board is powered.
• (I am not sure that this one is necessary) A program that creates dummy settings file (.ini), reads and changes its keys and values continuously.

This setup has approximately 80% success rate on my board. By the way, I am using ext2 file system for this step. Thank you all for helping me on this issue.