How do I protect SD card against unexpected power failures?

Question

I am working on a device that uses the Microchip MDDFS library to store data to an SD card. The logger will log data at a maximum rate of 1 entry (56bytes) every minute. The problem is the device may lose power at any time, potentially in the middle of a write sequence. I am wondering what is the best way to protect my data against corruption. I have found that if the file is open when the power is lost, all data that was written to the file after the last file-close is lost. I don't know if the same holds true if power is lost in the middle of the write sequence.

Since the write procedure doesn't happen very frequently I could open the file, write the data, and then close the file, every time data is logged. Would this approach damage the sd card over time?

Another approach could be to keep the file open but after every 10 or 50 writes I could close the file and then re-open it.

I could also buffer data in memory, then flush the data occasionally maybe after a kbyte or so.

The last idea I had was, in my circuit, I could add a large capacitor that would provide power to my pic/sd card long enough after the power is disconnected to quickly close the file. The problem with this approach is that the time it takes to close the file and/or save data is very inconsistent. From my understanding, this time can very depending on the current place in a flash page that the file is in.

Anyways, what would you guys suggest?

Answer 1

A few things can happen when you write data to a file. I'm going to describe the sequence that needs to happen for data to be safe, not necessarily library calls.

When you're writing, and adding on to the end of the file (normal write mode), you read the last block of the file into memory, modify it with your write data, and then write the whole block back to the SD card. If the block is full, then a new block must be found in the File Allocation Table (FAT). After finding a new block, the FAT must be updated, which is a read-modify-write cycle. If we're done with the file, then we need to update the file attributes (such as file length) in the root directory, which causes another read-modify-write cycle.

Minimizing your write time

• Make sure that the file is already holding your data when you write a sector. If you start out with a large file and overwrite the data instead of appending the data, the data will be safe as soon as the SD card sector write is finished. You can eliminate one to two read-modify-write cycles that way. My start-up code would write 0's to a file in sector increments until the SD card is full, and then rewind to the beginning of the file.

• Make the size of your data entries such that an integer number of entries fit in a sector. I would bump up your entries to 64 bytes. While this is less efficient, it will prevent you from needing to read-modify-write two sectors.

• Create a variant of the FSwrite function that allows you to write whole sectors. If you keep the whole sector in SRAM, then your cycle goes from "read-modify-write" to "modify-write"

Keep your PIC and SD power on as long as possible

• Big capacitors are good. 470uF should give you more than enough power to finish a write cycle.

• Make sure your power source won't suck the power out of your back-up capacitor! Add a diode if necessary.

Know when you're out of power

• A big power supply cap will give you 10ms or more to wrap things up with a SD card, but don't press your luck. Use a pin on your microcontroller to see if your power source is still good, and don't start a write if your source is dead.

Answer 2

One issue not yet mentioned with SD cards (or MMC, CompactFlash, etc.) is that while an SD card may appear to the host as a simple collection of 512-byte sectors which may be read and written in arbitrary order, flash devices generally store 528-byte pages in groups which are 32KB if not larger each, and the only operations supported are to either write to an otherwise-blank page, or to erase an entire group. To deal with this limitation, the controller on an SD card will keep a table which will allow any logical sector to be mapped to any physical page. When a request is made to write a sector, the controller will find a blank page somewhere on the chip and update the mapping with the new address of the sector in question. If blank pages get to be in scarce supply, or at various other times, the controller will move an erasable-group full of pages to new addresses, and then erase that entire group.

The significance of this is that the act of writing to a particular logical sector may require shuffling around the data from many logical sectors. If something goes wrong in that process, it could result in the corruption of any arbitrary sector--not just the sector the card was asked to write. A good SD card controller should be designed to perform the data-shuffling operations in such a way that if power is lost during a data-shuffle, it will be able to figure out which parts of the operation had completed and which had not, and consequently be able to finish off the operation properly. Unfortunately, I have no idea how one can tell whether the $5 SD card one picked up at a discount store will be any good in that regard.

To be sure, even if an SD card is absolutely perfect from the standpoint of ensuring that every write operation which was reported as having been completed will, in fact, survive a power failure (i.e. ensuring that whether or not all of the work the write would have caused is complete, enough has completed that the card will finish the operation when power is reapplied) that doesn't mean the host operating system won't have trouble if it performs some but not all of the data writes it intends. Nonetheless, it is important to bear in mind that if the SD card can't uphold its end of the "bargain", there's nothing that can be done on the host side software to prevent data loss from power failure.

Answer 3

I would also suggest using some sort of checksum to verify the data on the SD is correct whenever it needs to be read.

Answer 4

Perhaps this supercapacitor at Sparkfun would solve the problem.

Answer 5

Since the write procedure doesn't happen very frequently I could open the file, write the data, and then close the file, every time data is logged. Would this approach damage the sd card over time?

As with any engineering problem, you'll need to deal with tradeoffs here.

Is it critical that no-data be lost at all? Then I'd do the above. You'd have more damage done by losing the data than by ruining a card. You may want to do a stress test, of sorts, to determine how many times you could perform that operation before the card becomes corrupted. If your'e cool with the length of time it took before the card became unusuable, and it seems to be an acceptable length of time before changing the card, I'd go with that route.

Answer 6

If you want just to store data there is no need to a file system. the write operation will be done directly over the SPI by selecting the block adress. by doing that you minimize the write time and you the risk of data corruption.

Even in case of power loss and no luck, you will lose just one entry (which maybe acceptable on some system).

Answer 7

In addition to the other answers: Newer cards might support flushing the cache of the SD card and/or signalizing to the SD card that the host wants to power off. This is described in Part 1 of the Physical Layer Simplified Specification SD Card Specification (V8.00):

Chapter "4.17 Cache"/"5.8.2 Performance Enhancement Function" and "5.8.1. Power Management Function" ("Power Off Notification").

However, as far as I know, these functions are not supported in SPI mode, which is probably what you use. :(