Will an SPI flash memory chip have the same issues with non-atomic write operations as a dsPIC's internal EEPROM?

Question

A while back I had some intermittent trouble with the internal EEPROM of a dsPIC. Every so often, some value in the EEPROM would be found zeroed out on power-on. I tracked the problem down to when the chip lost power after the erase step of the write cycle, but before the write had completed. It was all about the timing of the power-down relative to the firmware execution, which was (in normal operation) random. I solved this by adding a buffer section to my EEPROM, to ensure that an incomplete write-cycle could be completed on restoration of power. I had to turn EEPROM writes into an atomic operation.

Now I'm using a different dsPIC without internal EEPROM, and I'm trying to use an external flash memory chip to store persistent data. I'm wondering if I should have similar concerns. Should I be worried that my external flash chip will power down mid-write and lose data, and write a fix for this in my firmware like I did for internal EEPROM? Or does the chip itself guarantee atomic write operations?

For further detail, my buffering technique defines an area of persistent memory that consists of three fields: address to write to, data to be written, and a READY flag. A "write" consists of four steps: write to buffer, set READY flag, write from buffer, clear READY flag. On power-up, you check the READY flag. If it's set, execute whatever's in the buffer. This worked well in EEPROM, but I'm not sure if it will work well in flash.

Answer 1

I have never heard of a flash memory chip (or processor with internal flash) that has sufficient energy storage internally to complete a write (or erase) cycle if external power should be removed. In other words, if you don't have control over when your system powers down, you always need to create a protocol that can detect and deal with any individual flash update operation that might have been interrupted.

One way around this is to provide the necessary energy storage (e.g., an electrolytic capacitor) on your board, such that you can detect an external power failure, yet still complete any write/erase operation that may have already started.

EDIT: Your write buffer concept could be used with the external flash, but it needs to be modified to take into account the larger erase granularity. According to the datasheet, the minimum erase size is one "sector" (4K bytes).

You'll need to reserve three sectors for your write buffer. One of these will hold your READY flag (call this the WB_R wector). The second will hold the sector address of the sector being updated (call this the WB_A sector). The third will hold the updated data for that sector (call this the WB_D sector).

To update any particular byte (or a group of bytes in a single sector), follow the following steps. We assume that WB_R is already erased.

1. Erase WB_A.
2. Locate the flash sector that contains the byte you want to change (call this the DEST sector).
3. Write the sector address of DEST to WB_A.
4. Erase WB_D.
5. Copy the contents of DEST to WB_D, but when you get to the byte(s) that you're changing, write the new value(s) to WB_D instead of the old value(s).
6. Set the READY flag in WB_R. Note that this means you change it to its non-erased state. Since the erased state is 0xFF, this means that you write 0x00.
7. Erase DEST (getting the sector address from WB_A).
8. Copy the contents of WB_D to DEST.
9. Erase WB_R.

On power up, check the READY flag, and if it's set (anything other than 0xFF — it may have been only partially written or partially erased), jump directly to step 7.

Note that with this algorithm, each of the write buffer sectors gets written and erased at least once for each write operation you do. This could become a problem if you do a lot (more than 100,000) of writes over the lifetime of the product. If that's the case, you'll need a more sophisticated wear-leveling algorithm.

Answer 2

Buffered write is not sufficient. You need to take a leaf from the file system and database guys here: you need a data structure in flash that can recover to a "good" state when one block is corrupted.

A typical way to do this is to ping-pong between two blocks. Make the last two or four bytes of the blocks be the "serial number" of the block, and the rest of your data in the rest of the block. When you write a new block, increment the serial number of the previous block by one, skipping the erase "0" value (which may be 0xff, depending on flash type) and write the new block with that serial number.

On power-up, read both blocks, and see which one has the later serial number (taking into account the wrap from 0xffff->0 and ignoring the skipped erase values.) Use that block. You may also want to add a CRC of your data to verify it wasn't corrupted in the middle (although if you put the serial at the end, that "shouldn't" be a problem.)

If you have complex data, you can expand upon this in the way that a database or file system will update a tree on disk, or even implement write logging.

Answer 3

This is an area where you need to sit down and carefully strategize things. Some details from the data sheet are:

1. 4k sector erase: \$400ms\$ worst case
2. 32k block erase: \$800ms\$ worst case
3. 64k block erase: \$1000ms\$ worst case
4. page write: \$3ms\$ worst case
5. first byte write: \$40\mu s\$ worst case
6. next byte write: \$12\mu s\$ worst case

It's probably a good idea, if you have control over this detail, to arrange local power (via a capacitor's stored charge) that will sustain itself within a "droop margin" you determine while critical writes take place. This doesn't have to be a full second, if you wisely use the first byte write timing (don't forget to include additional communication/setup times with that.) You could update just one or two bytes in a special page that signifies that a block or sector erase is being started, for example. This allows you to determine, if you experience a brown-out or reset, where you were last at so you could finish the process. You may need more than one "special page" too. But in any case, you need to thoroughly consider all cases!

Answer 4

If power is lost while a flash chip is erasing the block, robust software should assume that the contents of the block may arbitrarily change at any time unless or until the block is re-erased and an erasure cycle runs to completion. Even if the block still appears to hold old data, there's no guarantee that it will continue to do so for any length of time. Even if the block appears to be erased, there's no guarantee that programmed bits won't spontaneously "appear". I've seen a few processors with internal flash that included an ability to check whether bits were "truly" blanked or were "thoroughly" programmed, but I've never seen such functionality exposed by an external flash device.

If one wishes to be able to periodically store data to flash, and ensure that in case of power failure every update will either succeed completely or not at all, one must have at least three flash blocks, and define a protocol such that whenever a block is being erased, one can determine that based only upon the contents of the other two blocks. There are a variety of protocols for implementing this; I'll suggest a simple one here, assuming that the amount of information to be stored is a full block minus one minimum-size programmable unit, and three blocks are available, which I'll call X, Y, and Z.

Each block will have a "control" bits within it which is reserved for tracking validity/erasure status; I'll call those bits x, y, and z. During operation, the system will maintain the invariant that the block which holds correct data will have its control bit blank; the "preceding" block (X is preceded by Z) will have its control bit programmed. The control bits for the remaining block (the one "following" the one with valid data) will be irrelevant. If all the control bits are blank, nothing has ever been properly written; if all control bits are programmed, something has gotten seriously corrupted.

To write new data, erase the block following the one which holds correct data, then store new data into that block. Finally, as the last step, program the control bit of what used to be the current block. Until that control bit is programmed, nothing will care about the contents of the block that was just programmed. Once that bit is programmed, nothing will care about the contents of the block following the new block. Provided that the system has enough energy available to ensure that the programming of that one bit will either succeed or fail cleanly, reliable operation is assured in all power-loss scenarios.

Suppose that x is programmed, y is blank, and z is anything. Because the valid data block must have its own flag blank and the previous block's flag must be programmed, X cannot be a valid block (flag x is programmed), and Z cannot be a valid block (because flag y is programmed). Consequently, Y is the only block which can hold valid data. Block X holds the previous version of the data, and Z cannot be relied upon to hold anything. When it's necessary to store new data, code should start by erasing Z (regardless of whether it already appears blank), and programming all the data which is should contain. If power is lost at any time during this process, the system state will be the same as before it began (based on the flags, the contents of Z are presumed meaningless, so its contents do not affect system state at all).

Only after all the writes to Z are complete and it holds valid data should flag y be programmed. Once that flag is written, Z will be recognizable as the block which holds valid data since its own flag will be blank while the preceding blocks' flag (y) is programmed; the fact that y is now programmed will mean Y is no longer valid.

The next time it's necessary to store new data, block X should be erased and have data stored there; completion should be indicated by programming flag z. The time after that, Y should be erased and have data stored there, with completion indicated by programming flag x. It is vital that attempts to program flags x, y, and z either run to completion or have no effect, but those are the only operations which need to be "guaranteed atomic" at the hardware level. All other writes to memory will be done to a block whose contents will never even be looked at(*) unless they run to completion.

(*) The system generally won't be able to avoid accessing the invalid block, but the system's behavior will be unaffected by the value read.

BTW, if one isn't confident in the ability to ensure that flag writes run to completion, there are various approaches with redundant flag bits which might potentially help somewhat, but reliability will no longer be assured. Suppose, for example, that the system loses power while bit y is partially programmed so it will sometimes read as programmed but sometimes as blank. If on the first power-up, y reads as blank, the next update would erase Z. If during that erase, the system loses power and on the next power-up, y reads as programmed, the system would assume that Z is the valid block. If y had read as programmed both times, then Z would have been the valid block and the next block erased would have been X. If it had read as blank both times, then Z would have been correctly recognized the second time as being the invalid block. Although one might try to guard against these dangers by adding redundant flag bits, such approaches don't help much. One may design things so that it would be "unlikely" for partially-programmed flags to behave in troublesome fashion, but that is fundamentally different from the guarantee that if flag writes work atomically, nothing the chip could report for any other partially-written data would cause any trouble.