Datalogging on serial flash memory

Question

I am working on a datalogger that writes positions from GPS to serial flash W25Q64FV. I plan to write GPS data every second. I have couple of questions:

1. With this flash memory it is possible to write to any position from 1 to 256 bytes at once (page). Specification says the memory has 100,000 write/erase cycles. Is it counted per page or per byte? So if I write one byte at address X and then later another one at address X + 1 (but the same page) does it mean that I have only 99.998 writes left to that page? I would say no, but for sure I am asking. What if writing one byte internally rewrites complete page?
2. How to persistently keep address of last written byte? I could write it to Arduino's EEPROM on every update, but it also has "only" 100,000 write cycles so this would be reached much sooner. Other possibility could be to scan all written memory and look for empty byte(s) but this can probably take a while. Or maybe just to have some bitmap in the flash header marking full pages and I would only scan the last one. Any other suggestion?
3. Is there any special purpose of erasing blocks or complete flash memory? Can't I just write to memory in round-robin way? If I delete file on my HDD it won't write zeroes to all its bytes so couldn't it be same with flash?

Answer 1

First of all, the way flash works, in general, is that a write command can only change bits from one to zero. To return bits to one, you must erase an entire erase block. Therefore, you cannot just "write to it like a hard drive". On most flash devices, multiple writes can be made to the same block or even the same byte, again as long as you are only changing ones to zeros. (There are some flash devices where the manufacturer recommends against multiple writes to the same block, but this is rare.)

To answer your specific questions:

1) In general, the endurance is based on erase cycles on each block. You can write individual bits (to zero) as much as you want, but doing an erase to return bits to one will wear down those bits of flash.

2) One way is to use the first byte of each record as a status indicator. It starts, after erasure, as 0xFF. When you start writing the record, you write 0x7F to that byte (zero the high bit). Once you complete writing the record, you write 0x3F to that byte (zero the next highest bit). To mark a record as deleted, you write 0x00 to that byte. All of this takes advantage of the fact that you can always turn a bit from one to zero.

Later, when you're reading records, you only look at records with 0x3F in that byte. 0x7F means that the record writing was interrupted (power fail/reboot) and is invalid. Once an entire erase block has only 0x00, 0x7F and 0xFF blocks, it can be erased.

By the way, you can keep a map of used/free blocks in RAM so that you only have to scan when booting up.

3) See the top of this posting.

Answer 2

Having worked with flash and been bitten by various problems, I would suggest that it may be a good idea to design your data format in such a way that it will not be confused by anything that might appear in a block that was erased when power was lost. My present favored scheme is to use a rolling sequence of blocks, each of which has a byte which is programmed on all but the most recent block; the block following that block should be assumed contain garbage, and its content should be ignored. When the most recent block gets full, the following block should be erased and any necessary header information written. Once that is done, the former "most recent" block should have its flag byte programmed to indicate that it is no longer the most recent block (the newly-erased one is).

Note that with this scheme, there is no danger of a seemingly-valid data pattern appearing in a partially-erased block, or of software mistaking a partially-erased block for a blank one. It is not possible to avoid having code read the the "status" bit of a partially-erased block, but that block will be recognizable as garbage based purely on the contents of other non-garbage blocks.

Answer 3

1) A p/e-cycle for a sector would be you writing multiple values into that sector and erasing the whole sector at some point. So you could write all addresses in a sector and that would only count as one p/e-cycle if you only erase the sector once it is full.

2) Do you really have to persistently store the last memory location? I would probably just check for the last used Flash-location during device-power up and then keep the address counter in RAM.

3) Depending on your application you could just fill the device up and only erase once you have to. Or you could implement some sort of management that moves the start address around i.e. skip to the next block once the data has been transmitted to some external device. That could also be used as some simple form of wear leveling.

Answer 4

This is the way I do wareleveling on the EERPOM/FLASH.

Split your memory into two arrays, once for data and one as warelevel (I assume 16bit number for warelevel here, which works ok so long as you have < 65535 records).

To start out with all memory is should be 0xFF bytes. So the first write is a special case of both warelevel[0] and warelevel[1] being 0xFFFF, you write record data to data[0x0000] and you write warelevel[0x0000] = 0x0001.

On all subsequent writes you tranverse the warelevel array, looking for a non incrementing number. When you find one - you use that index for your data array, and update the warelevel array so that it is sequential.

So for the second write you will see that warelevel[0] = 0x0001, but warelevel[1] = 0xFFFF, which is not incremental. So you know the next index to write to is 1. So you write warelevel[1]=warelevel[0]+1, and write your data to data[1].

Then for the next write: warelevel[1]=0x0001 and warelevel[2]=0x0002, and warelevel[3]=0xFFFF, so now you know to write to address 3, and now you set warelevel[3] to 0x0003.

This continues on and one forever. Basically at anytime you can find the next position to write to by simply looking for a non incrementing number change in the warelevel array.

Then all you have to do is 'wrap around'. For example keep counting past 0xFFFF back to 0. With standard integer overflow, everything will work. The counter can go 0 to 0xFFFF, even if you have less than 0xFFFF records (However you need a bigger bit size counter if you have MORE than 0xFFFF records).

This will allow you to make sure every byte is written to an equal number of times. So the code is basically 'best use' regardless of if the life of of the memory is 10k 100k or 1M.

The downside is of course that you lose space to the warelevel array. The smaller your records - the more space you lose.

The advantages of this method is that you don't have to 'store' an index anywhere. So there is no memory location that gets smashed hard.