What is good practice for storing and manipulating variable data in a microcontroller circuit when the data may exceed microcontroller working memory?

In my application, I am designing an access control system around a microcontroller. A user inputs a key (RFID fob scan or keycode), and if its value matches a known/valid value then access is granted (to door, locker, etc). Since I am likely to use a Particle Photon or Argon, networking shall be used to update the list of valid keys over time. Keys should be stored in non-volatile memory in case of power loss or network outages.

For a small number of keys, it would be straightforward to store and manipulate keys as an array variable in code, but if the total number of keys is very large, this array variable may exceed available microcontroller application memory. In regular computer programming there is RAM and hard disk (or database), so I expect there is appropriate non-volatile memory I can attach to a microcontroller. If this understanding is correct, what would be considered good practice in this situation? I am also guessing that we want to keep the following factors in mind: speed of access, wear leveling, ease of interfacing.

  • 1
    \$\begingroup\$ i would not store the keys on the device unless you need offline access. All your mentioned problems would be solved if you auth over the net instead of on-device. If the lag is user-acceptable, virtually all the advantages (security, updating, wear) are on the side of not storing keys. You send the server the code over https, or use a digest authentication method to protect the keys from 3rd-party observers. \$\endgroup\$ – dandavis Apr 16 at 19:13
  • \$\begingroup\$ You can readily buy SPI NOR flash up to 32 megabytes or I2C EEPROM up to 128k bytes. You can also at a premium get MCUs with two megabytes of internal flash. Ideally you store one way hashes so a stolen device is less of a gold mine. \$\endgroup\$ – Chris Stratton Apr 16 at 19:24
  • \$\begingroup\$ Have you locked in on a particular MCU? For example, there are MCUs with build-in F(e)RAM, which doesn't require wear-leveling techniques (if you choose to do wear-leveling you may be able to use ideas from Linux's wear-leveling code which is open and doesn't violate patents.) An MCU with F(e)RAM would not by itself solve your limited memory issue. But F(e)RAM is also available in external chips -- I know because i use them, regularly. You might also consider setting up a policy limiting the key count and/or aging old keys. \$\endgroup\$ – jonk Apr 16 at 19:28
  • \$\begingroup\$ Or just go with a Beaglebone or similar clone that already has Linux. If you're at all facile with Linux, it'll save you from screwing around with patching in the SD card reader. If you want to go with a lesser microcontroller and an SD card, check your favorite RTOS vendor to see if they have a file system add-on. \$\endgroup\$ – TimWescott Apr 16 at 19:37
  • \$\begingroup\$ I'd avoid using an SD card for this, or really as a routine and required part of any embedded system. If you go with an MCU (ESP whatever?) you spend time building up what you need, while if you go with an embedded Linux you spend time removing and locking down things until only what you need remains. If you were going to use Linux for this, a "router" chip like an AR9331, MT7688, etc running from a ramdisk initialized from a compressed image on SPI flash probably makes more sense than a more desktop-like Beagle, Pi, Etc. \$\endgroup\$ – Chris Stratton Apr 16 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.