In general, how do I know if my project needs a file system? [closed]

Question

This question gives a good high-level overview of when an operating system is appropriate on an embedded platform. The three topics mentioned are networking requirements, GUI requirements, and file system requirements.

Networking and GUIs are obvious enough, but I am wondering when a file system is appropriate and/or required for an embedded system. What sort of embedded tasks lend themselves to a file system solution?

Maybe something like a data collection application, where previous measurements or log files need to be stored for future reference?

Answer 1

The most obvious situations where I find a file system desirable:

1. Whenever you have to exchange data with an external system (eg. a PC). Definitely a FAT file system is the easiest solution when using an SD card or USB stick (example: datalogger, preparing config files on a PC).

2. You use NAND storage. You need a file system that is NAND-aware, because all NAND chips have bad sectors and the file system has to work around them.

Answer 2

Operating Systems abstract things so different applications on the same machine do not need to care about each other.

File systems follow the same logic. If you need storage for multiple independent components that should not need to coordinate resource usage between each other, you need a mediating component, which would be a file system.

This can be a generic mapping from a (possibly hierarchical) primary key to a single data blob of arbitrary length, as most file systems, or it can contain additional features like secondary streams, indexes (e.g. artist search in MP3 folders), a record format like in Palm OS, ...

Answer 3

I worked on a project of remote data collection. While the system were supposed to be permanently connected through RF, we had to handle the case in which either the RF, the internet down or any other reason data couldn't be sent as we couldn't afford to loose data.

At that time we were using an RTOS with a Fat file system on a PIC24 and SD card. Unfortunately, this wasn't reliable as in case of power loss, it could happen to have the file system corrupted.

Also FAT is quite complex to handle with uC, especially when you start to have a lot of files.

Our solution at the end was to write data in raw into the SD card with the write/read sector being stored on the first 10 sector of the SD with randomisation to avoid flash wear.

The downside was that we had to use a special software in admin mode to read the data from a computer, although it was much more reliable than a Fat file system, and also much easier to handle on the PIC.

Looking back, I would probably now use an embedded Linux to handle this instead of a PIC as it saves a lots of time and headache. Connectivity is also much easier to handle on a linux than a uC, downside of linux is cost, power consumption, boot time and IO latency depending on what you do.

Answer 4

The only time you need a file system is when you have a requirement which mentions it, like "the system should provide the user with such and such files" or "the system configuration has to be stored in a config file".

In all other cases, a file system is a trade-off between software complexity and convenience. File systems let you store data without dealing with low-level details, are easily scalable, and (if you pick a compatible filesystem) make data exchange much easier.

Sometimes you can't afford complexity because your target platform is very limited in terms of memory and/or performance. Sometimes complexity gets in the way of achieving the desired reliability (as in a device which can be powered down at any time but doesn't tolerate to lose data). In these cases working with fixed data records instead of files should be considered.

If you can afford the complexity and need (or foresee that you may need) flexibility, abstraction or compatibility, using a file system is a sound idea.