I have a scenario where I have 2D arrays of variable sizes and various data types (double and boolean as of now). These arrays should be read and processed by a microcontroller which has 256KB of SRAM.

In my setup I plan to transfer these arrays from SD card (stored as files) to SRAM for processing. I currently can think of following possible aproaches

  1. These arrays are actually matrices from MATLAB. So I intend to save MATLAB variables as .mat and reuse code from matimport.c
  2. Save the data contained in the variables in an xml file. Use an XML parser for the target microcontroller
  3. Create custom binary files with associated metadata. So the program in microcontroller reads the metadata(rows, columns, data type)about the binary file and extract variable based on offset's computed from the meta data

Sol 1 looks easy and straight forward. But, I guess I need to use certain MATLAB external libraries and I suppose these precompiled libraries (such as libmat.lib, libmx.lib) are not available for ARM.

Sol 2 is easy if there are good parser's written for a microcontroller. Suggestions of XML parser's are welcome

Sol 3 involves slightly more work. For this method, I intend to write a C application that reads .mat files and create a custom binary file with metadata and actual data. And have another identical import function in the microcontroller.

Total size of all variables combined is the range of 20KB to 40KB. Therefore, SD card is used in the system.

Which among these techniques is well suited for a low footprint embedded solution ? My goal is to choose and go with a solution that utilizes the least amount of memory and one which is most flexible for expansion. Ofcourse, do suggest any other approaches to solve this problem.

PS. The device is not Arduino based. But, I nevertheless added it as Arduino Development comes under the realm of this question.


closed as too broad by Eugene Sh., Daniel Grillo, Voltage Spike, Bence Kaulics, uint128_t Jul 6 '16 at 2:39

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • \$\begingroup\$ Could you explain why you added the arduino tag? I fail to see any relationship, it's not even ARM-based. \$\endgroup\$ – Dmitry Grigoryev Jul 5 '16 at 15:30
  • \$\begingroup\$ Also, your first approach won't work NOT because you don't have libmat for ARM (perhaps you do, in open-source projects like scilab or octave), but because MATLAB runtime won't fit on a micro. \$\endgroup\$ – Dmitry Grigoryev Jul 5 '16 at 15:43
  • 1
    \$\begingroup\$ Sol 4) Dont save them as .mat, save them as .txt and then parse them in matlab when you read them in. Use simple approaches when prototyping \$\endgroup\$ – Voltage Spike Jul 5 '16 at 15:58
  • \$\begingroup\$ @DmitryGrigoryev There are a few non ATmega boards that are based on ARM. Totally agree that MATLAB runtime is not suitable for memory constrained devices. Did some research after posting this question \$\endgroup\$ – user115881 Jul 6 '16 at 9:08
  • \$\begingroup\$ @laptop2d Storing it in a binary file is more suitable. For ex. An integer would take the exact size of an integer in RAM. However, when the same integer is to be represented using ASCII, the memory required is large \$\endgroup\$ – user115881 Jul 6 '16 at 9:13

I suggest to put the complexity on the PC side and not on the microcontroller side.

You should store the file on the sd card as a binary image of the memory content you would like to get on the microcontroller. Storing in a defined binary format should be doable directly from Matlab.

All you have to do is to copy byte after byte the content of you sd card in a predefined area in the SRAM.

An even simpler form would be to avoid using a file system at all and use the SD card as a raw medium. Event easier at the microcontroller side...

Having the complexity at the PC side makes this much easier:

  • You can use whatever programming language you want and you are not restricted to what is available on the MCU. I would use Python for this but that's a personal preference.
  • It's easy to debug PC software in comparison to MCU one.
  • Your MCU would have less things to do and thus more available computational time to do something else (or go to sleep to save batteries)
  • \$\begingroup\$ Writing binary files directly from MATLAB sounds interesting. Suppose, the data is stored in row major. How can the interpreter in the MCU know when to switch to second row ? \$\endgroup\$ – user115881 Jul 5 '16 at 14:33
  • \$\begingroup\$ All the usual possibilities. A metadata chunk at the beginning defining the matrix size or a row delimiter. \$\endgroup\$ – Daniel Jul 5 '16 at 14:50
  • \$\begingroup\$ The matrix is going to be serialized anyway, so you might as well just use a metadata chunk to tell the micro how to interpret it. \$\endgroup\$ – Daniel Jul 5 '16 at 14:51