modular approaches are pretty handy in general (portable and clean), so I try to program modules as independent of any other modules as possible. Most of my approaches are based on a struct that describes the module itself. An initialization function sets the primary parameters, afterwards a handler (pointer to desriptive struct) is passed to whatever function inside the module is called.

Right now, I am wondering what the best approach of allocation memory for the struct describing a module may be. If possible, I'd like the following:

• Opaque struct, so the struct may only be altered by the use of provided interface functions
• Multiple instances

I see the following possibilities, that all conflict with one of my goals:

global declaration

multiple instances, allcoted by linker, but struct is not opaque

(#includes)
module_struct module;

void main(){
module_init(&module);
}


malloc

opaque struct, multiple instances, but allcotion on heap

in module.h:

typedef module_struct Module;


in module.c init function, malloc and return pointer to allocated memory

module_mem = malloc(sizeof(module_struct ));
/* initialize values here */
return module_mem;


in main.c

(#includes)
Module *module;

void main(){
module = module_init();
}


declaration in module

opaque struct, allocated by linker, only a predefined number of instances

keep the whole struct and memory internal to the module and never expose a handler or struct.

(#includes)

void main(){
module_init(_no_param_or_index_if_multiple_instances_possible_);
}


Is there an option to combine these somehow for opaque struct, linker instead of heap allocation and multiple/any number of instances?

solution

as proposed in some answers below, I think the best way is to:

1. reserve space for MODULE_MAX_INSTANCE_COUNT modules in the modules source file
2. do not define MODULE_MAX_INSTANCE_COUNT in the module itsself
3. add an #ifndef MODULE_MAX_INSTANCE_COUNT #error to the modules header file to make sure the modules user is aware of this limitation and defines the maximum number of instances wanted for the application
4. on initialization of an instance return either the memory address (*void) of the desricptive struct or the modules index (whatever you like more)
• Most embedded FW designers are avoiding the dynamic allocation to keep the memory usage deterministic and simple. Especially if it is bare-metal and does not have underlying OS to manage the memory. – Eugene Sh. Aug 15 '18 at 14:25
• Exactly, thats why I want the linker to do the allocations. – L. Heinrichs Aug 15 '18 at 14:28
• I'm not quite sure I understand... How could you have the memory allocated by the linker if you have a dynamic number of instances? That seems quite orthogonal to me. – jcaron Aug 15 '18 at 17:13
• Why not let the linker allocate one big memory pool, and do your own allocations from that, which also gives you the benefit of a zero-overhead allocator. You can make the pool object static to the file with the allocation function so it's private. In some of my code, I do all the allocations in the various init routines, then I print out afterward how much was allocated, so in the final production compile I set the pool to that exact size. – Lee Daniel Crocker Aug 15 '18 at 23:58
• If it's a compile-time decision, you could simply define the number in your Makefile or equivalent, and you're all set. The number would not be in the source of the module, but would be application specific, and you just use an instance number as a parameter. – jcaron Aug 16 '18 at 14:49

Is there an option to combine these somehow for anonymous struct, linker instead of heap allocation and multiple/any number of instances?

Sure there is. First, however, recognize that the "any number" of instances must be fixed, or at least an upper bound established, at compile time. This is a prerequisite for the instances to be statically allocated (what you're calling "linker allocation"). You might make the number adjustable without source modification by declaring a macro that specifies it.

Then the source file containing the actual struct declaration and all its associated functions also declares an array of instances with internal linkage. It provides either an array, with external linkage, of pointers to the instances or else a function for accessing the various pointers by index. The function variation is a bit more modular:

module.c

#include <module.h>

// 4 instances by default; can be overridden at compile time
#ifndef NUM_MODULE_INSTANCES
#define NUM_MODULE_INSTANCES 4
#endif

struct module {
int demo;
};

// has internal linkage, so is not directly visible from other files:
static struct module instances[NUM_MODULE_INSTANCES];

// module functions

struct module *module_init(unsigned index) {
instances[index].demo = 42;
return &instances[index];
}


I guess you're already familiar with how the header would then declare the struct as an incomplete type and declare all the functions (written in terms of pointers to that type). For example:

module.h

#ifndef MODULE_H
#define MODULE_H

struct module;

struct module *module_init(unsigned index);

// other functions ...

#endif


Now struct module is opaque in translation units other than module.c,* and you can access and use up to the number of instances defined at compile time without any dynamic allocation.

*Unless you copy its definition, of course. The point is that module.h does not do that.

• I think it's an odd design to pass the index from outside the class. When I implement memory pools like this, I let the index be a private counter, increasing by 1 for every allocated instance. Until you reach "NUM_MODULE_INSTANCES", where the constructor will return an out of memory error. – Lundin Aug 17 '18 at 6:16
• That's a fair point, @Lundin. That aspect of the design supposes that the indexes have inherent significance, which may or may not in fact be the case. It is the case, albeit trivially so, for the OP's starting case. Such significance, if it exists, could be further supported by providing a means to obtain an instance pointer without initializing. – John Bollinger Aug 17 '18 at 11:51
• So basically you reserve memory for n modules, no matter how many will be used, then return a pointer to the next unused element if the application initializes it. I guess that could work. – L. Heinrichs Aug 17 '18 at 17:05
• @L.Heinrichs Yes, since embedded systems are of a deterministic nature. There's no such thing as "endless amount of objects" nor "unknown amount of objects". Objects are often singletons too (hardware drivers), so there's often no need for the memory pool, as just one single instance of the object will exist. – Lundin Aug 20 '18 at 14:06
• I agree for most cases. The question had some theoretical interest as well. But I could use hundreds of 1-wire temperature sensors if there are enough IOs available (as the one example I can come up with now). – L. Heinrichs Aug 20 '18 at 15:08

I program small micro-controllers in C++, which achieves exactly what you want.

What you call a module is a C++ class, it can contain data (either externally accessible or not) and functions (likewise). The constructor (a dedicated function) initilializes it. The constructor can take run-time parameters or (my favourite) compile-time (template) parameters. The functions within the class implicitly get the class variable as first parameter. (Or, often my preference, the class can act as a hidden singleton, so all data is accessed without this overhead).

The class object can be global (so you know at link-time that everything will fit), or stack-local, presumably in the main. (I don't like C++ globals because of the undefined global initialization order, so I prefer stack-local).

My preferred programming style is that modules are static classes, and their (static) configuration is by template parameters. This avoids nearly all overhad and enables optimization. Combine this with a tool that calculates the stack size and you can sleep without worries :)

My talk on this way of coding in C++: Objects? No Thanks!

A lot of embedded / microcontroller programmers seem to dislike C++ because they think it would force them to use all of C++. That is absolutely not neccessary, and would be a very bad idea. (You probably don't use all of C either! Think heap, floating point, setjmp/longjmp, printf, ...)

In a comment Adam Haun mentions RAII and initialization. IMO RAII has more to do with deconstruction, but his point is valid: global objects will be constructed before your main starts, so they might work on invalid assumptions (like a main clock speed that will be changed lateron). That is one more reason NOT to use global code-initialized objects. (I use a linker script that will fail when I have global code-initialized objects.) IMO such 'objects' should be explicitly created and passed around. This includes a 'waiting' facility 'object' that provides a wait() function. In my setup this is 'object' that sets the clock speed of the chip.

Talking about RAII: that is one more C++ feature that is very usefull in small embedded systems, although not for the reason (memory deallocation) it most used for in largere systems (small embedded systems mostly don't use dynamic memory deallocation). Think of locking a resource: you can make the locked resource a wrapper object, and restrict access to the resource to be only possible via the locking wrapper. When the wrapper goes out of scope, the resource is unlocked. This prevents access without locking, and makes it much more unlikely to forget the unlocking. with some (template) magic it can be zero-overhead.

The original question didn't mention C, hence my C++-centric answer. If it realy must be C....

You could use macro trickery: declare your stucts publicly, so they have a type and can be allocated globally, but mangle the names of their components beyond usability, unless some macro is defined differently, which is the case in your module's .c file. For extra security you could use the compile time in the mangling.

Or have a public version of your struct that has nothing usefull in it, and have the private version (with usefull data) only in your .c file, and assert that they are the same size. A bit of make-file trickery could automate this.

• The type of programmer you describe would probably make a mess in any language. Macro's (present in C and C++) are one obvious way.

• Tooling can help to some extent. For my students I mandate a built script that specifies no-exceptions, no-rtti, and gives a linker error when either the heap is used or code-initialized globals are present. And it specifies warning=error and enables nearly all warnings.

• I encourage using templates, but with constexpr and concepts metaprogramming is less and less required.

• "confused Arduino programmers" I would very much like to replace the Arduino (wiring, replication of code in libraries) programming style with a modern C++ approach, which can be easiere, more secure, and produce faster and smaller code. If only I had the time and power....

• Thanks for this answer! Using C++ is an option, but we are using C in my company (what I havent mentioned explicitly). I have updated the question to let people know Im talking about C. – L. Heinrichs Aug 15 '18 at 14:51
• Why are you using (only) C? Maybe this gives you chance to convince them to at least consider C++... What you want is essentialy (a small part of) C++ realised in C. – Wouter van Ooijen Aug 15 '18 at 14:55
• What I do in my (first 'real' embedded hobby project) is initializing simple default in the constructor, and use a separate Init method for relevant classes. Another benefit is that I can pass stub pointers for unit testing. – Michel Keijzers Aug 15 '18 at 15:00
• @Michel for a hobby project you are free to choose the language? Take C++! – Wouter van Ooijen Aug 15 '18 at 15:02
• And while it is indeed perfectly possible to write good C++ programs for embedded, the problem is that some >50% of all embedded systems programmers out there are quacks, confused PC programmers, Arduino hobbyists etc etc. These kind of people simply cannot use a clean subset of C++, even if you explain it to their face. Give them C++ and before you know it, they will use the whole of STL, template metaprogramming, exception handling, multiple inheritance and so on. And the result is of course complete trash. This is, sadly, how some 8 out of 10 embedded C++ projects end up. – Lundin Aug 16 '18 at 15:09

I believe FreeRTOS (maybe another OS?) does something like what you're looking for by defining 2 different versions of the struct.
The 'real' one, used internally by the OS functions, and a 'fake' one which is the same size as the 'real' one, but doesn't have any useful members inside (just a bunch of int dummy1 and similar).
Only the 'fake' struct is exposed outside of the OS code, and this is used to allocate memory to static instances of the struct.
Internally, when functions in the OS are called, they are passed the address of the external 'fake' struct as a handle, and this is then typecast as a pointer to a 'real' struct so the OS functions can do what they need to do.

• Good idea, I guess i could use --- #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) --- BUILD_BUG_ON(sizeof(real_struct) != sizeof(fake_struct)) ---- – L. Heinrichs Aug 15 '18 at 14:47

Anonymous struct, so the struct may only be altered by the use of provided interface functions

In my opinion, this is pointless. You can put a comment there, but no point trying to hide it further.

C will never provide such high isolation, even if there is no declaration for the struct, it will be easy to accidentally overwrite it with e.g. mistaken memcpy() or buffer overflow.

Instead, just give the struct a name and trust other people to write good code also. It will also make debugging easier when the struct has a name you can use to refer to it.

Pure SW questions are better asked at https://stackoverflow.com.

The concept with exposing a struct of incomplete type to caller, like you describe, is often called "opaque type" or "opaque pointers" - anonymous struct means something else entirely.

The problem with this is that the caller won't be able to allocate instances of the object, only pointers to it. On a PC you would use malloc inside the objects "constructor", but malloc is a no-go in embedded systems.

So what you do in embedded is to provide a memory pool. You have a limited amount of RAM, so restricting the number of objects that can be created is usually not a problem.

See Static allocation of opaque data types over at SO.

• Ou thank you for clarifying the naming confusion on my end, ill adjust the OP. I was thinking of going to stack overflow, but decided i would like to target embedded programmers specifically. – L. Heinrichs Aug 17 '18 at 17:04