Are virtual-functions called at a high frequency suitable for embedded systems?

Question

I have a (reasonably powerful) 32-bit MCU. Would it be unwise to design my system using OOP principles - mainly virtual methods?

I have a virtual method which would be called around 1000/second. Could this be a problem?

Answer 1

Just like on X86, this is going to depend too much on the specific code that any one answer will be useful. It is also going to depend a lot on the processor (and the memory architecture that processor uses).

A virtual method call in general is a pointer to a vtable, which has a pointer to a function. If you are looking at something where a memory lookup is a single cycle, then this is maybe three instructions, and three cycles (two mov, and an add). If the memory model is more complicated, like on an X86, where a pull out of cache can have drastic consequences in timing, then you could be looking for more than 1000 cycles.

Virtual methods also include the memory overhead of all those vtables, which, depending on memory size and memory pointer size, could get large.

For 1000 calls a second, for a "beefy" 32bit MCU running at a few MHz, and for not a lot of work being done in all those calls. You are probably fine. But if you are doing 1000 calls a second and each call takes a ms to finish, the virtual method calls aren't going to be something you can afford.

Like everything in performance and programming: profile, profile, profile. (or measure, measure, measure.)

Answer 2

Andrew's answer is totally right, I just want to emphasize on a few point:

If you really need a virtual method (because the code to execute depends on the type of object), then, most probably, a virtual method call would actually be faster than most non-OOP alternatives like a sequence of "if else". Only a call to a C-style function pointer could be more efficient in terms of number of instructions, but if you look at the generated code, I'm sure you'll see you just gain a couple of CPU instructions (you gain the vtable lookup).

When correctly used, OOP doesn't induce significant efficiency penalties. In some cases, it can even be faster: exceptions, for example. Those are often seen as "evil" and forbidden in embedded development, precisely because they are seen as a performance penalty. This is true when an exception is actually raised, but this is the opposite for the nominal case (when you use the proper jump tables implementation, not the setlongjmp implementation): because you don't have to check the return value between each call to eventually return an error at the upper level, your code runs faster when there is no error (which is the path you usually want to optimize).

Answer 3

First, calling a function 1000 calls/seconds does not seem like high-frequency on a 32-bit CPU. You don't say what MCU is being used, but assuming something quite low-end like a 24MHz ARM Cortex-M0, 1000 calls/second is not many (I'd WAG roughly 500,000 virtual function calls/second, might be +/- 5x). If instead it is a high-performance MCU, then 1000 calls/second will likely be a tiny fraction of its available performance.

However, you may be suffering from 'premature optimisation'.

The classic wisdom on optimisation is Michael A. Jackson (computer scientist and software engineer)

The First Rule of Program Optimization: Don't do it.
The Second Rule of Program Optimization – For experts only: Don't do it yet.

Problems with worrying about software optimisation too early in a project include: code becoming more complex, harder to write, debug and change resulting in the delivery of a functioning system taking longer. The knock-on result may rob the developers and customers of time to optimise the system for the important use cases. Also, if performance wasn't actually a problem, but a more complex and performant system was built, the system likely cost more, and is harder to maintain.

Further, this indicates that worrying about the wrong area of performance may be worse than not worrying about performance at all. That isn't saying performance might not be an issue. Instead it is saying try to figure out if performance might become an issue, and try to get to a stage where there is solid evidence of the scale and cause of the performance issue.

There are examples of developers guessing where performance problems may be before a system was sufficiently developed to get actual measurements. The industry wisdom is developers are not good at doing this accurately. The majority of these examples are used to illustrate the dangers of premature optimisation; focusing on avoiding features, or engineering extra technology to remove a perceived performance problem. These activities are often shown to be misdirected, and increase the difficulty of building adequately performant systems.

However, if you think 1000 virtual function calls is actually going to be an issue, how could you prove or disprove the theory?

You could go ahead and build part of your system the way you believe is the easiest to design, code and test. For example build and use one of your virtual objects in the real context. Then you could measure it.

Don't build the whole system, but do build enough of the real system to have a meaningful test. Aim to do this early enough so that if it unearths a problem, you have time to change the approach. One advantage of this strategy is if there is no problem, you haven't wasted any time, and your system is the way you think it should be and you have meaningful evidence. This way the system has all of the qualities, like robustness, flexibility, etc that you require, without adding the complication of engineering a more performant, in this case, virtual-method-call-equivalent than the compiler will produce.

An alternative is to spend a small amount of time building a 'micro-benchmark', something cheap and quick to build, which lets you measure the performance of the feature you are concerned about.

As an example, build a few classes with a virtual method, call it 1000 times, and measure how long it takes. This is slightly harder than it might appear as a clever compiler might be able to optimise it away unless the code has a side effect that is used. So, for example, you may have to read the assembler to ensure it is testing the right thing.

A problem with micro-benchmarks is they can gave an unrealistic picture. They may run noticeably faster than similar code in a real context. So if the micro-benchmark suggests the feature under test is too slow, then you are likely to have a problem. Because micro-benchmarks are somewhat unrealistic, you might want to improve them as you progress, and retry them, so don't spend much more than a few hours on one.

Yet a different approach is to build performance tests into your testing framework. They should get run frequently, as you develop, test and check-in code. Those tests might be set with a tighter time tolerance than the actual system so that you get a bit of warning if you drift towards your performance threshold.

Summary: Don't just worry about something causing a performance problem, have a plan and an approach which enables you to get evidence and facts, and get that information early enough that it is useful, ie. if things are proven to be poor, you need enough time to fix it and deliver on time.

A function call is not just the call instruction, but also passing parameters, building a stack frame, returning from the function, and retrieving results. So even though a virtual method lookup and call might take, say, 3x cycles longer than a simple call, it might be less than 25% of actual function call overhead.

The function-body, ie. the useful work the function actually does is likely to dominate the time, and be far more than the function call overhead. This should be relatively easy to prove with a small micro-benchmark of virtual function call of an empty function (a couple of hours work), and later measuring a real object implementation of the virtual function.

Potentially bigger problems on an MCU may be:

• using new and the cost of memory management,
• accessing object data, rather than global data, especially if there are lots of objects or variables inside 'hot' loops, as every data access might be an indirect memory access (No I am not saying this is automatically 'a bad thing')

Often the biggest problems come from poor algorithms or data structures, or overly complex implementations, which might not show up until parts of the system are working.

Summary: There are lots of potential sources of performance problems. Try to figure out quick ways to determine if something is likely to be a 'killer', or sequence the development of the system so that the system itself is functional enough, early enough, to give meaningful evidence. Build simple performance tests that you can use continuously to ensure you can be confident that development is on an adequate path.

Edit:
Our advice may cause the system to be too complex and hence costly to build, or run too slow. So it might be worth considering how you would mitigate the risk that our advice is wrong.

Edit 2:
Program Optimisation takes time, and so costs money. Optimising a program usually reduces other qualities of the program, for example worsens maintainability. So optimising a program which works to specification is usually undesirable.

Summary: The simplest program which does the job to specification, is a good program.

Edit 3:
In general poor code quality, sometimes described as Code Smell, is a separate issue from program optimisation. Program optimisation is motivated by a overuse of resources ie. a lack of performance, and is identifiable because the program fails to meet is performance specifications.

Sometimes it may be hard to adequately define performance specifications until some of a system works, but that is a further reason to avoid spending time on optimisation before the system works.

Program code quality is more difficult to define, and is often identified by code reviews between competent peers. Preferably, program code quality should be examined and improved continuously. There is a balance between code quality and development productivity to be achieved, though.