Is it really a good-practice disable optimizations during the development and debugging phases?

Question

I've read Programming 16-Bit PIC Microcontrollers in C, and there is this affirmation in the book:

During the development and debugging phases of a project, though, it is always good practice to disable all optimizations as they might modify the structure of the code being analyzed and render single-stepping and breakpoint placement problematic.

I confess I was a little confused. I didn't understand if the author said that because of the C30 evaluation period or if it's really a good practice.

I'd like to know if you actually use this practice, and why?

Answer 1

This is pretty standard in software engineering as a whole - when you optimize code, the compiler is allowed to re-arrange things pretty much however it wants, as long as you can't tell any difference in operation. So, for instance, if you initialize a variable inside every iteration of a loop, and never change the variable inside the loop, the optimizer is allowed to move that initialization out of the loop, so that you're not wasting time with it.

It might also realize that you compute a number which you then don't do anything with before over-writing. In that case, it might eliminate the useless computation.

The problem with optimization is that you'll want to put a breakpoint on some piece of code, which the optimizer has moved or eliminated. In that case, the debugger can't do what you want (generally, it will put the breakpoint somewhere close). So, to make the generated code more closely resemble what you wrote, you turn off optimizations during debug - this insures that the code you want to break on is really there.

You need to be careful with this, however, as depending on your code, optimization can break things! In general code that is broken by a correctly functioning optimizer is really just buggy code that's getting away with something, so you usually want to figure out why the optimizer breaks it.

Answer 2

I've sent this question to Jack Ganssle and this is what he answered me:

Daniel,

I prefer to debug using whatever optimizations will be in the released code. NASA says "test what you fly, fly what you test." In other words, don't do testing and then change the code!

However, sometimes one has to turn off optimizations so that the debugger will work. I try to turn them off just in the modules I'm working on. For that reason I believe in keep files small, say a few hundred lines of code or so.

All the best, Jack

Answer 3

Depends, and this is generally true of all tools not just C30.

Optimizations often remove and/or restructure the code in various ways. Your switch statement may get reimplemented with a if/else construct or in some cases may be removed all together. y = x * 16 may get replaced with a series of left shifts, etc. although this last type of optimization can usually still be stepped through, its mostly the restructuring of control statement that gets ya.

This can make it impossible to step a debugger through your C code because the structures you defined in C no longer exists, they were replaced or re-ordered by the compiler into something the compiler believes will be faster or use less space. It can also make breakpoints impossible to set from the C listing since the instruction your breaking on may no longer exist. For example you may try to set a breakpoint inside an if statement, but the compiler may have removed that if. You may try to set a breakpoint inside a while or for loop but the compiler decided to unroll that loop so it no longer exists.

For this reason if you can debug with optimizations off, its usually easier. You should always retest with optimizations on. This is about the only way you'll find out that you missed an important volatile and its causing intermittent failures (or some other weirdness).

In the case of embedded development, you have to be careful with optimizations anyway. Specifically in sections of code that are timing critical, some interrupts for instance. In these cases you should either code the critical bits in assembly or use compiler directives to make sure these sections are not optimized so you know they have a fixed execution time or a fixed worst case run time.

The other gotcha can be fitting code into the uC, you may need code density optimizations to simply fit your code into the chip. This is one reason why its usually a good idea to start with the largest ROM capacity uC in a family and only choose a smaller one for manufacturing, after your code is locked.

Answer 4

Generally, I would debug with whatever settings I planned to release with. If I was going to release optimized code, I would debug with optimized code. If I was going to release unoptimized code, I would debug with unoptimized code. I do this for two reasons. First, optimizers can make significant enough timing differences to cause the end product to behave differently than unoptimized code. Second, even though most are pretty good, compiler vendors do make mistakes and optimized code can produce different results from unoptimized code. As a result, I like to get as much test time as possible with whatever setting I plan to release with.

That being said, optimizers can make debugging difficult as noted in the previous answers. If I find a particular section of code that is hard to debug, I will temporarily turn the optimizer off, do the debugging to get the code working, then turn the optimizer back on and test once more.

Answer 5

My normal strategy is to develop with the final optimisation ( max for size or speed as appropriate), but flip the optimisation off temporarily if I need to debug ot trace something. This reduces the risk of bugs surfacing as a result of changing optimisation levels.

A typical failure mode is when increasing optimisation causes previously unseen bugs to surface due to you not having declared variables as being volatile where necessary - this is essential to tell the compiler which things should not be 'optimised'.

Answer 6

it certainly makes sense in the case of break points... as the compiler can remove lots of statements that don't actually effect memory.

consider something like:

int i =0;

for (int j=0; j < 10; j++)
{
 i+=j;
}
return 0;

could be entirely optimized out (because i isn't ever read). it would look from the point of view of your breakpoint that it skipped all of that code, when it basically just wasn't even there.... I presume that is why in sleep type functions you will often see something like:

for (int j=delay; j != 0; j--)
{
    asm( " nop " );
    asm( " nop " );
}
return 0;

Answer 7

Use whatever form you are going to release with, debuggers and compiling for debugging hide a lot (A LOT) of bugs that you dont see until you compile for release. By then it is much harder to find those bugs, vs debugging as you go. 20 something years now and I never had a use for a gdb or other like debugger, no need to watch variables or single step. Hundreds to thousands of lines per day. So it is possible, dont be lead to think otherwise.

Compiling for debug then later compiling for release can and will take twice to more than twice the effort. If you get into a bind and have to use a tool like a debugger then compile for the debugger to work through the specific problem, then revert back to normal operation.

Other problems are also true like the optimizer makes the code faster so for embedded in particular your timing changes with compiler options and that can affect the functionality of your program, here again use the deliverable compiling choice during the whole phase. Compilers are programs too and have bugs and optimizers make mistakes and some dont have any faith in that. If that is the case there is nothing wrong with it compile without optimisation, just do it that way all the time. The path I prefer is to compile for optimization then if I suspect a compiler problem disable the optimization if that fixes it usually go back and forth sometimes examining the assembler output to figure out why.

Answer 8

I always develop code with -O0 (gcc option to turn optimization off). When I feel I'm at the point where I want to start letting things head more toward a release, I will start with -Os (optimize for size) as generally the more code you can keep in cache the better it'll be, even if it's not super-duper optimized.

I find that gdb works far better with -O0 code, and it's a lot easier to follow if you have to step into the assembly. Toggling between -O0 and -Os also lets you see what the compiler's doing to your code. It's quite an interesting education at times, and can also uncover compiler bugs... those nasty things that make you pull your hair out trying to figure out what's wrong with your code!

If I really need to, I'll start adding in -fdata-sections and -fcode-sections with --gc-sections, which allow the linker to remove entire functions and data segments that aren't actually used. There are a lot of little things you can tinker with to try to shrink things down further or make things faster, but by and large these are the only tricks I end up using, and anything that has to be smaller or faster I'll hand-assemble.

Answer 9

Yes, disabling optimizations during debugging has been best practice for a while now, for three reasons:

• (a) if you're going to single-step the program with a high-level debugger, it's slightly less confusing.
• (a) (obsolete) if you're going to single-step debug the program with an assembly-language debugger, it's much less confusing. (But why would you bother with this when you could be using a high-level debugger?)
• (b) (long obsolete) you're probably going to only run this particular executable once, then make some change and recompile. It's a waste of a person's time to wait an extra 10 minutes while the compiler "optimizes" this particular executable, when that's going to save less than 10 minutes of runtime. (This is no longer relevant with modern PCs that can compile a typical microcontroller executable, with full optimization, in less than 2 seconds).

Many people go even further in this direction, and ship with assertions turned on.

Answer 10

Simple: optimizations are time-intensive, and may be useless if you have to change that piece of code later on in development. So they may well be a waste of time and money.
They are useful for finished modules, however; parts of the code that most probably won't need changes anymore.

Answer 11

If you are using the debugger then I would disable optimizations and enable debug.

Personally I find that the PIC debugger causes more problems than it helps me fix.
I just use printf() to USART to debug my programs written in C18.

Answer 12

Most of the arguments against enabling optimization in your compiling come down to:

1. trouble with debugging (JTAG connectivity, breakpoints etc)
2. wrong software timing
3. sh*t stops working correctly

IMHO the first two are legit, the third not so much. It often means you have some bad code or are relying on unsafe exploitation of the language/implementation or maybe the author is just a fan of good ol' Uncle Undefined Behaviour.

The Embedded in Academia blog has a thing or two to say about about undefined behaviour, and this post is on how compilers exploit it: http://blog.regehr.org/archives/761