An alternative view : Microcontrollers don't run out of memory.
At least, not when programmed properly. Programming a microcontroller is not exactly like general purpose programming, to do it properly you have to be aware of its constraints and program accordingly. There are tools to help ensure this. Search them out and learn them - at least how to read linker scripts and warnings.
However as Majenko and others say, a badly programmed microcontroller may run out of memory, and then do anything including infinite loop (which at least gives the watchdog timer a chance to reset it. You did enable the watchdog timer, didn't you?)
Common programming rules for microcontrollers avoid this : for example, all memory is either allocated on the stack or statically (globally) allocated; "new" or "malloc" are forbidden. So is recursion, so that the maximum depth of subroutine nesting can be analysed and shown to fit in the available stack.
Thus the maximum storage required can be computed when the program is compiled or linked, and compared with the memory size (often encoded in the linker script) for the specific processor you are targetting.
Then the microcontroller may not run out of memory, but your program might. And in that case, you get to
- rewrite it, smaller, or
- pick a bigger processor (they are often available with different memory sizes).
One common set of rules for microcontroller programming is MISRA-C, adopted by the motor industry.
Best practice in my view is to use the SPARK-2014 subset of Ada. Ada actually targets small controllers like AVR, MSP430 and ARM Cortex reasonably well, and inherently provides a better model for microcontroller programming than C. But SPARK adds annotations to the program, in the form of comments, which describe what the program is doing.
Now the SPARK tools will analyze the program, including those annotations, and prove properties about it (or report potential errors). You don't have to waste time or code space dealing with erroneous memory accesses or integer overflows because they have been proven to never happen.
Although there is more up-front work involved with SPARK, experience is showing it can get to a product faster and cheaper because you don't spend time chasing mysterious reboots and other strange behaviour.
A comparison of MISRA-C and SPARK