Stability means essentially the same thing in control, electrical, mechanical engineering.
Impulse response (or step response) is a rough and ready test to see 'how stable' a stable closed loop system is. It cannot be used on an unstable system. Many overshoots => marginally stable, 'dead-beat' response => very stable, 'one tiny overshoot' => perfectly OK (unless you cannot tolerate any overshoot (so OK on my baking oven, not OK on my epoxy curing oven)), 'crawls up to the final value' => too stable.
However, if a system is marginally stable, how do you improve it? If a system is unstable to begin with, how do you test it? If your system is big, expensive or dangerous, do you dare even switch it on closed loop? It's easy enough to switch on a small electronic circuit and see, but what if it's a 100MW rolling mill speed controller, or a mag-lev train rail guidance loop?
This is where open-loop Bode plot testing comes in. You measure the gain and phase of the open loop system as you sweep the excitation frequencies over a large enough range to capture all the useful behaviour of the system.
Once you have the Bode plot, there are various methods you can use to predict the stability of the system once the loop has been closed.
For a very simple low order system (unity gain stable op-amps are like this) you can simply look for gain margin and phase margin in the region of the unity gain response. Note that this sort of op-amp stability, where one time constant dominates, is designed to be simple, to the detriment of other parameters like speed. You can get faster opamps that are not unity gain stable, but you need to know what you're doing.
For higher order systems (almost anything with a motor in it for instance), this simple graphical approach is not enough, and then you can crank up more subtle and mathematical methods like root locus plots and the Routh-Horowicz criterion, both made my head ache in my student days.
When a system is unstable, it will always tend to oscillate, even when the input is held at zero. The reason is there is noise always present in any real physical system, it's part of the physics and cannot be reduced below a certain magnitude. This noise contains energy at all frequencies, and will be amplified by the system and eventually grow until the system is crashing against the end-stops.