I'm not an expert on this, still learning myself, but in my search for an intuitive grasp on op-amps I've found it very helpful to think about closed-loop gain as percentage of feedback.
Unity gain is 100% feedback. In other words, if the output is off by 10mV, the error signal fed back to the inverting input is 10mV; 100% of the error.
If the op-amp is set up with a gain of 10, say with a 10k and 1k resistor, then the feedback is only 10%. An error of 10mV will only apply 1mV to the inverting input.
I'm not sure how accurate it is, but I think of op-amps as being "twitchy", meaning for a small input it wants to produce a big output (like over-react in the twitchy characterization). Accordingly, they're more stable when you "water down" the feedback you give them. And the op amp is not likely to be happy if you want to feed it a big input and expect it to make a teensy adjustment in its output, even if that fulfills the "make its inputs equal rule" of an ideal op-amp.
Increasing closed-loop gain (reducing feedback percentage) does reduce the op-amp's bandwidth though, a decade for each order of magnitude of closed-loop gain, so a balance must be struck. For example, a 10MHz op-amp would be reduced to 1MHz at 10% feedback (closed-loop gain of 10).
As I understand it, all op-amps are inherently unstable. The only question is whether you let the manufacturer compensate it for you or whether you want to do that yourself. Unity-gain stable op-amps are compensated by the manufacturer. This makes them easy to deploy in a lot of circuits as the designer need add no compensation circuitry. However it does close off the possibility of choosing a compensation network (circuit) perhaps more suitable for the specific application. So it's a design choice.