It depends. Capacitors are fairly complex components, despite their appearances, and quite a bit of consideration goes into applying them such that they last a long time. The below covers only a few points, and I'm not even touching the foil capacitors.
Tantalum capacitors must be used well within their rated voltage, pulse/ripple current and temperature ratings. They don't usually fail gracefully, but short out. They will usually hold their capacitance within tolerance until the very end. If the circuit is properly protected and the capacitor's failure resistance is low enough, nothing bad happens other than the circuit not working anymore. But often the capacitor will fail not as a "dead short", but with enough resistance to heat it up and ignite it. They burn for a short time but at a very high temperature. It's a catastrophic failure.
Electrolytic capacitors fail "slowly" and typically, but not always, tend to lose capacitance and become more and more resistive. Eventually, they end up as low capacitance capacitors with very high ESR - the approximate end condition is an open circuit. An aging electrolytic may also develop low impedance between the electrodes - not usually a short, but a resistance in the right range to turn the capacitor into a heater. The self-heating boils the electrolyte and causes it to vent and corrode anything it comes in contact with.
The electrolytics that fail low-impedance may appear fine with no DC bias, i.e. if you check them out on an LCR meter. The failure may be triggered only after the capacitor voltage has been brought up high enough. I've had a few large electrolytics from lab power supplies that behaved that way: mostly nominal at low voltage, shorts out once some threshold voltage is passed.
The biggest killer of electrolytic capacitors is heat - whether due to ambient temperature, nearby heat sources such as hot parts and heatsinks, or self-heating due to the ripple current. Remember that the electrolytic capacitor life ratings are at the maximum rated operating temperature. An electrolytic capacitor that's running cool - with internal temperature of, say, 35C - will retain usable ratings for tens of thousands of hours if it was well made. They are more likely to fail shorted when operating near the rated voltage and hot, but at least in my limited experience that was not a problem when running them close to room temperature. Using a 16V capacitor on a 15V circuit may be acceptable if the capacitor is cool during operation and the 15V is well regulated. If the "15V" comes straight from an unregulated rectifier, e.g. from a mains transformer, then it's not really 15V even if an RMS voltmeter indicates so - the peaks are higher, and as the capacitor ages and loses capacitance and gains ESR, the ripple will only grow and accelerate the failure.
Ceramic capacitor performance strongly depends on the type of dielectric used. Dielectrics other than NP0 and C0G have varying degrees of capacitance change with applied voltage. Small AC voltages may even increase the capacitance slightly, only for it to "fall off a cliff" when the AC amplitude or DC component is increased. When using anything but NP0/C0G ceramics above a couple of volts DC, always measure their capacitance using a capacitance meter that can accommodate bias. If your RLC meter doesn't allow that - many types get damaged with even small DC bias - feed a square wave into the capacitor through a resistor, and derive the capacitance from the time constant observed on a scope. E.g. for a 9V application on a 10V capacitor, apply a 9V DC + 0.2Vpp square wave through a resistor, and observe the voltage on the oscilloscope. The RC time constant observed, with the known series resistance, lets you figure out the capacitance. Ceramics can fail "hard open" or "hard short". Mechanical stress causes cracking and that leads to open failures, but sometimes the layers misalign in such a way that the capacitor shorts out. There are also random insulation failures that result in shorts, but that's rare if the capacitors are otherwise not overstressed - i.e. when the operating voltage, ripple current, and operating temperature are all a bit away from the recommended operating range.
Unless you're trying to save single cents from a very high volume device, you'll not save any money by "economizing" on capacitors. In low volume professional applications, ample derating is more than worth it. For example, in high current LDO circuits I derate tantalums by 50-70% on the operating voltage vs. voltage rating, 50%-90% or more on ripple current, and keep them well away from the temperature limits. For aluminum electrolytics, I derate operating voltage by 20%, ripple current by 75% or more. For ceramics, the voltage derating is usually limited by very steep price increase once you're past a certain voltage-capacity product for a given dielectric category. For bulk decoupling I use physically large ceramics. For local decoupling in modern digital logic/CPU applications, the parasitics often matter more than the exact capacitance value, so a small case size is very beneficial, up to a point.