One potential problem with using a capacitor to 'smooth' a PWM voltage for the LED, is the LED has a minimum forward voltage before it turns on sufficient to be visible.
Its brightness is not controlled by voltage, it is controlled by current, and the amount of time it is switched on (i.e. the PWM duty cycle).
The capacitor might reduce the 'smoothed' PWM voltage below the minimum forward voltage, so the LED would no longer be visible, even though it would be visible using exactly the same PWM signal directly (without the capacitor).
So it would reduce the brightness range over which the LED can be controlled.
AFAIK, the bigger killer of LEDs is heat leading to a significant temperature rise, and not switching.
Typically we want to drive an LED with a constant current (or something near, e.g. a resistor), so that it is protected from too much heat leading to temperature rise and permanent damage. Edit: Depending on how the capacitor is connected, a capacitor may actually reduce the effectiveness of the constant current circuitry.