I have a (unregulated) wall wart that feeds into a regulator circuit. The wall wart is rated 24V nominal, and measures 23V when open with a multimeter set to DC. When connected to the regulator it measures 35.5V, but the input capacitor on the regulator is only rated 35V. I don't/haven't seen any damage yet, but should I be worried over the long-term effects of this?
Your wall wart probably only has a diode rectifier. This explains why you measure 23V unloaded and a higher voltage when you connect the regulator circuit. In the latter case the input capacitor will load to approximately the peak value and that is what you measure.
If you operate an electrolytic capacitor (near or) above its rating, the lifetime will be drastically reduced. It may even explode when using above its rating and since you are probably measuring average DC with your multimeter, the peak voltage is probably even higher than 35.5V. I'd recommend at least a 50V replacement or replacing the wall wart.
Before replaceing the cap, investigate if the rest of the circuit (the regulator) is rated for voltages higer than 35V. An average LM75xx probably isn't. And remember that if you use a linear regulator, the difference between input and output voltage is being dissipated as heat, which can be a lot!