I've noticed that on older boards some of the electrolytic capacitors don't sit up straight, they're not totally vertical. I wonder why this happens, is it because they were soldered by hand and the person doing it placed them that way? Were they placed vertically and bent over time? Is it bad to the circuit in any way?
More stress is placed on the leads if there is no space between the part and the PCB. Electrolytics are more fragile than some parts because of the semi-liquid electrolyte, which can leak if there is tension on the lead, such as that caused by thermal cycling.
On some equipment, soft standoffs are used to lessen the strain, and prevent vibration. However, some manufacturers just leave a gap between part and PCB. Even if the part were vertical to start, vibration will knock it off axis. Also, as you state, hand assembly leads to random variation.
In my opinion the reason is quite easy to explain if you consider the manufacturing process.
On such a cheap (yellow coated paper type) PCB the capacitors are mounted by factory workers. These workers need to place the capacitors as quickly as possible at the correct place and observing the polarity. They have no time to bend the capacitor into an upright position. Also: it doesn't matter for the functionality of the electronics.
After mounting usually the soldering is done in a wave soldering machine which could also bend the leads (and the capacitor) in any direction.
The capacitors don't fit the board
In modern PCB design, the dimensions of the capacitor legs are known. The board is designed for that spacing, and production always uses that size of capacitor. With automated assembly, if you really and truly need a different size of capacitor then it's better to redesign the board to fit it. And more often than not, these days they'll be surface-mount anyway.
This was not the case, back in the day. Boards were assembled by hand - and because humans were involved, they could make things fit. There was no urgency to order components with exactly the same leg spacing as the holes in the board, and if it became cheaper to order a different size of component then the company absolutely would.
Then we get to how they were assembled. People used something like this.
You'd clamp the board in the frame and stuff all the components of roughly the same height in. Then you'd pull the padded lid down to hold the components in place as you flipped the board over to solder the underside. You'd repeat this in order of height; typically you'd have one go for all the resistors, diodes and ICs, and then a second round for capacitors and anything else taller. Naturally any component which wasn't firmly in place could fall a bit into the padding, or could get squashed and pushed over by the padding.
All this basically stopped when a production line with wave-soldering machines became the norm for soldering through-hole components. At that point you needed your components mounted reasonably securely, and leaving them waving in the wind wasn't going to work. Reliability studies also found that components tended to fail because of the mechanical stresses between the component legs and body, giving a further reason to mount them more securely.