Basically no substance is perfectly smooth, or inelastic. When two objects come together with any force, there is kinetic energy that needs to die down upon contact. So you get some "ringing", plus since the surface isn't smooth the contact makes/breaks a bit randomly until settling (e.g. even with a sliding switch the contact resistance will vary upon movement)
It happens with all switches; sliding, pushbutton, relay, etc, though sliding or wetted contacts (e.g. mercury) are better. It also happens upon opening the switch, not just on closing. Some are better than others - you can get some awful ones that will bounce for tens of ms, and some good ones that settle within <5ms.
To add more on the opening of the switch, it's due to the irregular surfaces opening in a not perfectly perpendicular manner. The widely varying resistance this creates appears similar to the bounce during closing. When pressure is released, one part of the contact may disengage, but another part may briefly obtain a lower resistance than it had previously, the more irregular the surface and the more of an angular force between the contacts, the more pronounced this effect is likely to be.
Although it looks smooth to our eye, the copper surface looks rather different under a microscope: