If you really try, you can press a button for only a few 10s of milliseconds. However, that is quite extreme and requires a deliberate attempt.
In my experience, 50 ms is a good debounce time. That is about the limit
where humans start to notice the delay if they aren't specifically looking for it. Most switches stop bouncing in 10-20 ms, but I have run into a few that bounced longer than that. Since humans don't notice a 50 ms delay, it's pretty much a no-brainer to debounce for at least that long, assuming the switch is being operated by a human.
If the switch is a mechanical sensor, then a faster response may be needed depending on its exact purpose. In that case a different debouncing algorithm may be appropriate, like react to the first state change immediately, but then block out new state changes for some interval.
I have used basic 50 ms debounce logic in many products, and not once has anyone complained or even mentioned they noticed the delay. To me the response feels instantaneous, although I know it's not. The processor usually has a 1 ms periodic interrupt for various other timing purposes anyway. To be clear, the debounce algorithm is that the new state of the switch is only made official if it has been seen that way for 50 consecutive 1 ms samples. This is easy to do and quite robust in my experience.