A primary factor when determining voltage levels for a differential bus is power consumption. The higher the voltage/bit rate is, the higher the power consumption is (this should be obvious to the reader). In particular, power consumption is amplified when you have very high speed signals, or multiple load points. If you think of the same issue in the other direction, a higher voltage level will be harder to achieve from the driver perspective thus will limit transmission speed. Current mode driving (which ensures the speed) used in many modern buses, USB included, allows lower voltage swings on the data lines.
On another note, reflections or signaling imperfections will result in over/undershoots. If you already have an intrinsically high voltage on the bus, the superimposed (and higher power) transients may not be tolerable by the device. That power also goes in vain. The extreme case of this phenomenon is when you disconnect the antenna from an RF transmitter. If you have enough power in the transmitter you will jeopardize the radio. You can take other factors, like EMI, into consideration as well. How about the dissapated heat in termination? For a given Z0 more volatge, more heat.
That is why the Low/Full speed USB uses 3.3V, USB 2.0 and later uses the even lower 800/400mv. We usually want to apply the lowest voltage that makes sense for the specific interface. Be reminded that many high speed interfaces (such as ethernet, can, hdmi, pci, lvds, and many more) all use low voltage signals in the same tier.