If you power the LEDs directly from the USB power they will draw less current, since the resistor inside is designed for an higher voltage. So you don't risk overloading the USB (those LEDs won't draw nowhere near 500mA, even at 12V).
The problem you will probably face is that at 5V they could be quite dim, or could even be off. But you could try and see if they light up sufficiently for your taste.
Disclaimer: I never took one of those apart, so I assume they have a limiting resistor inside (it's probably the cheapest option), but they could also have some sort of integrated constant current regulator chip instead (unlikely but possible). In other words, the LED itself could be one of those designed to be powered directly from 12V.
Anyway the best thing to do to test them is to hook them to a bench power supply set for 5V output and measure the current they draws using a multimeter.
If you don't have a bench power supply, an easy alternative is to put 3 new alkaline AA or AAA batteries (they are very common everywhere) in series: they have a nominal voltage of 1.5V, but if they are new they could source something more like 1.6V: you will end up with 3x1.6V=4.8V with which you could test your LEDs quite safely. Note that USB standard requires that the USB power lines are 5V±0.25V, so testing at 4.8V is quite reasonable.
BTW, what do you really mean with "... to bump up the voltage of the USB port..."?
If you plan to increase the voltage on the power lines of the USB bus with some additional circuitry, then DON'T DO THAT. That will damage your motherboard.