The answer depends quite a lot on the device you're powering. Many modern electronics use a linear voltage regulator which provides the operating voltage from the input voltage. This type of device often has a fairly wide range of acceptable input voltages - a 5V regulator like the LM7805 can provide a regulated 5V output with an input voltage between 7V and 35V. The regulator will draw the same current from the supply as the load draws (generally less than 1A) and dissipate the excess power (obeying P=IV) as heat. A voltage below 9V is very unlikely to damage your device, so you could probably try it and see.
What this means is that there's probably a voltage range with which the device will work happily. If there's no specification or manual which defines this, opening up the device and inspecting the power input of the PCB would probably reveal a voltage regulator of some sort. You could look up the datasheet for this to get a better answer. Remember that if you do choose a voltage above 9V, the amount of heat generated in the regulator will increase, which could cause it to fail if it was already being operated towards the current limit.
There's another consideration which has to be made when using lithium batteries - safety. Deeply discharging lithium batteries can permanently damage them and even present a fire hazard. You can buy lithium battery packs which incorporate small ICs to automatically cut them off when the voltage drops too low.