Yes, USB charger is a regulated power supply. USB devices expect around 5 V on the Vcc line. Do note that the regulation on some of the cheaper USB power supplies is pretty bad, especially if you're near the power supply's maximum current.
The best way to be sure if you need a regulator is to check the regulator and what your power supply is providing. If you have a multimeter, use it in the voltage mode on the two outer pins if the USB connector of the power supply. You should get around 5 V.
In the guide you linked to, the regulator used is of 7805 pattern. Here 78 is the series number and 05 is the output voltage of the particular regulator.
Such regulators need higher input voltage of around 7 to 8 volts (2 to 3 volts above the output voltage) to be able to regulate properly. Basically, this type of regulator, called linear regulator, turns the excess voltage into heat and provides 5 V output (for 7805). If the input voltage is too low, it won't be able to give correct output voltage.