When using a wall adapter the most important rating is the one on the adapter.
If you are soldering something to another something that had a USB connector it might be a good idea to put a label onto it "Warning, uses more than 500mA" in case in 3 years you think "Hey, plug that in!". That said, any PC/Laptop built 2005-ish or later should have protection circuitry that just switches off the group of USBs that has the overloaded port, as per the standards prescribed for that.
One thing to watch out for is thin, flimsy wires, like those cheap 50cent USB charge cables. They will gives you voltage drops that'll make you cry at 2A. The one cable you should be able to fully trust is the cable that came with the charger/phone, as that should be dimensioned for the total 2A.
Again there, the main risk is of things not working as expected, something wanting 5V might get only 4V if you go over 500mA with a very cheap thin wire, 9 out of 10 cases it'll just not work or work badly.
All in all: Go for it, but always remember it might exceed USB standard drain and as such never turn an idea-train like this into a mass product before researching the required power sensing standards. But for one-offs, toy away!