The input numbers are a maximum or worst case scenario that the manufacturer wants you to take into consideration, they do not reflect the power draw at 100% of the time. It could be for example, some surge current when first plugging it in due to capacitors charging up, or just some huge margin.
Also, the average input current at 100V would be more than double that of when plugged at 240V.
Let's do the reverse calculations, from the output to the input:
5V and 0.7A gives 3.5W output. If you assume 50% efficiency, that's 7W on the input.
At 100V, that's 0.07A and 0.03A at 240V. (Much less than 0.15A)
Added to that, your phone does not draw 0.7A at 5V all the time that its charging, so in practice, the power draw is a lot lower.
Power supply efficiency:
This interesting article from 2012 tests a dozen chargers, from brand names to counterfeit ones, and the efficiency ranges from 60 to 80% (note: "vampire" indicates the no-load power consumption):
One way of guesstimating the power usage from phone charging would be:
1 - Estimate how much charge your phone holds. Let's say 2000mAh 3.7V battery, so ~ 8Wh
2 - Lets say you fully charge your phone every day.
3 - Lets assume the charger circuitry in your phone has 80% efficiency and the USB PSU 60%. So charging your phone wastes 50% of the energy.
That's 16Wh per day. ~6kWh per year. This does not take into account the power draw of the charger when your phone is not plugged in, but on the other hand I assumed pretty bad numbers for the rest of the points.
This article from 2013 by Forbes uses 5.45Wh as the battery energy, does not take power losses into account and arrives at a result of 2kWh.
Battery charging curve:
You can see in the graph below that maximum current draw falls drastically after the first hour(s) of charge. So even the numbers of 0.07A and 0.03A are the maximum for a brief time.