Japan is a bit of an exception- they adopted 100/200VAC for residential power rather than 120/240VAC. As with North America, the higher voltage is not typically present on sockets all over the place, but is wired specifically for high power consumption appliances such as central air conditioners. If you buy a house with a gas range, there will probably be a 240VAC circuit roughed in, just in case you want to go electric.
As far as I can tell (via Wikipedia and their references), the 120V nucleated from the US in the 1880s. The present 3-wire system (120/240, previously a bit less) was introduced in the 1880s as well.
Starting with one utility in Germany, the mains voltage was doubled to 220V, and the European standard spread from there. As to why not 300 or 480V, safety would be the answer. Early systems were at the same voltage as earlier DC systems, which would have been more dangerous than AC for the same voltage.
So now we have North American homes wired with 120VAC to most wall outlets and major appliances typically get 240VAC (or 208 in apartment buildings because of 3-phase distribution). European homes get 230VAC for everything unless they are lucky enough to have 380VAC.
The only major disadvantage of the split system from the consumer point of view is that portable high power consumption devices such as powerful kettles are not available due to the limitations of the wiring and receptacles. I can buy a 1500W kettle, but my European friends can get a 3kW kettle and have their Earl Grey tea ready much faster. The same gauge wire as in Europe can be run to major appliances such as stoves, range tops, electric dryers and central air conditioners, optionally with a neutral to run things like controls and small lamps.
There may be slightly more capital cost in the wiring, however the lower voltage is safer so the receptacles and cords and so on can be made a bit cheaper.
But basically there's no reason to change the voltage in either case.
The big improvement would come from going to distribution of 3-phase which would allow longer life devices and more efficient motors.
(of course industrial and commercial applications can and do often use different voltages from residences, for example in Canada 600VAC is used industrially, which explains the 346VAC ballasts used for lighting in an office I frequent- the wall switches are similar to 120VAC ones but made better).