It may be helpful to separate two usages of the term power.
The first is the technical usage. According to Ohm's Law, this is current multiplied by voltage: \$IE\$. (Or \$I^2R\$, or \$\frac{E^2}{R}\$.)
The second is the common misuse of the word in everyday speech. People say things like, "It's not plugged in, I need power!" or "Power up that amplifier." It's a convenient expression compared to "It's not plugged in, I need electrical energy."
Power is a rate: how quickly something uses energy. In electrical terms, it's measured in watts. Watts, in turn, are Joules (energy) per second. If a circuit uses 2 watts, it's using 2 joules per second. If the circuit requires 1 ampere of current, we can know (using Ohm's law) that the voltage must be (E = P / I) 2 volts. Another way to look at it might be to say that it is using 1 ampere at 2 volts every second, but that is cumbersome and more easily stated using watts.
Energy is a quantity of something. In electronics, a joule is a the energy required to produce one watt of power for one second. More specifically, it's how much work is required to move one coulomb (a lot of electrons) of electric charge at one volt.
Energy doesn't necessarily have to be electric in nature. It can also be thermal, gravitational, kinetic, acoustic, etc. For example, you could move an object on a table by using sound waves to vibrate it.
You can have some amount of available energy that, when delivered (or consumed) in a short duration, produces a high amount of power. Delivered or consumed slowly, produces a small amount of power.
A battery stores energy, in chemical form, which can be used at different rates. A remote control for a television, for example, draws very little current. Batteries in such a device last a very long time. You could say that a remote is a low power device. By comparison, a small electric motor using the same battery would be a high power device, because it consumes energy more quickly.
Keep in mind, power is still watts, and watts are a defined unit (joules per second). So when we say something is high or low power, we are speaking relatively. That same electric motor would be considered a low power device, if you were comparing it to a much larger motor.
A 100 watt light bulb requires more power than a 50 watt bulb. It doesn't really matter whether it is low voltage and high current, or high current and low voltage. The wattage is a product of the two, however it is arrived at.
So, circling back to the first part of your question: You're never supplying a load with power, technically. You supply energy and it consumes it at some rate, which in turn is power. But as previously mentioned, people tend to use power as a substitute for other concepts.
So what is the difference between an amplifier and a power amplifier?
Amplifier is a very general term. It is simply a category of circuit or device that increases the magnitude of a signal. It might be an operational (differential) amplifier, an audio amplifier (microphone, instrument, headphones, etc.), radio frequency amplifier, and so on. (To say nothing of amplifiers non-electrical in nature, like fluid or mechanical. A jack to raise your vehicle could be considered a type of amplifier).
The answer is "it depends." Remember how the term "power" gets thrown around inappropriately?
If you're talking about an operational amplifier in terms of the portion of a circuit, it only amplifies voltage. It's definitely not a power amplifier.
Any amplifier that amplifies audio, is technically an audio power amplifier. The output from such an amplifier has both voltage (amplitude) and current. Since P = IE, increasing either voltage or current will, by definition, increase power.
Class A amplifiers, are always power amplifiers. If you encounter one labeled only as an "amplifier," it is just someone taking a shortcut and not calling it an "audio power amplifier." (Similar to how we might say "fill up the car with gas" instead of "fill up the car with unleaded gasoline.") The specifics have simply been omitted.
<br>to add a paragraph space. With Markdown, simply enter two carriage returns and paragraph space will be added for you. In English, there's no space before punctuation such as a question or exclamation mark. Finally, show thanks by voting on and accepting answers rather than including it in your question. \$\endgroup\$