> Unless of course it's not just the current that matters, and that
> voltage matters more than just producing a certain amount of current

In real circuits *both* matter. An IC, for instance, needs a certain minimum voltage to work (silicon diodes, for example, don't conduct at all below about 0.7V). On the other hand, too high a *voltage* may destroy semiconductor structures just by the force it exerts on the electrons. (Really high voltages can even cause arcs through normally insulating parts, permanently damaging the insulation. Much lower voltages can cause similiar effects through the miniscule insulating structures in ICs.)

Then, as per V=RxI, or I=V/R, higher voltage often forces more current through a given conducting structure, which technically is mostly not a problem by itself, but a higher current through any non-superconductor causes higher power loss in the conductor which heats the conductor and may ultimately cause irreversible thermal damage.

So, practically, we need to maintain the voltage between the lower and upper bound for the circuit to operate. Too low and it won't work, too high and it may burn out.

As others have answered, most non-trivial circuitry does not act like a constant (ohmic) resistance but varies its apparent impedance over time. Digital ICs (CMOS), for example, often briefly consume much more power on the edges of their clock signal than in between, so powering via some sort of constant *current* source will force higher voltage *and* too much current through them at times and/or not enough at other times.

The analogy with water (pressure=voltage,flow=current) holds: If you had a source that always forces as constant *flow* (current) of, say, 1 liter per second (Ampère) through your piping, what would happen if you shut a valve between the source and the drain? The source would increase the pressure until the flow is at 1 liter per second again, which may happen only after the (shut) valve or other parts of the piping break.