I'm going to refer to voltage as "tension" throughout this answer.
The English word "tension" (in the electrical sense) is much older than "voltage," and it was probably used long before scientists were able to formulate a clear definition of tension.
Alessandro Volta certainly had some idea of tension. A handful of sources on the Internet say that Volta described a law of capacitance (stating that in a capacitor, tension is proportional to charge) in 1776. I haven't been able to find any primary sources for this, and I have no idea whether he used the word "tension," or a different word like "potential." In any case, he certainly never called it "voltage."
Michael Faraday published a book titled Experimental Researches in Electricity in 1839. The book uses the word "tension" in a few different places. For example, in the section "Identity of Electricities derived from Different Sources," Faraday writes:
Tension.—When a voltaic battery of 100 pairs of plates has its extremities examined by the ordinary electrometer, it is well known that they are found positive and negative, the gold leaves at the same extremity repelling each other, the gold leaves at different extremities attracting each other, even when half an inch or more of air intervenes.
That ordinary electricity is discharged by points with facility through air; that it is readily transmitted through highly rarefied air; and also through heated air, as for instance a flame; is due to its high tension.
It's not obvious exactly why Faraday chose to use the word "tension" rather than some other word, but he was probably imitating earlier authors. I don't think Faraday had a very clear idea of just what tension is.
In the year 1861, Latimer Clark and Sir Charles Bright proposed a unit of measurement for what they called "electrical tension, potential or electromotive force," in their paper "Measurement of electrical quantities and resistance." Their proposed unit was the "ohma," and it never caught on.
No later than 1873, as described in the Wikipedia article "Volt," a new measurement of tension had been defined, and, as you know, this new measurement was called a "volt." I haven't been able to find where the idea for a volt came from, though.
Finally, since the unit of electric tension was called a volt, speakers of English started to refer to tension as "voltage" instead. (The Online Etymology Dictionary says that the word is from 1882, but doesn't give a source.) For one reason or another, the word "voltage" ended up becoming so popular that the older word, "tension," has been all but forgotten.
(As TypeIA mentioned in a comment, English is pretty much the only European language that uses a word similar to "voltage." Most other languages use either a word related to "tension," or a completely different word like "napięcie.")