Following Neil's answer, yes you can do that, but you usually won't.
As said, it's necessary not to build an LC oscillator with vastly differently sized reactive and capacitive elements, and finding an inductor in the same order of magnitude will be a bit expensive (if you can't scavenge one from something else).
Also: If you want an oscillator, you're often actually interested in an exact, stable frequency oscillation.
Now, most electrolytic capacitors are actually sold with a 20% value tolerance. That's not a great start to hit an exact frequency. You say you've got matched pairs – but are these really matched, or do they just carry the same specification?
Also, electrolytics are usually mainly used as relatively long-term, large value "energy storage" in the power supply of loads. As such, they're optimized for high capacity density, but not for low equivalent series resistance, so instead of this typical Colpitts
simulate this circuit – Schematic created using CircuitLab
You'd have to consider the circuit including the parasitic series resistances; and also, electrolytics capacitors aren't totally flat over all frequencies, so if you don't operate at low frequencies, subtract a few percents from the nominal capacity value:
simulate this circuit
Suddenly, your oscillator is dampened, and the the fact that there's voltage drops over R1par and R2par means that the transistor isn't quite working at the same Uce, which also means the amount of energy stored per oscillation will change. This makes stable operation a bit tricky.
Regarding Polarization/Orientation: as usual, make sure the caps are oriented the same way as the biasing.