The short answer is (as you suspect) yes.
...as long as you connect the primaries and secondaries the same way.
If it's really a bank of three single-phase transformers, taking one out of the circuit and reconnecting as you described will give you an open delta. The power rating will of course be less, as there's one less winding. There are some other considerations if it's a three-phase transformer with all windings on a common core.
There's no need to restrict it to an ungrounded system. The voltage and phase relationships are still maintained.
On the primary side of course you don't want to ground one phase if the source isn't connected that way.
On the secondary side, I have seen some industrial installations with a 3-phase, 3-wire service, supplied from a pair of transformers in open delta with the corner grounded. Also common is to have one of the transformers larger, with a center tap. It supplies 120/240V single-phase loads. The smaller transformer gives the third phase for motor loads.
See if you can discern what's going on in this Google street view image and how a center-tap-grounded open delta system is derived from two phases of a grounded wye: