You can try to measure the voltage by any means you have. But consider that there is no such thing as truly floating.
Modern GPIO pads on MCUs have many dozens of semiconductor structures attached to it, in buffer, output (disabled) buffer, ESD structures, etc., and every structure (diode, transistor/switch) would have certain leakage (conductance), which can be in both directions, to Vcc rail, or ground. So the sum of leakages is hardly predictable. Manufacturers only guarantee that the leakage won't exceed, say, +-1 uA, which comes to about 3 MOhms and above. So you would need a voltmeter/probe with 100 MOhms input impedance, but the measurement won't make much sense due to the above considerations. When you try to use a 20-K DMM or even 1 MOhm scope probe, the instrument input impedance will drag the pin down and you will read zeros.
Many MCU vendors have "weak pulls" to create definite voltages on unconfigured pins during power-up sequence, to avoid uncertainties.
For some details on GPIO leakages see this answer.
The simplest way to determine is a GPIO pin is "floating" is to put a 10-MOhm standard scope probe on it, and touch the pin with finger. If you see 50-60 Hz rail-to-rail waveform, the pin is floating.