The input offset voltage is fairly constant (yes, a positive or negative fairly constant value) at a given common-mode voltage. It should not change with output voltage (so not proportional to output voltage). So in the case of a closed-loop inverting amplifier the output should be -Rf/Rin*Vos. Of course it will vary with chip temperature and maybe other things like time and perhaps even how much you bend the PCB.
If you have a non-inverting amplifier then you mix up the common mode rejection of the amplifier and the input offset voltage at a fixed common-mode input voltage.
It is not generally possible to measure the input offset voltage of a modern op-amp open-loop as your equation implies. The gain of a modern precision op-amp is very high and not well controlled (it might vary by 2:1 from unit to unit). So if the gain is (say) 3E6 and the input offset voltage is 25uV then the output would saturate most likely (75V). Below is a typical method of measuring input offset voltage from an Analog Devices application note:
The auxiliary op-amp is an integrator that forces the output of the Device Under Test to close to 0V. The output at TP1 is 1000x the input offset voltage, so it is easy to measure with a multimeter.
One consequence of Vos possibly being either polarity is the behavior of something like a single-supply non-inverting amplifier. If the Vos is positive then the output will be Vos*(1+Rf/Rin) with 0V in, but if the Vos is negative, the output will sit close to 0V until an input voltage exceeding -Vos*(1+Rf/Rin) is applied.