This means that the last 15mV of its ideal 0-5V range are unused, so its range is really 0.15-4.85V.
will typically swing within 15 millivolts of the power supply rails with no output load.
Emphasis mine. So, add a load to the output and you will probably dampen that oscillation.
Note that this isn't what you'd normally see from a rotary encoder. Because that is not an analog sensor at all, actually. It only has an analog output for convenience reasons. Internally, it's more or less a digital counter with a digital-to-analog converter. Probably, that conversion is done with a PWM unit, followed by an analog low-pass RC filter, and that filters "better" when you load it. (Unless you design it with a sharper filter, which they seemingly didn't; the analog model is really just a PWM model + one resistor + one capacitor, most likely.)
Is this common to all analog sensors?
No, yes. I mean. Yes.
You get noise in every system that is warmer than absolute zero. In an analog system, you notice that as fluctuation away from the actual signal value.
If so, why is this the case?
Hm, imagine solid matter as a somewhat structured clutter of atom cores with electron bands, in which the electrons kind of "dissolve" and just contribute to the band as a whole. These bands exist as a statement of "at energy level soandso and impulse soandso, there's a x% probability of electron".
If you now apply a small vibration (that's what temperature is, the energy of mass bouncing around very quickly, but overall staying in the same place, otherwise the thing wouldn't be solid), you kinda disturb the whole distribution of electron probabilities.
That has consequences: because in some energy-impulse position that prior had a 0% chance of electron, you might now have a 90% chance of electron. That means that overall, the "likely" positions of electrons shift.
That's effectively a current. You see noise currents due to thermal noise, and that implies noise voltages.
Thermal noise isn't the only kind of noise: anything that conducts is kind of an antenna – probably a pretty bad one, but for some frequencies, an OK one. So you pick up radio waves, too, for example.
There's way more sources of noise than I could list here – but in the analog world, you can't make a decision on whether they're part of the signal or not.
In the digital world, you restrict yourself to discrete voltages, say 0 V, 1 V, 2 V and 3 V. If your receiving end sees "1.1 V", it can reasonably apply a correction to that. Sure, there's some likelihood that it's not a noisy 1 V (+0.1 V noise), but say a noise 2 V (-0.9 V noise) or a 0 V (+1.1 V noise) originally, but for most noise types, lower values for noise amplitude are more likely than higher, so this question can be answered with a maximum likelihood approach of "take the nearest discrete value".
That's why, whenever feasible (technologically and economically), an EE will choose to place the ADC as close as possible to the analog sensor: Where your 0.1 V noise on the digital line make no difference at all, it would be a 2% error on your analog 0 to 5 V sensor.