An additional caveat... sometimes you actually want randomness.
For example:
In cryptographic (security/authenticity) applications, pure "unguessable" randomness is required. Using a converter's LSB's (those below noise floor) is a quick way to generate lots of purely random numbers quickly.
When the ADC hardware is available for other purposes (sensors and the like), it's a quick-and-easy way to seed secure communication.