For any given loop, it will have a frequency response. If the frequency response is 'peaky', then we'll often say the loop has marginal stability. This will amplify the input noise, and as the gain is high over a small range of frequencies, it will tend to filter it and make the output look oscillatory.
However, a properly designed loop will have minimal or no peaking, and will minimise this effect.
If the frequency response goes to infinity at some frequency, then we'd say it was unstable, and it would oscillate at that frequency, but with very large amplitude, so that wouldn't account for small oscillations.
If it's a digital implementation, then it will have a finite resolution in its internal representation of the parameters. As it's a feedback system, there is the opportunity for limit cycles to occur. If the word length is short enough, these limit cycles may be visible.