Could anyone explain limit cycle related spurious tones? It is stated that a higher order modulator is better than low level one because of limit cycle related spurious tones, but I don't have picture of the limit cycle. Does it mean the tones in DC input and this tones in DC with period appear to be an increasing high frequency tones?
That's easy to answer with a general answer and immensely hard to answer in full detail that people have been writing books and papers about it for decades.
I've had some personal experience of this about 10 years ago when I implemented what was meant to be a simple and cheap SD converter (other side of same coin) with a mix of hardware and software and discovered a whole new world of opportunities to get unexpected results in the midst of normal behaviour.
Here's the easy general answer to your question:
VERY roughly limit cycle oscillations (or spurious outputs or whatever) are spurious products that appear in an SD system in the presence of a constant input signal that should ideally be converted 'perefctly'. The constant input may be a DC level of a sinusoid or a mix of sinusoids which have a periodic function which interacts 'in some way' [tm] with the SD system due (probably) to the sampling period of the SD system not matching the period of the input system OR the SD having several stable states which it can flip between at certain points in the cycle. (Thing eg model railway circuit with many subloops and points which are switched randomly at certain locations).
That answer can be summarised as"makes funny outputs unexpectedly as a result of its nin-mlinera nature" and may sound excessively naive and simplistic.
Glimpse into deep mysteries:
This very useful slideshow fromBerkley EE247 lecture 24 does a nice job of providing a "demystified" explanation and purports to provide a solution. It says:
Limit Cycle Oscillation
BUT !!! - the following suggests that causes or solutions may not be quite that straight forward.
Here's a sample of perhaps the best related content from page 100: