You will find that it has "fast mode" options for the ADC peripheral, and that you will need to use lower resolution, like 8 bit, to get the maximum speed listed. The higher the resolution of the ADC the slower and more power used during conversion.
There is a whole application note on the use of ADC in the XMEGA series by Atmel by the way. AVR1300
From reading that briefly, it seems you need to have a XMEGA A series chip to get the maximum 2Mhz ADC clock rate, and even then to clock get 8 bit resolution samples at 2Msps the source impedance needs to be under 1k Ohms. See graph on page 14.