Audio outputs accepting stereo 3.5mm TRS connectors will have 2 separate channels - the tip for left signal channel and the ring for right channel. Thus on a minimum, you need a dual pole (two channels) double throw (two devices) switch to toggle between the Xbox and PC. For the safety of the circuits in the devices, one output device must be disconnected first from the speaker before the other will be connected. To do that you need to specify a "break before make"/BBM switch. The sleeves of both devices will share the same ground going to the speakers.
The switch that you found is a DPDT (dual pole double throw) and from its datasheet (model STA 206 P) has an on-off-on contact pattern or 3 positions (device one, none, device two) which satisfies the BBM requirement. You can use that switch and ignore contact ratings - the current of the audio signal is magnitudes smaller than the typical capacity of miniature toggle switches.