The device you are looking for is part of a whole system called a Software Defined Radio, or SDR for short. There are plenty of USB devices that can act as the input to an SDR system by sampling RF for processing on a PC.
One I know a little bit about is the HackRF Jawbreaker. You can find the official site here with ample technical detail at Github. The project is entirely open source and, with a case, looks like the image below.
Most USB devices used for SDR will allow you to plug your own antenna in. There are a number of projects looking at using DVB-T dongles to sample RF, as their chipsets lend themselves quite nicely to the task. If you can find decent software for your PC then this could be an inexpensive way of getting into SDR.
It is important to note that these USB devices are not generally "spectrum analysers" in the same sense as a dedicated benchtop spectrum analyser. Over a limited frequency bandwidth the USB devices can be persuaded to act like a spectrum analyser, for example a quote from the HackRF Kickstarter page:
The maximum bandwidth of HackRF is 20 MHz, about 10 times the bandwidth of TV tuner dongles popular for SDR.
If you are looking for a device that can sample near-DC to >1Ghz in real-time then you'll need to investigate USB/GPIB capable spectrum analysers.