Obviously, temperature range. If you get one that will go very cold (say much below -20°C), different refrigerants have to be used, and it may not have as good efficiency or slew rate. You probably want to be able to go at least to 125°C on the high side.
Ease of monitoring, ease of getting wires in and out, interface between your DUT and the chamber programming to allow shutdown, emergency stop and other safety features. It's nice to have pre-programmed routines for MIL-whatever, but it's essential to be able program what you need (maybe you're doing cycling for burn-in).
If you do sensitive things you might be worried about mechanical vibration. They tend to be noisy so they're not nice to have right next to you. Humidity is a bit of pain, and may require plumbing-in to work well.
This is not the place to recommend suppliers, but I was able to find a fairly large new one for only about $15K with all the features I wanted (not bench top, it has half a small room dedicated to it). At maybe 2' x 2' x 2' cube inside, it's still too small for some things. Of course if you feel lucky you could try eBay, but... and there are u$ed equipment dealers. Some chambers use LN2 to cool rapidly- there are certain hassles and safety issues involved in dealing with cryogens if you don't already do that in-house.