Many novices seem to want to measure the current output capability of power supplies by setting their meter to read current, then connecting it directly across the power supply terminals. YOU MUST NEVER CONNECT AN AMMETER DIRECTLY ACROSS A POWER SUPPLY Doing so will cause very large currents to flow, and may permanently damage the meter, power supply, or user!
If you keep the meter in voltage mode, and are careful to not short the test leads or power supply wiring, you should be able to measure the supply voltages without any hazard.
Many PC power supplies (and other switch-mode supplies) require a minimum load to operate correctly. Be sure you have a suitable load connected during your tests (or any time the supply is on.)