In theory yes, connecting to 5V and ground wires of an ATX supply provides 5V to your circuit.
In practice, there are a few issues. The first one is that the power supply won't provide the main output rails before it is turned on by your circuit. When it is in standby, the only output is the standby 5V which may not provide enough current.
A second, but more larger issue is, that ATX power supplies are generally not ideal for custom hobby projects. They can have problems if you only load one output and have no load at all on other outputs. They may not turn on or stay on with no or very low load. And they may turn off if you suddenly remove all or most load it has. So as a workaround, extra dummy load resistors and capacitors may need to be added to each output rail.