I had the same question when I first came across this and here's how I think of it. The whole point of Thevenin is to reduce a complicated circuit into a much simpler circuit consisting if a voltage (Vth) and a resistance (Rth). If you define Vth to be the O.C. voltage at the output of your complicated circuit then consider how this voltage might be formed by current through your complicated circuit flowing across some (as yet) unknown resistance.
To find this unknown resistance you might visualize the Vth generating current that flows back into your complicated circuit and think of all the paths it would take.
Now think of this Thevenin current attempting to make its way along the branch with the current source. An ideal current source will flow its current in any attached load and its model has an infinite resistance in parallel (in a real current source, this resistance is not infinite and will draw a varying current as the load resistance varies). You're only interested in the resistance of this path, so you switch off the current source but it's (infinite) parallel resistance remains. Since the resistance is infinite, no current will flow - it's regarded as an open circuit.
A voltage source is different. It's modelled as having a theoretically tiny (i.e., shorted) resistance in series. Current flowing through this voltage source encounters this tiny (i.e., non existant) resistance. And again, when you turn off the voltage source you're left with this infinitely tiny (i.e., short) resistance.
In reality, current flows around the circuit and out through your open circuit Vth, but it encounters the same resistances as if you had flowed the current into the circuit with the voltage and current sources off. It's you that defines Rth by following the established rules. But, just remember that what's happening is you're finding your back box resistance (Rth) that's establishing some voltage at the output (Vth). Hope that helps!