I can't use KVL since i don't know the voltage rise of the current source after removing the load.
In comments you asked,
Would it be possible to further simplify this circuit?
You can further simplify this circuit by removing the voltage source and resistor, leaving just the current source. Any elements in series with an ideal current source have no effect on the behavior of the circuit.
So, back to your main question,
How could I find Thevenin's voltage in this circuit?
You cannot. This circuit behaves exactly like an ideal current source, and therefore it does not have a Thevenin equivalent.
Note: I am ignoring the ground symbol in your schematic, assuming it is only there to make the Falstad simulator happy. If it is an intentional part of the design then you can not form a thevenin equivalent for this circuit because it is not a 1-port network --- the ground symbol gives a third connection point that might affect any external circuit that also has a ground symbol.