Looks like all I needed is a 0.1uF capacitor I had lying around and connect them between the signal capacitor (Q3 in question) and +ve:
Now:
- It doesn't falsely trigger no matter how many times I touch the circuit
- No effect from EMI
As mentioned by @Tim Williams in comment, the bigger the value of the capacitor the longer you have to press the button and also longer activation time.
For example, I replaced the 100nF capacitor with a 1000uF capacitor, and here's what happens:
The activation time is also higher:
With a bigger capacitor (4700uF):
So for now the 100nF capacitor works perfectly and I ran the circuit for 12 hours straight without getting it latched.