You will need an additional "switch" in series with the load circuit to disconnect the load. Q1 Cannot disconnect the load because of the manner in which it is connected in its present role.
A P Channel FET "Q2" (which could be of the same type as Q1) connected in series with Q1 d-s but with opposite polarity can provide the switching.
A voltage sensitive comparator then drives Q2 gate low when battery voltage is >= 3.2V. A TLV431 would do this with 3 added resistors - 2 form a voltage reference divider for the TL431 reference input and one forms a pullup for Q2 gate to turn it off when the TLV431 is not turned on. The reference input resistors should be as large as bias currents allow to minimise battery drain.
A TLV431 is specified rather than a TL431 as it has lower minimum regulating current requirements. If ultra low batteruy drain bia R1 + R2 is required there are shunt regulators that need only about 1 uA bias currents in R1, R2

simulate this circuit – Schematic created using CircuitLab