If the battery is correctly placed, voltage between drain and gate will always be about 4V. So MOSFET will be passing and current could flow in both directions.
I would pay attention to the following : If your lithium battery is too low the Vdg voltage may not be enough to turn on the MOSFET fully. Current will be able to flow but in the worse case it will flow through the intrinsic diode causing voltage drop and power loss. In this particular case you won't be able to charge the battery (intrinsic diode will block the current). So check the threshold voltage that turn your p-Mos on to be sure that it fits with your battery voltage.