Input offset means the op amp can see a difference of >6mV between it's inputs. The output can still be over 6mV. You need a rail to rail opamp which goes down to zero volts at its output. I don't know if there's anything cheap which can do that.
Here's a tutorial on input offset voltage https://www.analog.com/media/en/training-seminars/tutorials/MT-037.pdf
The usual way to get a zero output is to use a balanced dual supply. And, then some kind of protection circuit to make sure output is within uC input limits. This makes every single op amp ever made go down to zero volts.
Or, one can add the input with a bias voltage (65mV in this case) and then feed it to the op amp. But, this will likely introduce even more error.
If you can find an opamp that does have 0 V output, but has a high input offset, you can still use the LMV324. Set a gain on the LMV324 and feed it into the other opamp.
[The OP07 is not a rail to rail opamp. From the datasheet, with ±15V supply, output swing is ±12.5V max. That means, if power supply is single rail 5V, output will vary from +1.5V to +3.5V max, instead of 65mV to 4.99V for the LMV324 (typical), or even 5mV to 3.5V for the LM358 (typical)]
added At 3.3V single rail supply for the opamp, for measuring battery voltage, the LM358 is fine. I would try to increase the supply voltage a bit if possible to account for inter chip variations, as 3.3V leaves exactly 3.3-1.8=1.5V headroom (which is the limit for this opamp).
As Doodle and others say below, in reality a battery will never drop to zero volts, so rail to rail opamp is not really required.
Input null would be great for accuracy. However, if the OP is using a uC, the calibration can possibly also be done in software, if the input offset for the particular chip is stable (this is an assumption).