Most ICs have ESD protection diodes on their inputs. That means there is a diode from the input to the positive supply and from the negative supply to the input. These are reverse biased in normal operation. Their purpose is to clip out of range voltages that could damage the rest of the IC.
When the opamp is unpowered, the positive supply is supposedly at zero, so applying more than one diode drop to any input will cause significant current to flow in that input and also sortof possibly apply power to the device thru the diode. That's not a reliable way to power the device, and it could result in latchup and other bad things.
Another point is why are you buffering a battery voltage? Most likely the battery can deliver significantly more current than the opamp can. The buffered output might be a little lower impedance if not exceeding the current limit, but in a battery powered device this current will still come from the battery. The buffer may hold the output voltage close to the input voltage, but the drain on that output voltage will lower the battery voltage at least as much if the battery were connected directly. This isn't making a lot of sense.