You can, but as per @Dampmaskin's answer, it's not really the best plan.
Why not do as often seen in new electronic devices, simpleand inserst a thick plastic slip put between a battery and one of its the terminals (or between two batteries). If the slip is long enough to be seen outside the closed battery box, and a clearly visible sign says something like "remove before use", you can just pull the slip to "reactivate" the device.
You can also use posterboard or any other relatively rigid but thin non-conductive material to accomplish the same effect.