I recently measured a brand new Duracell AA battery which is labeled as 1.5 V. The reading was 2.3 V. Is this ever normal for a fully charged battery, or is my multimeter broken?
Most likely cause is that you need to replace the battery in your multimeter.
When the battery is very weak, the voltage drops to the point where the internal reference can no longer maintain regulation and it, in turn, also drops. A low reference voltage results in a high reading, which is exactly what you are seeing.
Usually there will be a low battery indicator in the display, but you may have not noticed. Personally, I think the display should be blanked rather than display an erroneous reading, but many meters do not do that.