This falls under the topic of "permissive changes". The FCC has a document called PERMISSIVE CHANGE POLICY which outlines what can or cannot be changed without requiring a new FCC ID.
Under section 2 c). it says, "Part substitution – electrically identical parts may be substituted."
I would assume that the two batteries are the same voltage, but perhaps different capacities (mAh) or chemistries? For example, allowing for either rechargeable or non-rechargeable batteries in the same package (such as two AA's) shouldn't be a problem. However if the two battery variants are substantially different in size, it could be argued that would affect the RF performance of the device (since they have a metal casing) and require re-testing.
Your RF testing house may be able to give you some guidance on whether different size batteries would affect the results.