Batteries are commonly sealed as air contact would increase the speed at which they dry out and die. An exception to this rule are Air-batteries which need the air to function properly.
The epoxy-dipping might interfere with the internal pressure relief valve in case something goes wrong inside the battery. So the cell might rupture with a higher pressure - but I suppose the epoxy coating is not strong enough to make a difference. You won't have any problems with swelling of the cell and cracking the coating, as cylindrical cells don't do that.
Not entirely sure on the chemical compatibility off all the materials involved. Do you have a datasheet of the chemicals available? Are there any combinations listed which are to be avoided?
I guess it'll be fine - just never heard of someone doing it, but I also haven't found any source for it being bad or good.
One thing I found was 3M advertising liquid epoxy adhesives in a brochure as a great way for battery assembly applications. So the batteries might be glued with epoxy to something, guess coating it should be fine.