electronics voltage sensitivity

I have a 12v battery directly powering a handful of electronic devices (router, modem, hard drives).
These vary from 0.5a~2a so I have breakers on the (+) lines into the 12v dc power plug.
I'm noticing when I charge the battery through a 120v wall charger, the battery outputs up to 14.1v and during discharge it dips to 12v.

Since I would want to keep a constant, what device would I need to keep a constant 12v output from the battery during charging and discharge?
Would this dc/dc boost be a device that could help? http://www.mini-box.com/DCDC-USB-200?sc=8&category=981

edit i think this actually solves my issue Clean 12v car power