First of all, despite the initial paraphraph in Andy aka's answer, your use of 9.81 m/s^2 is correct (if not well explained): You correctly converted your sensor's output noise amplitude from voltage units to the corresponding noise amplitude in acceleration units. Where you seem to be stuck is how to make sense of it.
To use the output of your accelerometer, you will pass it through some kind of (likely digital) filter or other algorithm. After all, the direct output is an instantaneous acceleration where, as you calculated, the noise alone means any individual value on its own will typically be off by 4 mm/s^2 and hence larger than the deceleration due to the drag you are interested in. So how much can you lower this noise by making more than one measurement?
The most obvious thing would be to average measurements, but that may be more difficult than it sounds, unless you can guarantee that your sensor is always perfectly aligned with the motion of your satellite. However, it is something that can be calculated relatively easily, without referring to things you'll learn later in aerospace engineering. But since it should interest you, let me at least drop the keywords: I'm thinking about sensor fusion and Kalman filtering.
So let's hypothetically say you have one full orbit to determine your acceleration value (this won't work in practice due to sensor drift, see the next paragraph). Essentially, you will be integrating (or averaging) your sensor output. The noise level you will see depends on what the actual, frequency-dependent noise is after taking into account only the frequencies that pass through your (integrating or averaging) filter. Andy aka's answer tells you the answer for one specific case assuming you have white (frequency-independent) noise and a specific frequency cut-off value. A better prediction will take the frequency-dependence of the noise into account. Just remember to calculate in what makes physical sense, noise power (the squares of noise amplitude), and only convert to what makes sense to us electrical engineers after you're done calculating/filtering/integrating. That way you'll likely get the right result immediately and conveniently avoid having to ask "what is root-Hz, and how do you calculate with it." At sub-Hz frequencies, the frequency dependence of your noise will almost certainly be 1/f noise, and your accelerometer datasheet may actually list a value for it.
Now for the caveat: Your frequency range for using an entire orbit for averaging, sub-mHz, is so low that the noise will probably have an even worse frequency dependence than 1/f. Because it's just too slow to measure conveniently, engineers often invent other names for it, talking about drift instead. Of concern is both the kind of drift that a theoretical scientist might prefer to call some worse-than-1/f-noise and that induced by environmental factors like temperature changes. Have a look at what all of this does in your application (the relevant figures are likely specified in your accelerometer's datasheet as worst-case limits for temperature-output coupling and longterm drift). If you're clever, you may find ways to custom design an algorithm (or complicated nonlinear filter) that removes part of the drift by comparing "coming" and "going" parts of the orbit, or rotating the sensor to be in and then opposed to the direction of motion. At some point, knowing something about Kalman filters may help...
