Presumably you're on Earth fairly near the surface. Find a table (or a machinist's surface plate). You can use a spirit level to make sure it's reasonably close to level. If you point the accelerometer vertically one way you get +1g and the other way you get -1g, so if you get it close to vertical each way, measure the mV and divide the (signed) difference by two you'll get the sensitivity in mV/g.
Add the two and you'll get some measure of the offset (but it will depend on hysteresis and nonlinearity in your accelerometer).
Trying to measure the offset by pointing the accelerometer horizontally is a lot more difficult because it's a sin(\$\theta\$) function rather than a cos(\$\theta\$) function so a relatively small angular error results in a relatively large error in g. For the same reason, small errors in angle don't matter so much when measuring the sensitivity with the accelerometer close to vertical in either direction.
To calibrate it in between 0g and +/-1g precisely you can use a machinists's index head, or less accurately, machinist's rotary table.