Power: As Janka pointed out in his comments, your power supply must be within spec per the datasheet. As you've shown it, it doesn't meet that spec. Now, you can power it with a single positive supply (4.8V <= V+ <= 10V) on V+ and 0V at V-, but then you will have to bias the input to accommodate the asymmetrical power, and AC couple your source. It's all about tradeoffs.
Offset: Your gain is 2k (2001). Per the datasheet, max input offset voltage can be as much as +/- 0.25mV. Assuming worst case, that will add 0.5V to your output. With a signal swing of 1mv pk, your 1Hz output is 2.0v pk. Add the 0.5V offset, and if you're powering with +/- 2.4V, you will be in saturation on signal peaks. If your signal is 1mv p-p, then you have head room.
Input Balance: Add a 1k resistor at the opamp's (+) signal input. This will minimize additional offset caused by input bias current. With gains this high, these seemingly small values can stack up.