Regardless of the output of your existing encoder, you'll never get what you want from it. In order to get 1 second of arc resolution, the encoder will need 360 x 60 x 60 ppr, or just about 1.3 million ppr.
If you have analog (sine) outputs, you might optimistically hope for x100 interpolation, but that still gives you 60k ppr, or more than an order of magnitude less than you need. Plus, units with nominal resolution this low don't ordinarily have good sine outputs.
What total angular range do you need to produce and measure? If you need 360 degree azimuth capabilities, you might look into high-end telescope mounts. This, http://www.bhphotovideo.com/bnh/controller/home?O=&sku=1059016&gclid=CjwKEAjw96aqBRDNhM6MtJfE-wYSJADiMfggqATnClg6dU22LzlKee3LraQYf7j2-gto9peKS5BnLxoC0-Hw_wcB&Q=&is=REG&A=details for instance, is rated for a resolution of .06 seconds, although it has a (correctible) periodic error of 5 seconds. At $2500, it may be a little rich for your blood.
Alternatively, you can look into motorized rotary stages, again for serious bucks. Edmund Scientific, for instance, has http://www.edmundoptics.com/optomechanics/positioning-stages-slides/motorized-positioners/zaber-motorized-rotary-stage-system/3305/ which will give you 4.1 urad resolution, which is about what you want.
If you're looking for a shaft angle encoder, you can get what you want. Here's an example http://www.gpi-encoders.com/PDF/9480H.pdf Note the size. Also note that price is not listed. I'd expect multi-kilobucks, although I might be wrong.
Please be aware that, what ever you wind up with, you will need to provide extraordinary levels of mechanical precision. The slightest bit of runout will cause problems. Dimensional changes in your system due to temperature variations will also be a problem. Lots of luck.