Basic parameters
To summarize, you want to log a analog signal of up to 100 mV amplitude at 10 kHz sample rate. That's quite doable.
You want to collect data for up to "weeks". Let's see how much data that is. You didn't specify resolution or signal to noise ratio, but let's say 12 bits/sample, or 1.5 bytes/sample. That's 15 kBytes/second = 900 kBytes/minute = 54 MBytes/day = 380 MBytes/week = 1.5 GBytes/month. That's also doable.
Amplification
I do not want to amplify the signal because that would introduce distortion.
That's just silly nonsense. Amplifying this signal can be done relatively easily without adding distortion at the level of 12 bits/sample. How do you think they take even smaller signals from air microphones and ultimately get to the volts or 10s of volts signals to drive HiFi audio speakers? This has been a well solved problems for many decades, even before there were transistors and opamps.
Basic design approach
I want to attach the recording mechanism right to the hydrophone in a water-tight box. I can make the housing, but I do not know how to record the data, or what could be used to record and digitize such a low voltage signal.
The basic strategy is to amplify the signal to maximize it within the voltage range of a A/D, then grab samples from this A/D and store them in a flash memory.
Distortion requirements
In general, the goal is to to collect and digitize the low voltage signal as purely as possible, minimizing any noise or distortion.
Without numbers, that's no spec at all. Surely there are other tradeoffs, like cost, space, and power, just to name a few obvious ones. Don't say "the best you can do" since that's meaningless. Best in what context? $1,000,000 budget? Surely you know something about the signal. Since you haven't given us any guidance, I'm going to proceed assuming 12 bits/sample is good enough, because probably it is.
Use a microcontroller
The process of grabbing samples from the A/D and stashing them in a flash memory would certainly benefit from using a programmable part like a microcontroller. Plenty of microcontrollers have 12 bit A/Ds built in, which makes things easy.
Amplifier parameters
It's not clear whether your "0.099 volts" maximum amplitude is RMS, peak, or peak to peak. For sake of example, I'll round off the "0.099 volts" to 100 mV and assume that's peak. That means the raw hydrophone signal goes from -100 mV to +100 mV, with a range of 200 mV.
The microcontroller will run from 3.3 V, and let's say you run the A/D from a separate 3.0 V reference to ensure it's nice and clean. You therefore want to expand a 200 mV range to a 3.0 V range, which means you need a gain of 15. Again, that is really no problem, and won't add any meaningful distortion or noise.
The amplifier will also contain a high pass filter, or "DC blocking" filter. You need to specify what your lowest frequency of interest is. For example, HiFi audio considers everything below 20 Hz unhearable. You are therefore free to throw it away if the end result is for human hearing. The DC blocking filter will also allow you to offset the midpoint of the amplified signal to the middle of the A/D voltage range, or 1.5 V. This is again easy to do electrically.
Sampling and aliasing
Since you are sampling at 10 kHz rate, the maximum theoretical frequency you can capture is 5 kHz. Any input frequencies over that will fold back to lower frequencies as "aliases". This is something you want to avoid. To do this, you need to eliminate frequencies above half the sample rate from the signal before you sample it.
Making analog filters sharp and accurate is difficult, but at your sample rates is not so hard digitally. Assuming you want flat frequency response close to the 5 kHz limit within reason, I would actually sample significantly faster, then filter and reduce the sample rate digitally. This is done often enough it has a special name, which is decimation. For example, you might sample at 100 kHz (still easy for 12 bit A/Ds in microcontrollers to do), low pass filter digitally so that 5 kHz and above is attenuated to oblivion, then pick 10 kHz samples out of the resulting stream.
This allows for a sloppy but effective analog filter before the 100 kHz sampling. For example, three poles of simple R-C filtering with rolloffs at 10 kHz attenuates by 125 (-42 dB) by the 50 kHz aliasing limit of the 100 kHz sample stream. Most likely your microphone and other physical things in your system will attenuate significantly at that frequency too. I'm assuming you don't have to worry about a full amplitude 50 kHz signal being received, so -42 dB at 50 kHz is good enough.
Storage
Once you have the 10 kHz sample stream, you have to store it somewhere. With today's technology, that is best done in a flash memory. You need less than 2 GBytes, so there are many options available. At that relatively small size, you can use flash chips directly, managed by the microcontroller. Some commercial memories, like SD cards, can be interfaced with over something like a SPI bus, which is something the microcontroller will have a built-in peripheral to handle. If you are willing to make your own device to read the data off this flash card, the you don't have to use a file system. Otherwise, you probably need to implement something like FAT32 with just a single file at a known location.
Power: The real problem
You need to consider the power requirements. Let's say you need 100 mA at 3.3 V to run all this, and that the power system from battery to ultimate consumption is 80% efficient. 100 mA at 3.3 V means you are consuming 330 mW. Taking the 80% efficiency into account means you need a battery that can supply 413 mW. That's 413 mJ/second, 25 J/minute, 1.5 kJ/hour, 36 kJ/day, and 1 MJ/month. This is your real problem, not amplifying or sampling or storing the signal.
Let's put this in perspective. First let's look at how big a lithium battery this implies. Assuming average 3.5 V out of the battery, you need (1 MJ)/(3.5 V) = 286 kAs = 80 Ah. That could be, for example, twenty 4 Ah cells bundled together. That's quite a bundle, and will be the bulk of the size and weight of your overall device, but it is at least within the realm of reasonably doable.
Again, you really should be focusing on the power requirements. That's your biggest problem, and where good engineering can make the most difference.
