You need to know the short circuit current and open circuit voltage of your solar panel to be able to tell for sure, but NiMH is a good option here since they self-limit any oversharge well given that the current is low enough, and at 80 mA, it should be. NiCd is even better in this regard, but they are terrible otherwise. If your open circuit voltage is much higher than 1.6 V/cell, I would at least think about it and test it first.
Perhaps you could put a bit less stress on the batteries by using an LDO like your suggested LM317 to limit the peak voltage. If it's a one-off, I would not bother. If it's mass production, you need to find and test the worst case scenario in this regard and check the datasheet of your batteries.
Normal NiMH charging is done with either negative delta V detection or temperature sensor to terminate the charge. The former is possible in your case but the current is too low for the latter.
