What you propose is OK as long as the battery is not charged beyond its maximum voltage rating. For really critical applications, that maximum voltage rating must be adjusted as a function of temperature. Otherwise, use the lowest allowed voltage over the temperature range the battery will encounter.
The charge current must also be limited, especially when the battery is low. However, for a solar panel, that can be achieved by sizing the panel so that it can't over-current the battery with maximum sunlight.
Also be careful about the charger drawing down the battery when there is no power from the solar panel. Even something like a voltage divider to feed a fraction of the battery voltage into a comparator or A/D can drain the battery over time. A diode in series is one way to deal with this, but then you have to be careful about the maximum voltage setting. A Schottky diode may drop a few 400 mV at full charging current, but may drop almost nothing at low currents when the battery is nearly full.