Is it possible to damage phone battery when charging with a current source?
Yes as it is also possible to damage a battery with a voltage source.
But your battery is charged by a charging circuit that behaves as a current source (when the battery is quite empty) and later as a voltage source (when the battery is almost full).
Sure, now you're confused !
Damage occurs when the current or voltage is too high. Simply prevent that and you will be fine.
But actually your headline is not your real question. The charging circuits are in the phones and it should stay that way for safety reasons.
All you need to do is provide 5 V to the phones. This is what planned you to do already.
The fact that the PV module can generate a high current or voltage is irrelevant, it should generate 5 V DC for the USB and that's it. Sure at the solar-panel side of the PV controller the voltage and current will vary but a proper PV controller can handle that. It will simply convert the energy to 5 V or 12 V at the output and the phones will simply take as much current as they need.
If you cannot find a charge controller that can provide 5 V, you can use one that provides 12 V and use DCDC buck converters like this one
to efficiently convert that 12 V into the 5 V you need.