Don't go with round machine pins. Yes they are higher quality but are available with fewer options and right now you need those options. Go with square headers instead.
Get right-angle long-tailed male headers to go on the board (don't go female on the board because they are available in fewer options).
Then get crimp housing with female crimp contacts.
I would get a small flat screwdriver and carefully wedge it between the gaps on the sides.
I have encountered connectors like this that are actually not connectors, they are wire to board connections and can only be unsoldered. You can tell if it looks like there are wires soldered on the underside of the board or metal posts (both will be soldered, if it's ...
This is not a good idea overall.
Avoid soldering straight to a PCB in case it needs to be replaced. Use connectors instead.
In case you do solder straight onto a PCB, then do so directly in the vias. Remove the socket strip, it fills no purpose. The solder job on the picture is poor and the presence of the awkward socket strip is likely partially to blame ...
Pins 21 and 22 are the i2c pins. So I’d say that is a QWIIC connector or whatever the other suppliers call it. You can purchase various boards from Sparkfun et al that plug into that. I have a little oled module using that interface.
These are the right ones. No other type will fit on. Some one have a center locking pin, if the contacts have a hole in the middle, they hold better than cheapest variants. You have to read the datasheet of the C14 plug and select the correct width of the contact.
I don’t know what housing or heatsink you are using or even power supply, but if the connector is a problem, you need help and they need something fool proof.
My simple solution would be to attach them to small obsolete circular CPU heatsink so that a fan is not needed for 5W.
For power I would derate 25% approx and run two Red LEDs in series with a Molex ...