I use these quite a lot. When making a circuit where any high-frequency signals need to be measured with minimal disruption to the circuit, placing a cheap BNC jack as a test point is a very easy way to get a good solid connection to the probe. It also provides better ground integrity than the traditional ground spring (and far better than the grounding clip that's basically an antenna), and better mechanical stability--you can insert the probe and leave it there without it falling over or straining anything on the board. Handy when you have three or four different points that all need to be probed at once, and you don't have three or four hands to hold them all!
If I were to use a BNC cable here instead, the cable's capacitance would create a major problem--I'm trying to measure high-speed events, and adding so much capacitance would ruin the signal. So instead, I use a scope probe, with its high input impedance; 10 MΩ is typical of a 10x probe, with 9 MΩ of that being located right at the tip of the probe. This isolates the cable capacitance from the circuit well enough for the measurements I need to take.
But if a 50R termination is not allowed, why would there be a BNC connector in the first place?
Because I'm designing the circuit, I know I have these adapters handy, and I put a BNC jack there specifically to use them. I usually add something on the silkscreen to the effect of "high impedance scope probe only", because plugging in a cable there would ruin the circuit characteristics.
BNC seems like a weird choice for a connector [for high voltages].
You're right, it is. And that's not what I would generally use these for, though I have used them for high voltages when there was really no other option. It hasn't broken yet, but I'm definitely using the parts outside their ratings. It's an acceptable sacrifice in pursuit of science.