When I worked at 1 and 2.5 GHz, we used these connectors without consideration for stub effects. Now, working at 25 GHz , we would just about never use them (even the variants with coax connector types appropriate for 25 GHz). However, your operating band is somewhere between there so these rules of thumb aren't especially helpful.
I agree with Andy's analysis that, especially if you trim the stub protruding from the opposite side of the board, this connector is likely to work well at 6 GHz. But you haven't said what your application is and whether you have particularly stringent requirements for VSWR or for insertion loss flatness across a wide frequency band. So I'll add a few suggestions.
Using an edge-mount SMA connector would produce a smaller discontinuity between the pcb trace and the connector.
Routing the trace on the opposite side (or, on a buried layer close to the opposite side) of the board from where the connector is mounted would produce a smaller discontinuity.
In comments you asked,
Do you have any suggestions regarding the ground planes underneath the connector?
I'd pull all copper away (maybe by 3 - 5 mm?) from the stub part of the connectors center pin. (The stub is the part that isn't on the path from the trace to the other side of the connector). This will reduce capacitance between the pin and other nets (mainly power and ground) which I'd expect to reduce the impedance discontinuity. (The trade-off is it could increase radiation from the stub)
If you want to get really fancy, and your signals are not particularly wide-band, you could break out an EM simulation and probably find a way to add an inductive discontinuity (neck down the trace as it approaches the connector) to your trace to compensate for the capacitive discontinuity from the stub --- but I wouldn't advise doing this without being able to optimize it in simulation, and it's probably not justified if your application doesn't have especially strict VSWR requirements.