The SWD protocol is device agnostic, and in fact there are only a small number of variants of the SWD to internal memory-mapped bridge components. The protocol provides an abstracted mechanism to perform accesses to on-chip components, and also a discovery mechanism to identify which components are present.
This means that all SWD debug hardware should be able to interface to all modern Arm chips, providing they operate at the correct voltage.
The external device also determines the clock speed, so there is no restriction about using a low performance adaptor with a high clock speed device.
Where you can observe issues is the software driving your debug probe. Even assuming the software has support for the probe, your software will need to recognise the target processor and sufficient of it's memory map to function. For example, the debug port access transfers to a Cortex-M and a Cortex-A are not identical (even though they follow standard templates). The ID register values for a Cortex-M33 won't be recognised by 3 year old software, etc.
You can also find debug harwdare or software which is locked to specific device types or manufacturers, but they still use the standard protocol.