In the datasheet there's a note regarding that section
The note says:
Wanted signal level at PIN = -64 dBm. Two interferers with equal input
power are used. The interferer closest in frequency is not modulated,
the other interferer is modulated equal with the wanted signal. The
input power of the interferers where the sensitivity equals BER = 0.1%
PIMD,1M IMD performance, 1 Msps, 3rd, 4th, and 5th offset channel: -33 dBm
means that the power level of both interferers can have a power of up to -33 dBm.
Since the wanted signal is at -64 dBm, those interferers can be 64 - 33 = 31 dB higher in power than the wanted signal.
Under those conditions the BER (Bit to Error rate) is 0.1% (1 bit per 1000 bits can be wrong). More accurate: the chip's manufacturer guarantees that the BER will be 0.1% or better under these conditions. It is impossible to design for an exact BER, I know, I worked on BER and Bluetooth many years ago.
You cannot draw conclusions from this for your transmitting at Y = X + 3 case because in the specified case the carrier at + 3 MHz is unmodulated. You ask about a modulated carrier which makes it a different scenario.
Also there is no 33 dB in this. What is mentioned is -33 dBm so 33 dB less power than 1 milliWatt. -33 dBm is an absolute power level, the level which the interferers can be for the stated conditions for this specification point.
Such a specification point might look weird, like it is some "random" condition. Well it isn't. It is usually a result of the Bluetooth specification and then translated to include antenna gain and antenna switch losses and such.