I've been working through something similar, myself. Not having Matlab, I've had to do everything with [sigh] Excel. So my setup may take a little explaining...
Start with a column counting from 0 to 4095. Emulate a 50MHz ADC by filling the next column with time increments. After that, a few columns that make sine waves, based on the time. I selected 11.5MHz, 12.5MHz(desired signal), 13Mhz and 14MHz signals.
Sum up the sine waves, label that column FFT_IN. Use Tool to do a FFT Transform, plot the magnitude of the result: a nice little spectrum. That's your INPUT.
Next: Demodulate to get that 12.5MHz. Make "I" stream by multiplying FFT_IN by: 0,1,0,-1.
"Q" is: 1,0,-1,0. Result: samples alternate between I and Q, with every second sample in a stream inverted. (Optional: I also run a little FIR on this data)
Merge the I and Q streams into complex samples, and do another FFT. Surprise: the signal starts at DC (was 12.5MHz) with signals present at 500KHz, 1MHz, and 1.5MHz.
Why? Because the 11MHz signal has been "reflected" back into the positive frequency domain.