Just to add a bit to Andy's answer (a mathematical approach). In order to understand why, you need to be familiar with the frequency response. An opamp has input and stray capacitance on the inputs, which reduces the closed-loop bandwidth, as stated in the answer.

simulate this circuit – Schematic created using CircuitLab
Without going into the math, you can find the loop gain (which you'd use to find gain and phase margin for stability purposes), and it turns out to be:
$$\text{Loop Gain}=A_{ol}\dfrac{R_1}{R_1+R_F}\dfrac{1}{\frac{s}{\omega_p}+1} $$
Now, the opamp open loop gain, \$A_{ol}\$ is frequency dependent and we could model it as a 2 pole system:
$$ A_{ol}=\dfrac{A_{DC}}{(\frac{s}{\omega_1}+1)(\frac{s}{\omega_2}+1)}$$
From this, you know that if the pole due to the input capacitance (\$\omega_p\$) is close to the \$\omega_2\$ pole, you are adding an extra 90 degree phase shift and that puts you closer to instability. In the ideal case, where \$C_p\to 0\$, this pole is far away from the second pole of the open loop opamp gain, but as you increase the resistors' values, the pole may move to a bad spot. That is why, from a math standpoint, you may have to reduce the resistor values to avoid this.
In order to compensate for this, you may place a capacitor in parallel with the feedback resistor (as you have it), and then choose \$R_1C_p=R_FC_F\$ and that will cancel out (ideally) the effect of the pole caused by the parasitic capacitance. You could go through the math and derive this, I just didn't want to expand a lot more on what already has a good answer by Andy.