The issue you've heard about is that noise couples easily to any node that has a high impedance to ground at the noise frequency. Here, that would be your inverting input, as you say.
There are a couple things you can do. The simplest is to make the high-impedance node less high-impedance; right now, its impedance is something close to 50 kΩ, but you can reduce that by an order of magnitude by using 10 kΩ resistors instead of 100 kΩ. Of course, this increases the loading on your op amp, and reduces the circuit's input impedance, so you can't just go all the way down to using 1 Ω resistors.
Another method, commonly used in high-precision circuits, is to create a guard ring around the high-impedance node. This consists of a trace surrounding the node that's driven with a low impedance to a fixed voltage (ideally driven to the same voltage as the high-impedance node's voltage, which also eliminates leakage currents, but any fixed voltage is better than nothing). This shields the high-Z node from outside interference, because the guard ring will absorb the incoming noise instead of the high-Z node. For this circuit, the guard ring can simply be grounded.
Keep the high-Z node as small as possible. This reduces the coupling to external noise, and makes the guard ring easier to implement. Here, the guard ring would need to be positioned like this:
simulate this circuit – Schematic created using CircuitLab
It needs to cover the right side of R1, the left side of R2, and the inverting input of the op amp. That means that ideally, R1 and R2 should be as close to the op amp as possible, otherwise you'd need to extend the guard all the way down a cable or something.
It can also help to have the guard ring extend to below (and even above, if possible) the high-Z node as well, on a multi-layer PCB. If this is impractical, or adds too much capacitance to the node, you can also try putting cutouts in power planes below the node to reduce power noise coupling into the node.
