As an example, the OPA227 has 3.5nV/sqrt(Hz) input voltage noise density (at 10Hz). Its input impedance is 10MΩ, which has an equivalent Johnson noise of 1μV/sqrt(Hz) at room temperature - much larger than what the datasheet quotes.
An op-amp input might be connected to a voltage source that has some output impedance and that impedance is usually minutely small compared to the input impedance of the op-amp. This means that the net impedance is pretty close to the source impedance.
Hence the thermal noise at that input is pretty much determined by the source's output impedance.
For the OP227 it is usually the common-mode input impedance that is effectively loading the source and that is 1 Gohm. That puts it into even more perspective but, how high could the source impedance be you might ask - given that the offset current could be up to 10 nA, a source impedance of 1 Mohm would produce a DC error at the input of 10 mV and that is about a thousand times more than the inherent input offset voltage for the device.
So, nobody would really want to use this device with a source impedance of more than about 10 kohm else why pick an op-amp that has such a good offset voltage spec and then wreck the application performance with such a high source impedance.
So, with a source impedance of about 10 kohm you can see that this dominates over the internal impedance of the input (1 Gohm) by a mile.
Finally, the input voltage noise density has nothing to do with the input impedance and its potential thermal noise. If you are trying to make sense of one by looking at the other then please don't because they are unrelated.