In the relevant part of the infrared spectrum for this application\$^1\$, air has a high transmittance:

[![absorption spectrum for air][1]][1]

(taken from [here][2] licensed under  [Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License][3])

So wherever you point an IR temperature sensor, the output will be dominated by the surface you are pointing at and not the air in between.

If you try and point an IR sensor in the sky (no clouds) you will get a a very low reading (mine showed -40°C but that is the lower range end), and not the air temperature.

So you need some surface at room temperature to measure the room temperature. Do not use shiny metals, as those reflect IR radiation. Normal glass can be used, so you could hang a decorative looking thing in the room and measure the surface of that - provided you have a small enough measuring cone (so it only hits your wanted surface and not something behind it).


----------
\$^1\$: For -40°C the peak in IR is around 15µm, for 50°C the peak is around 8µm.

  [1]: https://i.sstatic.net/PAOI6.gif
  [2]: http://chem.libretexts.org/LibreTexts/University_of_California_Davis/UCD_Chem_115_Lab_Manual/Lab_3%3A_Fourier_Transform_Infrared
  [3]: https://creativecommons.org/licenses/by-nc-sa/3.0/us/