I have zero experience so if this is a stupid question please at least be kind. It seems obvious that I couldn't run the fan on one 3.7v battery but then I'm not an electronics guy. So I was thinking I'd need to step down the voltage from 2 batteries? Would the fan run in 6v without burning it up? Is there such a thing as a 5v battery? Is need as long a run time as possible. I'd also need a switch for it but I think I can figure that out.
It seems obvious that I couldn't run the fan on one 3.7v battery
No it does not, many motors which are designed for 5 V will also run on 3.7 V. The same is true for fans. So I would just try and see if it works.
Also, a 3.7 V Li-Ion cell will be have a voltage around 4.2 V when fully charged. That 3.7 V is the voltage when the battery is almost empty.
If one cell doesn't work then I would use a simple switched mode converter module which are cheap, efficient and should "just work" even for people with little experience with electronics.
From my experience, you can use a voltage regulator such as the common LM7805 or even a step down switching converter. But the simpler way would be to add a resistor in series with the fan. To calculate resistance, you need the current the fan draws and the voltage drop of the load. So if the fan runs on 5v, that is your voltage drop. So 3.7 x 2 would be 7.4v. So in order to drop the voltage to 5v, take 7.4 - 5, which is 2.4, and divide it by how much current the fan draws. So it should be 2.4 divided by the current draw. The current draw is most of the time written on the top of the fan itself, so do 2.4 divided by that number. Find a resistor that value or close to it and put it in series with the fan. Just to be safe, I would use a 3 watt resistor, but 1 watt should work fine.