At the end of the day, the board house will determine what tolerances. This information is usually posted on a manufactures website, for example:
Nominal Hole Size vs available Drill Bit Sizes
The defined tolerances start with the drill bits, these are available
in standard size steps of 0.05mm and any hole defined in a CAD system
that does not meet this 0.05mm step will be rounded to the nearest
step.
It is possible to have smaller drill bit size steps however, their
high cost these usually outweighs their advantage.
Looking at it from another point of view, we are talking about a
rounding with a maximum value of 0.025mm or 25µm.
This should not cause any downstream process issues.
https://www.eurocircuits.com/finished-hole-size-tolerances/
Another manufacturer specifies different IPC tolerances.
https://www.protoexpress.com/kb/drills-and-through-hole-plating/
Another thing to consider is what you need in your design, a low speed pcb with a microcontroller and large smt pads probably doesn't need to have tight tolerances. A design with a BGA with vias between pads and impedance control will, the tighter tolerance will add expense because setup times and material costs will increase. This is especially true in high volume runs.
I would aslo keep the tolerances a bit loose if you don't need it.