What is the range of currents that I can safely draw out of a 9V battery without damaging it?
I am planning to design an experiment and I don't want to draw too much of current out of a battery and damage it.
This depends widely on the battery chemistry, size and manufacturer.
It is directly dependent to the serial resistance of the battery and the heat generated during discharge.
You should find the specific battery datasheet and look for the discharge current.
Some datasheet will give you a value in Amp, other will give you a value in C, like "2.5C" which refers to the battery Amp Capacity in A or mA. If the battery is 2500mA, 2C would mean 5Amp, 0.5C would mean 1.25Amp.
For standard 9V batteries, this information is not always available, but these batteries are not designed to deliver high load.
This datasheet shows a discharge rate of 150mA continuous, at this rate the battery would be done in about 8 hours.
Given this is a high end battery, I would not get over 100mA discharge rate for a standard 9V battery.