I have an 8 cell 9.6v 600mah battery. If I take out 2 cell, making it a 7.2v battery 6 cell battery will the mah change? Also can I still use the same charger that came with the battery if I change the battery?
The mAh will remain the same (the Wh will obviously be lower though), since it will be made from 8 * 1.2V 600mAh cells initially. You will have a 7.2V 600mAh battery. You will probably need to change the charger though (depending on the charger and battery type which you have not specified - some may work okay)
The same charger will indeed work for NiCd and NiMH batteries as they are charged via current; it not work with LiPo and Lion as they are charged to a specific cell voltage.
Yes, your milliampere-hour (mAh) will remain 600.... however your total energy capacity aka watt-hours (Wh) will be reduced.