I am wondering if an 18.5V, 3.5A charger be able to charger a 18V battery without any bad side effects? The old charger (not functioning anymore) was 24V 0.2A. I know the charger must have a higher voltage rating than the battery to be able to completely charge it.
Probably not. The voltage spec written on the side of a battery pack is usually nominal voltage during discharging, not maximum voltage during charging. The wall adapter input to the battery charger needs to be the maximum battery voltage during charging, plus however much drops across the battery charger voltage regulator, plus the drop in protection circuitry & wiring resistance etc., plus enough that if the regulator is operating at the low end of the spec variation range (e.g. +/- 5%) it's still high enough.
If the original wall adapter was 24V, you probably need to find another 24V adapter. It's possible the charger will work with a somewhat lower (or higher) voltage, but as the only information we have is what was specified by the manufacturer, we have to take their word for it.
There are many dangers in haphazardly replacing one adapter by another when it comes to batteries.
Not only is enough voltage overhead important, but the charging current is important.
If the battery pack has electronics inside to regulate the charge current and voltage, you can probably replace the old 24V by a new 24V of 0.2A or more. But often in simple battery-adapter systems the adapter is all the advanced electronics in existence, even with Lithium Ion/Polymer, where there are many other reasons to make the battery itself intelligent.
In that case you are going to need something that delivers the same output curve as the original. Same peak current, same voltage/current drop-off curves. For example if you'd just connect a 24V 4A supply and the battery is a 200mAh 18V Li-Ion pack without current-limiting electronics you stand to create ionized gas or even a (violent) explosion, unless the cells are very, very high grade.
Look, you have to be really careful when charging batteries. Looks like you are not even aware of what the chemistry type of chemistry the battery is. Stop for a while and try to get the grasp of things I'm going to say below.
You have to know the battery chemistry. Charging depends upon that. All types of batteries are changed in completely different ways. They have different current limits and voltage limits upon exceeded, the battery will explode. I am not asking you not to charge. I am asking you to first learn about your battery. Its charging methods and then try to use proper methods to charge it. BatteryUniversity.com is a good website that teachers about main battery chemistries.
The other thing is, there are certain phases of battery charging such as Constant Current CC, Constant Voltage CV etc. So your charger needs to have both current and voltage control capability. If you do not use them properly, you might end up damaging or critically decreasing the battery life. So get to know your battery first. Also go to ti.com. They have charging ICs with reference designs you can easily get started with. Good luck.