NimH charging is damaging the batteries
STOP! Really stop flying you drone NOW until you get a proper lipo charger.
A lipo charger is not expensive. An cheap Chinese charger is anywhere from $5 to $30 (depending on how many cells you need to charge). Even the cheapest lipo charger would be significantly safer than charging with a NimH charger no matter how expensive that NimH charger is.
With a lipo charger you should be able to simply charge and leave it. Literally millions of people charge their lipo powered iPhones and Samsungs without waiting.
A lipo battery that's being used a lot should last two years. Unused lipos should last ten years or so.
Also, buy a low voltage alarm or learn to time your flights. DO NOT drain the batteries to lower than 3.2V if you want them to last two years. If you do their lifespan will be reduced significantly. My personal limit is 3.4V and I usually fly until 3.6V (which may be 3.2V or less when in flight).
Sometimes you will get a bad lipo. It could be manufacturing defects or badly stored or it could just be old stock. But it should not be more than 1% of the batteries you buy. I've personally have only ever gotten one bad battery.
Finally, if you're worried about fire hazard make sure you charge away from anything combustible. You can get fireproof bags at hobby stores. Or you can just get a metal toolbox to charge your battery in. Lipo fires burn hot and spark a lot but they don't last long. So your main concern should be to limit the spread of sparks if the battery combusts. I personally charge outdoors or in a corner of the room away from all furniture.