The goal of the erase routine is to have the charge state of each bit be independent of the previously written data. While a chip's own circuitry may be unable to distinguish a bit which is 70% charged from one which is 98% charged, or a bit which is 30% charged from one which is 2% charged, someone with sufficiently sophisticated probing equipment may be able to make such distinctions.
If a memory chip is designed in such a fashion that attempting to erase some area of memory will activate a "discharging" circuit for a fixed amount of time, then bits which used to be "charged" before the erase operation will end up with more charge afterward than bits which were not (the discharging circuitry can't drain all the charge; rather, turning it on for some number of microseconds will drain off about half the charge). Since the erase circuitry for memory chips cannot generally be controlled on a per-bit basis, and since leaving it on long enough to eliminate all detectable charge residue would stress the device, the erase circuitry is not in and of itself generally sufficient to reliably destroy information in such a way that sufficiently-sophisticated equipment could not recover it.
The ideal way to erase information on a chip would be to repeatedly write random data patterns to it and then erase them. Unfortunately, that is not always practical. For a device which would allows the area holding a key to be erased without disturbing anything else, that would be the desirable. Performing an erase and then overwriting the data is also pretty good, however. Performing an erase followed and then writing all zeroes would probably be about as good as writing random data, but simply zeroing out those bits that aren't yet written (in a single operation, and without doing the erase first) would not. Actually, the ideal approach for a single write/erase cycle might be to zero out bits in random groups, then perform an erase, and then zero out bits in different random groups, but that approach would only be work on devices which allow bytes to be programmed multiple times between erase cycles. I suspect that the use of random data rather than all zeroes is intended to force the performance of an erase cycle.
The way bits are physically written to flash devices is such that there will often be some unpredictable variations in the charge state of "zero" bits. Thus, writing zeroes to a flash device without erasing it first will probably do a better job of making the information unrecoverable than would writing zeroes to an EEPROM without erasing it first. Further, erasing a flash chip generally requires that it be programmed with all zeroes first (chips handle that operation internally). That, combined with the fact that it may not be possible to erase just the part of the chip containing a key, is probably what drives the decision to use "fill with zeroes" as the proper approach for erasing keys from flash.
