This is not an easy thing to do. You'd have to prepare an extra on-board programmer as well as code to communicate with it; it's possible this has been done before, but I haven't seen it.
Nick Alexeev's solution would be correct if the select level on the reset pin differed from the programming level, but they're identical in low voltage serial programming, which is the default method. This means the AVRs would end up responding in parallel, possibly damaging their MISO output buffers, and making it impossible to address them individually. The very first time, while all the AVRs are erased, you may get away with leaving the reset pins high instead of low (as the default state of the I/Os is floating). But that basically breaks down on the programming further chips, unless the early ones know to avoid the SPI pins.
pault's solution is closer, but it's not actually enough to manipulate the reset lines individually (although obviously the master AVR needs its own); it's the MISO and SCK lines that need to be separated. The first so the slave AVRs cannot damage each other, the second so they can be individually addressed.
Since you need many SCK lines, this means you probably can't use the SPI port for talking to the slave devices; besides, you need the first SPI to even program the master AVR. That in turn means bitbanging (controlling by software) the programming pins, in addition to reprogramming the serial interface, so the programming firmware modifications are notable.
An option is to use some form of I/O extender chip instead of the master AVR, and let the host do the bitbanging through it. This is likely to be slower, but speed is rarely a problem on chips this size.
In short, it's usually easier just to put an ICSP connector per AVR instead. Atmel simply didn't design the ICSP protocol for multiple chips, although many other SPI chips do support it (via chip selects or daisy chaining). JTAG TAP is similarly desined for daisy chaining.