With an unregulated power supply, the voltage will vary with the current draw. Motor speed is proportional to voltage, so any given motor will experience speed variation according to the current draw of the other motors. When each motor is switched on, it will draw a higher current than the normal running current. Depending on the motor design, that current could be much higher than the normal running current. With the very small motors described, the high current will likely be very brief and not extremely high. Even with brief surges of current, the speed variation of the other motors may be noticeable.
If the power supply has electronic over-current protection, switching-on motors may cause the power supply to shut off or cut the voltage to protect itself.
If the power supply has enough capacity and speed variations are not a problem, switching several motors on and off should not be a problem. The voltage in a car is not particularly well regulated, but the supply has a large capacity. The many motors for fans, windows, seat adjusters etc. turn on and off with no difficulty. However the voltage drops considerably and the lights dim if the starter motor is operated with the lights on.
I missed the mention of motor driver, one for each motor I assume. With proper use of the motor drivers, any change in the supply voltage due to varying load or raw input supply voltage can be compensated for. Attempting to accelerate a motor too quickly or overloading a motor could still shut the supply down.