When I connect the circuit, output of 5.020V drops to 4.8V
The lowest value reached might be even lower than 4.8V. If that value was measured with a multimeter (DVM) then it's unlikely the 4.8V reading was the lowest voltage reached, due to the low sample rate of DVM. [See later update below - an even lower voltage of 4V was subsequently reported, and the measurements were done using a UNI-T UT61E multimeter.]
You would need to use an oscilloscope (or similar high sample-rate device) to record the voltage at the RPi itself, looking for short dips in voltage, to understand more about how the power output from that "battery shield" behaves during the RPi boot.
The problem might be mitigated by adding additional bulk capacitance at the output of the "battery shield" (or on the power pins of the RPi). Also minimising the length of any wiring (inductance) between the "battery shield" and RPi can only help.
Update:
As mentioned in comments below, the "battery shield" output voltage was seen to "even drop to 4V once". So, as I said in a reply to that, there is a fundamental incompatibility between the power output from that "battery shield" and the power requirements of the connected devices (RPi Zero and display).
In comments, Bruce Abbott has kindly found a review of that "battery shield" on YouTube by user "Great Scott!" here:
https://www.youtube.com/watch?v=aND0j2Y2IkM
Thanks to Bruce for finding that. The video is worth watching and it shows the output voltage collapses at an output current of only around 300mA (which is the "guess-timated" current drain of the attached devices in this case). Also there is no over-discharge protection and no output short-circuit protection.
Summary:
Therefore the actual output current capability of 5V at 300mA from these "battery shield" boards as shown in that video, does not match their claimed specification of providing 5V at 1A. If the current required for the display and RPi during boot was enough (even momentarily) to cause the "battery shield" output voltage to drop significantly, as shown in that video, then that would explain why the RPi is unable to boot successfully.
If I simply plug USB port, the circuit works well.
A typical PC USB 2.x host port will supply at least 500mA, even without enumeration of the connected device (although host ports are not required to do that). The video gives evidence that the "battery shield" supplies an even lower current of only 300mA, before the voltage drops below that usable for the RPi. This would explain why connected devices which require between 300mA to 500mA could behave correctly when powered from a PC USB port, but not when powered from that "battery shield".
