How can z80 uses a 4-bit ALU and return results in a single clock cycle?

Question

According to Ken Sheriff's blog post Z80 ALU is 4-bits wide. If we take a look at page 8 and 9 of Z80 User Guide we will find the following information:

Clock states T3 and T4 of a fetch cycle are used to refresh dynamic memories. The CPU uses this time to decode and execute the fetched instruction so that no other concurrent operation can be performed.

We also know that many instructions are performed in 1 machine cycle with 4 T-states, such as ADD r, SUB r, etc.

Considering the following facts:

• Z80 data bus is 8-bits wide while its ALU is 4-bits wide
• Some instructions that use the ALU are executed in 4 clock cycles (T-states)
• We know that T1 and T2 are used for fetching opcode from memory and we need T3 to decode the opcode

How is it possible that a 4-bit ALU produces a 8-bit result in just a single clock cycle?

Answer 1

The instruction actually ends at the end of T2 of the M1 cycle that reads the next instruction, as explained at z80.info

So, an instruction such as: SUB r can be broken down like this:

M1 / T1
M1 / T2 : read opcode
M1 / T3 : opcode is decoded as SUB r. A is loaded into op1 latch
M1 / T4 : r is loaded into op2 latch
------------
M1 / T1 : low nibble of substraction is calculated and stored in result low latch
M1 / T2 : high nibble of substraction is calculated and written to register file,
          result low latch is written to register file, 
          next opcode is read from memory

This overlapping scheme is only used if the operation writes to a register. If it writes to memory, no overlapping can be happened because that would collide with the M1 cycle trying to read an opcode.

Another example: INC HL. This increments a 16 bit register, so the 4-bit ALU must be used four times. INC HL has one M1 cycle that lasts 6 clock cycles. Its breakdown is this (the number of clock cycles for every machine cycle is taken from the Mostek MK3880 Z80 Technical Manual):

M1 / T1 :
M1 / T2 : read opcode
M1 / T3 : opcode is decoded as INC HL
M1 / T4 : L is loaded into op1 latch
M1 / T5 : low nibble of L is incremented and stored into result low latch
M1 / T6 : low nibble of H is loaded into op1 low latch,
          high nibble of L is incremented and written back to register file
------------
M1 / T1 : high nibble of H is loaded into op1 high latch,
          result low latch (new value of low nibble of L) is 
                               written back to register file
M1 / T2 : high nibble of H is incremented and written back to register file,
          read next opcode, etc...

ADD HL,DE : it lasts 11 clock cycles with the following breakdown: 4+4+3

M1 / T1 :
M1 / T2 : read opcode
M1 / T3 : opcode is decoded as ADD HL,DE
M1 / T4 : L is loaded into op1 latch
------------
M2 / T1 : E is loaded into op2 latch
M2 / T2 : low nibble of L+E is calculated and stored into result low latch
M2 / T3 : high nibble of L+E is calculated and written back to register file,
          low nibble is also written to register file
M2 / T4 : H is loaded into op1 latch
------------
M3 / T1 : D is loaded into op2 latch
M3 / T2 : low nibble of H+D is calculated and stored into result low latch
M3 / T3 : high nibble of H+D is calculated and written back to register file,
          low nibble is also written to register file

Answer 2

I've skimmed the document Ken Sheriff's Blog references, "Zilog Oral History Panel on the Founding of the Company and the Development of the Z80 Microprocessor".

On page 10, it mentions small amounts of pipelining; for example "At first I introduced the pipeline 4-bit ALU"

It may only need one extra T state, where ALU execution overlaps some other operation for everything to fit.

So, in the "Zilog Oral History Panel on the Founding of the Company and the Development of the Z80 Microprocessor" document, "Masatoshi Shima" seems to describe enough about the relationship between processor functions that it may explain how it worked. It appears to explain things in some detail, and my reading is there is a small amount of pipelining, sufficient to explain how they squeezed everything into 4 T states.

EDIT:
I would encourage anyone interested in the history of Microprocessor development to read that document. It is fascinating. My thanks to @GabrielOshiro for highlighting it.