How does PCIe Endpoint device advertise the BAR memory space requirement to the host?

Question

In many PCIe training material, there is details walkthrough on how the BAR is being programmed. I think I probably can understand some portion of that, but somehow there is a missing pieces that I couldn't digest well.

System software must first determine the size and type of the address space requested by a device

Above is true as the host has no idea what is the required address space from the device initially. So, since the size of the address space need to be determined by the endpoint and supposed this piece of info will be passed over to the host through the first time configuration space read?

If above is correct, how is this information is passed over to the host initially? The explanation stated about uninitialized state of the BAR which mean initially the top 20 bits of the BAR is X. So I couldn't think of through what operation the required address space information is being passed over to the host?

Next, why the host still need to write 1 in the upper 20 bits during configuration space based on earlier given information from the device and later read it back to determine the targeted range of the memory space?

I'm not sure where do I start to get confuse, but the PCIe book I read does not explain how the endpoint device pass the targeted address space information to the host and I assume this info is residing in the same 4 byte BAR field. Although I'm very sure I mis-interpret something wrongly, so I hope at least someone can enlighten me the specific process how the endpoint device to pass over / advertise the targeted address space to host so that I can appreciate why there is a need of step to writing 1'b1 in the top 20 bits of the BAR field.

Still my question is if in the first place there is already have way for endpoint device to pass over the targeted address space info to the host, why there is still need a step to write number of 1 to the top 20 bits and later readback. Why can't directly based on the first configuration read info to the host and the host simply append the address offset to it which is the last step of the BAR programming?

Answer 1

The Configuration Address Space of each PCIe target is always accessible to the PCIe Host. The register addresses within the CAS are fixed to the offsets defined in the PCIe spec'. The Host always knows their addresses.

The Base Address Registers (BARs) are within the CAS at known addresses. Each BAR contains sufficient Base Address bits from the MSB downwards to define the start address of the block when aligned to a boundary the size of the block.

The PCIe bus transfer data in units of 32-bit dwords, so PCIe addresses always have bits 1:0 as 00. BAR bits 1:0 are therefore free for other functions. The smallest address range that can be allocated is 4 KB, so BARs do not contain bits 11:2, only MSB down to 12.

So for example, a card needing 256 KB of memory space would provide a BAR with:

• bits 31:18 as RW, to hold the base address
• bits 17:12 as RO, always reading zeroes

During configuration, the Host determines the size of the required address range by:

• writing all 1's to BAR bits 31:12
• reading back the BAR and checking which bits could be written to 1

In the 256 KB example, it would write 0xFFFF_F000 and read back 0xFFFE_0000. Bits 17:12 stayed at 0's when written with 1's, which corresponds to 256 KB (2^18 bits = 256 KB).

This gives the Host the address range size. Once it has this for all PCIe target devices, it can then allocate the actual base addresses and write them to the BAR RW bits. The BAR enable bits are also written to 1 and the system can start.

The BAR is a compact way for a Host to specify a base address while discovering a size. This is no different to what a person does when looking at the schematic for a microprocessor with memories and an address decoder.

For example, the below circuit is a simple Z80 system with 32 KB ROM and 32 KB RAM. The Z80 has a 16-bit address space, with 16 address pins for accessing 64 KB of 8-bit memory.

We can see that the address decoder uses only address bit 15 to determine which chip to access. We can see the address bits 14:0 connect directly to the chips. So we can deduce that each chip occupies 32 KB of address space as it uses 15 address lines (2^15 = 32,768 = 32 KB).

We can deduce that the chips are arranged on 32 KB address boundaries because the lowest address bit used in address decoding is bit 15. So bit 15 is, like our programmable BAR bits, used to determine where in the address map the chip appears.