Does PCIe 100ms boot-up time requirement including the entire FPGA device?

Question

we know most of the FPGA vendor have way to ensure the entire FPGA device can be configured within the 100ms time. However, since most of the PCIe solution is a harden block (dedicated circuitry) sitting inside the same FPGA device. I supposed the entire link training should be totally independent from the FPGA core.

In this case, I assume partial configuration just need to be done earlier to wake up the PCIe circuitry and ensure the LTSSM operation happen to fulfill the 100ms boot-up time requirement.

However, why it looks like the 100ms boot time requirement also apply to the entire FPGA device especially on the core fabric (This means the FPGA core also has to be ready within 100ms)? Is this really necessary? If there a hidden reason in behind?

Answer 1

I suggest you to look into Intel Stratix 10 FPGA datasheet to estimate FPGA configuration times with specific configuration schemes (ASx4 or AVSTx8 x16 x32) for ~50% or 100 % logic utilization, you will realize that large FPGAs now take way more than 100 mS to configure. So the 100 mS requirement is not enough to finish configuration. In fact the configuration bitstream in Stratix10 is compressed and is decompressed inside the FPGA chip using a SDM block - Secure Device Manager. The bitstream size also depends upon logic utilization. The more the logic utilization, the larger the bitstream, the longer it takes to configure. This compression is done to reduce configuration times. From Stratix10 onwards, Intel uses a processor to manage configuration and ton of other features it is called an SDM.

Regarding using PCIe in FPGA - I think you are referring to the capability of configuring FPGA via PCIe. If that is not the case, you are simply referring to PCIe spec when used in conjunction with an FPGA.

If you are referring to configuring FPGA via PCIe, then check below extract from CvP User Guide for more details about the topic.

Stratix 10 configuration via Protocol (CvP) User Guide https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug-s10-cvp.pdf

Stratix 10 configuration User Guide https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/hb/stratix-10/ug-s10-config.pdf

CvP uses an external PCIe* host device as a Root Port to configure the Intel Stratix 10 device over the PCIe link. You can specify up to a x16 PCIe link. Typically, the bitstream compression ratio and the SDM input buffer data rate, not the PCIe link width, limit the configuration data rate. Intel Stratix 10 devices support two CvP modes, CvP initialization and CvP update. CvP initialization process includes the following two steps:

1. CvP configures the FPGA periphery image which includes I/O and hard IP blocks, including the PCIe IP. CvP uses quad SPI memory in AS x4 mode to configure the FPGA fabric. Because the PCIe IP is in the periphery image, PCIe link training establishes the PCIe link of the CvP PCIe IP before the core fabric configures.
2. The host device uses the CvP PCIe link to configure your design in the core fabric. CvP update mode updates the FPGA core image using the PCIe link already established from a previous full chip configuration or CvP initialization configuration. After the Intel Stratix 10 enters user mode, you can use the CvP update mode to reconfigure the FPGA fabric. This mode has the following advantages: • Allows to change core algorithms logic blocks. • Provides a mechanism for standard updates as a part of a release process. • Customizes core processing for different components that are part of a complex system. For both CvP initialization and CvP update modes, the maximum data rate depends on the PCIe generation and number of lanes. For Intel Stratix 10 SoC devices, CvP is only supported in FPGA configuration first mode. For more information refer to the Intel Stratix 10 Configuration via Protocol (CvP) Implementation User Guide.

Answer 2

The hard PCIe endpoint configuration space is part of the FPGA configuration process; items such as capabilities, memory size required (which requires a write and read of the BARs) and class code (to name but 3 items) all need to be configured prior to being accessed.

Prior to configuration, the configuration space does not really contain valid values so yes, FPGA configuration needs to be completed prior to the PCIe boot time requirement.

In some complex designs I have deliberately held the system processor in reset until FPGA configuration has completed; all these devices (well, all the ones I have used with a PCIe endpoint) have a signaling protocol to identify that configuration has successfully completed.