Exact recreation of Xilinx FPGA binaries from source control

Question

I'm software developer in a small shop where there's only been one EE guy responsible for a series of FPGA designs spanning a decade, almost all of which target the Spartan line, specifically the XC3S5000.

I'm looking for the community's opinion on some of the EE guy's assertions which I find hard to believe:

1.) Many builds must be created and tested (without changing the input files) because a particular binary often fails testing due to "timing issues." (verbatim)

Me: Is frequent failure of an FPGA design expected and this incredibly slow process considered normal? I can't imagine repeatedly compiling the same traditional software development language and hoping the output will be right *this time*. The HDL seems underspecified if this is the case.

2.) Because the creation process is subject to "random" algorithms (I assume this is placement and routing) it is "very hard" to recreate the binary from just the input HDL.

Me: Surely there is an initial seed provided to this pseudo-random algorithm which can be queried after a successful build? The exact good binary could then be recreated using the seed later. This seed could optionally be checked in to source control.

3.) Because the Xilinx-provided tools may change (be upgraded, etc.) even if an exact recreation is possible from a known good seed and the old tools, a new set of tools may create a binary which fails testing. EE guy doesn't see any use in investigating how to find the random seed used for a build, and how to provide it to a rebuild.

Me: How likely is a build to break upon introduction of new tools?

4.) The binary build process disallows any post-build manipulation of the bytes of the binary due to checksumming and the inability to write to a known offset into this binary.

Me: You can't embed information into an existing binary?

5.) He insists on checking in both the final product and NGC files.

Me: Is there ever a reason to store NGC files?

Notes:

*FPGA binaries will eventually be stored in a different place, but definitely not in a text-centric version control system.

*We use a fragile version scheme to query what FPGA is currently running in our system. It is useful for software to work around hardware bugs by checking if the FPGA is of a certain version. We can read both a monotonically increasing 16-bit number and a build date from the FPGA, but these numbers don't sync directly with the version control system in which the binary is checked in.

*I've looked over the following questions already: What files/directories are needed to recreate a Xilinx PlanAhead project?

List of Xilinx file suffixes (for ISE)

Answer 1

I used to work in a job that had a contract requirement that we could reconstruct our integrated circuit designs and simulations for twenty years. That was a battle we fought over and over again, so I have had some experience with the painful process of "archiving" a complex design.

1) If the design has tight timing constraints then it may be necessary to synthesize/optimize a number of times before all of the timing issues are uncovered and the timing constraints are "fully and correctly specified" as Dave Tweed puts it. This is particularly true if you incorporate any third-party IP, or if the person doing the synthesis is not the author of the HDL.

2) While the synthesis process is deterministic it is also subject to a vast number of variables. Some of these are hidden from the user. While I would say you should be able to get the same binary if you run exactly the same tools on the same computer on the same HDL with the same constraints on the same day, I would be hesitant to guarantee that any given binary could be reproduced exactly next week.

3) This is most certainly true. New versions of the design tools routinely break existing designs. It's not about some seed for a random number generator, it's about changes in the optimization algorithms and changes in the characterization data of the physical part. Because these changes are proprietary information they are largely hidden from the user and can appear to have a random nature.

4) This is typically true. The binary bitstream is the end result of the entire synthesis and optimization process. Most vendors hide the format of this data and may even encrypt it, which is why you can't find open-source FPGA design tools that go all of the way to the chip.

5) Is there ever any reason to store the NGC? I'll turn that around: can you justify that there is no information whatsoever in the NGC that cannot be obtained from whatever you are calling the "final design"?

Answer 2

It sounds like your "EE guy" has inadequately specified the timing constraints for the design tools (in the .UCF file), if some runs meet timing and some do not. All of his assertions that you cite appear to be workarounds for this situation.

When the timing constraints are fully and correctly specified, the Xilinx tools will work as hard as necessary to make sure they're met (including margins to account for variations due to process, voltage and temperature). Yes, successive runs will be different in minor details because of the randomness used in the optimization process, but every run that completes will either meet timing, or the tools will explicitly tell you where there was a problem.

Answer 3

I would agree issue #1 could be incomplete usage of timing constraints, though perhaps only if your requirements are reasonable for the hardware, vs only met in lucky cases.

Issue #4 may or may not be true, depending on the unspecified precise needs and circumstance. Xilinx provides the Data2MEM tool which can change block ROM contents of a compiled bitstream - something often useful for updating software for a built in processor, or tables/keys. However, it is not compatible with the encryption and compression options. And it isn't of use for anything that is not a block memory. (Last I checked - which was a few years ago - Altera did not have an equivalent, meaning you really did have to rebuild those designs to change block memory contents)

For most of the other issues (irreproducibility of precise binary at least across build environments and toolchain version changes, toolchain version compatibility issues), your engineer likely has a valid point. You might find it very informative to try some simple FPGA projects to get an idea of how different this is from developing for a stored-program computer, especially if you are used to open source toolchains.