what is the most difficult part of creating a DDR2 or DDR3 memory controller inside an FPGA?

Question

In this day and age we can just use the memory IP provided to us by the FPGA vendor be it a soft IP or hard IP. This makes it almost trivial to communicate with high speed memory devices like DDR2 and DDR3 RAMs from the FPGA side. The PCB layout is still a massive challenge though and there is no doubt about this at all.

Many years ago I read that once upon a time, the engineers had to design a memory controller for high speed RAMs themself for an FPGA being used in their design. I am not sure how often this happened. I remember reading that at that time, the most difficult part of the memory controller design was the PHY.

From memory I can remember that it gave a reason as well. The reason it gave is that the PHY required very precise control of propagation delays and this required some sort of novel idea to be achieved in FPGA that has this issue with a nondeterministic propagation delay between different cells every time we compile the design.

Is this true? If this is true, how was this problem solved? I have never found details of this anywhere but I am curious to know what special steps engineers took to solve such a problem. It seems that they had to specify very tight timing constraints for specific paths and let the fitter do the rest, but if all it took was some SDC constraints, that does not appear too difficult to me in theory.

Answer 1

The problem was solved by the FPGA vendors realizing that this was an important thing to be able to do, and they began incorporating the necessary hardware into every I/O pad's logic. The key element is a programmable delay line, and the vendor's memory controller IP includes the logic to calibrate the delays automatically.

For example, look at Xilinx UG388, "Spartan-6 FPGA Memory Controller User Guide", Chapter 4, "MCB Operation", where it talks about the startup sequence and self-calibration. The key element is called IDELAY. You can learn more about this primitive in UG381, "Spartan-6 FPGA SelectIO Resources User Guide", p. 70, "I/O Delay Overview".

Answer 2

I had the privilege of being a software & systems guy working on a project back when DDR in FPGA was new -- this would have been late 1990's, early 2000's. The way this was accomplished then was by locking down the position of the gates in the FPGA. Basically, this is a level of hand-holding that isn't usually necessary, but which the tools allowed (and may, still, I don't know). By locking the relationship of the gates to the I/O pins, the various delays were guaranteed to be preserved.

I'm not sure if this still needs to be done or not.

Answer 3

I had the pleasure of making my own DDR controller on a Cyclone IV a few years ago. The main motivation was that the tools no longer contained DDR[1] controller IP at the time, and the application was a little bit unique in that the memory was being used as an enormous lookup table with truly random addressing, and needed consistent timing for every access. This actually simplified the state machine logic quite a bit since I didn't have to worry about keeping the rows precharged or anything like that: Just precharge the row, read the data, end the sequence. Rinse and repeat.

From a digital state machine standpoint, it really wasn't that challenging. The memory datasheets have pretty good state machine flow diagrams, and just generally understanding that you need to schedule in your refreshes to meet the specs. You can either have an evenly distributed refreshes or one big burst of refreshes.

You have to understand the low level interface hardware available at the FPGA IO blocks. You'll need to find a bi-directional DDR block, where the output write data is clocked out by your FPGA clock and the input data is clocked in from the DDR strobes. Every FPGA vendor and family does this a little bit differently. I highly recommend simulating to make sure you're doing it right.

The next hardest part is the timing constraints. The bidirectional data pins from can be tricky when it comes to writing IO timing constraints, especially when the outputs and inputs are clocked from different sources, and again, each vendor has slight nuances on how to write them properly.

Finally it's just a matter of meeting timing. You can use the IODELAYs if available, use PLLs with adjustable phases, just whatever you need to do to meet timing.