0
\$\begingroup\$

I've decided to start a small project over this break for creating a very simple GPU. As far as the GPU will go for now is simply receiving pixels and outputting pixels to SDRAM. In addition, there is another part of the circuit that will get pixel data from the SDRAM and output it to an LCD (I've already made the timing controller for the LCD, just need an interface to the SDRAM which makes sense). The last part somehow does a video overlay with a stream from some sort of CCD/CMOS sensor. As for the SDRAM, I've made controllers before and have enough of an understanding of VHDL to do this, so that's a non-issue. My main problem comes in with the timing.

First off, the LCD runs at 9MHz so that's fine so it takes ~111ns per pixel with a 25ns setup time before the falling edge. Suppose I'm running the SDRAM at 100MHz. After a burst setup, I can get 10ns per pixel, so as long as we're in a burst, there's more than enough time to get the data. But this speed is only in a burst. If I were to load the row every single time, I would be cutting it close. This means that during the time where I actively need to read data for the LCD, I am completely held by it. No writes.

The only time I can do the writes are during the columns (~480 col) ~4100ns of Hsync in each line (~280 lines) and the ~525000ns during Vsync. Suppose that worst case, my GPU needs to write to every pixel of the screen. This means that I can do ~410 pixels per Hsync, which leaves maybe a hundred or so pixels left that I need to write during the Vsync. This write data will be writing to a 2nd frame area in the memory (hence, double buffering).

That timing works out, ... assuming my GPU has its own (large) pixel cache during it's waiting times and so on so forth. That's an issue I'm going to deal with later. My main issue right now is the video overlay. In addition to what's currently happening, I have my pixel data from the CCD/CMOS. After my worst case GPU usage, I have less than ~525000ns to write 480x280 (~1344400ns) which is definitely impossible by an order of a bit less than 3.

Am I doing something terribly wrong? How does the computer I have in front of me right now do this? The display is SO much larger and I doubt that SDRAMS with a MUCH shorter latency exist (maybe DDR). :(

\$\endgroup\$
  • \$\begingroup\$ Have you considered dual-port RAM? This is basically RAM with two interfaces to the same memory. Your CPU can load and write to the memory at whatever speed it wants, and the LCD reads it's contents through the other interface at it's native speed. \$\endgroup\$ – Connor Wolf Dec 15 '11 at 11:27
  • \$\begingroup\$ True, something VRAM like is also an option. Though, is that was they use in standard graphics cards these days? I thought they used a faster form of DDR SDRAM, but not necessarily dual-ported. My main problem is that if I can't do what they're doing, but with a GPU that doesn't do much and a display that is low resolution, then I'm definitely not designing correctly. \$\endgroup\$ – Koma Dec 15 '11 at 14:58
0
\$\begingroup\$

I see two questions here, each with it's own answer.

First how to multiplex the SDRAM. My suggestion would be to grab a bunch of output pixels from the SDRAM in a burst, stuff them in a smaller external memory or FIFO (easy if your logic implementation platform is an FPGA) and then feed them to the LCD at its slower rate, during which time the GPU can use the SDRAM. When you run out of queued pixels, claim the SDRAM back from the GPU and get another bunch.

Second, how to do the overlay. Quite simply, for best performance don't commit the overlay data to the SDRAM, instead, put it in it's own memory and multiplex it in front of (or transparently mixed with) the output of the SDRAM under the control of a state machine with programmable size parameters.

|improve this answer|||||
\$\endgroup\$
  • \$\begingroup\$ Using a FIFO or similar method seems to be the way to go. I'm even thinking of setting this up in a VRAM scheme where at every Hsync I load an entire row into a separate buffer which is read out to the display. This would let my SDRAM be free for the majority of the time. As for the overlay, I was thinking of muxing in front with the select being combinational logic derived from the current pixel color (so a masking scheme). If the current pixel color is black, then use the CCD/CMOS data instead. I think that this may be how it's normally done in computers. \$\endgroup\$ – Koma Dec 15 '11 at 6:02
  • \$\begingroup\$ Thinking about it some more, I need to clear the screen to black each time regardless (I'm not sure if standard SDRAM has that capability) so I might as well set the screen color to the CCD/CMOS output. This would force me to tripple buffer. 1: Write image sensor data, 2: Write GPU data, 3: Output to display. Is this method unheard of? \$\endgroup\$ – Koma Dec 15 '11 at 6:16
  • \$\begingroup\$ Try to avoid clearing the screen. I think you are missing something (or not explaining something rather unusual) about the interaction of the CCD, GPU, and display buffer. What is the actual use case? \$\endgroup\$ – Chris Stratton Dec 15 '11 at 6:31
  • \$\begingroup\$ Sorry if I'm being unclear. On the display should be a video stream from the image sensor, but with other shapes drawn ontop. These shapes come from the GPU (triangles, points, lines). Normally, during every frame, the screen should be cleared to get ready for the next frame. The hardware needs to do this clear because whatever that is giving commands to the GPU doesn't know about the internal structure of the framebuffer. Instead of clear, I want to write image sensor data as the default 'color'. \$\endgroup\$ – Koma Dec 15 '11 at 7:24
  • \$\begingroup\$ You can do it that way, but it could be more efficient to have one circuit for displaying the video, and another independent one that generates framebuffer graphics, with a block of logic with rules for combining them pixel by pixel in real time - such as graphics always in front with some color (0,0,0) meaning transparent to reveal the video behind. You might also look into having ping-pong frame buffers (one GPU'ing, one showing, then swap), and if you really want to erase all the time you could perhaps erase the memory incrementally as you read it onto the LCD. \$\endgroup\$ – Chris Stratton Dec 15 '11 at 8:40
0
\$\begingroup\$

Explaining exactly what to do is way beyond what we can do in a forum like this. That being said, you need buffers or FIFOs. Every read and write to SDRAM must be a burst where every word of the burst is utilized. Data would be temporarily stored in FIFOs or other types of buffers to accommodate the burst transfers.

The FIFOs also help deal with the arbitration delays when multiple things are trying to access SDRAM at the same time.

The other way to speed things up is to use a wider data bus to SDRAM.

|improve this answer|||||
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.