Before anyone say something negative about my post, yes, i have already read a lot of other posts before i public this question (well maybe not A LOT) but still i have doubts.

I'd be really glad if someone can guide me, first i'll explain what i want to achieve and what i know.

My purpose is to make an edge detector with pure VHDL (HLS is a valid option) an FPGA with Vivado Design Suite (with pure i mean without using any external help like pre built IP cores or toolkits form xilinx, etc), im between a Zybo fpga with zynq 7000 msoc, or an kintex with enough RAM to load a low quality image.

I already have some VHDL knowledge but i have never worked with image processing with a FPGA.

Here i go.

  1. IS it even possible to make a real time color image processing in FPGA without using any external Software or Hardware, like MATLAB, and to be able to read an image from a camera and after the processing, display it on a screen with VGA connection or something like that? AND of course, synthesizable.

  2. First, which is the best (in terms of difficulty and performance) format file to load an image to fpga BRAM. I have read some posts saying that .bin, .hex, .mif and .coe, but which is the right one? I pretend to read a grayscale image, and after that, a color image.

  3. Could someone give me an example (some explaining would be perfect) in vhdl to read a .coe/hex/etc file? Or link me to a good source of knowledge on how to do that.

  4. Is there any book about image processing with fpga with some examples?

and thats it.

I would really be grateful if someone could guide me.

  • 3
    \$\begingroup\$ "IS it even possible to make a real time color image processing in FPGA?" I sure hope so, because that's what I've been doing for the past several years! "best format file to load an image?" If your image is in a file, you're not doing real-time. Use a simulator (not a real FPGA), and use it to read your file data. \$\endgroup\$
    – Dave Tweed
    Sep 25 '20 at 3:54
  • 1
    \$\begingroup\$ @DaveTweed Maybe you've just been living a delusion thinking you have for the past few years and have actually been doing something completely different but still getting paid for it somehow. \$\endgroup\$
    – DKNguyen
    Sep 25 '20 at 4:52
  • \$\begingroup\$ i liked your answer, one question, what other ways i have to save the image if in real time? i mean the image took from the camera have to be on certain format file to be "readable" or how would you do it? \$\endgroup\$
    – esencia
    Sep 25 '20 at 5:45
  • \$\begingroup\$ In the projects I've been working on, we either use a commercial frame grabber (CameraLink, SDI, DVI, or HDMI interface), or send the video to a separate dedicated chip for H.264 compression and then pass the data to the host PC via PCIe. \$\endgroup\$
    – Dave Tweed
    Sep 25 '20 at 11:07

The answer, as usual, is "it depends".

Your main obstacle is memory size. FPGA block ram tends to be on the order of a few hundred kilobytes to tens of megabytes in the high end chips, and it's organized differently, as hundreds of small patches rather than one big chunk.

So an algorithm that operates on a big area would synthesize to a single copy of that algorithm, an address decoder that takes the upper bits and decides which patch of RAM to address, and lots of interconnects. At runtime, most of the chip would be idle, with a small part running with a slow clock, because the interconnects are rather long. You can optimize a bit by adding pipelines and designing your algorithm to operate on tiles (which is doable for edge detection, although difficult on tile boundaries), but in general memory size will still be too little.

A sensible FPGA based design would probably use external DDR memory, but we are in the midrange segment there already.

VGA output is not a big deal, you just need a set of DACs and a tunable clock source to generate the pixel clock (the FPGA onboard PLLs are usually not flexible enough). For HDMI, you just need differential outputs that are fast enough (mid-range segment again) and a clock source, so that is even less effort.

Generally, you wouldn't use the block ram initialization formats to load image data, these are used in FPGA initialization only. Instead, you'd leave the block ram uninitialized during load and implement a small state machine that can be used to transfer data in. To get data from a camera that would probably be a MIPI interface.

One thing you can probably do is stream the image processing. For edge detection, you mostly need to compare adjacent pixels, which you get basically for free horizontally as the pixels are transferred right after another, and with a small buffer holding a few lines for vertical and diagonal tests.

  • \$\begingroup\$ Thank you for the given information, one question, i havent use MIPI interface, but beside that, the image took from a camera, how can i read it? i thought the only way of storing an image to a FPGA was as a file with specific format. \$\endgroup\$
    – esencia
    Sep 25 '20 at 5:52
  • \$\begingroup\$ @esencia a file is just a way of representing data. FPGAs process data. Ergo an FPGA can process any file, you just have to design a processor to read and decode that file from some storage source (MMC? SD Card? NOR/NAND flash? MIPI source? up to you). \$\endgroup\$ Sep 25 '20 at 9:00
  • \$\begingroup\$ You should probably lead with your last paragraph about stream processing. You can do a lot of image processing without ever needing to store a whole frame. There are only certain classes of operations that actually require frame buffers -- time domain filtering, frame rate adjustments, interlace/deinterlace, to name a few examples. And when you do need a frame buffer, even relatively low-end FPGAs can interface to an external DDR SDRAM chip or two. A single 512 Mbit chip can hold four 2k x 2k x 32-bit frame buffers. \$\endgroup\$
    – Dave Tweed
    Sep 25 '20 at 11:46

Yes of course it's possible. But rather than starting with a camera interface you might want to look if your FPGA board already has an Ethernet port, and find a sample project allowing you to connect to that and simply transfer files. Zynq will definitely have something suitable as a starting point. It may be in HLS but that won't stop you using VHDL for the good stuff.

"Reading a hex file" in VHDL is something you'd do IN SIMULATION to load testcases : synthesis has no idea of files at runtime, (but you could load one default image to initialise a memory at synthesis time)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.