A debayer block inputs image from a CCD Bayer filter and then estimates the missing color information for each pixel. The data comes from a camera. How should one go about writing a testbench for something like this?
First understand what the problem specification (your first sentence) REALLY means.
In other words, given a full colour image(1), what does the Bayer filter generate from it (2)? What information is missing (3)? What can a Debayer generate given only the Bayer filter output (4)? How close to the original image is the Bayer output(5)?
You now have 5 sample images generated from an original - possibly in Gimp or Photoshop or Matlab - or by a VHDL package from an input file (XPM format for simple reading).
Your testbench should at least read file (2) and file (4), feed (2) into the Debayer, and compare its output(6) with (4). If the output and (4) are identical, the testbench has passed. Otherwise, it is often useful to write both output (6) and error (7) = (6) - (4) for analysis in graphics tools.
For example, if the VHDL algorithm rounds differently, the rounding errors are visible in the error output (7).
The above assumes you are only testing the quality of the VHDL implementation of the Debayer algorithm. If you are testing the algorithm itself (which is usually done before the VHDL stage, perhaps in Matlab or Python) then you would also use files (3) and (5) to see how close your Debayer algorithm can get to the original.