Hardware test of FPGA, thousands of combinations?

Question

My colleagues longtime ago have designed an FPGA (using VHDL) for a special purpose. Another colleague prepared the test scenarios.

Now they want me to test this FPGA board following these written scenarios. The test bench is ready. So I have nothing to do except reading these scenarios, changing the Analog switches placed in the test bench (ADC is used in the test bench) and see the DIGITAL output on a monitor (an application was developed to show the result of FPGA computation on pc monitor)

My question is:

What is the name of this type of testing?

I searched several hours in Amazon book store to find in chapters of books written for FPGA design, something about real way of testing an FPGA with no success.

I have 5 analog inputs and the ADC will create let's say 20 digital variables for the FPGA. These test scenarios specific for this FPGA test not all the input combination (2^20=1048576)but at least 1000 combination.

So I should perform 1000 tests manually:

reading the scenario, manually changing the analog switch values, read the digital result on monitor)

But how about FPGA’s that have more inputs? should we test hardwarely all the possible combinations of the inputs?

Please help me. If you know a chapter of a book explaining the way we test physically (hardware) the FPGA or if we use other ways I’ll appreciate. The keywords help me too

Answer 1

When done in simulation, this type of testing would be called "directed testing". In other words...

If I put in data A, I expect result B.

If I out in data C, I expect result D.

etc...

(where all of the sets of data (A, C, ...) and results (B, D, ...) have been determined beforehand.

Normally one would try to find a way to eliminate the human (that's reading and checking the result on the monitor) from the loop - partly because humans are less reliable than computers, and partly for speed reasons.

However, the "constrained random testing" mentioned by others is generally the way to go if possible. In this, you generate randomised input, work out what the result should be for that input (using an independent model of the design), and check that the result from the FPGA matches what you expected. You will generally get much better coverage this way than by directed testing (although directed testing can still be useful for "hello world"-type bringup tests and for hitting specific corner cases.

For constrained random testing, you really need an automated mechanism for checking the result.

Answer 2

The two broad flavors of testing are directed testing and randomized testing.

For directed testing, you design each test. You might vary the order of the tests, but there are no new tests unless you write them. This is good for go/no-go testing or system emulation where you create a lot of complex directed tests. Best case in this scenario is to randomize the order of the tests.

Randomized testing randomizes one or more input variables to the device under test. This is largely regarded as the best way to do a lot of testing.

Constrained random testing puts bounds on these variables.

On the other side of your bench, you'll want to model what those inputs should produce as expected outputs. Because you're using an ADC, there may be variance in the digital output and you'll need to account for that in your model. It would be best to make automatic measurements of the digital output, perhaps you could even ask the design team for a test path from the FPGA and read the back from a register.