How to think like a Verification Engineer

Before directly start writing test scenarios understand and clarify specifications thoroughly. For example

1. Is this a normal vending machine or a digital one?
2. How many products it can handle? (3/5/7 etc)
3. Does it accept both coins and cash? If both then which all coins or cash are accepting (1/2/5/10 etc..)
4. Does it accept digital payments/Credit Cards? If yes which all are acceptable?
5. Is this having a normal keypad or touchpad?

After clarifying all the doubts you are having after understanding specifications you should start thinking or writing test scenarios to verify a particular design.

Sample Scenarios to verify coffee vending machine

1. Verify the input mechanism for coffee ingredients-milk, water, coffee beans/powder, etc
2. Verify that the quantity of hot water, milk, coffee powder per serving is correct
3. Verify the effect of suddenly switching off the machine or cutting the power. The machine should stop in that situation and in power resumption, the remaining coffee should not get come out of the nozzle.
4. Verify that coffee should not leak when not in operation
5. Verify the amount of coffee served in single-serving is as per specification
6. Verify that the digital display displays correct information
7. Check if the machine can be switched on and off using the power buttons
8. Check for the indicator lights when the machine is switched on-off
9. Verify that the functioning of all the buttons work properly when pressed
10. Verify that the complete quantity of coffee should get poured in a single operation, no residual coffee should be present in the nozzle
11. Verify the mechanism to clean the system work correctly- foamer
12. Verify that the coffee served has the same and correct temperature each time it is served by the machine
13. Verify that system should display an error when it runs out of ingredients
14. Verify that pressing the coffee button multiple times leads to multiple serving of coffee
15. Verify that there is a passage for residual/extra coffee in the machine
16. Verify that machine should not make too much sound when in operation
17. Performance test – Check the amount of time the machine takes to serve a single serving of coffee
18. Performance test – Check the performance of the machine when used continuously until the ingredients run out of the requirements
19. Negative Test – Check the functioning of the coffee machine when two/multiple buttons are pressed simultaneously
20. Negative Test – Check the functioning of the coffee machine if the ingredient container’s capacity is exceeded

A 2x2 port packet switch specifications:

Specification: There are two input and output ports A and B as shown above. Each port can receive packets of variable size between 64 and 1518 bytes. Each packet will have a 4 Byte Source Address and 4 Byte destination address along with data and a 4 Byte CRC computed across the packet as shown below. The packet will be switched to one of the output ports based on the Destination Address.

How will you verify the design? How will you generate stimulus and checkers? What will be some of your corner cases for verification?

Answer:

For these kinds of questions where a design specification is given, the first step is to understand the design specification and clarify any questions with the interviewer. The next step is to identify the scenarios to be verified and come up with a verification plan and strategy document. This should list down the features/scenarios to be verified, what methodologies can be used to verify (directed/constrained random, coverage, assertion, etc.), how to check for correctness, etc. Further, details should be provided on how the stimulus can be generated and how checking can be done. Another aspect is to think through all design features and identify corner cases that need to be verified.

Now, let’s try to list down how this simple router design can be verified

Following are some of the scenarios that need to be verified:

1. Check for proper switching of packets from Port A to both output ports based on the destination address.
2. Check for different packet sizes - minimum size, maximum size, and random sizes in between will be good.
3. Check for all possible values of Source and Destination Address.
4. Check for different data patterns.
5. Check for streaming packets (back to back with no delay, few cycles delay, and large cycles delay), same size packets streamed, or different size packets streamed.
6. Check for correct CRC functionality by expecting the correct CRC.
7. Check where some bits of SA/DA or data or even the CRC are corrupted.

What more can you think of?

Now, in order to verify the above scenarios, we need to design a constrained random packet generator and we also need a scoreboard/checker that checks for packet correctness and correct switching behavior. If the tests are random, we will also need to write some coverage monitors that make sure all the important scenarios mentioned above are getting covered.

If an interviewer wants to test you more, then he can also continue with questions asking you to actually write a SystemVerilog packet generator code or a checker or driver, etc.

A single port RAM has only a single port for read and write. So, it can only do a read or a write at any given point of time. Other design specifications that need to be considered for verification are the RAM size, and width of address and data bus.

Based on this, following are some of the scenarios that should to be verified:

1. Single Read and Write behaves correctly,
2. Back to Back reads or writes to the same address and different addresses,
3. Writes followed by reads to same address back to back,
4. Reads followed by writes to same address back to back,
5. Verifying the boundary of RAM sizes - reads and writes,
6. Verifying the different patterns of writing into memory location like writing all zeros, all ones, alternating zero/one, walking ones/zeros patterns.

If you are further requested to define a verification environment, you can consider scenarios like above and define whether a directed or constrained random environment will work better and how the stimulus generator and checkers can be designed.

Following are the scenarios that need to be verified for this given ALU design:

1. Verify that all individual operations work (Add, Sub, Increment, AND, and OR) by driving the two operands A and B, and driving the select lines for each of the operations.
2. Verify that if select lines are between 110-111, then no operation happens.
3. For each of the above instructions, select minimum and maximum values of A and B and combinations. Given that A and B are 4 bit, the maximum value possible is 4’b1111
4. Verify the overflow and underflow cases for ADD and SUB cases. If both A and B are 4’b1111, overflow happens for ADD, while if the value of B is greater than A, underflow happens for SUB.
5. Verify wrap-around cases for increment instructions. If A=4’b1111, increment should cause a value of 0.
6. Once individual scenarios are verified, create a random sequence of opcodes to verify that the effect of one operation does not affect the following ones. Check for sequences where the same opcodes repeat more than once or different opcodes repeat in different patterns.
7. To create a stimulus, you can design a random opcode and operand generator and a simple driver. To check for results, a simple model or ALU can be written and the results can be compared against the same.