The Practical Verification Debug Guide: A Repeatable Method (+ AI-Assisted Debug)
DEBUG | UVM | SYSTEMVERILOG | AI-ASSISTED VERIFICATION
Updated November 2026 · By The Art of Verification
Quick answer: A repeatable verification debug method is a fixed order of five questions. 1) Was the stimulus legal? 2) Did it reach the DUT pins? 3) Did the design respond correctly? 4) Did the checker judge it correctly? 5) Do the log and waveform agree? Stop at the first “no”. That door holds your root cause. AI tools can now sort logs and suggest the door, but an engineer still confirms the cycle in the waveform before any fix.
Key takeaways
- Debug is the biggest single slice of a verification engineer’s week. Method beats talent.
- Walk five doors in order: legality, delivery, design, checking, consistency.
- Read the first UVM_ERROR, not the last. Later errors are usually echoes.
- In the waveform, start at the symptom and walk backward.
- Group regression failures by signature before you debug any single test.
- AI is great at reading and sorting. You still own the verdict.
Why does debug skill decide your schedule?
Ask where a chip schedule really goes. The answer is rarely “writing tests”. It is figuring out why tests failed.
- The Wilson Research Group functional verification study found verification engineers spend more time in debug than in any other activity, 44% of their time in the 2018 edition, up from 39% in 2016 (Siemens Verification Horizons).
- The 2024 edition reported only 14% of projects hit first silicon success, the lowest in two decades (Verification Academy).
Put those two numbers together. Nearly half your week goes to debug, and most chips still need another spin. So the way you debug is not a side skill. It is the job.
Two engineers with the same tools can work at very different speeds:
- The slow one opens the waveform first, drowns in thousands of signals, and loses an afternoon per failure.
- The fast one narrows the problem to one layer before opening any viewer, and finds root cause in minutes.
There is a second cost too. A regression full of false failures trains the team to ignore red. Then a real bug walks through in the noise. Good debug keeps the regression honest.
What is a good analogy for debug in verification?
Pick whichever one sticks. They all say the same thing: check in order, from cheap to expensive.
1. The missing letter
Your letter never reached your friend. You do not blame the post office first. You check:
- Did I write the right address? (door 1, legal stimulus)
- Did I actually drop it in the box? (door 2, delivery)
- Did the post office route it right? (door 3, design)
- Did my friend check the right mailbox? (door 4, checker)
- Am I sure I am remembering the right day? (door 5, consistency)
Blaming the design first is like blaming the post office before checking the address.
2. The doctor who checks vitals before the MRI
A good doctor takes temperature and blood pressure before ordering an MRI. The waveform is your MRI. It shows everything, it is slow, and it is useless if you do not know where to look. Logs and assertions are your vitals. Read them first.
And when the thermometer says 45°C but the patient looks fine, you check the thermometer. That is a monitor bug.
3. The match replay
When a volleyball team reviews a lost point, nobody rewinds to the warm-up. You start at the point that was lost and go back one touch at a time: bad set, late pass, wrong rotation. That is exactly how to read a waveform. Start at the mismatch. Walk back one cause at a time.
What is the five-door debug method?
Every failure lives behind one of five doors. Walk them in order and close each one until only the guilty one is open.
| Door | Question | Where to look | Typical guilty party |
|---|---|---|---|
| 1. Legality | Was the stimulus legal? | Solved item print, constraints, sequence | Missing or wrong constraint |
| 2. Delivery | Did it reach the DUT pins? | Driver log, interface signals | Virtual interface or config_db |
| 3. Design | Did the DUT respond correctly? | First failing assertion, RTL waveform | RTL bug |
| 4. Checking | Did the checker judge correctly? | Scoreboard, reference model, monitor | Testbench bug |
| 5. Consistency | Do log and waveform agree? | Seed, timestamps, tool versions | Stale dump or wrong seed |
Why the order matters: each door you clear removes a whole layer from the suspect list. Skip ahead to the RTL waveform and every layer stays a suspect. That is why unstructured debug feels endless.
The idea underneath: a failure has one first cause and many downstream effects. Logs and waves show effects everywhere. Your only job is to find the earliest cycle where correct became incorrect.
If all five doors say yes, your test expectation is wrong. Go back to the spec, not the RTL.
What comes before and after the five doors?
The five doors find the root cause. Two steps around them make the whole job repeatable: one before you start, one after you finish.

Step 0: reproduce and shrink
- Reproduce with the saved seed and the exact command line. A failure you cannot repeat can only be guessed at.
- Shrink it: fewest transactions, shortest run, one agent if you can. A 2,000-cycle failure is a different job from a 2-million-cycle one.
- Check the run got your settings. Print every plusarg and define the test reads in one banner line at time 0 (more in the mistakes below).
- Analogy: a doctor asks you to show when it hurts before ordering any scan.
Think in hypotheses
- Before you open a tool, write one line: “I think it is X, and if so I will see Y at time T.”
- If Y is not there, X is wrong. Cross it off and write the next one.
- This turns hours of scrolling into a short list of yes or no checks.
Step 6: fix, lock it in, report
- Write the bug report so someone else can act on it: seed, command, first error line, the one signal and time that prove it, and which door.
- Lock it in with an assertion, a check or a test that fails if the bug comes back.
- Ask why the bench missed it earlier. A missing check is a second bug.
How do you build a UVM testbench that explains its own failures?
Debug speed is designed in, not bolted on. Your component boundaries are your debug boundaries.

What each component should do for you:
- Sequence: print the solved item and reject illegal combinations. This moves most door 1 problems out of debug entirely.
- Driver: log every item it drives, with the cycle. Driver logged it but pins stayed flat? Door 2, the interface is not connected.
- Monitor: passive and complete. Rebuild every transaction and timestamp it. A monitor that misses one creates a fake mismatch.
- Scoreboard: on a mismatch, print expected, actual, and the source transaction. Snapshot shared model state before you compare.
- Everyone: use consistent message IDs so a simple grep finds the story.
New to the plumbing? Start with the typical UVM testbench architecture, the build a UVM environment series, sequencer and driver communication, and passing a virtual interface with uvm_config_db.
What SystemVerilog code makes debug faster?
Honesty note: the code below follows the SystemVerilog and UVM standards and common practice. It was not captured from a specific simulator run. Compile it in your own flow before relying on it.
Door 1: print what was really sent
Print the solved transaction, not the one you think you sent. Bad, good, best:
// BAD: you print your intent, not the solved values.
// An illegal randomized value stays invisible.
`uvm_info("SEQ", "sending a 4-beat burst to 0x1000", UVM_LOW)
`uvm_do(item)
// GOOD: print the solved item after randomization.
function void post_randomize();
`uvm_info("SEQ", $sformatf("addr=%0h len=%0d size=%0d burst=%0d",
addr, len, size, burst), UVM_LOW)
endfunction
// BEST: print it AND self-check legality, so an illegal
// stimulus fails loudly here, not later as a fake design bug.
function void post_randomize();
`uvm_info("SEQ", $sformatf("addr=%0h len=%0d size=%0d burst=%0d",
addr, len, size, burst), UVM_LOW)
if (addr[11:0] + (len + 1) * (1 << size) > 13'h1000)
`uvm_error("SEQ_ILLEGAL", "AXI INCR burst crosses a 4KB boundary")
endfunction
If randomize() itself returns 0, nothing legal gets sent at all. That is a different problem with its own clues in the log: see why SystemVerilog randomization fails for the six causes and a 5-minute triage.
- Why it exists: an illegal burst makes the design misbehave correctly. Without this print you chase a ghost.
- Next step: move the check into a shared protocol checker so every sequence gets it for free. The rules are in the AXI protocol verification guide.
Door 3: let an assertion mark the crime scene
// Fails at the exact cycle ack is late.
property p_req_then_ack;
@(posedge clk) disable iff (!rst_n)
$rose(req) |-> ##[1:5] ack;
endproperty
a_req_ack: assert property (p_req_then_ack)
else `uvm_error("PROT_REQ_ACK", "ack did not follow req within 5 cycles")
// Prove the precondition actually happens.
c_req: cover property (@(posedge clk) disable iff (!rst_n) req);
Why $rose(req) and not plain req: while req stays high, a plain req |-> starts a new attempt every cycle, so one long request can raise errors that have nothing to do with the design. Trigger on the start of the request. This exact property is run on VCS in the planted-bug test below.
- An assertion turns a long backward trace into one log line with a timestamp.
- The cover matters. An assertion whose precondition never fires proves nothing. Formal tools hit the same vacuity trap; see formal vs simulation-based verification.
- Bind assertions to the RTL module so they travel with the design into every test.
Door 4: a scoreboard a reviewer can trust
// Snapshot first, compare second, print everything on a miss.
function void check_item(axi_item act);
axi_item exp = model.predict_snapshot(act); // copy, not a live handle
if (!act.compare(exp))
`uvm_error("SB_MISMATCH", $sformatf("\nEXP: %s\nACT: %s\nSRC: %s",
exp.convert2string(), act.convert2string(), act.src_txn_id))
endfunction
For a worked example where counters, not error lines, separate a design bug from a bench bug, see async FIFO verification part 1.
Does the method work? Three planted bugs, real VCS output
To show what each door looks like in a real log, here is one small UVM bench with a bug planted behind doors 1, 3 and 4. It is a 16-word memory with a req/ack handshake, a driver, a monitor, a scoreboard, and the checks from the section above. The plusarg +BUG=N picks the bug. All runs: Synopsys VCS X-2025.06-SP1, UVM 1.2.
| Run | Planted bug | Door | Caught by | Scoreboard |
|---|---|---|---|---|
+BUG=0 | none | none | nothing to catch | items=16 errors=0 |
+BUG=1 | 4KB constraint switched off | 1, legality | 6 SEQ_ILLEGAL errors from post_randomize() | 0 errors |
+BUG=3 | DUT acks after 6 cycles, spec says 5 | 3, design | 16 PROT_REQ_ACK assertion errors | 0 errors |
+BUG=4 | monitor reuses one object | 4, checking | 16 SB_MISMATCH errors | 16 errors, design correct |
The three planted bugs, as they appear in the code:
// +BUG=1 (sequence): the legality constraint is switched off
if (bug == 1) req.c_4k.constraint_mode(0);
// +BUG=3 (DUT): ack comes after 6 cycles instead of 2
initial lat = (planted_bug() == 3) ? 6 : 2;
// +BUG=4 (monitor): one object is reused for every transfer
if (!reuse || t == null) t = bus_item::type_id::create("t");
+BUG=1: door 1, illegal stimulus
UVM_INFO testbench.sv(181) @ 0: uvm_test_top [TEST] planted bug: 1
UVM_ERROR testbench.sv(65) @ 95: uvm_test_top.env.sqr@@s.req [SEQ_ILLEGAL] burst at ffc, 4 beats of 2 bytes crosses a 4KB boundary
UVM_ERROR testbench.sv(65) @ 175: uvm_test_top.env.sqr@@s.req [SEQ_ILLEGAL] burst at fd8, 13 beats of 4 bytes crosses a 4KB boundary
...
UVM_INFO testbench.sv(152) @ 655: uvm_test_top.env.sb [SB] items=16 errors=0
UVM_ERROR : 6
Six of 16 bursts were illegal, and the scoreboard said nothing, because this small DUT does not care about 4KB. A real AXI slave would, and the failure would look like a design bug. Only the check in post_randomize() names the true cause.
+BUG=3: door 3, a late design
UVM_INFO testbench.sv(181) @ 0: uvm_test_top [TEST] planted bug: 3
"testbench.sv", 44: tb.bif.a_req_ack: started at 35ns failed at 85ns
Offending 'ack'
UVM_ERROR testbench.sv(45) @ 85: reporter [PROT_REQ_ACK] ack did not follow req within 5 cycles
...
UVM_INFO testbench.sv(152) @ 1295: uvm_test_top.env.sb [SB] items=16 errors=0
UVM_ERROR : 16
The assertion fired on every transfer, at the exact cycle the window closed. The scoreboard still passed: late data is still correct data. The only other hint was that the run took 1295 ns instead of 655.
+BUG=4: door 4, the design is innocent
UVM_INFO testbench.sv(181) @ 0: uvm_test_top [TEST] planted bug: 4
UVM_ERROR testbench.sv(148) @ 655: uvm_test_top.env.sb [SB_MISMATCH] item 0: RD word=0 data=1d57d3d8, expected xxxxxxxx
UVM_ERROR testbench.sv(148) @ 655: uvm_test_top.env.sb [SB_MISMATCH] item 1: RD word=0 data=1d57d3d8, expected xxxxxxxx
...
UVM_ERROR testbench.sv(148) @ 655: uvm_test_top.env.sb [SB_MISMATCH] item 15: RD word=0 data=1d57d3d8, expected xxxxxxxx
UVM_INFO testbench.sv(152) @ 655: uvm_test_top.env.sb [SB] items=16 errors=16
Sixteen errors, and the DUT is correct. The clue is in the errors themselves: all 16 print the same item. Sixteen different transfers cannot look identical, so the checker is holding one object sixteen times. The fix is one line: create a new object per transfer in the monitor, or clone before you store. More on this trap in UVM TLM explained.
What the three runs teach
- The message ID points to the door.
SEQ_ILLEGAL,PROT_REQ_ACKandSB_MISMATCHeach sent us to a different place before we opened a waveform. - A scoreboard is one door, not the whole check. Two of the three bugs passed it.
- When every error prints the same thing, suspect the checker before the design.
We later reused this bench as the baseline against an AI-written testbench on a design with four hidden bugs. It caught 2; the AI bench caught 4, and the reason was stimulus, not checking. See the experiment, logs and files.
Stuck on a scoreboard mismatch? See the 6 causes and a 5-minute triage, with five planted bugs and their VCS logs.
How do you read a waveform without getting lost?

- Jump to the failing time from the first error. Never scroll forward from time zero.
- Add only the signals of the failing interface plus the first failing assertion.
- Walk back one cause at a time. Ask “what made this value wrong?” and move one hop upstream.
- Drop a marker at each hop so the next engineer can follow your trail.
- Name the signal and the exact cycle in your bug report. “It looks wrong around the middle” helps nobody.
Door 2 quick read:
| What you see | What it means | Next step |
|---|---|---|
| Driver logs a send, pins stay flat | Virtual interface not connected | Check uvm_config_db set and get paths |
| Pins toggle, no ready | DUT not accepting | Go to door 3 |
| Handshake completes | Delivery is fine | Go to door 3 |
How do you read UVM reports fast?
The report summary tells you the shape of the failure before you open anything. Read top down. The first UVM_ERROR is usually the cause.
| Report clue | Likely cause |
|---|---|
| Many errors at the same timestamp | One root event, checks cascading |
| Error, long silence, then timeout | Design hung after the first error |
| Only a UVM_FATAL about config | Environment build problem, not a design bug |
| Objection drop warning | Test ended before stimulus finished |
| Failure vanishes with full dump on | Race or uninitialized value, timing shifted |
Run the regression at low verbosity for speed. Rerun only the failing seed at high verbosity for detail.
UVM command-line switches worth knowing
| Switch | Shows | Use it when |
|---|---|---|
+UVM_VERBOSITY=UVM_HIGH | More uvm_info messages | You need door 1 prints, such as the solved item |
+UVM_OBJECTION_TRACE | Every raise and drop | The test ends at 0 ns, or never ends |
+UVM_CONFIG_DB_TRACE | Every set and get | A get fails or returns the wrong value |
+UVM_PHASE_TRACE | Phase start and end | The run is stuck between phases |
+UVM_MAX_QUIT_COUNT=1 | Stops at the first error | You want the first error to be the last line |
uvm_top.print_topology() | The component tree | A component is missing or extra |
factory.print() | Overrides in force | An override did not take |
Verbosity levels and IDs in detail: UVM verbosity and reporting.
What are the most common RTL and testbench bugs?
RTL bugs, by symptom
| Symptom | Common RTL cause | How to confirm |
|---|---|---|
| Data right, one cycle late | Missing or extra pipeline stage | Count register stages in the wave |
| Fails on some seeds only | Uninitialized register or race | Look for X right after reset |
| Fails after back-to-back requests | FIFO full or empty off by one | Watch occupancy at the boundary |
| Hang under heavy load | Arbitration deadlock | Check grants when all agents request |
| Wrong only across clock domains | Missing synchronizer | See the CDC verification guide |
| Passes RTL, fails gates | X-propagation or timing | See the gate-level simulation guide |
Testbench bugs that look like design bugs
- Virtual interface never set, so the driver drives nothing.
- Scoreboard compares before the design has produced the result.
- Objection dropped too early. The test ends mid-traffic and looks like a hang.
- Reference model misses a legal reordering and flags correct behavior.
- Coverage sampled on the wrong edge, so a real hit looks like a hole.
- Factory override forgotten. The old component runs and your fix “does nothing”.
Rule of thumb: design bugs tend to be narrow. Testbench bugs tend to be broad. When unrelated tests fail the same odd way, suspect shared testbench code first. Treating your own testbench as a suspect is a sign of a senior engineer. More on that mindset in the thought process behind great verification.
Coverage symptoms that send you the wrong way
| Coverage symptom | What it usually means |
|---|---|
| Hole in a bin you know you drove | Sample edge or condition is wrong |
| Hole in a truly rare bin | Stimulus never got there. Likely bug hiding place. |
| Sudden drop after a merge | A covergroup got disconnected |
Background: types of coverage metrics, functional coverage guidelines, and the coverage closure guide.
How do you triage a big regression in one morning?
Picture the morning after a nightly run on a subsystem with a CPU, a DMA engine and a memory controller. Two hundred red tests. Coffee is still hot.

- Bucket first. Group by first UVM_ERROR text and first failing assertion, with numbers and timestamps stripped out.
- Read the buckets, not the tests. 40 share an objection warning from a shared base test. 90 hit the same AXI assertion. 70 are scattered.
- Debug one seed per bucket. The shortest, lowest-seed failure in each group.
- Check what changed. Diff RTL and testbench files since the last green run.
- Re-bucket after fixes land. Most scattered failures are cascades and vanish on their own.
Keep debug cheap while you do it:
| Setting | Runtime cost | When to use |
|---|---|---|
| Full dump, whole run | Several times slower | Rare, hard bugs only |
| Windowed dump around failure | Small | Default for reruns |
| High verbosity, whole regression | Huge logs, slower | Avoid. Rerun one seed instead. |
| Assertion-only run | Fast | First-pass protocol triage |
Always save the seed and command line for every job. A failure you cannot reproduce is a rumor, not a bug.
How can AI help with verification debug?
This part changed a lot in 2026. All three big EDA vendors now ship or preview AI agents aimed at debug:
- Siemens announced the Questa One Agentic Toolkit in February 2026, including a Debug Agent that correlates waveforms, assertions, coverage and logs to point at suspicious transitions and suggest failure mechanisms for an engineer to review (Siemens).
- Synopsys unveiled a Verification AgentEngineer in March 2026 that reads logs, groups errors, and checks the likely cause against waveforms. Synopsys also reported a 25% to 40% cut in debug cycle time in an early evaluation (Tom’s Hardware).
- Cadence opened early access to its ChipStack agents in February 2026, aimed at front-end design and verification loops.
Two honest caveats. These numbers are vendor-reported and measured against different baselines. And every one of these tools keeps a human approving the final root cause. That tells you where the real line sits.
Here is the way I think about it: AI is a very fast junior engineer who has read every log but never seen the spec. Brilliant at reading and sorting. Not yet trusted to sign off. So give it the reading work, and keep the judgment.

For the wider picture beyond debug (test selection, coverage steering, drafting, and the guardrails that keep each honest), see AI in ASIC verification: a practical method, with limits.
The AI-assisted debug flow, step by step
- Make your logs machine-readable. One line per transaction, stable message IDs, seed and test name in the header. Good logs help humans and AI equally.
- Cluster failures automatically. Normalize the first error line and bucket. You do not even need an LLM for this step (script below).
- Ask the AI for a door, not a fix. Feed it the first error, about 200 lines of context, and the diff since the last green run. Ask which of the five doors is most likely guilty and why.
- Human gate. You open the waveform at the cycle it named and confirm or reject. No fix goes in without this.
- Fix and lock. Patch, add an assertion or check, rerun the exact seed, then the bucket.
- Feed it back. Store the signature, door, and root cause. That history makes the next suggestion better.
Step 2 script: bucket failures by first error
# bucket_failures.py : group regression failures by normalized first error
import re, sys, collections, pathlib
ERR = re.compile(r'^(UVM_ERROR|UVM_FATAL)\s.*@')
NORM = [(r'@\s*[\d.]+\s*[munpf]?s?', '@T'), # timestamps
(r"(0x|'h)[0-9a-fA-F_]+", 'HEX'), # hex with a prefix
(r'\b[0-9a-fA-F]{8,}\b', 'HEX'), # bare %h words such as deadbeef
(r'\b(?=[0-9a-fA-F]*\d)[0-9a-fA-F]{4,}\b', 'HEX'), # shorter hex with a digit
(r'\b\d+\b', 'N')] # decimal numbers
def signature(text):
for line in text.splitlines():
if ERR.match(line):
for pat, rep in NORM:
line = re.sub(pat, rep, line)
return line.strip()[:160]
return 'NO_UVM_ERROR (check timeout or crash)'
buckets = collections.defaultdict(list)
for log in pathlib.Path(sys.argv[1]).glob('*/sim.log'):
buckets[signature(log.read_text(errors='ignore'))].append(log.parent.name)
for sig, tests in sorted(buckets.items(), key=lambda kv: -len(kv[1])):
print(f'{len(tests):4d} {sig}\n e.g. {tests[0]}')
Tested on 7 sample logs. UVM prints hex with %h, with no 0x, so the two bare-hex patterns above matter: without them, one scoreboard failure printed as data=1a2b3c4d and data=deadbeef split into three buckets. With them, 7 failures group into 4 causes:
3 UVM_ERROR testbench.sv(N) @T: uvm_test_top.env.sb [SB_MISMATCH] item N: RD word=N data=HEX, expected HEX
2 UVM_ERROR testbench.sv(N) @T: reporter [PROT_REQ_ACK] ack did not follow req within N cycles
1 NO_UVM_ERROR (check timeout or crash)
1 UVM_FATAL testbench.sv(N) @T: uvm_test_top.env [NOVIF] no vif
Step 3 prompt: keep the AI inside the five doors
You are helping debug a UVM simulation failure.
Use the five-door order: 1 legality, 2 delivery, 3 design,
4 checker, 5 consistency.
Inputs: first error + 200 lines of log, solved item print,
git diff since last passing run.
Answer with:
- Most likely door and a confidence (low / medium / high)
- The exact log line numbers that support it
- One signal and one sim time I should check in the waveform
- What evidence would prove you wrong
Do not propose an RTL fix.
Why “what would prove you wrong”? Because AI tools sound just as sure when they are guessing. Asking for line numbers and a falsifying check turns a confident paragraph into something you can verify in two minutes.
Where AI saves time, door by door
AI is fastest where the work is reading and comparing many lines. It is slowest, and least trusted, where the work is knowing what the spec means.
| Door | What AI does well | What you still check | Analogy |
|---|---|---|---|
| 1. Legality | Scans thousands of solved-item prints and flags values outside the ranges you give it | The spec rule itself | A proofreader who catches every misspelt word but did not write the dictionary |
| 2. Delivery | Compares the driver log with the monitor log and lists items sent but never seen; spots a missing setting in the banner line | The pins in the waveform | A parcel tracker that shows where the box stopped |
| 3. Design | Lines up the first assertion failure with nearby signal changes and names a time to look at | The cycle in the waveform, and what the spec really says | A smoke alarm: it tells you which room, you find the fire |
| 4. Checking | Notices patterns across many errors: the same item every time, the same expected value, errors only at the end of the test | The monitor and scoreboard code | Noticing that every complaint letter is in the same handwriting |
| 5. Consistency | Matches log times to waveform times | The waveform itself | A second pair of eyes on the timeline |
| Regression | Groups hundreds of failures by first error | One seed per group | Sorting the post before anyone opens a letter |
Worked example: the +BUG=4 log
Here is the prompt above applied to the real +BUG=4 run from this page. The inputs are the first error with the log around it, the solved-item print, and the change since the last passing run (the monitor’s object handling).
A good answer looks like this. We wrote it as a model answer to show the shape to expect; it is not the output of an AI tool:
Door: 4 (checking). Confidence: high.
Evidence: lines 4 to 19 of the log are 16 SB_MISMATCH errors, and every one
prints the same item: "RD word=0 data=1d57d3d8". Sixteen different transfers
cannot be identical, so the scoreboard is likely holding one object 16 times.
All 16 errors come at 655 ns, in check_phase, after traffic stopped.
Check in the waveform: rdata at any two read acks. If it differs between
transfers, the DUT is returning different data and the checker is wrong.
What would prove me wrong: rdata is 1d57d3d8 on every read ack. Then the
design is at fault, not the checker.
Why this is quick for a tool and slow for a person: each of the 16 lines on its own looks like a design bug. The clue is only in the pattern across them, and comparing many lines is exactly the reading work a tool does in seconds. You still spend two minutes in the waveform to confirm it before touching any code.
How to measure the gain on your own team
Independent numbers on AI debug speed are thin. The figures above are vendor-reported, so measure your own. Three numbers are enough:
- Time to first root cause. For each failure, from first look to the door you confirm. Compare the median before and after, over a few weeks.
- Door hit rate. How often the first door the AI named was the right one. If it is wrong more often than right, change what you feed it before you blame the tool.
- Failures per group. Failures in a regression divided by the groups they fall into. On the 7 sample logs on this page, the fixed script made 4 groups: 7 failures, 4 causes to read.
Write the three numbers down every week. If they do not move after a month, the tool is adding steps, not removing them.
Guardrails that matter
- Never paste RTL, specs or logs into a public chatbot. Use your company’s approved, on-prem or enterprise setup.
- Demand evidence. No line numbers, no trust.
- Watch for the “clean story” trap. A tidy explanation that does not match the waveform is still wrong.
- Measure it. Track time to first root cause and how often the suggested door was right. If it is not saving time after a month, change the inputs.
Typical debug patterns at IP, subsystem and SoC level
Three patterns that come up again and again. They are composites, not one team’s story, and one of them is the +BUG=4 run above.
IP level: a flag one cycle late
- Symptom: a mismatch only when reads and writes land in the same cycle near full.
- Door 3: the full flag rises one cycle late, so one write lands in a full FIFO.
- Usual root cause: full decoded from the current count, not the next count.
- Lock it in: an assertion that a write is never accepted when full.
Subsystem level: the design is innocent
- Symptom: scoreboard errors on one seed in a few hundred, and the waveform looks fine.
- Door 4: the checker compares a handle that another thread has already changed.
- Lesson: clear the checker before a long RTL hunt. The
+BUG=4run shows exactly this.
SoC level: the hang that is not a hang
- Symptom: a test never ends after a merge, with a long tail of errors.
- Move: read the first error, then run with
+UVM_OBJECTION_TRACE. - Usual root cause: a sequence drops its objection early, or a component raises one per packet and never drops the last. One forum user found 34 objections still raised by a scoreboard.
More SoC context in key areas of SoC verification and the SoC verification flow.
Common debug mistakes, from real forum threads
Retold in our own words, each with more than one fix.
- Treating objections as an end-of-test switch only. A user asked how objections help debug at all (thread).
- Fix A: know the two failure shapes. Nobody raises, so the test ends at 0 ns; or someone never drops, so it hangs.
- Fix B: run with
+UVM_OBJECTION_TRACEand read who raised last. - Fix C: raise and drop in the test, not in drivers, monitors or scoreboards.
- A regression that never ends in CI (thread).
- Fix A: a per-test timeout (
set_timeoutor+UVM_TIMEOUT) so CI always gets a log. - Fix B: a heartbeat or progress print, so the log shows where it stopped.
- Fix C: then find the real cause. Replies in that thread stress that a timeout only hides it.
- Fix A: a per-test timeout (
- Objections raised per packet in the scoreboard. A test never ended with 34 objections still raised (thread).
- Fix A: let the test own the objection and use
drain_timefor data still in flight. - Fix B: check scoreboard queues in
check_phaseinstead of holding the phase open.
- Fix A: let the test own the objection and use
- Stored transactions that change after you store them (thread).
- Fix A: create a new object per transaction in the monitor.
- Fix B: clone in the subscriber before you store it.
- See it happen: the
+BUG=4run above.
- Failures that change with the simulator or with dumping. Two blocks writing and reading the same variable at the same time give different results on different tools (thread).
- Fix A: drive and sample through clocking blocks.
- Fix B: keep testbench code in step with one clock edge (thread).
- Debugging a run that never got your settings. While testing this page, three runs meant to plant bugs all ran clean:
+BUG=Nhad gone into compile options, not run options.- Fix A: print every plusarg and define the test reads in one banner line at time 0, and read that line first.
- Fix B: give each setting a compile-time fallback (
+define+BUG=N) so either box works. - Fix C: when a “buggy” run passes, suspect the run settings (door 2) before you celebrate.
Debug interview questions for verification engineers
- A test fails. What do you check first?
Whether the stimulus was legal, by printing the solved transaction. - Design bug or testbench bug, how do you decide?
Design bugs are narrow. Broad failures across unrelated tests point at shared testbench code. - Thousands of signals. Where do you start?
At the failing time, walking backward from the first failing assertion. - Why read the first UVM error and not the last?
The first is usually the root. Later ones are cascades. - Lead-level: 100 tests fail the same way after a merge.
Treat it as one cause. Read one representative’s first error and diff the changed files. - How would you use AI in debug safely?
For clustering and log reading on approved tools, with evidence required and a human confirming in the waveform.
FAQ
What is the first step in debugging a failing UVM test?
Reproduce it with the saved seed and command line, then print the solved transaction to check the stimulus was legal.
Should I always open the waveform first?
No. The waveform is door 3. Clear legality and delivery first using logs.
Why does my failure disappear when I turn on waveform dumping?
Dumping can shift timing. That is a strong hint of a race or an uninitialized value.
How do I tell a monitor bug from a design bug?
If the DUT pins show the correct value but the scoreboard disagrees, suspect the monitor or reference model.
How do I debug a simulation hang?
Find the last activity before silence, check for a dropped objection, and look at arbitration grants when every agent is requesting.
Can AI find root cause in chip verification on its own?
Not reliably yet. 2026 agents from Siemens, Synopsys and Cadence speed up log reading, clustering and waveform correlation, but all keep an engineer approving the final root cause.
What is the fastest way to triage a big regression?
Bucket failures by normalized first error, debug one seed per bucket, and diff changes since the last green run.
What is the five-door debug method?
Five questions in a fixed order: was the stimulus legal, did it reach the DUT, did the design respond correctly, did the checker judge correctly, and do the log and waveform agree. Stop at the first “no”; the root cause is behind that door.
How do I debug a UVM test that never ends?
Run with +UVM_OBJECTION_TRACE, find the component that raised last and never dropped, and keep objections in the test only. Add a timeout so the run always produces a log.
How do I group hundreds of regression failures?
Take the first error of each log, replace times, hex values and numbers with placeholders, and count identical signatures. Then debug one seed per group.
Printable debug checklists
Debug checklist (every failure)
- Reproduce with the recorded seed and command line.
- Door 1: print and judge the solved stimulus.
- Door 2: driver log vs interface pins.
- Door 3: first failing assertion, then trace backward.
- Door 4: expected vs actual, decide which to trust.
- Door 5: seed and timestamps match the log.
Testbench checklist
- Sequences print solved items and reject illegal ones.
- Driver and monitor log each transaction with a timestamp.
- Scoreboard snapshots shared state before compare.
- Message IDs are consistent and grep-friendly.
- Assertion messages name the rule that broke.
RTL checklist
- Counters and FIFO flags checked at boundaries.
- Every register has a reset value. No X after reset.
- Every clock crossing has a reviewed synchronizer.
- Internal invariants carry designer assertions.
Regression and AI checklist
- Seed and command line saved per job.
- Windowed dump by default.
- Failures bucketed by signature before triage.
- AI suggestions come with log line numbers.
- No fix without a human waveform check.
Signoff checklist
- Every root cause added an assertion or check.
- No open failure without a documented root cause.
- Coverage drops after merges were investigated.
Summary
Debug is an order, not a gift. Legality, delivery, design, checking, consistency. Print before you blame. Read the first error. Walk the waveform backward. Group before you dig. Let AI do the reading, and keep the verdict for yourself.
The engineer who finds bugs fast is rarely the smartest one in the room. Just the most orderly one. 🔍
References
- Part 8: The 2018 Wilson Research Group Functional Verification Study, Siemens Verification Horizons: debug at 44% of verification engineers’ time, up from 39% in 2016
- 2024 Siemens EDA and Wilson Research Group IC/ASIC functional verification trend report
- IEEE Std 1800-2017 (assertions, clocking blocks) and the Accellera UVM 1.2 command-line options
- Verification Academy forum threads linked in the mistakes section
Related reading and learning path
- Typical UVM Testbench Architecture
- AXI Protocol Verification Guide
- CDC Verification Guide
- Gate-Level Simulation Guide
- Formal vs functional verification
- Coverage Closure Guide
- The Thought Process Behind Great Verification
- UVM Verbosity and Reporting
- UVM TLM: Ports, Exports, Analysis Ports and FIFOs
- Functional Coverage in SystemVerilog
- UVM Interview Questions by Level
Part of the learning path Verification fundamentals

3 Comments
Comments are closed.