Microsoft PowerPoint – HW_Verification_Lecture_5
Hardware Verification –
Copyright By PowCoder代写 加微信 powcoder
Autumn term 2022
• Introduction to Formal Verification
• What is formal verification?
• How formal verification algorithms work
• Properties
• Assertions
• Cover points
• Constraints
• Assumptions
• Clock and reset definitions
• Proven properties
• Bounded or inconclusive proofs
• Counterexamples for properties that fail or cover points that are not reachable
• What is coverage?
• Code Coverage
• Functional Coverage
• Statistical Coverage
• Collecting coverage in Questasim
• Coverage demo using Questasim
What is Coverage?
• The most effective way to verify complex designs is by using
constrained random testing
• But how do you know how good your random testing is?
• Coverage is a measure of which features of your design have been
• It also lets you measure progress towards your goal of complete verification
• First step is to write a Verification Plan
• Make a list of what needs testing and how you will test it
• This will guide you in writing Functional Coverage points (see later)
Coverage Flow
Specification
Verification
Design Coverage
Coverage AnalysisDebug
From: Spear & Tumbush, SystemVerilog for Verification
Types of Coverage
CODE COVERAGE
Automatically collected by simulator for “free”
Collects information about the lines of RTL
Lines hit
Branches hit
Expressions hit
Signal toggles hit
States and transitions in a FSM
FUNCTIONAL COVERAGE
Automatically collected by simulator
based on user input
Collects information about the user-
defined scenarios or requirements
Shows success (or failure) of constrained
random (or directed) stimulus
Allows linkage of requirements that span
multiple features
STATISTICAL COVERAGE
Collects engineer defined statistics within a simulation and across a group of simulations
Enables a greater understanding of what is really happening within the simulations
Enables fine tuning of constraints to improve stimulus
Functional/code coverage may show that we have executed all instructions, but might not
show that we have executed 1 instruction 1 billion times and all instructions 100 times
always @* begin
if (a==b) begin
z = x + y;
else if ((a > b) || (a > c)) begin
z = x – y;
else begin
Code Coverage
// Statement
// Statement
// Statement
// Statement
// Statement
// Branch (if)
// Branch (else) // Expression
// Condition
// Condition
Good initial metric of RTL code, but does not tell us about the interactions/relationships
between unrelated pieces of code.
Code Coverage – what does it tell us?
• Code coverage measures how thoroughly your tests exercise the RTL
• But it doesn’t say anything about covering your Verification Plan
• It won’t find bugs in your implementation
• It won’t tell you if you’ve forgotten to implement something
• For example, you might forget to implement the SUB instruction in an
• If you never test your design with a subtract operation, you may still get 100%
code coverage
• So 100% code coverage is a basic requirement
• But you also need functional coverage and checkers for effective verification
Functional Coverage
• ADD r0, r1, r2
• SUB r4, r5, r0
• MUL r7, r8, r9
• Functional Coverage is a user-written encapsulation of a requirement of the DUT.
• It covers testing against the specification – will the design behave correctly?
• The amalgamation of all these requirements gives us our overall functional coverage
• The following example is a simple 6 instruction, 10 register CPU:
Instruction Coverage = 50% hit
ADD SUB MUL AND OR NOT
Destination Coverage = 30% hit
r0 r4 r7 r8 r9r1 r2 r3 r5 r6
Functional Coverage Flow
Specification
Verification
Design Coverage
Coverage AnalysisDebug
From: Spear & Tumbush, SystemVerilog for Verification
Metrics – Tracking the Bug Rate Curve
1st Release
Day number
Functional Coverage Strategies
• Think about what needs testing
• FIFO – test empty and full states, not every state of FIFO
• Counter – test that it starts counting, test what happens when it overflows
• Be economical with functional coverage points
• You will need to analyse the reports!
• Measure progress towards completion
Functional versus Code Coverage
% FUNCTIONAL COVERAGE
Good coverage –
but check bug rate
Start of project –
more testing
Is all functionality
functional coverage
points – or there is
redundant code
Statistical Coverage
Analyse data collected during simulations, to ensure a good spread of coverage
Enables more efficient use of verification cycles and compute resource
Enables a greater understanding of what is really happening within a simulation
Enables fine tuning of constraints to improve stimulus
Makes use of visualisation and data science
Instruction
ADD SUB CMP MUL DIV
Coverage: Are we done?
• “Once we get to 100% Code Coverage and 100% Functional Coverage, are we
• No! Functional Coverage is only as good as the coverage points you define.
There may be bugs in scenarios or relationships between features that have not
been thought of and have not yet been stimulated by the constrained random
• “How do we solve this?”
• We can’t ever prove that we’re finished and there are no more bugs – the large
state space and high complexity of most designs makes this impossible
• We need to demonstrate a very high level of confidence in the robustness of the
• Deciding that we have reached the required level of confidence is very hard
• Metrics are the key to successful verification
Writing Functional Coverage – Cover Points
• Cover points sample values of variables and expressions
• Multiple cover points form a cover group
• Multiplier example – Verification Plan says to test all operand values
• So declare a class for the random variables
program automatic multiplier_tb
#(parameter WIDTH = 5) // Multiplier width in bits
(mult_if.TEST multif); // Uses an interface
class Operand_values; // Declare a class to randomise the operand values
rand logic [WIDTH-1:0] i;
rand logic [WIDTH-1:0] j;
Operand_values opvals; // Instantiate the class
Writing Cover Groups
• Write a cover group to cover range of values of operand ‘a’
• Can create user-defined bins; otherwise default bins are used
• Use separate bins for “corner cases” – in this case zero and max value
covergroup cover_a_values; // Cover point to check range of operand a values
coverpoint multif.a {
bins zero = {0};
bins lo = {[1:7]};
bins med = {[8:23]};
bins hi = {[24:30]};
bins max = {31};
Writing Cover Groups
• Instantiate cover group in testbench code
• Create random values of both operands
• Gather coverage with
initial begin
cover_a_values cova; // Cover point to be sampled
cova = new(); // Instantiate cover group
opvals = new(); // Allocate a random operand values object
@multif.cb; // Wait for a positive clock edge
assert (opvals.randomize) else $fatal; // Create random operands
multif.a=opvals.i; // Drive multiplier with random values
multif.b=opvals.j;
cova.sample(); // Collect coverage of operand a values
More about coverage bins
• SystemVerilog will create automatic bins if you don’t define them
• Limit the number of automatic bins with auto_bin_max
• In example shown, this would create 2 bins for the 32 possible values
of operand ‘a’
• 1st bin with values 0-15
• 2nd bin with values 16-31
covergroup cover_a_values;
coverpoint multif.a { option.auto_bin_max = 2;}
Conditional coverage
• You can turn off coverage with the ‘iff’ keyword
• For example, to stop collecting coverage during reset
• Or you can explicitly start and stop coverage
covergroup cover_a_values;
coverpoint multif.a iff (multif.rst_n);
cova = new(); // Instantiate cover group
cova.stop();
wait (multif.rst_n == 1);
cova.start();
Transition coverage
• State transitions can be specified for a cover point
• Multiple transitions can be specified using ranges, e.g. (0,1 => 2,3)
• Here we are checking whether operand ‘a’ goes from 31 to 0 or from
covergroup cover_a_transitions;
coverpoint multif.a (
bins trans = (31 => 0), (0 => 31)};
Wildcards and ignore_bins
• Use wildcard to group multiple states
• Use ignore_bins to exclude values from coverage measurement
covergroup cover_a_transitions;
coverpoint multif.a {
wildcard bins even_values = {5’b????0}; }
covergroup cover_a_transitions;
coverpoint multif.a {
ignore_bins lowvalues = {0,1,2,3};
Cross Coverage
• Use cross coverage to measure what values were seen for 2 or more cover
points at the same time
• For example, ALU operation types crossed with source registers
• Needs NxM cross bins if one variable has N values and the other M values
class Operand_values;
rand logic [WIDTH-1:0] i;
rand logic [WIDTH-1:0] j;
Operand_values opvals;
covergroup cross_a_and_b;
vala: coverpoint opvals.i;
valb: coverpoint opvals.j;
cross vala, valb;
Excluding cross coverage bins
• You can exclude bins using ignore_bins
• But with cross coverage, you specify the cover point with binsof and the
values to ignore with intersect
covergroup cross_a_and_b;
vala: coverpoint opvals.i;
valb: coverpoint opvals.j;
cross vala, valb {
ignore_bins high_ab = binsof (vala) intersect {16:31} &&
binsof (valb) intersect {16:31};
Collecting coverage in Questasim
• Compile code with +cover switch to enable code coverage
• Run simulation with –coverage switch to collect code coverage
vlog -work work +acc=blnr +cover -noincr -timescale 1ns/1ps -f multiplier.vc
vopt -work work multiplier_top -o work_opt
vsim -coverage work_opt -gui
Reporting coverage in Questasim
To get code coverage report:
Tools -> Coverage Report ->
(creates a report.txt or report.html)
To get functional coverage report:
Tools -> Coverage Save
(saves a ucdb file on exit)
vcover report -details -html multiplier.ucdb
vcover report -details multiplier.ucdb
Coverage HTML Report
• Code coverage and functional coverage are important metrics
• Coverage on its own is not enough
• Complemented by checkers, formal verification, directed tests
• Statistical coverage can be used to check the spread of coverage
• Statistical coverage is not required for the main h/w coursework
Coverage – Summary
Software Verification lecture next week,
Thursday 17th November
Dr Wickerson will present the lecture on Teams
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com