The second implementation of the seminar course Verification with UVM by Tampere University is finished and it is time to look back on experiences and the lessons learned. We will continue fine-tuning the course content for a new regular course System-on-Chip Verification starting in 2022.
Part of the excitement in SoC design is the anticipation of whether our product will work. The design phase is long, but the final confirmation cannot be gained before the chips arrive from the expensive fabrication process. A single bug can turn the dreams of a successful product into bankruptcy. Compared to software testing, the visibility into the final product is limited, and special measures must be taken to verify that the product would fulfill its requirements.
Verification takes time and effort
Verification is a process of ensuring that the design meets its specification. With today’s SoCs containing dozens of subsystems and billions of transistors, it is not a trivial task. In a larger SoC design, verification can be a separate project led by a dedicated team of skilled verification engineers. A recent study on functional verification by Wilson Research Group found out that companies spent 56% of project time on verification on average, but even extreme values of over 80% were reported.
A typical way to verify the functionality is to simulate the system with additional code to model the operating environment: a testbench. The testbench generates the input stimulus for the design under test (DUT) and checks the output by comparing it to a golden reference value. This method is usually referred to as functional verification. There are other methods too, such as formal verification, where the functionality correctness is proven mathematically without any simulations.
Functional verification tries to answer not only the question “Does it work” but also “How much have we tested?” To build confidence, the testbench should exercise the DUT in unimaginable ways to catch all the bugs. Functional coverage is an important metric for tracking the process.
Writing a testbench that covers all the corner cases is hard and time-consuming, while chips keep getting more complex. Verification engineers are doing all they can to keep up, but still a verification gap is forming between the design and verification productivity. Traditional RTL (register transfer level) languages such as VHDL and Verilog have been developed with design work in mind, and although already familiar to users, they are not sufficient for the increased productivity requirements.
SystemVerilog is the dominating verification language
The growth of the verification gap has been tackled on multiple fronts. Introduction of more efficient verification languages, increased level of abstraction, new paradigms, and enhanced reuse have all played their part in making modern SoC products possible. The evolution of verification languages has led to dominance of SystemVerilog, which started as a verification-targeted extension to Verilog borrowing object-oriented properties from the software languages. It also offers design-oriented improvements over the older Verilog language on the side.
A common language does not guarantee that verification engineers can work efficiently and easily share their work. Specific verification methodologies can be thought of as reference manuals and libraries of testbench building block templates. Using the methodologies ensures that verification engineers can focus on the task at hand and produce testbenches that are built to be reused. The current standard Universal Verification Methodology (UVM) is built on SystemVerilog and it has been widely adopted in the industry – over 70% of design projects have reported to be using SystemVerilog and UVM in verification.
The combination of SystemVerilog and UVM offers a wide range of tools for efficient testbench design. The key design principle is to separate the tests from the testbench structure (environment) so that different testcases with different input stimuli can be run without modifying the environment. Small subsystem-level environments can be later integrated hierarchically to a system level testbench. Multiple abstraction levels and concepts such as constrained randomization and coverage collection allow a rapid increase in test effectiveness.
Verification with UVM course
Tampere University has recognized the increasing demand for skilled verification engineers. Modern verification concepts, UVM, and the SystemVerilog language are important additions to every RTL designer’s toolbox, and therefore we started proof testing a new seminar course “Verification with UVM”. The course is targeted to master’s level students with expertise in digital design, but we offer it also for professionals in the industry through open university.
The course is arranged in seminar form, starting with lectures and continuing with student seminar presentations and hands-on exercises. The lectures introduce verification, SystemVerilog language, and the basics of UVM environment. Students then dive deeper into a single UVM concept or other verification-related topic in their presentations. There is a large variety of possible seminar topics, so that students can participate in deciding the complementary course content based on their own interests.
The hands-on exercises start with a short introduction to SystemVerilog as design language. Students are familiar with VHDL, so they already know the concepts of RTL design. This exercise introduces them to the SystemVerilog syntax of writing combinational and sequential components.
The UVM Basic exercises form the core content of the course. In this module, students build a complete UVM testbench step by step. They implement coverage collection and automatic checking into their environment, and further improve the test coverage by creating a new constrained random test. The quality of the final testbench is proven by searching for nasty bugs injected to the working DUT. Even though the sample DUT design is simple, finding these bugs would require a well-built UVM environment, determination, and some detective skills. Small bonus exercises guide the most enthusiastic students to further improve their testbench with interesting details.
The practical side of the course concludes with the dreaded Final Assignment. In this task, students first examine a real-world example of an integration level testbench that combines multiple block-level environments together, and then reuse these components for a new DUT. Most of the components needed are already there but integrating them together requires UVM expertise.
This was the second implementation of the seminar course. Students are now finishing their Final Assignment. Even though the corona restrictions forced us to work remotely, students showed their interest in the topics. The quality of presentations was outstanding, and discussion was lively! Thank you to all the participants, a seminar course like this cannot be successful without your contribution.
The final lesson on the course was that UVM is an important tool but not the silver bullet. Near the end of the course, student presentations introduced some of the famous hardware problems: the floating-point division bug in Intel Pentium processors and the Meltdown/Spectre vulnerabilities. These are examples of problems that could not have been fixed by UVM.
To broaden the view on verification, there were also presentations on assertion-based verification, formal verification, and gate-level simulation. A combination of all these concepts is required for a successful product. Knowing the basics prepares the students to work on verification projects. Finally, the presentation on verification planning gave us valuable hints on how to use the simulation tools to implement regression testing and automatically track the status of the project. This is a part of the verification project that is often overlooked.
Now when the course is finished, students have an overall view of all the most important tools used for SoC verification. They are familiar with the SystemVerilog language and know how to construct a functional testbench with UVM, but also know when to use other complementary verification tools such as formal verification. Verification is an interesting field with a variety of topics to master and the target of the course was to support studying these further. Our student presentation on Formal verification ended with the claim “A design verification engineer does not necessarily need to know UVM” – what a motivational speech!
For the next implementation, we should continue extending the course more outside the world of UVM. New technologies like Portable Stimulus Standard (PSS) have emerged to break the status quo and deserve to be mentioned. Also, hands-on work with formal verification was not possible this year, but it will surely be an interesting new exercise.
COMP.CE.420 System-on-Chip Verification starts in 2022
Based on the seminar course testing, the verification course is ready to be added as a permanent member of our course catalog. Starting from the spring 2022, we will build on these experiences and continue training new verification engineers on a new course COMP.CE.420 System-on-Chip Verification.
Verification is a challenging and rewarding field filled with discovery. The evolution of methods and tools ensure a life-long learning experience, and you have a chance to participate in the method development process with your own innovations too! In addition to all this, verification engineers typically get the best in-depth view into all the design internals and become experts on how the systems work in practice.
Welcome onboard!
Written by Arto Oinonen
