Organisatorisches Verification Technology gy Prof. Dr.-Ing. Hans - - PowerPoint PPT Presentation

1 Organisatorisches Verification Technology

gy

• Prof. Dr.-Ing. Hans Eveking

C t S t G Computer Systems Group Darmstadt University of Technology

eveking@rs.tu-darmstadt.de

2 Organisatorisches Verification Technology Computer Systems Group

gy

• Prof. Dr.-Ing. H. Eveking

"H t d i di it l h d ith t b "How to design digital hardware without bugs, and know it"

3 Organisatorisches Verification Technology

Computer Systems Group

gy

• Prof. Dr.-Ing. H. Eveking



Prerequisites:  Basic knowledge in Boolean algebra + digital circuits  Basic knowledge in Boolean algebra + digital circuits (will repeat some basics next week)



Keywords:  Digital Systems  EDA (Electronic Design Automation)  Design Methodology



References: R idl l i d d  Rapidly evolving area, no standard text  No book covers all aspects, some aspects are not covered by any book covered by any book  References will be given chapterwise

4

Verification Technology

Computer Systems Group

gy

• Prof. Dr.-Ing. H. Eveking



A few words about me ...  PhD in EE / habilitation in CS  PhD in EE / habilitation in CS  ´91-´95 professor ("Design Methodologies") in CS-

• Dept. of J.W.Goethe-University, Frankfurt

p y,  Since ´95 professor ("Computer Systems") in EE&IT

• Dept. of Darmstadt Univ. of Technology

5 Organisatorisches Verification Technology

Computer Systems Group

gy

• Prof. Dr.-Ing. H. Eveking



Lectures:  Wednesday 11.40-12.25 S3 06/052 y  Thursday 11.40-13.10 S3 06/053



Exercises:  Wednesday 12.30-13.15 S3 06/052  Start: to be announced



Printed collection of slides will be distributed or are available in Room S3 06/329 (secretary)

 also available on the web



Written exams in August-October 2011/April 2012  Simple questions, but 50% correct answers are required

6

Verification Technology

Computer Systems Group 

More verification ...

gy

• Prof. Dr.-Ing. H. Eveking



More verification ...

last 15.7. 17.10. last week Summer term Winter term 10.-14.10. Lab (0+3) We are here term "Computer Systems Lab" Verification with industrial (Siemens/Infineon/OneSpin S l ti ) t l here Guest-lecture on industrial verification: Solutions) tools

• Dr. Claudia Blank (Intel)

7

• 0. Introduction:

f

Computer Systems Group

The Verification Problem

Verification Technology

Content

0.1 What is correctness? 0 2 Protpotyping Synthesis Extraction Simulation 0.2 Protpotyping, Synthesis, Extraction, Simulation, Emulation 0.3 The Simulation Crisis 0.3 The Simulation Crisis 0.4 Formal Verification

8

0.1 What is correctness?



Example Logic-Verification: show that two circuits 0.1 What is correctness?



Example Logic-Verification: show that two circuits implement the same Boolean function a g

=1

b

1

a a

&

g a b

&

g

& &

b

&

9 0.1 What is correctness?



Correctness is relative to a specification We can not say that the network of NAND-gates — We can not say that the network of NAND-gates is correct by itself



A specification defines the meaning of correctness A specification defines the meaning of correctness — In the example, we have as a specification that the network of NAND-gates implements the XOR function g = a  b



In the following, we consider only "design correctness" (we do not discuss problems involved in the physical (we do not discuss problems involved in the physical realization of a design)



Verification establishes the correctness of a design



Verification establishes the correctness of a design

 In the hardware domain, "testing" means to

, g detect defects due to the manufacturing process

10 0.1 What is correctness?



Correctness is a logical concept



In this lecture we consider correctness only at the



In this lecture, we consider correctness only at the logic level

 Consider only digital circuits  Consider only digital circuits  No treatment of analog circuits

11 0.1 What is correctness?

1   a a

Correctness

1   a a

Laws of Logic Mind Logical Organisation Logic Logic IT- System

A d t B s d E

A

   

 

   

y N t Laws of physics Physical Nature Physical Components Analysis

12 0.1 What is correctness?

— Example: old mobile phone

Analog part Digital Signal- P RF- Interface Audio- Interface Processor Display Interface Interface Micro- Controller Display SIM card ... Digital part

13 0.1 What is correctness?



The relevance of logical design correctness will be illustrated by means of two examples: y p  Pentium-Bug  Ariane 501 Ariane 501

14 0.1 What is correctness?



The effect of the Pentium bug at Intel $

0 9 1

$ 480.000.000 loss

0 7 0,8 0,9 0,5 0,6 0,7 0,3 0,4 0,5 0,1 0,2 0,3 0,1 1Q92 3Q92 1Q93 3Q93 1Q94 3Q94 1Q95 3Q95

15 0.1 What is correctness?



The "Pentium-Bug":  FP division algorithm of 1st generation of the Pentium  FP division algorithm of 1st generation of the Pentium processor had a bug  The problem was difficult to detect (theoreticians working with very large prime numbers discovered the bug)  P bl t d t t d b f P ti I  Problem was not detected before many Pentium I were sold ...  Intel was forced to take back the erroneous chips  Intel was forced to take back the erroneous chips

 Hardware with errors is not accepted by the

community (in contrast to software ...) y ( )



Pure "logical" design error (not a physical one)



Physical exchange of Pentium-Chips was a loss of Physical exchange of Pentium Chips was a loss of $ 480.000.000 for Intel

 No patches for hardware ...

16 0.1 What is correctness?



History: Intel´s design roadmap in June 1992:

1980 1990 1985 1995

# Trans.

286

130.000

386

500.000

486 586

1.200.000 3.000.000

586 686

7.000.000

786

20.000.000

17 0.1 What is correctness?



The situation of the hardware designer:  The number of transistors per chip quadruples every 3  The number of transistors per chip quadruples every 3 years  At the same time, the time-to-market has to be reduced At the same time, the time to market has to be reduced  "Quality" software has ~ 1 undetected error per 1k LOC (lines of code) — A design of an ASIC in VHDL ( a standard hardware description language) has easily 100k LOC LOC  A redesign costs ~ 250 k€ (mask costs)  Th OS h t th fi t f t d  The OS has to run on the first manufactured processor chip!  How to get "zero-defect" VLSI ?  How to get zero-defect VLSI ?

18 0.1 What is correctness?



First launch of Ariane 501:  An overflow-situation in th i ti t the navigation computer was not handled correctly  The overflow resulted in a  The overflow resulted in a diagnosis message  The diagnosis message was The diagnosis message was wrongly interpreted as position information by the central computer computer  The central computer tried to correct the "wrong" position correct the wrong position by a sudden modification of the steering by > 20°  Cost: $ 500.000.000  "Purely logical" error

19 0.1 What is correctness?



Much more examples of design errors which were partially detected by formal verification techniques: y q  Motorola Fire-Chip Airbag-Controller: was able to fire the airbag in certain situations when the car was started  AMD K6 Processor: bug similar to Pentium  ...

20 0.1 What is correctness?



Example problems during the design of Pentium 4 (source: Bentley/Gray Intel Corp.): ( y y p )

 "RTL Coding (18.1%)—These were things like typos, cut and paste errors, incorrect assertions (instrumentation) in the SRTL code, or the designer misunderstood what he/she was supposed to the designer misunderstood what he/she was supposed to implement.  Microarchitecture (25.1%)—This covered several categories: problems in the microarchitecture definition architects not problems in the microarchitecture definition, architects not communicating their expectations clearly to designers, and incorrect documentation of algorithms, protocols, etc.  L i /Mi d Ch (18 4%) Th b th t  Logic/Microcode Changes (18.4%)—These were bugs that

• ccurred because: the design was changed, usually to fix bugs or

timing problems, or state was not properly cleared or initialized at reset or these ere b gs related to clock gating reset, or these were bugs related to clock gating.  Architecture (2.8%)—Certain features were not defined until late in the project. This led to shoehorning them into working functionality."

21 0.1 What is correctness?



Two main sources of catastrophic failures:  Complexity of algorithms  Complexity of algorithms  Unforeseen interaction of parts

22 0.1 What is correctness?

T t bilit Correctness Testability Low Cost Design Quality Low Cost Maintainability Maintainability Low Power D d bilit Low Power Consumption Dependability Security

23 0.1 What is correctness?



Correctness is essential  To avoid costly design iterations and recalls  To avoid costly design iterations and recalls  To ensure functionality in safety-critical applications

24

0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation S l th d t bt i d i t 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation



Several methods to obtain design correctness:

25 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation



Prototyping  Build one or more physical prototypes  Build one or more physical prototypes  Debug the prototypes  More desasters:  More desasters: — Mainframe design around 1980 3 prototypes debugged 3 years at 24/7/52 by — 3 prototypes, debugged 3 years at 24/7/52 by means of oscilloscopes, etc. — Problem: design complexity (asychronous Problem: design complexity (asychronous design, new pipelined design) plus unaccessibility of test points due to newly i t d d t f hi h i t t d i it introduced types of higher integrated circuits

 No prototyping of VLSI chips  No prototyping of VLSI chips

26 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation



Synthesis  Automatic generation of a hardware design by  Automatic generation of a hardware design by means of an EDA tool Specification Synthesis Hardware Implementation  Avoids manual design and the errors of human designers („correctness by construction“)  But: is the synthesis program correct?

 Synthesis results should never be trusted

27 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation



Extraction  Abstraction of a high-level specification on the  Abstraction of a high-level specification on the basis of a low-level implementation — Example: extraction of a gate-network from Example: extraction of a gate network from a transistor-netlist Specification Specification Extraction Hardware Implementation p

28 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation

Example: design flow as a combination of synthesis, extraction, verification (used at BULL, AT&T, ...): Specification Synthesis Verification FSM 1 FSM 2



RT- description Synthesis Extraction p Synthesis Transistor netlist Transistor netlist Synthesis Layout Extraction

29 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation



Simulation  "Virtual prototype" in the computer  Virtual prototype in the computer  Problems: Development of simulation stimuli (~test cases) — Development of simulation stimuli (~test cases) — Interpretation of simulation results Incompleteness — Incompleteness Inspection by designer Stimuli Results g Simulator Stimuli Results

30 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation

 Example: ALU: 2*32 data-, 13 control-inputs

 77 inputs

277 1 5*1023 cases!

 77 inputs  277 ~ 1.5*1023 cases!

— 1 GHz Pentium, 1ns per case, ~ 3.15*1016 cases per year P lif ti 4 — Processor lifetime ~ 4 years

 Corresponds to 8.3*10-7 of all possible cases!  A ALU

b i l t d h ti l !

 A ALU can never be simulated exhaustively !

32



32



32 13

31 0.2 Prototyping, Synthesis, Extraction, Simulation, Emulation



Emulation: use hardware accelerators  Replace (parts of) the software model by special  Replace (parts of) the software model by special hardware  Employ "real" hardware Employ real hardware  Use programmable, FPGA-based hardware — Execution time ~ execution time of real design Execution time execution time of real design — Observability as good as with a simulator

32

0.3 The Simulation Crisis 0.3 The Simulation Crisis # Test-Vectors Simulation time Simulation time # G t # Gates

33 0.3 The Simulation Crisis



Remark:  In the hardware domain "testing" means "normally":  In the hardware domain, testing means normally : check for manufacturing defects, not design errors!

 Hardware testing is a very interesting and

a d a e test g s a e y te est g a d challenging research area with extremely important industrial applications!  However, talking about simulation techniques it is common to use terms like "test", "test cases", etc. also if we look for design errors also if we look for design errors

34 0.3 The Simulation Crisis

RTL D i RT-level simulation RTL D i RT-level simulation RTL-Design #1 RTL-Design #2 Gate-level Gate-level Gate-level Gate level simulation Gate level simulation Gate level simulation

Gate-level

Gate-level simulation Gate-level simulation Gate-level simulation simulation simulation simulation Scan-path, clock-tree insertion Manual change

Gate-level

35 0.3 The Simulation Crisis



Example: Unisys ECL  CMOS redesign of a processor of 395,000 gates , g



Time spent after one engineering change:  5-12 days design of test vectors 5 12 days design of test vectors  1-2 days of simulation  1-3 days analysis of simulation results  1 3 days analysis of simulation results

36 0.3 The Simulation Crisis



Extremely high cost of simulation on "computer-farms"

 MIPS R 4000: 100 Workstations on 200 days (Hennessy 1995)  MIPS R 4000: 100 Workstations on 200 days (Hennessy, 1995)  IBM: 500 Workstations during design time



Verilog RTL source for SUN SPARC microprocessor



Verilog RTL source for SUN SPARC microprocessor  1995: 300k lines  1999: 1 8M lines 6x increase  1999: 1.8M lines 6x increase



Verilog simulation for SUN SPARC microprocessor  1995: 2*109 cycles  1995: 2*109 cycles  1999: 200+ *109 cycles, 100x increase!!!

 I t l P

ti 4 l th d k t ti

 Intel Pentium 4: several thousand work stations — 2.4*1011 RT-simulation cycles

 50 t

100 t ti !

 50 to 100 years computer time !

Source: Anant Agrawal, Sun Microsystems, presentation to EDAC Meeting April 11, 2000rh

37

Source: R. Camposano, EDA Forum 2002

38

Source: R. Camposano, EDA Forum 2002

39 0.3 The Simulation Crisis



10 ... 1000 cycles per second are typical simulation speeds  Boot-procedure of an operating systems amounts to  Boot-procedure of an operating systems amounts to several weeks of simulation time



MPEG-2 Video Processor MPEG 2 Video Processor  Decoding one frame ~ 3M clock cycles  Logic simulation ~ 16 days  Logic simulation 16 days



Echo-Canceling ASIC  1 minute real-time ~ 1 year simulation time  1 minute real time 1 year simulation time



100MHz Pentium  Boot procedure = 6*109 cyles ~ 19 years of simulation  Boot procedure = 6 10 cyles 19 years of simulation

40 0.3 The Simulation Crisis



The cost for verification today is 70-80% of the total design cost g

41 0.3 The Simulation Crisis



Simulation techniques  Directed test  Directed test — Hand-coded 0/1 sequences of input values +: You can start immediately! — +: You can start immediately! — -: Difficult detection of corner cases : Incompleteness — -: Incompleteness Inspection by designer Stimuli Results g DUV Stimuli Results DUV

(Design Under Verification)

42 0.3 The Simulation Crisis

 Testbench creation — The testbench replaces the environment of the design under verification Written e g in VHDL or Verilog — Written, e.g., in VHDL or Verilog — +: Improved coverage Additi l t k b ll > 100k li f d — -: Additional task, may be well > 100k lines of code — -: Start of verification delayed Testbench DUV DUV

(Design Under Verification)

43 0.3 The Simulation Crisis

 Constrained random simulation — Automatic generation of simulation stimuli — Stimuli = random pattern observing specified constraints constraints — +: Creation of simulation stimuli much easier + M id d — +: More cases considered — -: Corner cases difficult to reach (20-input AND) Testbench Input t DUV generator DUV

(Design Under Verification)

Driver Results

44 0.3 The Simulation Crisis

 Monitors — Automatic inspection of simulation results — Reports violation of desired behavior specified by properties (assertions) or automata by properties (assertions) or automata — +: No inspection of simulation results by designer necessary designer necessary Testbench Input t Monitor DUV generator Monitor DUV

(Design Under Verification)

Driver Results

45 0.3 The Simulation Crisis

— Very simple example: write 3 values A, B, and C into registers X, Y, and Z so that X > Y > Z X, Y, and Z so that X > Y > Z Testbench Random t Monitors X>Y>Z Sorting generator Results X>Y>Z Sorting Device A,B,C Results for X, Y, Z

46 0.3 The Simulation Crisis

Coverage 

Simulation Coverage

Coverage

100%

Simulation Coverage  When can I stop testing?  Coverage metrics

design time

 Coverage metrics  Different types of coverage metrics (cf. software testing): g) — Code coverage (addresses code execution, statement/condition/path coverage) — Assertion coverage (addresses assertion activation) F ti l ( dd d i i t t) — Functional coverage (addresses design intent) — ...

47

0.4 Formal Verification 0.4 Formal Verification



Simulation is also based on a formal model, e.g., a VHDL description of the Hardware



Formal verification = mathematical modeling + exact calculations as in other engineering disciplines as in other engineering disciplines



Formal verification promises  Qualitatively better results  Qualitatively better results  Completeness  Faster results  Faster results

 Better tools and techniques

48 0.4 Formal Verification



Formal verifcation requires a precise formal specification as a definition of correctness p g = a  b a a

&

g a b

&

g

& &

b

&

49 0.4 Formal Verification



Given a specification, a complete proof of correctness can be given by, e.g., deriving canonical representations g y, g , g p S ifi ti b Specification 1 1 a g = a  b

=

1 a 1 b a a

&

g 1 1 a a b

&

g

& &

Implementation b

&

50 0.4 Formal Verification

Problem: exponential growth

• f the representation in

x3

• f the representation in

the # variables

1 3 5 4 7 9 8 11 13 12 15 25 24 27 29 28 31 17 16 19 21 20 23 x0 2 3 6 7 10 11 14 15 26 27 30 31 18 19 22 23 x2 x1 x2 x4 4 8 12 x3 x2 1 2 3 5 6 7 9 10 11 13 14 15 x0 x1 1 2 3 5 4 6 7 x0 2 x1 2 6 10 14 x2 x1 1 3

x0

2 3 4 5 ...

51 0.4 Formal Verification



One possible solution (Chapter 1): representation by means

• f Binary Decision Diagrams (BDD's)

S ifi ti a

1

y g ( ) — Linear in the # variables for many circuits Specification g = a  b b b

1 1

=

1 a a

&

g

=

a

1

a b

&

g

& &

b b

1 1

Implementation b

&

1

52

RT-level simulation + property verification RT-level simulation + property verification RTL-Design + property verification RTL-Design + property verification Formal Verification RTL-Design #1 #2 Implementation verification Gate-level Design #1 1 Gate-level Design #1 2 Gate-level Design #1 3 Scan-path, clock-tree insertion Manual change #1.1 #1.2 #1.3 Equivalence Gate-level Design #1 1 Gate-level Design #1 2 Gate-level Design #1 3 Equivalence verification Scan-path, clock-tree insertion Manual change #1.1 #1.2 #1.3

53 0.4 Formal Verification



3 basic types of formal verification  Implementation verification  Implementation verification g = a  b Specification Implementation a g = a  b Implementation Verification a a

&

g

& &

Implementation b b

& & &

b

54 0.4 Formal Verification

 Equivalence Verification Implementation 1 Implementation 2 Equivalence Verification

 Equivalence verification works on the

Verification

 Equivalence verification works on the

same level, e.g., gate-level

 Implementation and equivalence

p q verification relate two complete descriptions

55 0.4 Formal Verification

 Property Verification Property

Never: all lights are green

Property Verification Implementation Controller

56 0.4 Formal Verification



State-of-the-art:  Equivalence verification  Equivalence verification — Most successful! Industrially used for circuits with many millions of — Industrially used for circuits with many millions of gates — Not done by simulation anymore (would take Not done by simulation anymore (would take weeks)  Implementation verification — Industrially used to check RT- (register-transfer-)

• vs. gate-level synthesis results (~ equivalence ver.)

— But: not able to cope with many synthesis

• ptimizations

57 0.4 Formal Verification



State-of-the-art (cont'd.):  Property verification  Property verification — Model-checking successfully used for hardware as well as for software well as for software — Completeness problem: when do I have written enough properties? — What are the essential properties of a design?

58 0.4 Formal Verification



A simplistic view of the current design situation RT L l

VDHL, Verilog

Block Block Chip Level

Blocks of

Gate Level RT Level Block #1 Block #n

• c s o

500k-1M gates

Layout Level Gate Level

Automated synthesis

Layout Level

59 0.4 Formal Verification



The V-model of a design process

System S ifi ti C, SystemC, SystemVerilog, Word Specification Word, ...

RT L l

VDHL, Verilog

Block Block Chip Level

Blocks of

Gate Level RT Level Block #1 Block #n

• c s o

500k-1M gates

Layout Level Gate Level

Automated synthesis

Layout Level

60 0.4 Formal Verification



A simplistic view of the current design situation

System S ifi ti C, SystemC, SystemVerilog, Word Specification Word, ...

RT L l

VDHL, Verilog

Block Block Chip Level

Blocks of

Gate Level RT Level Block #1 Block #n

• c s o

500k-1M gates

Layout Level Gate Level

Automated synthesis Equivalence verification

Layout Level

61 0.4 Formal Verification



A simplistic view of the current design situation

Simulation of C, SystemC, SystemVerilog, Word Simulation of the complete chip System S ifi ti Word, ... Property verification

• f a block

Specification

RT L l

VDHL, Verilog

Block Block Chip Level

Blocks of

• f a block

Gate Level RT Level Block #1 Block #n

• c s o

500k-1M gates

Layout Level Gate Level

Automated synthesis Equivalence verification

Layout Level

62 0.4 Formal Verification



The „Formal Verification Community“ EDA Companies

(Cadence, Synopsys, Mentor, Verisity 0-In OneSpin )

Manufacturers

(Intel, IBM, Fujitsu, Motorola, Infineon ) Verisity, 0-In, OneSpin, ...) Infineon, ...)

Research groups at g p Universities

63 0.4 Formal Verification



"Formal verification groups" to support designers and develop new techniques and methodologies (Intel, AT&T, p q g ( , , IBM, Motorola, HP, Compaq, Siemens, Infineon, Philips, Fujitsu, Bosch, ...)

64

Verification Technology

Fachgebiet Rechnersysteme

gy

• Prof. Dr.-Ing. H. Eveking

How to show the correctness of

Content



Introduction: The Verification Problem



Logic-Verification correctness of Gate networks



Bit-Vector and Word-Level Verification



Sequential Circuit Verification Gate networks Circuits with



Model-Checking



Verification of Processors Circuits with storage elements



... Big circuits!

65

Verification Technology

Fachgebiet Rechnersysteme

gy

• Prof. Dr.-Ing. H. Eveking

Objectives



Understand the verification problem



Understand basic solutions of the verification problem



Understand basic solutions of the verification problem



Understand the basic principles of EDA (electronic design automation) algorithms used for verification design automation) algorithms used for verification



Learn to apply verification techniques to designs



Learn to assess the chances and limitations of modern



Learn to assess the chances and limitations of modern verification technology