Why formal verification remains on the fringes of commercial development Arvind Computer Science & Artificial Intelligence Laboratory Massachusetts Institute of Technology WG2.8, Park City, Utah June 16, 2008 May 27, 2008 L1-1 http://csg.csail.mit.edu/arvind
A designer’s perspective The goal is to design systems that meet some criteria such as cost, performance, power, compatibility, robustness, … The design effort and the time-to- market matter ($$$) Can formal methods help? May 27, 2008 L1-2 http://csg.csail.mit.edu/arvind
Examples IP Lookup in a router challenging Increasingly 802.11a Transmitter H.264 Video Codec OOO Processors Cache Coherence Protocols May 27, 2008 L1-3 http://csg.csail.mit.edu/arvind
Example 1: Simple deterministic functionality Internet router LC Line Card (LC) Arbitration Packet Processor Control SRAM Processor (lookup table) Switch Queue IP Lookup Manager Exit functions A packet is routed based on the “Longest Prefix Match” (LPM) of it’s IP address with LC entries in a routing table Line rate and the order of LC arrival must be maintained line rate ⇒ 15Mpps for 10GE May 27, 2008 L1-4 http://csg.csail.mit.edu/arvind
“C” version of LPM int 0 lpm (IPA ipa) 0 0 … /* 3 memory lookups */ … { int p; … 0 /* Level 0: 8 bits */ p = RAM [ipa[31:24]]; … 2 8 -1 if (isLeaf(p)) return value(p); /* Level 1: 8 bits */ p = RAM [ipa[23:16]]; if (isLeaf(p)) return value(p); /* Level 2: 8 bits */ Not obvious from the C p = RAM [ptr(p) + ipa [15:8]]; if (isLeaf(p)) return value(p); code how to deal with /* Level 3: 8 bits */ - memory latency p = RAM [ptr(p) + ipa [7:0]]; return value(p); - pipelining /* must be a leaf */ } Memory latency Must process a packet every 1/15 µ s or 67 ns ~30ns to 40ns Must sustain 4 memory dependent lookups in 67 ns Real LPM algorithms are more complex May 27, 2008 L1-5 http://csg.csail.mit.edu/arvind
An implementation: Circular pipeline outQ inQ yes enter? RAM done? no fifo Does the look up produce the right answer? Easy: check it against the C program Performance concern: Are there any “dead cycles”? Has direct impact on memory cost Do answers come out in the right order? Is it even possible to express in a given logic? Alternative: The designer tags input messages and checks that the tags are produced in order May 27, 2008 L1-6 http://csg.csail.mit.edu/arvind
Example 2: Dealing with Noise 802.11a Transmitter headers 24 data Controller Scrambler Encoder Uncoded bits Interleaver Mapper Cyclic IFFT Extend Must produce one OFDM symbol accounts for 85% area (64 Complex Numbers) every 4 µ sec May 27, 2008 L1-7 http://csg.csail.mit.edu/arvind
Verification Issues Control is straightforward Small amounts of testing against the C code is sufficient, provided the arithmetic is implemented correctly C code may have to be instrumented to capture the intermediate values in the FIFOs No corner cases in the computation in various blocks High-confidence with a few correct packets Still may be worthwhile proving that the (non standard) arithmetic library is implemented correctly May 27, 2008 L1-8 http://csg.csail.mit.edu/arvind
802.11a transceiver: Higher-level correctness Does the receiver actually recover the full class of corrupted packets as defined in the standard? Designers totally ignore this issue This incorrectness is likely to have no impact on sales Who would know? If we really wanted to test for this, we could do it by generating the maximally-correctable corrupted traffic All these are purely academic questions! May 27, 2008 L1-9 http://csg.csail.mit.edu/arvind
Example 3: Lossy encodings H.264 Video Decoder Errors don’t Parse Inverse Compresse NAL matter much + Quant d Bits unwrap CAVLC Transformation Frames Inter Intra Deblock Prediction Prediction Filter Ref Frames The standard is 400+ pages of English; the standard implementation is 80K lines of convoluted C. Each is incomplete! Only viable correctness criterion is bit-level matching against the reference implementation on sample videos Parallelization is more complicated than what one may guess based on the dataflow diagram because of data-dependencies and feedback May 27, 2008 L1-10 http://csg.csail.mit.edu/arvind
H.264 Decoder: Implementation Parse Inverse Compresse NAL + Quant d Bits unwrap CAVLC Transformation Frames Inter Intra Deblock Prediction Prediction Filter Ref Frames Different requirements for different environments QVGA 320x240p (30 fps) DVD 720x480p HD DVD 1280x720p (60-75 fps) Each context requires a different amount of parallelism in different blocks Modular refinement is necessary Verifying the correctness of refinements requires traditional formal techniques (pipeline abstraction, etc.) May 27, 2008 L1-11 http://csg.csail.mit.edu/arvind
Example 4: Absolute Correctness is required Microprocessor design Register Empty E File Waiting W Dispatched Di Get operands Writeback for instr Killed K results Done Do Re-Order Buffer State Instruction Operand 1 Operand 2 Result Instr - V - V - - E E Instr - V - V - - Get a ready Head W Instr A V 0 V 0 - ALU instr ALU W Instr B V 0 V 0 - Unit(s) W Instr C V 0 V 0 - Put ALU instr Decode Insert an W Instr D V 0 V 0 - results in ROB instr into Tail E Unit Instr - V - V - - ROB E Instr - V - V - - E Instr - V - V - - Get a ready MEM instr E Instr - V - V - - MEM Instr - V - V - - E Unit(s) Resolve E Instr - V - V - - Put MEM instr branches Instr - V - V - - E results in ROB E Instr - V - V - - Instr - V - V - - E E Instr - V - V - - May 27, 2008 L1-12 http://csg.csail.mit.edu/arvind
“Automated” Processor Verification Models are abstracted from (real) designs UCLID – Bryant (CMU) : OOO Processor hand translated into CLU logic (synthetic) Cadence SMV - McMillian : Tomasulo Algorithm (hand written model. synthetic) ACL – Jay Moore: (Translate into Lisp) … Some property of the manually abstracted model is verified Great emphasis (and progress) on automated decision procedures Since abstraction is not automated it is not clear what is being verified! BAT[Manolios et al] is a move in the right direction May 27, 2008 L1-13 http://csg.csail.mit.edu/arvind
Automatic extraction of abstract models from designs expressed in Verilog or C or SystemC is a lost cause May 27, 2008 L1-14 http://csg.csail.mit.edu/arvind
Example 5: nondeterministic specifications Cache Coherence It took Joe Stoy more than 6 months to learn PVS and show that some of the proofs in Xiaowei Shen’s thesis were correct This technology is not ready for design engineers May 27, 2008 L1-15 http://csg.csail.mit.edu/arvind
Model Checking CC is one of the most popular applications of model checking The abstract protocol needs to be abstracted more to avoid state explosion For example, only 3 CPUs, 2 addresses There is a separate burden of proof why the abstraction is correct Nevertheless model checking is a very useful debugging aid for the verification of abstract CC protocols May 27, 2008 L1-16 http://csg.csail.mit.edu/arvind
Implementation Design is expressed in some notation which is NOT used directly to generate an implementation The problem of verification of the actual protocol remains formidable Testing cannot uncover all bugs because of the huge non-deterministic space Proving the correctness of cache coherence protocol implementations remains a challenging problem May 27, 2008 L1-17 http://csg.csail.mit.edu/arvind
Summary The degree of correctness required depends upon the application The real success of a formal Different applications require vastly different formal and informal techniques technique is when it is used ubiquitously without the Formal tools must be tied directly to high-level design languages designer being aware of it e.g., type systems Formal techniques should be presented as debugging aids during the design process A designer is unlikely to do any thing for the sake of helping the post design verification May 27, 2008 L1-18 http://csg.csail.mit.edu/arvind
Recommend
More recommend
