Proving a Concurrent Program Correct by Demonstrating It Does - PowerPoint PPT Presentation

Proving a Concurrent Program Correct by Demonstrating It Does Nothing Bernhard Kragl IST Austria doi: 10.1007/978-3-319-96145-3_5 https://github.com/boogie-org/boogie Shaz Qadeer Microsoft https://www.rise4fun.com/civl

Credits Cormac Flanagan Stephen N. Freund Serdar Tasiran Tayfun Elmas Chris Hawblitzel

Program verification Program 𝒬 Verifier Is 𝒬 safe? Error report Proof

Program verification Invariants Program Templates 𝒬 Proof structure … Verifier Is 𝒬 safe? Error report Proof

Reasoning about transition systems • Transition system 𝑊𝑏𝑠, 𝐽𝑜𝑗𝑢, 𝑂𝑓𝑦𝑢, 𝑇𝑏𝑔𝑓 𝑊𝑏𝑠 (Variables) 𝐽𝑜𝑗𝑢 (Initial state predicate over 𝑊𝑏𝑠 ) 𝑂𝑓𝑦𝑢 (Transition predicate over 𝑊𝑏𝑠 ∪ 𝑊𝑏𝑠′ ) 𝑇𝑏𝑔𝑓 (Safety predicate over 𝑊𝑏𝑠 ) • Inductive invariant 𝐽𝑜𝑤 𝐽𝑜𝑗𝑢 ⇒ 𝐽𝑜𝑤 (Initialization) 𝐽𝑜𝑤 ∧ 𝑂𝑓𝑦𝑢 ⇒ 𝐽𝑜𝑤 ′ (Preservation) 𝐽𝑜𝑤 ⇒ 𝑇𝑏𝑔𝑓 (Safety)

Structured program vs. Transition relation Init: 𝑞𝑑 = 𝑞𝑑 1 = 𝑞𝑑 2 = 𝑏 Next: 𝑞𝑑 = 𝑏 ∧ 𝑞𝑑 ′ = 𝑞𝑑 1 ′ = 𝑞𝑑 2 ′ = 𝑐 ∧ 𝑦 ′ = 0 ∧ 𝑓𝑟 𝑚, 𝑢 1 , 𝑢 2 a: x := 0 ′ = 𝑑 ∧ ¬𝑚 ∧ 𝑚 ′ ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 2 , 𝑦, 𝑢 1 , 𝑢 2 𝑞𝑑 1 = 𝑐 ∧ 𝑞𝑑 1 b: acquire(l) acquire(l) ′ = 𝑒 ∧ 𝑢 1 ′ = 𝑦 ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 2 , 𝑚, 𝑦, 𝑢 2 𝑞𝑑 1 = 𝑑 ∧ 𝑞𝑑 1 c: t1 := x t2 := x ′ = 𝑓 ∧ 𝑢 1 ′ = 𝑢 1 + 1 ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 2 , 𝑚, 𝑦, 𝑢 2 𝑞𝑑 1 = 𝑒 ∧ 𝑞𝑑 1 d: t1 := t1+1 t2 := t2+1 ′ = 𝑔 ∧ 𝑦 ′ = 𝑢 1 ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 2 , 𝑚, 𝑢 1 , 𝑢 2 𝑞𝑑 1 = 𝑓 ∧ 𝑞𝑑 1 e: x := t1 x := t2 ′ = 𝑕 ∧ ¬𝑚 ′ ∧ 𝑓𝑟(𝑞𝑑, 𝑞𝑑 2 , 𝑦, 𝑢 1 , 𝑢 2 ) 𝑞𝑑 1 = 𝑔 ∧ 𝑞𝑑 1 f: release(l) release(l) ′ = 𝑑 ∧ ¬𝑚 ∧ 𝑚 ′ ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 1 , 𝑦, 𝑢 1 , 𝑢 2 𝑞𝑑 2 = 𝑐 ∧ 𝑞𝑑 2 ′ = 𝑒 ∧ 𝑢 2 ′ = 𝑦 ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 1 , 𝑚, 𝑦, 𝑢 1 g: assert x = 2 𝑞𝑑 2 = 𝑑 ∧ 𝑞𝑑 2 ′ = 𝑓 ∧ 𝑢 2 ′ = 𝑢 2 + 1 ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 1 , 𝑚, 𝑦, 𝑢 1 𝑞𝑑 2 = 𝑒 ∧ 𝑞𝑑 2 ′ = 𝑔 ∧ 𝑦 ′ = 𝑢 2 ∧ 𝑓𝑟 𝑞𝑑, 𝑞𝑑 1 , 𝑚, 𝑢 1 , 𝑢 2 𝑞𝑑 2 = 𝑓 ∧ 𝑞𝑑 2 ′ = 𝑕 ∧ ¬𝑚 ′ ∧ 𝑓𝑟(𝑞𝑑, 𝑞𝑑 1 , 𝑦, 𝑢 1 , 𝑢 2 ) 𝑞𝑑 2 = 𝑔 ∧ 𝑞𝑑 2 Procedures and dynamic thread creation 𝑞𝑑 1 = 𝑞𝑑 2 = 𝑕 ∧ 𝑞𝑑′ = 𝑕 ∧ 𝑓𝑟(𝑞𝑑 1 , 𝑞𝑑 2 , 𝑚, 𝑦, 𝑢 1 , 𝑢 2 ) complicate transition relation further! Safe: 𝑞𝑑 = 𝑕 ⇒ 𝑦 = 2

Interference freedom and Owicki-Gries Ψ 1 : 𝑄 1 𝐷 1 {𝑅 1 } Ψ 2 : 𝑄 2 𝐷 2 𝑅 2 Ψ 1 , Ψ 2 interference free 𝑄 1 ∧ 𝑄 2 𝐷 1 ∥ 𝐷 2 𝑅 1 ∧ 𝑅 2 • Example: 𝑦 = 0 𝑦 ≔ 𝑦 + 1 ∥ 𝑦 ≔ 𝑦 + 2 𝑦 = 3 𝑦 = 0 𝑦 = 0 𝑦 = 0 Interference freedom: 𝑄 1 : {𝑦 = 0 ∨ 𝑦 = 2} 𝑄 2 : {𝑦 = 0 ∨ 𝑦 = 1} 𝑄 1 ∧ 𝑄 2 𝑦 ≔ 𝑦 + 2 𝑄 𝑄 2 ∧ 𝑄 1 𝑦 ≔ 𝑦 + 1 𝑄 2 1 𝑦 ≔ 𝑦 + 1 𝑦 ≔ 𝑦 + 2 𝑅 1 ∧ 𝑄 2 𝑦 ≔ 𝑦 + 2 𝑅 1 𝑅 2 ∧ 𝑄 1 𝑦 ≔ 𝑦 + 1 𝑅 2 𝑅 1 : 𝑦 = 1 ∨ 𝑦 = 3 𝑅 2 : 𝑦 = 2 ∨ 𝑦 = 3 𝑅 1 ∧ 𝑅 2 𝑦 = 3

Ghost variables Need to refer to other thread’s state • local variables • program counter • Example: 𝑦 = 0 𝑦 ≔ 𝑦 + 1 ∥ 𝑦 ≔ 𝑦 + 1 𝑦 = 2 𝑦 = 0 𝑒𝑝𝑜𝑓 1 ≔ 𝑔𝑏𝑚𝑡𝑓; 𝑒𝑝𝑜𝑓 2 ≔ 𝑔𝑏𝑚𝑡𝑓 𝑄 1 : ¬𝑒𝑝𝑜𝑓 1 ∧ ¬𝑒𝑝𝑜𝑓 2 ⇒ 𝑦 = 0 ∧ 𝑒𝑝𝑜𝑓 2 ⇒ 𝑦 = 1 𝑄 2 : ¬𝑒𝑝𝑜𝑓 2 ∧ ¬𝑒𝑝𝑜𝑓 1 ⇒ 𝑦 = 0 ∧ 𝑒𝑝𝑜𝑓 1 ⇒ 𝑦 = 1 𝑦 ≔ 𝑦 + 1; 𝑒𝑝𝑜𝑓 1 ≔ 𝑢𝑠𝑣𝑓 𝑦 ≔ 𝑦 + 1; 𝑒𝑝𝑜𝑓 2 ≔ 𝑢𝑠𝑣𝑓 𝑅 1 : 𝑒𝑝𝑜𝑓 1 ∧ ¬𝑒𝑝𝑜𝑓 2 ⇒ 𝑦 = 1 ∧ 𝑒𝑝𝑜𝑓 2 ⇒ 𝑦 = 2 𝑅 2 : 𝑒𝑝𝑜𝑓 2 ∧ ¬𝑒𝑝𝑜𝑓 1 ⇒ 𝑦 = 1 ∧ 𝑒𝑝𝑜𝑓 1 ⇒ 𝑦 = 2 𝑦 = 2

Rely/Guarantee Rely/Guarantee specifications 𝐷 ⊨ (𝑄, 𝑆, 𝐻, 𝑅) for individual threads and composition rule allow for modular proofs of loosely-coupled systems. 𝑦 ≥ 0 𝑆 = 𝐻 = 𝑦 ′ ≥ 𝑦 𝑦 ≔ 𝑦 + 1 ∥ 𝑦 ≔ 𝑦 + 1 𝑄 = 𝑅 = 𝑦 ≥ 0 𝑦 ≥ 0

Multi-layered refinement proofs [skip] || 𝑄 1 ≼ 𝑄 2 ≼ ⋯ ≼ 𝑄 𝑜−1 ≼ 𝑄 𝑄 𝑜 is safe 𝑜 𝑄 1 is safe Advantages of structured proofs: Better for humans: easier to construct and maintain Better for computers: localized/small checks  easier to automate Programs that do nothing cannot go wrong

Refinement is well-studied • Logic • 𝑄 𝑦, 𝑦 ′ ⇒ 𝑅(𝑦, 𝑦 ′ ) • Labeled transition systems • Language containment • Simulation (forward, backward, upward, downward, diagonal, sideways, …) • Bisimulation (vanilla, mint, lavender, barbed, triangulated, complicated, …) • …

Refinement is difficult for programs • Programs are complicated • Complex control and data • Gap between program syntax and abstractions • … especially for concurrent programs • … especially for interactive proof construction

CIVL: Construct correct concurrent programs layer by layer procedure P(…) { S } • Operates on program syntax S1; S2 • Organizes proof as a sequence of program if (e) S1 else S2 layers with increasingly coarse-grained while (e) S atomic actions call P • All layers and supporting invariants async call P expressed together in one textual unit call P1 || P2 • Automatically-generated verification call A conditions

Gated atomic actions [Elmas, Q, Tasiran 2009] (Gate, Transition) single-state predicate two-state predicate Command Gate Transition Lock specification 𝑦 ′ = 𝑦 + 𝑧 ∧ 𝑧 ′ = 𝑧 𝑢𝑠𝑣𝑓 x := x+y var lock : ThreadID ∪ {nil} 𝑧 ′ = 𝑧 𝑢𝑠𝑣𝑓 havoc x Acquire(): [assume lock == nil; lock := tid] 𝑦 ′ = 𝑦 ∧ 𝑧 ′ = 𝑧 𝑦 < 𝑧 assert x<y Release(): [assert lock == tid; lock := nil] 𝑦 < 𝑧 ∧ 𝑦 ′ = 𝑦 ∧ 𝑧 ′ = 𝑧 𝑢𝑠𝑣𝑓 assume x<y • Unifies precondition and postcondition • Primitive for modeling a (concrete or abstract) concurrent program

Operational semantics ℓ, 𝑡 ⋅ • Program configuration 𝑕, 𝑔 ⊎ 𝒰 • Transition relation ⇒ between configurations (and failure configuration ⊥ ) ⇒ ∗ ⊥ • Safety: ¬∃𝑕ℓ: 𝑕, ℓ, 𝑁𝑏𝑗𝑜 ⇒ ∗ ⊥ • 𝐻𝑝𝑝𝑒 𝑄 = 𝑕 ¬∃ℓ: 𝑕, ℓ, 𝑁𝑏𝑗𝑜 ⇒ ∗ 𝑕 ′ , ∅ 𝑕, 𝑕 ′ • 𝑈𝑠𝑏𝑜𝑡 𝑄 = ∃ℓ: 𝑕, ℓ, 𝑁𝑏𝑗𝑜 • 𝑄 1 ≼ 𝑄 2 : (1) 𝐻𝑝𝑝𝑒 𝑄 2 ⊆ 𝐻𝑝𝑝𝑒 𝑄 (2) 𝐻𝑝𝑝𝑒(𝑄 2 ) ∘ 𝑈𝑠𝑏𝑜𝑡 𝑄 1 ⊆ 𝑈𝑠𝑏𝑜𝑡(𝑄 2 ) 1 𝑄 2 preserves failures 𝑄 2 preserves final states

var x; IncrBy2() = call IncrBy2() [ x := x + 2 ] var x; Incr() = Op() { call Op() || Op() [ x := x + 1 ] call Incr() } var lock; Acquire() = Release() = Op() { call Op() || Op() var x; [ assume lock == nil; [ assert lock == tid; var t; lock := tid; ] lock := nil; ] call Acquire(); t := x; x := t + 1; call Release(); } var b; Acquire() { Release() { Op() { call Op() || Op() var x; while (true) b := 0; var t; if (CAS(b, 0, 1)) break; } call Acquire(); } t := x; x := t + 1; call Release(); }

const c >= 0; var x; call Main(); Main() { // Create c threads // each executing Incr  [assert x ≥ 0] } Incr() { acquire(); assert x ≥ 0; x := x + 1; release(); }

const c >= 0; var x; call Main(); Main() { x := 0; // Create c threads  [ ] // each executing Incr } Incr() { acquire(); assert x ≥ 0; x := x + 1; release(); }

Programs constructed with CIVL • Concurrent garbage collector [Hawblitzel, Petrank, Q, Tasiran 2015] • FastTrack2 race-detection algorithm [Flanagan, Freund, Wilcox 2018] • Lock-protected memory atop TSO [Hawblitzel] • Thread-local heap atop shared heap [Hawblitzel, Q] • Two-phase commit [K, Q, Henzinger 2018] • Work-stealing queue, Treiber stack, Ticket, …

Program layers in CIVL • A CIVL program denotes a sequence of concurrent programs (layers) • chained together by a refinement-preserving transformation • Transformation between program layers combines • Atomization: Transform statement S into atomic block [S] • Summarization: Transform atomic block [S] into atomic action A • Abstraction: Replace atomic action A with atomic action B

Right and left movers [Lipton 1975] Integer a “Semaphore” “wait” “signal” P(a) = [assume a > 0; a := a - 1] V(a) = [a := a + 1] right mover (R) left mover (L) Y V(a) P(a) X S 1 S 2 S 3 S 1 S 2 S 3 V(a) Y X P(a) S 1 T 2 S 3 S 1 T 2 S 3 Sequence R*;(N+  ); L* is atomic . . . . R* X N Y L* S 0 S 5 . . . . R* X N L* Y S 0 S 5