piCoq: Parallel Regression Proving for Large-Scale Verification - - PowerPoint PPT Presentation

▶
picoq parallel regression proving for large scale
SMART_READER_LITE
LIVE PREVIEW

piCoq: Parallel Regression Proving for Large-Scale Verification - - PowerPoint PPT Presentation

Mar 15, 2023 224 likes •894 views

piCoq: Parallel Regression Proving for Large-Scale Verification Projects piCoq: Parallel Regression Proving for Large-Scale Verification Projects Karl Palmskog , Ahmet Celik, and Milos Gligoric The University of Texas at Austin, USA 1 / 29

slide-1
SLIDE 1

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

Karl Palmskog, Ahmet Celik, and Milos Gligoric The University of Texas at Austin, USA

1 / 29

slide-2
SLIDE 2

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Introduction

Verification Using Proof Assistants

1 encode definitions in (higher-order) formalism 2 prove propositions interactively using powerful tactics 3 check soundness of every low-level step user logic engine proof checker proof assistant tactics subgoals proof

examples: Coq, HOL4, HOL Light, Isabelle/HOL, Lean, Nuprl, ...

2 / 29

slide-3
SLIDE 3

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Introduction

Some Large-Scale Proof Assistant Projects

Project Year Assistant Check Time LOC 4-Color Theorem 2005 Coq hours 60k Odd Order Theorem 2012 Coq hours 150k Kepler Conjecture 2015 HOL Light days 500k CompCert C compiler 2009 Coq tens of mins 40k Cogent (BilbyFS) 2016 Isabelle/HOL hours 14k Verdi (Raft consensus) 2016 Coq tens of mins 50k problem: long proof checking times

3 / 29

slide-4
SLIDE 4

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Introduction

Proof Engineering Techniques For Effective Proving

Proof selection: check only proofs affected by changes file/module selection proof selection Examples: Make, Isabelle [ITP ’14], iCoq [ASE ’17]

4 / 29

slide-5
SLIDE 5

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Introduction

Proof Engineering Techniques For Effective Proving

Proof selection: check only proofs affected by changes file/module selection proof selection Examples: Make, Isabelle [ITP ’14], iCoq [ASE ’17] Proof parallelization: leverage multi-core hardware parallel checking of proofs parallel checking of files Examples: Make, Isabelle [ITP ’13], Coq [ITP ’15]

4 / 29

slide-6
SLIDE 6

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Introduction

Our Contributions

taxonomy of regression proving techniques that leverage both selection and parallelism implementation of techniques in tool, piCoq, that supports Coq projects (useful for CI, e.g., Travis on GitHub) evaluation using piCoq on six open source projects (23 kLOC over 22 revisions per project, on average)

5 / 29

slide-7
SLIDE 7

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

The Coq Proof Assistant (1985-present)

based on constructive dependent type theory Gallina – programming/specification language Ltac – proof tactic language small trusted core checker for programs & proofs

6 / 29

slide-8
SLIDE 8

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Source File Example

Require Import List. Require Import ListUtil. Import ListNotations. Fixpoint dedup A A_eq_dec (xs : list A) : list A := match xs with | [] ⇒ [] | x :: xs ⇒ if in_dec A_eq_dec x xs then dedup A A_eq_dec xs else x :: dedup A A_eq_dec xs end. Lemma remove_dedup : forall A A_eq_dec (x : A) xs, remove A_eq_dec x (dedup A A_eq_dec xs) = dedup A A_eq_dec (remove A_eq_dec x xs). Proof. induction xs; intros; auto; simpl. repeat (try case in_dec; try case A_eq_dec; simpl; intuition); auto using f_equal.

  • exfalso. apply n0. apply remove_preserve; auto.
  • exfalso. apply n. apply in_remove in i; intuition.

Qed.

Dedup.v

7 / 29

slide-9
SLIDE 9

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Source File Example

Require Import List. Require Import ListUtil. Import ListNotations. Fixpoint dedup A A_eq_dec (xs : list A) : list A := match xs with | [] ⇒ [] | x :: xs ⇒ if in_dec A_eq_dec x xs then dedup A A_eq_dec xs else x :: dedup A A_eq_dec xs end. Lemma remove_dedup : forall A A_eq_dec (x : A) xs, remove A_eq_dec x (dedup A A_eq_dec xs) = dedup A A_eq_dec (remove A_eq_dec x xs). Proof. induction xs; intros; auto; simpl. repeat (try case in_dec; try case A_eq_dec; simpl; intuition); auto using f_equal.

  • exfalso. apply n0. apply remove_preserve; auto.
  • exfalso. apply n. apply in_remove in i; intuition.

Qed.

Dedup.v

Require statements expressing file dependencies.

7 / 29

slide-10
SLIDE 10

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Source File Example

Require Import List. Require Import ListUtil. Import ListNotations. Fixpoint dedup A A_eq_dec (xs : list A) : list A := match xs with | [] ⇒ [] | x :: xs ⇒ if in_dec A_eq_dec x xs then dedup A A_eq_dec xs else x :: dedup A A_eq_dec xs end. Lemma remove_dedup : forall A A_eq_dec (x : A) xs, remove A_eq_dec x (dedup A A_eq_dec xs) = dedup A A_eq_dec (remove A_eq_dec x xs). Proof. induction xs; intros; auto; simpl. repeat (try case in_dec; try case A_eq_dec; simpl; intuition); auto using f_equal.

  • exfalso. apply n0. apply remove_preserve; auto.
  • exfalso. apply n. apply in_remove in i; intuition.

Qed.

Dedup.v

Definition of a recursive function to remove duplicate list elements in Gallina.

7 / 29

slide-11
SLIDE 11

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Source File Example

Require Import List. Require Import ListUtil. Import ListNotations. Fixpoint dedup A A_eq_dec (xs : list A) : list A := match xs with | [] ⇒ [] | x :: xs ⇒ if in_dec A_eq_dec x xs then dedup A A_eq_dec xs else x :: dedup A A_eq_dec xs end. Lemma remove_dedup : forall A A_eq_dec (x : A) xs, remove A_eq_dec x (dedup A A_eq_dec xs) = dedup A A_eq_dec (remove A_eq_dec x xs). Proof. induction xs; intros; auto; simpl. repeat (try case in_dec; try case A_eq_dec; simpl; intuition); auto using f_equal.

  • exfalso. apply n0. apply remove_preserve; auto.
  • exfalso. apply n. apply in_remove in i; intuition.

Qed.

Dedup.v

Statement (type) of a lemma in Gallina.

7 / 29

slide-12
SLIDE 12

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Source File Example

Require Import List. Require Import ListUtil. Import ListNotations. Fixpoint dedup A A_eq_dec (xs : list A) : list A := match xs with | [] ⇒ [] | x :: xs ⇒ if in_dec A_eq_dec x xs then dedup A A_eq_dec xs else x :: dedup A A_eq_dec xs end. Lemma remove_dedup : forall A A_eq_dec (x : A) xs, remove A_eq_dec x (dedup A A_eq_dec xs) = dedup A A_eq_dec (remove A_eq_dec x xs). Proof. induction xs; intros; auto; simpl. repeat (try case in_dec; try case A_eq_dec; simpl; intuition); auto using f_equal.

  • exfalso. apply n0. apply remove_preserve; auto.
  • exfalso. apply n. apply in_remove in i; intuition.

Qed.

Dedup.v

Proof script in Ltac – potentially time-consuming to process.

7 / 29

slide-13
SLIDE 13

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Proof-Checking Toolchain

Legacy Top-Down Proof Checking (1990s) coqc: compilation of source .v files to binary .vo files .vo files contain functions and all proofs file-level parallelism via Make

8 / 29

slide-14
SLIDE 14

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Background

Coq Proof-Checking Toolchain

Legacy Top-Down Proof Checking (1990s) coqc: compilation of source .v files to binary .vo files .vo files contain functions and all proofs file-level parallelism via Make Quick Compilation and Asynchronous Checking (2015) coqc -quick: compilation of .v files to binary .vio files .vio files contain functions and proof tasks proof tasks checkable asynchronously in parallel

8 / 29

slide-15
SLIDE 15

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

Regression Proving Modes for Coq (Taxonomy)

Parallelization Selection Granularity None Files Proofs File level f·none f·file N/A Proof level p·none p·file p·icoq

9 / 29

slide-16
SLIDE 16

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·none Mode: File-Level Parallelization, No Selection

Parallelization Selection Granularity None Files Proofs File level f·none f·file N/A Proof level p·none p·file p·icoq

legacy mode used in most GitHub Coq projects no overhead from proof task management or dep. tracking parallelism restricted by file dependency graph

  • verhead from writing proofs to disk

10 / 29

slide-17
SLIDE 17

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·none Mode in Practice

ListUtil.v Dedup.v RemoveAll.v

11 / 29

slide-18
SLIDE 18

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

11 / 29

slide-19
SLIDE 19

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vo remove preserve, in remove

11 / 29

slide-20
SLIDE 20

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vo remove preserve, in remove 2 Dedup.vo dedup, remove dedup 2 RemoveAll.vo remove all, remove all in, remove all preserve

11 / 29

slide-21
SLIDE 21

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·none Mode: Proof-Level Parallelization, No Selection

Parallelization Selection Granularity None Files Proofs File level f·none f·file N/A Proof level p·none p·file p·icoq

legacy mode used in some GitHub Coq projects

  • verhead from proof task management

parallelism (largely) unrestricted by file dependency graph no overhead from writing proofs to disk and dep. tracking

12 / 29

slide-22
SLIDE 22

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

13 / 29

slide-23
SLIDE 23

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove

13 / 29

slide-24
SLIDE 24

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove 2 Dedup.vio dedup, remove dedup 2 RemoveAll.vio remove all, remove all in, remove all preserve

13 / 29

slide-25
SLIDE 25

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·none Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove 2 Dedup.vio dedup, remove dedup 2 RemoveAll.vio remove all, remove all in, remove all preserve 3 checking remove preserve 3 checking in remove 3 checking remove dedup 3 checking remove all in 3 checking remove all preserve

13 / 29

slide-26
SLIDE 26

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·file Mode: File-Level Parallelization, File Selection

Parallelization Selection Granularity None Files Proofs File level f·none f·file N/A Proof level p·none p·file p·icoq

novel mode that persists file checksums

  • verhead from file dependency tracking

parallelism restricted by file dependency graph

  • verhead from writing proofs to disk

14 / 29

slide-27
SLIDE 27

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

15 / 29

slide-28
SLIDE 28

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

15 / 29

slide-29
SLIDE 29

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vo remove preserve, in remove

15 / 29

slide-30
SLIDE 30

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

f·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vo remove preserve, in remove 2 Dedup.vo dedup, remove dedup

15 / 29

slide-31
SLIDE 31

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·file Mode: Proof-Level Parallelism, File Selection

Parallelization Selection Granularity None Files Proofs File level f·none f·file N/A Proof level p·none p·file p·icoq

novel mode that persists file checksums

  • verhead from file dependency tracking

parallelism (mostly) unrestricted by file dependency graph no overhead from writing proofs to disk

16 / 29

slide-32
SLIDE 32

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

17 / 29

slide-33
SLIDE 33

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

17 / 29

slide-34
SLIDE 34

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove

17 / 29

slide-35
SLIDE 35

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove 2 RemoveAll.vio remove all, remove all in, remove all preserve

17 / 29

slide-36
SLIDE 36

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·file Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove 2 RemoveAll.vio remove all, remove all in, remove all preserve 3 checking remove all in 3 checking remove all preserve

17 / 29

slide-37
SLIDE 37

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode: Proof-Level Parallelism, Proof Selection

Parallelization Selection Granularity None Files Proofs File level f·none f·file N/A Proof level p·none p·file p·icoq

novel mode that persists file & proof checksums

  • verhead from file & proof dependency tracking

parallelism (mostly) unrestricted by file dependency graph no overhead from writing proofs to disk

18 / 29

slide-38
SLIDE 38

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

19 / 29

slide-39
SLIDE 39

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

19 / 29

slide-40
SLIDE 40

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all

19 / 29

slide-41
SLIDE 41

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove

19 / 29

slide-42
SLIDE 42

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove 2 Dedup.vio dedup, remove dedup 2 RemoveAll.vio remove all, remove all in, remove all preserve

19 / 29

slide-43
SLIDE 43

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Mode in Practice

ListUtil.v remove preserve in remove Dedup.v dedup remove dedup RemoveAll.v remove all preserve remove all in remove all Phase Task Definitions and Lemmas 1 ListUtil.vio remove preserve, in remove 2 Dedup.vio dedup, remove dedup 2 RemoveAll.vio remove all, remove all in, remove all preserve 3 checking in remove 3 checking remove dedup 3 checking remove all in

19 / 29

slide-44
SLIDE 44

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Workflow with 4-way Parallelization

file dep. graph .v files proof dep. graph Analysis Checking Collection

20 / 29

slide-45
SLIDE 45

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Workflow with 4-way Parallelization

file dep. graph .v files proof dep. graph Analysis compilation commands .vio files affected proofs Checking Collection

20 / 29

slide-46
SLIDE 46

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Workflow with 4-way Parallelization

file dep. graph .v files proof dep. graph Analysis compilation commands .vio files affected proofs Checking proof dependencies proof- checking commands Collection

20 / 29

slide-47
SLIDE 47

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

p·icoq Workflow with 4-way Parallelization

file dep. graph .v files proof dep. graph Analysis compilation commands .vio files affected proofs Checking proof dependencies proof- checking commands Collection new dep. graphs storage

20 / 29

slide-48
SLIDE 48

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

piCoq Tool Implementation

extension of iCoq toolchain (Java, OCaml, bash) Java task executor for parallel compilation of .vo/.vio files extension of coqc for parallel checking of (and dependency extraction from) specific proofs across files

$ coqc -schedule -vio -task -depends -checking 4 \ file1.vio 1,15,16 \ file2.vio 3 ,10 ,11 ,13 ,20

21 / 29

slide-49
SLIDE 49

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

piCoq Tool Implementation

extension of iCoq toolchain (Java, OCaml, bash) Java task executor for parallel compilation of .vo/.vio files extension of coqc for parallel checking of (and dependency extraction from) specific proofs across files

$ coqc -schedule -vio -task -depends -checking 4 \ file1.vio 1,15,16 \ file2.vio 3 ,10 ,11 ,13 ,20

21 / 29

slide-50
SLIDE 50

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Proof Checking Modes

piCoq Tool Implementation

extension of iCoq toolchain (Java, OCaml, bash) Java task executor for parallel compilation of .vo/.vio files extension of coqc for parallel checking of (and dependency extraction from) specific proofs across files

$ coqc -schedule -vio -task -depends -checking 4 \ file1.vio 1,15,16 \ file2.vio 3 ,10 ,11 ,13 ,20

21 / 29

slide-51
SLIDE 51

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Evaluation: Open Source Git-Based Projects

Project LOC Domain Coquelicot 38260 real number analysis Finmap 5661 finite sets and maps Flocq 24786 floating-point arithmetic Fomegac 2637 formal system metatheory Surface Effects 9621 functional programming languages Verdi 56147 distributed systems

  • 137112

Avg. 22852.00

22 / 29

slide-52
SLIDE 52

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Evaluation: Open Source Git-Based Projects

Project LOC #Revs. #Files #Proof Tasks Coquelicot 38260 24 29 1660 Finmap 5661 23 4 959 Flocq 24786 23 40 943 Fomegac 2637 14 13 156 Surface Effects 9621 24 15 289 Verdi 56147 24 222 2756

  • 137112

132 323 6763 Avg. 22852.00 22.00 53.83 1127.16

23 / 29

slide-53
SLIDE 53

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Evaluation Details

Experiment machine: Intel Core i7-6700 CPU @ 3.40GHz 4 CPU cores 16 GB memory Ubuntu Linux 17.04 Coq 8.5 Evaluation Setup every build starts from scratch (version control) up to 4 parallel jobs/processes dependency metadata persisted between revisions

24 / 29

slide-54
SLIDE 54

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Results with 4-way Parallelization: Coquelicot

20 40 befc2aa5 65187c83 af196a6f 96153bf0 30893b49 7c74911f cc869c0d a94e2add ac474859 8f7b7c3f 51ccec65 a21157ac 67c0544f 99d324c8 02aa4835 ef18b8b5 18363068 a43e920b f25236ff e49f02aa fc2b7663 a22616bb a035d3bd 680ca587

Revision Time [s]

f·none p·none f·file p·file p·icoq 25 / 29

slide-55
SLIDE 55

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Results with 4-way Parallelization: Fomegac

10 20 30 0af6b37e ac790c93 a9bcb72a 78a261b5 79442b65 ca71b96c 4c1f4b3b 99f338f0 62d1a9d5 5f156d72 4b9edb51 b5020661 81d7d688 7a654d7c

Revision Time [s]

f·none p·none f·file p·file p·icoq 26 / 29

slide-56
SLIDE 56

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Speedups over f·none for 4-way Parallel Checking

Coquelicot Finmap Flocq Fomegac Surface Effects Verdi 1 2 3 4 5 6 7 8 9 Speedup factor over f·none f·none p·none f·file p·file p·icoq “How much faster modes are than the default mode, for each project”

27 / 29

slide-57
SLIDE 57

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Evaluation

Speedups from Sequential to 4-way Parallel Checking

Coquelicot Finmap Flocq Fomegac Surface Effects Verdi 0.5 1 1.5 2 2.5 3 3.5 Speedup factor over sequential f·none p·none f·file p·file p·icoq “Effect of parallelism on each mode and project”

28 / 29

slide-58
SLIDE 58

piCoq: Parallel Regression Proving for Large-Scale Verification Projects Conclusion

Conclusion

taxonomy of regression proving modes implementation of modes for Coq in tool piCoq eval shows speedups for parallelism/selection (up to 28.6×) Contact us: Karl Palmskog, palmskog@utexas.edu Ahmet Celik, ahmetcelik@utexas.edu Milos Gligoric, gligoric@utexas.edu Resources: Website: http://cozy.ece.utexas.edu/icoq/ GitHub: https://github.com/proofengineering/icoq This work was partially supported by the US National Science Foundation under Grants Nos. CCF-1566363 and CCF-1652517.

29 / 29

slide-59
SLIDE 59

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

f·none vs. p·none for 4-way Parallel Checking

Coquelicot Finmap Flocq Fomegac Surface Effects Verdi 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

30 / 29

slide-60
SLIDE 60

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

f·none vs. f·file for 4-way Parallel Checking

Coquelicot Finmap Flocq Fomegac Surface Effects Verdi 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

31 / 29

slide-61
SLIDE 61

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

p·none vs. p·file for 4-way Parallel Checking

Coquelicot Finmap Flocq Fomegac Surface Effects Verdi 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

32 / 29

slide-62
SLIDE 62

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

p·file vs. p·icoq for 4-way Parallel Checking

Coquelicot Finmap Flocq Fomegac Surface Effects Verdi 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

33 / 29

slide-63
SLIDE 63

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

Regression Proving in Evolving Projects

Typical proving scenario:

1 change definition or lemma statement 2 begin process of re-checking all proofs 3 checking fails much later (for seemingly unrelated proof)

Typical testing scenario:

1 change method statements or method signature 2 begin process of re-running all tests 3 testing fails much later (for seemingly unrelated test)

34 / 29

slide-64
SLIDE 64

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

Software Engineering Techniques For Effective Testing

Test selection: run only tests affected by changes file/class selection method selection hybrid Examples: Ekstazi [ISSTA ’15], STARTS [ASE ’17], HyRTS [ICSE ’18] Test parallelization: leverage multi-core hardware parallel threads parallel processes (VM forking) hybrid Examples: Gradle, Maven, JUnit

35 / 29

slide-65
SLIDE 65

piCoq: Parallel Regression Proving for Large-Scale Verification Projects

Regression Proving vs. Regression Testing

proof checking is deterministic proof checking has no side effects (e.g., I/O)

  • nly file-level deps. relevant for (asynch) proof checking

36 / 29