Sled gehammer Hell The Day after Jud gment Jasmin C. Blanchette TU - - PowerPoint PPT Presentation

▶
sled gehammer hell
SMART_READER_LITE
LIVE PREVIEW

Sled gehammer Hell The Day after Jud gment Jasmin C. Blanchette TU - - PowerPoint PPT Presentation

Oct 23, 2022 442 likes •1.18k views

Sled gehammer Hell The Day after Jud gment Jasmin C. Blanchette TU Mnchen Larry Paulson Jia Meng Kong Susanto Claire Quigley Markus Wenzel Fabian Immler Philipp Meyer Sascha Bhme Sledgehammer Sledgehammer Relevance filter Sledgehammer

slide-1
SLIDE 1

The Day after Jud gment

Jasmin C. Blanchette TU München

Sled gehammer Hell

slide-2
SLIDE 2
slide-3
SLIDE 3

Larry Paulson Jia Meng Kong Susanto Claire Quigley Markus Wenzel Fabian Immler Philipp Meyer Sascha Böhme

slide-4
SLIDE 4
slide-5
SLIDE 5

Sledgehammer

slide-6
SLIDE 6

Relevance filter Sledgehammer

slide-7
SLIDE 7

Relevance filter ATP translation Sledgehammer

slide-8
SLIDE 8

Relevance filter ATP translation E

Metis proof

SPASS

Metis proof Metis proof

Vampire SInE

Metis proof

Sledgehammer

slide-9
SLIDE 9

Relevance filter ATP translation E

Metis proof

SPASS

Metis proof Metis proof

Vampire SInE

Metis proof

Z3 CVC3 Yices SMT tr. SMT translation

Metis

  • r SMT

proof Metis

  • r SMT

proof Metis

  • r SMT

proof

Sledgehammer

slide-10
SLIDE 10

Relevance filter ATP translation E

Metis proof

SPASS

Metis proof Metis proof

Vampire SInE

Metis proof

Z3 CVC3 Yices SMT tr. SMT translation

Metis

  • r SMT

proof Metis

  • r SMT

proof Metis

  • r SMT

proof

Relevance filter Sledgehammer Sledgehammer

slide-11
SLIDE 11
slide-12
SLIDE 12

rev [a, b] = [b, a]

slide-13
SLIDE 13

rev [a, b] = [b, a]

by (metis Cons_eq_appendI eq_Nil_appendI rev.simps(2) rev_singleton_conv)

slide-14
SLIDE 14

2010

slide-15
SLIDE 15

54%

46%

3 ATPs x 30s

2010

slide-16
SLIDE 16

54%

46%

3 ATPs x 30s

66% 34%

3 ATPs x 30 s nontrivial goals

2010

slide-17
SLIDE 17

54%

46%

3 ATPs x 30s

66% 34%

3 ATPs x 30 s nontrivial goals

2010

39%

61%

(4 ATPs + 3 SMTs) x 30s

54% 46%

(4 ATPs + 3 SMTs) x 30s nontrivial goals

2011

slide-18
SLIDE 18

Issue #1: Too Many Facts Issue #2: Encoding Overhead Issue #3: Large Terms

slide-19
SLIDE 19

Issue #1: Too Many Facts

slide-20
SLIDE 20
slide-21
SLIDE 21

Conjecture: … c … d … e ...

slide-22
SLIDE 22

Conjecture: … c … d … e ... Lemma 1: … c … f ... Lemma 2: … f … g ... Lemma 3: … g … h ...

slide-23
SLIDE 23

Conjecture: … c … d … e ... Lemma 1: … c … f ... Lemma 2: … f … g ... Lemma 3: … g … h ...

✓

slide-24
SLIDE 24

Conjecture: … c … d … e ... Lemma 1: … c … f ... Lemma 2: … f … g ... Lemma 3: … g … h ...

✓ ✓

slide-25
SLIDE 25

Conjecture: … c … d … e ... Lemma 1: … c … f ... Lemma 2: … f … g ... Lemma 3: … g … h ...

✓ ✓ ✓

slide-26
SLIDE 26

The More Facts...

slide-27
SLIDE 27

The More Facts...

100 200 300 400 500 600 700 800 900 1000

Success Rate

E 1.2

slide-28
SLIDE 28

The More Facts...

100 200 300 400 500 600 700 800 900 1000

Success Rate

E 1.2

slide-29
SLIDE 29

E 1.2 The More Facts...

100 200 300 400 500 600 700 800 900 1000

Success Rate

slide-30
SLIDE 30

100 200 300 400 500 600 700 800 900 1000

Success Rate

SPASS 3.7 The More Facts...

slide-31
SLIDE 31

100 200 300 400 500 600 700 800 900 1000

Success Rate

SPASS 3.7 The More Facts...

slide-32
SLIDE 32

100 200 300 400 500 600 700 800 900 1000

Success Rate

The More Facts... Vampire 1.0

slide-33
SLIDE 33

100 200 300 400 500 600 700 800 900 1000

Success Rate

The More Facts... Vampire 1.0

slide-34
SLIDE 34

The More Facts...

100 200 300 400 500 600 700 800 900 1000

Success Rate

Z3 2.15

slide-35
SLIDE 35

The More Facts...

100 200 300 400 500 600 700 800 900 1000

Success Rate

Z3 2.15

slide-36
SLIDE 36

E SPASS Vampire Z3

slide-37
SLIDE 37

E SPASS Vampire Z3

slide-38
SLIDE 38

How Effective is the Relevance Filter?

slide-39
SLIDE 39

How Effective is the Relevance Filter?

0-9 100-109 200-209 300-309 400-409 490-499

Uses

slide-40
SLIDE 40

Experiments

slide-41
SLIDE 41

Experiments

Z3 Weights

slide-42
SLIDE 42

Experiments

Z3 Weights ……..……… +0.4 pp

slide-43
SLIDE 43

Experiments

Z3 Weights Z3 Triggers ……..……… +0.4 pp

slide-44
SLIDE 44

Experiments

Z3 Weights Z3 Triggers ……..……… +0.4 pp ……..……… +0.1 pp

slide-45
SLIDE 45

Experiments

Z3 Weights Z3 Triggers Z3 “Slicing” ……..……… +0.4 pp ……..……… +0.1 pp

slide-46
SLIDE 46

Experiments

Z3 Weights Z3 Triggers Z3 “Slicing” ……..……… +0.4 pp ……..……… +0.1 pp ........….…… +2.1 pp

slide-47
SLIDE 47

Experiments

Z3 Weights E Weights Z3 Triggers Z3 “Slicing” ……..……… +0.4 pp ……..……… +0.1 pp ........….…… +2.1 pp

slide-48
SLIDE 48

Experiments

Z3 Weights E Weights Z3 Triggers Z3 “Slicing” ……..……… +0.4 pp ……..……… +0.1 pp ........….…… +2.1 pp ..……………. +1.4 pp

slide-49
SLIDE 49

Issue #2: Encoding Overhead

slide-50
SLIDE 50

HOL to FOL

slide-51
SLIDE 51

HOL to FOL

Application Operatorxxxxx suc(N) app(suc, N)

slide-52
SLIDE 52

HOL to FOL

Type Information suc(N) ti(suc(ti(N, nat)), nat) Application Operatorxxxxx suc(N) app(suc, N)

slide-53
SLIDE 53

HOL to FOL

Type Information suc(N) ti(suc(ti(N, nat)), nat) Application Operatorxxxxx suc(N) app(suc, N) ti(app(ti(suc, fun(nat, nat)), ti(N, nat)), nat)

slide-54
SLIDE 54
slide-55
SLIDE 55

No App. App. No Types... 32K 41K Types... 60K 107K

Problem Size

slide-56
SLIDE 56

No App. App. No Types... 32K 41K Types... 60K 107K

Problem Size

No App. App. No Types... 1 s 15 s Types... 172 s ???? s

Solving Time (E 1.2)

slide-57
SLIDE 57

E SPASS Vampire Z3

slide-58
SLIDE 58

0% 10% 20% 30% 40% 50% 60% 70%

Arrow FFT FTA Hoare Jinja NS QE S2S SN All

E SPASS Vampire Z3

slide-59
SLIDE 59

0% 10% 20% 30% 40% 50% 60% 70%

Arrow FFT FTA Hoare Jinja NS QE S2S SN All

E SPASS Vampire Z3

Claim: Sorts Rock

x (Types)

slide-60
SLIDE 60

0% 10% 20% 30% 40% 50% 60% 70%

Arrow Jinja NS SN

E SPASS Vampire Z3

Claim: Sorts Rock

x (Types)

slide-61
SLIDE 61

Issue #3: Large Terms

slide-62
SLIDE 62

1 + … + 1 = (n::nat) rev [x1, …, xn] = [xn, …, x1] map f [x1, …, xn] = [f x1, …, f xn]

slide-63
SLIDE 63

1 + … + 1 = (n::nat)

0 s 0,2 s 0,4 s 0,6 s 0,8 s 1 2 3 4 5 6 7 8 9 10

E SPASS Vampire Z3

rev [x1, …, xn] = [xn, …, x1] map f [x1, …, xn] = [f x1, …, f xn]

slide-64
SLIDE 64

1 + … + 1 = (n::nat)

0 s 0,2 s 0,4 s 0,6 s 0,8 s 1 2 3 4 5 6 7 8 9 10

E SPASS Vampire Z3

rev [x1, …, xn] = [xn, …, x1]

0 s 7,5 s 15,0 s 22,5 s 30,0 s 1 2 3 4 5 6 7 8 9 10

Vampire Z3 E SPASS

map f [x1, …, xn] = [f x1, …, f xn]

slide-65
SLIDE 65

1 + … + 1 = (n::nat)

0 s 0,2 s 0,4 s 0,6 s 0,8 s 1 2 3 4 5 6 7 8 9 10

E SPASS Vampire Z3

rev [x1, …, xn] = [xn, …, x1]

0 s 7,5 s 15,0 s 22,5 s 30,0 s 1 2 3 4 5 6 7 8 9 10

Vampire Z3 E SPASS

map f [x1, …, xn] = [f x1, …, f xn]

Vampire E Z3

0 s 2,5 s 5,0 s 7,5 s 10,0 s 1 2 3 4 5 6 7 8 9 10

SPASS

slide-66
SLIDE 66

1 + … + 1 = (n::nat)

0 s 0,2 s 0,4 s 0,6 s 0,8 s 1 2 3 4 5 6 7 8 9 10

E SPASS Vampire Z3

rev [x1, …, xn] = [xn, …, x1]

0 s 7,5 s 15,0 s 22,5 s 30,0 s 1 2 3 4 5 6 7 8 9 10

Vampire Z3 E SPASS

map f [x1, …, xn] = [f x1, …, f xn]

Vampire E Z3

0 s 2,5 s 5,0 s 7,5 s 10,0 s 1 2 3 4 5 6 7 8 9 10

SPASS

(but simp can solve all of these within 10 ms…)

slide-67
SLIDE 67

Future Work

slide-68
SLIDE 68

Isabelle ATP / SMT

Future Work

slide-69
SLIDE 69

Improvexx Relevance Filterxx Lightenxx Translationxx Providexx Extralogical Info.xx

Isabelle ATP / SMT

Future Work

slide-70
SLIDE 70

Improvexx Relevance Filterxx Lightenxx Translationxx Providexx Extralogical Info.xx Handle Large Axiom Bases Support Types Exploit Extralogical Info.

Isabelle ATP / SMT

Future Work

slide-71
SLIDE 71

The Day after Jud gment

Jasmin C. Blanchette

blanchette@in.tum.de

Sled gehammer Hell