Towards Machine Learning Induction for Isabelle/HOL This work was - - PowerPoint PPT Presentation

towards machine learning induction for isabelle hol
SMART_READER_LITE
LIVE PREVIEW

Towards Machine Learning Induction for Isabelle/HOL This work was - - PowerPoint PPT Presentation

git clone https://github.com/data61/PSL Towards Machine Learning Induction for Isabelle/HOL This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466). Yutaka Nagashima University of Innsbruck Czech


slide-1
SLIDE 1

Towards Machine Learning Induction for Isabelle/HOL

Yutaka Nagashima

University of Innsbruck Czech Technical University

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

git clone https://github.com/data61/PSL

slide-2
SLIDE 2

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

git clone https://github.com/data61/PSL

slide-3
SLIDE 3

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Who is Isabelle?

git clone https://github.com/data61/PSL

slide-4
SLIDE 4

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Why induction? Who is Isabelle?

git clone https://github.com/data61/PSL

slide-5
SLIDE 5

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Why induction? Who is Isabelle?

git clone https://github.com/data61/PSL

https://www.logic.at/staff/gramlich/

  • Prof. Bernhard Gramlich

ITP (Inductive Theorem Proving) problems are at the heart of many verification and reasoning tasks in

slide-6
SLIDE 6

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Why induction? Who is Isabelle?

git clone https://github.com/data61/PSL

we are convinced that substantial progress in ITP will take time.

https://www.logic.at/staff/gramlich/

  • Prof. Bernhard Gramlich

ITP (Inductive Theorem Proving) problems are at the heart of many verification and reasoning tasks in

slide-7
SLIDE 7

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Why induction? Who is Isabelle?

git clone https://github.com/data61/PSL

we are convinced that substantial progress in ITP will take time.

https://www.logic.at/staff/gramlich/

  • Prof. Bernhard Gramlich

ITP (Inductive Theorem Proving) problems are at the heart of many verification and reasoning tasks in spectacular breakthroughs are unrealistic, in view of the enormous problems and the inherent difficulty of inductive theorem proving.

Strategic Issues, Problems and Challenges in Inductive Theorem Proving

slide-8
SLIDE 8

Why induction? we are convinced that substantial progress in ITP will take time. ITP (Inductive Theorem Proving) problems are at the heart of many verification and reasoning tasks in

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Who is Isabelle?

git clone https://github.com/data61/PSL

spectacular breakthroughs are unrealistic, in view of the enormous problems and the inherent difficulty of inductive theorem proving.

https://www.logic.at/staff/gramlich/

  • Prof. Bernhard Gramlich

Strategic Issues, Problems and Challenges in Inductive Theorem Proving

Challenge accepted!

slide-9
SLIDE 9

Why induction? we are convinced that substantial progress in ITP will take time. ITP (Inductive Theorem Proving) problems are at the heart of many verification and reasoning tasks in

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Who is Isabelle?

git clone https://github.com/data61/PSL

spectacular breakthroughs are unrealistic, in view of the enormous problems and the inherent difficulty of inductive theorem proving.

https://www.logic.at/staff/gramlich/

  • Prof. Bernhard Gramlich

Strategic Issues, Problems and Challenges in Inductive Theorem Proving

Challenge accepted!

The time has come!

slide-10
SLIDE 10

Why induction? we are convinced that substantial progress in ITP will take time. ITP (Inductive Theorem Proving) problems are at the heart of many verification and reasoning tasks in

Towards Machine Learning Induction for Isabelle/HOL

This work was supported by the project AI&Reasoning (reg. no. CZ.02.1.01/0.0/0.0/15_003/0000466).

?

Who is Isabelle?

git clone https://github.com/data61/PSL

spectacular breakthroughs are unrealistic, in view of the enormous problems and the inherent difficulty of inductive theorem proving.

https://www.logic.at/staff/gramlich/

  • Prof. Bernhard Gramlich

Strategic Issues, Problems and Challenges in Inductive Theorem Proving

Challenge accepted!

The time has come!

…or is coming soon.

slide-11
SLIDE 11

Interactive theorem proving with Isabelle/HOL

git clone https://github.com/data61/PSL

slide-12
SLIDE 12

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context

git clone https://github.com/data61/PSL

slide-13
SLIDE 13

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context subgoals error-message

git clone https://github.com/data61/PSL

slide-14
SLIDE 14

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context subgoals error-message

git clone https://github.com/data61/PSL

slide-15
SLIDE 15

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

git clone https://github.com/data61/PSL

slide-16
SLIDE 16

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

git clone https://github.com/data61/PSL

slide-17
SLIDE 17

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

git clone https://github.com/data61/PSL

slide-18
SLIDE 18

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

git clone https://github.com/data61/PSL

slide-19
SLIDE 19

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

git clone https://github.com/data61/PSL

slide-20
SLIDE 20

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

It's blatantly clear You stupid machine, that what I tell you is true (Michael Norrish)

git clone https://github.com/data61/PSL

slide-21
SLIDE 21

Interactive theorem proving with Isabelle/HOL

tactic / proof method proof goal context no sub-goal! subgoals error-message

It's blatantly clear You stupid machine, that what I tell you is true (Michael Norrish)

git clone https://github.com/data61/PSL

DEMO!

slide-22
SLIDE 22

goal

Dynamic ( Induct ) Auto IsSolved

git clone https://github.com/data61/PSL

slide-23
SLIDE 23

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct)

git clone https://github.com/data61/PSL

slide-24
SLIDE 24

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct)

git clone https://github.com/data61/PSL

slide-25
SLIDE 25

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto)

git clone https://github.com/data61/PSL

slide-26
SLIDE 26

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto)

git clone https://github.com/data61/PSL

slide-27
SLIDE 27

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto)

git clone https://github.com/data61/PSL

slide-28
SLIDE 28

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto)

git clone https://github.com/data61/PSL

slide-29
SLIDE 29

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs)

git clone https://github.com/data61/PSL

slide-30
SLIDE 30

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs)

git clone https://github.com/data61/PSL

slide-31
SLIDE 31

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs)

git clone https://github.com/data61/PSL

slide-32
SLIDE 32

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs)

git clone https://github.com/data61/PSL

slide-33
SLIDE 33

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto)

git clone https://github.com/data61/PSL

slide-34
SLIDE 34

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto)

git clone https://github.com/data61/PSL

slide-35
SLIDE 35

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto)

git clone https://github.com/data61/PSL

slide-36
SLIDE 36

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto)

git clone https://github.com/data61/PSL

slide-37
SLIDE 37

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto) apply (induct xs rule: Demo.sep.induct)

git clone https://github.com/data61/PSL

slide-38
SLIDE 38

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto) apply (induct xs rule: Demo.sep.induct)

git clone https://github.com/data61/PSL

slide-39
SLIDE 39

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto) apply (induct xs rule: Demo.sep.induct) apply (auto)

git clone https://github.com/data61/PSL

slide-40
SLIDE 40

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto) apply (induct xs rule: Demo.sep.induct) apply (auto)

git clone https://github.com/data61/PSL

slide-41
SLIDE 41

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto) apply (induct xs rule: Demo.sep.induct) apply (auto) done

git clone https://github.com/data61/PSL

slide-42
SLIDE 42

goal

Dynamic ( Induct ) Auto IsSolved

apply (induct) apply (auto) apply (induct xs) apply (auto) apply (induct xs rule: Demo.sep.induct) apply (auto) done

git clone https://github.com/data61/PSL

slide-43
SLIDE 43

Try_Hard: the default strategy

strategy Basic = Ors [ Auto_Solve, Blast_Solve, FF_Solve, Thens [IntroClasses, Auto_Solve], Thens [Transfer, Auto_Solve], Thens [Normalization, IsSolved], Thens [DInduct, Auto_Solve], Thens [Hammer, IsSolved], Thens [DCases, Auto_Solve], Thens [DCoinduction, Auto_Solve], Thens [Auto, RepeatN(Hammer), IsSolved], Thens [DAuto, IsSolved]] strategy Try_Hard = Ors [Thens [Subgoal, Basic], Thens [DInductTac, Auto_Solve], Thens [DCaseTac, Auto_Solve], Thens [Subgoal, Advanced], Thens [DCaseTac, Solve_Many], Thens [DInductTac, Solve_Many] ]

16 percentage point performance improvement compared to sledgehammer PaMpeR: Proof Method Recommendation but the search space explodes

git clone https://github.com/data61/PSL

slide-44
SLIDE 44

preparation phase recommendation phase

How does PaMpeR work?

slide-45
SLIDE 45

preparation phase recommendation phase

?

proof state proof engineer

How does PaMpeR work?

large proof corpora AFP and standard library

slide-46
SLIDE 46

preparation phase recommendation phase

?

proof state proof engineer

How does PaMpeR work?

large proof corpora AFP and standard library

Archive of Formal Proofs (https://www.isa-afp.org)

slide-47
SLIDE 47

preparation phase recommendation phase

?

proof state proof engineer

How does PaMpeR work?

large proof corpora AFP and standard library

slide-48
SLIDE 48

preparation phase recommendation phase

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions large proof corpora AFP and standard library

slide-49
SLIDE 49

preparation phase recommendation phase

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions :: ( tactic_name, [ bool ] ) database ( 425334 data points ) large proof corpora AFP and standard library

slide-50
SLIDE 50

preprocess decision tree construction preparation phase recommendation phase

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions :: ( tactic_name, [ bool ] ) database ( 425334 data points ) large proof corpora AFP and standard library

slide-51
SLIDE 51

preprocess decision tree construction preparation phase recommendation phase fast feature extractor

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions :: ( tactic_name, [ bool ] ) database ( 425334 data points ) large proof corpora AFP and standard library

slide-52
SLIDE 52

preprocess decision tree construction feature vector preparation phase recommendation phase fast feature extractor

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions :: ( tactic_name, [ bool ] ) database ( 425334 data points ) large proof corpora AFP and standard library

slide-53
SLIDE 53

preprocess decision tree construction feature vector proof method recommendation lookup preparation phase recommendation phase fast feature extractor

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions :: ( tactic_name, [ bool ] ) database ( 425334 data points ) large proof corpora AFP and standard library

slide-54
SLIDE 54

preprocess decision tree construction feature vector proof method recommendation lookup preparation phase recommendation phase fast feature extractor

?

proof state proof engineer

How does PaMpeR work?

full feature extractor 6021 CPU hours 108 assertions :: ( tactic_name, [ bool ] ) database ( 425334 data points ) large proof corpora AFP and standard library

?

slide-55
SLIDE 55

AITP2018 review

?

anonymous reviewer

git clone https://github.com/data61/PSL

slide-56
SLIDE 56

AITP2018 review

Proof Method Recommendation, PaMpeR!

?

anonymous reviewer

git clone https://github.com/data61/PSL

slide-57
SLIDE 57

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper.

?

anonymous reviewer

git clone https://github.com/data61/PSL

slide-58
SLIDE 58

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper.

?

anonymous reviewer

git clone https://github.com/data61/PSL

slide-59
SLIDE 59

New users of Isabelle are facing many challenges from

  • writing their first definitions,
  • stating suitable theorem statements, and
  • producing properly structured proofs.

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper.

?

anonymous reviewer

git clone https://github.com/data61/PSL

slide-60
SLIDE 60

New users of Isabelle are facing many challenges from

  • writing their first definitions,
  • stating suitable theorem statements, and
  • producing properly structured proofs.

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper. Proof methods are merely the bits at the bottom of that.

?

anonymous reviewer

git clone https://github.com/data61/PSL

slide-61
SLIDE 61

New users of Isabelle are facing many challenges from

  • writing their first definitions,
  • stating suitable theorem statements, and
  • producing properly structured proofs.

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper. Proof methods are merely the bits at the bottom of that.

?

anonymous reviewer

I was writing how to prove not how to specify!

git clone https://github.com/data61/PSL

slide-62
SLIDE 62

New users of Isabelle are facing many challenges from

  • writing their first definitions,
  • stating suitable theorem statements, and
  • producing properly structured proofs.

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper. Proof methods are merely the bits at the bottom of that.

?

anonymous reviewer

I was writing how to prove not how to specify!

git clone https://github.com/data61/PSL

slide-63
SLIDE 63

New users of Isabelle are facing many challenges from

  • writing their first definitions,
  • stating suitable theorem statements, and
  • producing properly structured proofs.

AITP2018 review

Proof Method Recommendation, PaMpeR!

I have doubts about various approaches proposed in the paper. Proof methods are merely the bits at the bottom of that.

?

anonymous reviewer

I was writing how to prove not how to specify! Proof Goal Transformer, PGT!

git clone https://github.com/data61/PSL

slide-64
SLIDE 64

PSL with PGT

PGT

git clone https://github.com/data61/PSL

slide-65
SLIDE 65

PSL with PGT

PGT strategy proof goal sub-optimal for proof automation context

PGT

git clone https://github.com/data61/PSL

slide-66
SLIDE 66

PSL with PGT

PGT strategy proof goal sub-optimal for proof automation context

PGT

tactic / sub-tool proof goal context

git clone https://github.com/data61/PSL

slide-67
SLIDE 67

PSL with PGT

PGT strategy proof goal sub-optimal for proof automation context proved theorem / subgoals / message

PGT

tactic / sub-tool proof goal context

git clone https://github.com/data61/PSL

slide-68
SLIDE 68

PSL with PGT

PGT strategy proof goal sub-optimal for proof automation context proof for the original goal, and auxiliary lemma

  • ptimal for proof automation

proved theorem / subgoals / message

PGT

tactic / sub-tool proof goal context

git clone https://github.com/data61/PSL

slide-69
SLIDE 69

PSL with PGT

PGT strategy proof goal sub-optimal for proof automation context proof for the original goal, and auxiliary lemma

  • ptimal for proof automation

proved theorem / subgoals / message

PGT

tactic / sub-tool proof goal context

DEMO!

git clone https://github.com/data61/PSL

slide-70
SLIDE 70

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-71
SLIDE 71

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-72
SLIDE 72

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-73
SLIDE 73

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-74
SLIDE 74

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-75
SLIDE 75

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-76
SLIDE 76

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-77
SLIDE 77

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-78
SLIDE 78

goal Conjecture Fastforce DInd DInd Quickcheck

git clone https://github.com/data61/PSL

slide-79
SLIDE 79

Success story

git clone https://github.com/data61/PSL

PSL can find how to apply induction for easy problems.

slide-80
SLIDE 80

Success story

git clone https://github.com/data61/PSL

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use.

slide-81
SLIDE 81

Success story

git clone https://github.com/data61/PSL

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

slide-82
SLIDE 82

Success story

git clone https://github.com/data61/PSL

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

CADE2017

slide-83
SLIDE 83

Success story

git clone https://github.com/data61/PSL

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

CADE2017 ASE2018

slide-84
SLIDE 84

Success story

git clone https://github.com/data61/PSL

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

CADE2017 ASE2018 CICM2018

(best system award)

slide-85
SLIDE 85

PSL can find how to apply induction for easy problems.

Too good to be true?

git clone https://github.com/data61/PSL

PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

slide-86
SLIDE 86

PSL can find how to apply induction for easy problems.

Too good to be true?

git clone https://github.com/data61/PSL

PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

  • n

l y i f P S L c

  • m

p l e t e s a p r

  • f

s e a r c h

  • n

l y i f P S L w i t h P G T c

  • m

p l e t e s a p r

  • f

s e a r c h

slide-87
SLIDE 87

PSL can find how to apply induction for easy problems.

Too good to be true?

git clone https://github.com/data61/PSL

b u t P a M p e R d

  • e

s n

  • t

r e c

  • m

m e n d a r g u m e n t s f

  • r

p r

  • f

m e t h

  • d

s PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

  • n

l y i f P S L c

  • m

p l e t e s a p r

  • f

s e a r c h

  • n

l y i f P S L w i t h P G T c

  • m

p l e t e s a p r

  • f

s e a r c h

slide-88
SLIDE 88

PSL can find how to apply induction for easy problems.

Too good to be true?

git clone https://github.com/data61/PSL

b u t P a M p e R d

  • e

s n

  • t

r e c

  • m

m e n d a r g u m e n t s f

  • r

p r

  • f

m e t h

  • d

s PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

  • n

l y i f P S L c

  • m

p l e t e s a p r

  • f

s e a r c h

  • n

l y i f P S L w i t h P G T c

  • m

p l e t e s a p r

  • f

s e a r c h Recommend how to apply induction without completing a proof.

slide-89
SLIDE 89

PSL can find how to apply induction for easy problems.

Too good to be true?

git clone https://github.com/data61/PSL

b u t P a M p e R d

  • e

s n

  • t

r e c

  • m

m e n d a r g u m e n t s f

  • r

p r

  • f

m e t h

  • d

s PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas.

  • n

l y i f P S L c

  • m

p l e t e s a p r

  • f

s e a r c h

  • n

l y i f P S L w i t h P G T c

  • m

p l e t e s a p r

  • f

s e a r c h Recommend how to apply induction without completing a proof. MeLoId: Machine Learning Induction

slide-90
SLIDE 90

How does MeLoId work?

[ apply(induct s), apply(induct t), apply(induct u), apply(induct s t arbitrary: u), … ] decision tree construction lookup preparation phase recommendation phase fast feature extractor

?

proof state proof engineer full feature extractor active mining about 40 assertions written in ML large proof corpora AFP and standard library lemma “foo x y = bar x y” apply(induct x arbitrary: y) [ ( apply(induct x arbitrary: y), used ), ( apply(induct y arbitrary: x), not ), ( apply(induct arbitrary: y), used ), ( apply(induct x rule: bar.induct), not ),… ] [ ( [1,0,0,1,…1], used ), ( [0,1,0,1,…1], not ), ( [1,1,0,0,…1], used ), ( [0,1,0,0,…1], not ), … ] lemma “f s t ==> g s u” Dynamic (Induct) [ [1,1,0,1,…1], [0,0,0,1,…1], [1,1,1,0,…1], [1,1,0,1,…1], … ] [ (0.3, apply(induct s t arbitrary: u)) (0.2, apply(induct s t)), (0.15, apply(induct t arbitrary: u)), (0.11, apply(induct u)), … ]

slide-91
SLIDE 91

How does MeLoId work?

[ apply(induct s), apply(induct t), apply(induct u), apply(induct s t arbitrary: u), … ] decision tree construction lookup preparation phase recommendation phase fast feature extractor

?

proof state proof engineer full feature extractor active mining about 40 assertions written in ML large proof corpora AFP and standard library lemma “foo x y = bar x y” apply(induct x arbitrary: y) [ ( apply(induct x arbitrary: y), used ), ( apply(induct y arbitrary: x), not ), ( apply(induct arbitrary: y), used ), ( apply(induct x rule: bar.induct), not ),… ] [ ( [1,0,0,1,…1], used ), ( [0,1,0,1,…1], not ), ( [1,1,0,0,…1], used ), ( [0,1,0,0,…1], not ), … ] lemma “f s t ==> g s u” Dynamic (Induct) [ [1,1,0,1,…1], [0,0,0,1,…1], [1,1,1,0,…1], [1,1,0,1,…1], … ] [ (0.3, apply(induct s t arbitrary: u)) (0.2, apply(induct s t)), (0.15, apply(induct t arbitrary: u)), (0.11, apply(induct u)), … ]

Writing useful assertions in ML is very tricky. => Domain specific language for writing assertions!

slide-92
SLIDE 92

How does MeLoId work?

[ apply(induct s), apply(induct t), apply(induct u), apply(induct s t arbitrary: u), … ] decision tree construction lookup preparation phase recommendation phase fast feature extractor

?

proof state proof engineer full feature extractor active mining about 40 assertions written in ML large proof corpora AFP and standard library lemma “foo x y = bar x y” apply(induct x arbitrary: y) [ ( apply(induct x arbitrary: y), used ), ( apply(induct y arbitrary: x), not ), ( apply(induct arbitrary: y), used ), ( apply(induct x rule: bar.induct), not ),… ] [ ( [1,0,0,1,…1], used ), ( [0,1,0,1,…1], not ), ( [1,1,0,0,…1], used ), ( [0,1,0,0,…1], not ), … ] lemma “f s t ==> g s u” Dynamic (Induct) [ [1,1,0,1,…1], [0,0,0,1,…1], [1,1,1,0,…1], [1,1,0,1,…1], … ] [ (0.3, apply(induct s t arbitrary: u)) (0.2, apply(induct s t)), (0.15, apply(induct t arbitrary: u)), (0.11, apply(induct u)), … ]

Writing useful assertions in ML is very tricky. => Domain specific language for writing assertions!

WIP!

slide-93
SLIDE 93

Thank you!

git clone https://github.com/data61/PSL

slide-94
SLIDE 94

Thank you!

Leave a star at GitHub for PSL!

git clone https://github.com/data61/PSL

slide-95
SLIDE 95

Thank you!

Leave a star at GitHub for PSL!

git clone https://github.com/data61/PSL

Let’s write a review paper “AITP deserves High-Performance Computing, Too!”

slide-96
SLIDE 96

Thank you!

Leave a star at GitHub for PSL!

git clone https://github.com/data61/PSL

Let’s write a review paper “AITP deserves High-Performance Computing, Too!”

PaMpeR’s feature extractor?

slide-97
SLIDE 97

Time

git clone https://github.com/data61/PSL

slide-98
SLIDE 98

Time

1986~ Isabelle

git clone https://github.com/data61/PSL

slide-99
SLIDE 99

Time

1986~ Isabelle 2004~ AFP

git clone https://github.com/data61/PSL

slide-100
SLIDE 100

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR

git clone https://github.com/data61/PSL

slide-101
SLIDE 101

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP

git clone https://github.com/data61/PSL

slide-102
SLIDE 102

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP 2018 PaMpeR’s data extraction

git clone https://github.com/data61/PSL

slide-103
SLIDE 103

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP 2019 definition of the “sep” function 2018 PaMpeR’s data extraction

git clone https://github.com/data61/PSL

slide-104
SLIDE 104

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP 2019 definition of the “sep” function 2018 PaMpeR’s data extraction lemma “map f (sep x xs) = sep (f x) (map f xs)"

git clone https://github.com/data61/PSL

slide-105
SLIDE 105

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP 2019 definition of the “sep” function 2018 PaMpeR’s data extraction lemma “map f (sep x xs) = sep (f x) (map f xs)" AITP2019 which_method?

git clone https://github.com/data61/PSL

slide-106
SLIDE 106

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP 2019 definition of the “sep” function 2018 PaMpeR’s data extraction lemma “map f (sep x xs) = sep (f x) (map f xs)" AITP2019 which_method?

PaMpeR’s feature extractor has to be able to analyze things (e.g. “sep”) that do not exist yet!

git clone https://github.com/data61/PSL

slide-107
SLIDE 107

Time

1986~ Isabelle 2004~ AFP 2017~ PaMpeR 2018~ more articles in the AFP 2019 definition of the “sep” function 2018 PaMpeR’s data extraction lemma “map f (sep x xs) = sep (f x) (map f xs)" AITP2019 which_method?

DEMO! PaMpeR’s feature extractor has to be able to analyze things (e.g. “sep”) that do not exist yet!

git clone https://github.com/data61/PSL

slide-108
SLIDE 108

Feature extractor?

slide-109
SLIDE 109

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

slide-110
SLIDE 110

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

assertion 27: if the outermost constant is the HOL equality?

slide-111
SLIDE 111

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

assertion 27: if the outermost constant is the HOL equality?

slide-112
SLIDE 112

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier?

slide-113
SLIDE 113

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier?

slide-114
SLIDE 114

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”?

slide-115
SLIDE 115

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc.

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”?

slide-116
SLIDE 116

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc. assertion 10: the context has a related recursive simplification rule?

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”?

slide-117
SLIDE 117

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc. assertion 10: the context has a related recursive simplification rule?

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”?

slide-118
SLIDE 118

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc. assertion 10: the context has a related recursive simplification rule?

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”?

slide-119
SLIDE 119

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc. assertion 10: the context has a related recursive simplification rule?

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”? assertion 58: the context has a constant defined with the “fun” keyword?

slide-120
SLIDE 120

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc. assertion 10: the context has a related recursive simplification rule?

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”? assertion 58: the context has a constant defined with the “fun” keyword?

slide-121
SLIDE 121

automatically proves and saves many auxiliary lemmas in the context sep.simps, sep.induct, sep.elims, etc. assertion 10: the context has a related recursive simplification rule?

Feature extractor?

assertion 27: if the outermost constant is the HOL equality? assertion 32: if the outermost constant is the HOL existential quantifier? assertion 93: if the goal has a term of type “real”? assertion 58: the context has a constant defined with the “fun” keyword?

[…,1,…,1,…0,…,1,…0,…]

10th 27th 32nd 58th 93rd

resulting feature vector:

slide-122
SLIDE 122

What assertions I wanted to write / wrote…

slide-123
SLIDE 123

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01:

slide-124
SLIDE 124

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01:

p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l !

slide-125
SLIDE 125

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01:

p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l !

slide-126
SLIDE 126

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01: Assertion 02: Do induction on argument number i if the function is defined by recursion in argument number i?

p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l !

slide-127
SLIDE 127

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01: Assertion 02: Do induction on argument number i if the function is defined by recursion in argument number i?

definition of constants! p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l !

slide-128
SLIDE 128

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01: Assertion 02: Do induction on argument number i if the function is defined by recursion in argument number i? Assertion03: Are induction variables appear at the deepest level in the syntax tree?

definition of constants! p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l !

slide-129
SLIDE 129

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01: Assertion 02: Do induction on argument number i if the function is defined by recursion in argument number i? Assertion03: Are induction variables appear at the deepest level in the syntax tree?

definition of constants! depth? un-currying! p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l !

slide-130
SLIDE 130

What assertions I wanted to write / wrote…

check if the induction variables (x and xs) are arguments of the constant (sep) that has an auxiliary lemma (sep.induct). If the induct method uses an auxiliary lemma (sep.induct) … Assertion 01: Assertion 02: Do induction on argument number i if the function is defined by recursion in argument number i? Assertion03: Are induction variables appear at the deepest level in the syntax tree?

definition of constants! depth? un-currying! p

  • s

i t i

  • n
  • f

a r g u m e n t s r e l a t i v e t

  • c

e r t a i n c

  • n

s t a n t s ! I n d u c t i

  • n

v a r i a b l e s ( x a n d x s ) a p p e a r m u l t i p l e t i m e s i n t h e g

  • a

l ! P x y ==> Q y z ==> R z w

slide-131
SLIDE 131

Time

git clone https://github.com/data61/PSL

slide-132
SLIDE 132

Time 2017: PSL

git clone https://github.com/data61/PSL

slide-133
SLIDE 133

Time 2017: PSL 2019: define the “sep” function

git clone https://github.com/data61/PSL

slide-134
SLIDE 134

Time 2017: PSL 2019: define the “sep” function

git clone https://github.com/data61/PSL

2019: define the “DInd” strategy

slide-135
SLIDE 135

Time 2017: PSL 2019: define the “sep” function 2019:

git clone https://github.com/data61/PSL

2019: define the “DInd” strategy

slide-136
SLIDE 136

Time 2017: PSL 2019: define the “sep” function 2019:

git clone https://github.com/data61/PSL

At the time of development (2017), PSL does not know about

  • user defined constants (e.g. “sep”) or
  • user defined proof strategies (e.g. DInd).

2019: define the “DInd” strategy