Hammer for Coq: Automation for Dependent Type Theory ukasz Czajka, - - PowerPoint PPT Presentation

▶

Apr 21, 2023 313 likes •790 views

Hammer for Coq: Automation for Dependent Type Theory ukasz Czajka, University of Copenhagen Cezary Kaliszyk, University of Innsbruck 29 March 2018 http://cl-informatik.uibk.ac.at/cek/coqhammer/ 1 / 16 Interactive Proof in Type Theory

SLIDE 1

Hammer for Coq: Automation for Dependent Type Theory

Łukasz Czajka, University of Copenhagen Cezary Kaliszyk, University of Innsbruck 29 March 2018

http://cl-informatik.uibk.ac.at/cek/coqhammer/

1/16

SLIDE 2

Interactive Proof in Type Theory

· Practical problem

http://cl-informatik.uibk.ac.at/cek/coqhammer/

2/16

SLIDE 3

Interactive Proof in Type Theory

· Practical problem

· large parts of proofs are tedious

http://cl-informatik.uibk.ac.at/cek/coqhammer/

2/16

SLIDE 4

Interactive Proof in Type Theory

· Practical problem

· large parts of proofs are tedious

· Automation for Interactive Proof

· Proof search: intuition, firstorder, · Decision Procedures: congruence, fourier, ring, omega, SMTCoq, ...

http://cl-informatik.uibk.ac.at/cek/coqhammer/

2/16

SLIDE 5

Interactive Proof in Type Theory

· Practical problem

· large parts of proofs are tedious

· Automation for Interactive Proof

· Proof search: intuition, firstorder, · Decision Procedures: congruence, fourier, ring, omega, SMTCoq, ...

· AI/ATP techniques: Hammers

· MizAR for Mizar · Sledgehammer for Isabelle/HOL · HOL(y)Hammer for HOL Light and HOL4 · CoqHammer for Coq

http://cl-informatik.uibk.ac.at/cek/coqhammer/

2/16

SLIDE 6

Hammers

· Hammer goal: provide efficient automated reasoning using facts from a large library.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

3/16

SLIDE 7

Hammers

· Hammer goal: provide efficient automated reasoning using facts from a large library. · Strong relevance filtering.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

3/16

SLIDE 8

Hammers

· Hammer goal: provide efficient automated reasoning using facts from a large library. · Strong relevance filtering. · Usable library search “modulo simple reasoning”.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

3/16

SLIDE 9

Hammers

· Hammer goal: provide efficient automated reasoning using facts from a large library. · Strong relevance filtering. · Usable library search “modulo simple reasoning”.

· We may not know the name of the lemma we want to apply.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

3/16

SLIDE 10

Hammers

· Hammer goal: provide efficient automated reasoning using facts from a large library. · Strong relevance filtering. · Usable library search “modulo simple reasoning”.

· We may not know the name of the lemma we want to apply. · There may be many equivalent formulations of the lemma – which one is used in the library?

http://cl-informatik.uibk.ac.at/cek/coqhammer/

3/16

SLIDE 11

Hammers

· Hammer goal: provide efficient automated reasoning using facts from a large library. · Strong relevance filtering. · Usable library search “modulo simple reasoning”.

· We may not know the name of the lemma we want to apply. · There may be many equivalent formulations of the lemma – which one is used in the library? · The exact lemma may not exist in the library, but it may “trivially” follow from a few other lemmas in the library.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

3/16

SLIDE 12

Hammer Overview

Proof Assistant Hammer ATP Current Goal TPTP ITP Proof ATP Proof

http://cl-informatik.uibk.ac.at/cek/coqhammer/

4/16

SLIDE 13

Hammers

Hammers work in three phases.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

5/16

SLIDE 14

Hammers

Hammers work in three phases. · Using machine-learning and AI techniques perform premise-selection: select about a few hundred to 1-2 thousand lemmas that are likely to be needed in the proof of the conjecture.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

5/16

SLIDE 15

Hammers

Hammers work in three phases. · Using machine-learning and AI techniques perform premise-selection: select about a few hundred to 1-2 thousand lemmas that are likely to be needed in the proof of the conjecture. · Translate the selected lemmas, together with the conjecture, from the logic of the ITP to a format accepted by powerful external automated theorem provers (ATPs) – most commonly untyped first-order logic with equality.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

5/16

SLIDE 16

Hammers

Hammers work in three phases. · Using machine-learning and AI techniques perform premise-selection: select about a few hundred to 1-2 thousand lemmas that are likely to be needed in the proof of the conjecture. · Translate the selected lemmas, together with the conjecture, from the logic of the ITP to a format accepted by powerful external automated theorem provers (ATPs) – most commonly untyped first-order logic with equality. Run the ATP(s) on the result of the translation.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

5/16

SLIDE 17

Hammers

Hammers work in three phases. · Using machine-learning and AI techniques perform premise-selection: select about a few hundred to 1-2 thousand lemmas that are likely to be needed in the proof of the conjecture. · Translate the selected lemmas, together with the conjecture, from the logic of the ITP to a format accepted by powerful external automated theorem provers (ATPs) – most commonly untyped first-order logic with equality. Run the ATP(s) on the result of the translation. · Reprove the conjecture in the logic of the ITP , using the information

btained in the ATP runs.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

5/16

SLIDE 18

Hammers

Hammers work in three phases. · Using machine-learning and AI techniques perform premise-selection: select about a few hundred to 1-2 thousand lemmas that are likely to be needed in the proof of the conjecture. · Translate the selected lemmas, together with the conjecture, from the logic of the ITP to a format accepted by powerful external automated theorem provers (ATPs) – most commonly untyped first-order logic with equality. Run the ATP(s) on the result of the translation. · Reprove the conjecture in the logic of the ITP , using the information

btained in the ATP runs. Typically, a list of (usually a few) lemmas

needed by an ATP to prove the conjecture is obtained from an ATP run, and we try to reprove the goal from these lemmas.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

5/16

SLIDE 19

Evaluations

Top-level goals: · HOL(y)Hammer

· Flyspeck text formalization: 47% · Similar results for HOL4 · Slightly weaker for CakeML

http://cl-informatik.uibk.ac.at/cek/coqhammer/

6/16

SLIDE 20

Evaluations

Top-level goals: · HOL(y)Hammer

· Flyspeck text formalization: 47% · Similar results for HOL4 · Slightly weaker for CakeML

· Sledgehammer

· Probability theory: 40% · Term rewriting: 44% · Java threads: 59%

http://cl-informatik.uibk.ac.at/cek/coqhammer/

6/16

SLIDE 21

Evaluations

Top-level goals: · HOL(y)Hammer

· Flyspeck text formalization: 47% · Similar results for HOL4 · Slightly weaker for CakeML

· Sledgehammer

· Probability theory: 40% · Term rewriting: 44% · Java threads: 59%

· MizAR

· Mizar Mathematical Library: 44%

http://cl-informatik.uibk.ac.at/cek/coqhammer/

6/16

SLIDE 22

Evaluations

Top-level goals: · HOL(y)Hammer

· Flyspeck text formalization: 47% · Similar results for HOL4 · Slightly weaker for CakeML

· Sledgehammer

· Probability theory: 40% · Term rewriting: 44% · Java threads: 59%

· MizAR

· Mizar Mathematical Library: 44%

· CoqHammer

· Coq standard library: 40%

http://cl-informatik.uibk.ac.at/cek/coqhammer/

6/16

SLIDE 23

CoqHammer demo

examples/imp.v

http://cl-informatik.uibk.ac.at/cek/coqhammer/

7/16

SLIDE 24

CoqHammer: premise selection

· Learning done each time the plugin is invoked (to include all accessible facts).

http://cl-informatik.uibk.ac.at/cek/coqhammer/

8/16

SLIDE 25

CoqHammer: premise selection

· Learning done each time the plugin is invoked (to include all accessible facts). · Two machine-learning filters: k-NN and naive Bayes.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

8/16

SLIDE 26

CoqHammer: premise selection

· Learning done each time the plugin is invoked (to include all accessible facts). · Two machine-learning filters: k-NN and naive Bayes. · Re-uses the HOLyHammer efficient implementation (also adapted by Sledgehammer).

http://cl-informatik.uibk.ac.at/cek/coqhammer/

8/16

SLIDE 27

Translation: target logic

Target logic: untyped FOL with equality.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

9/16

SLIDE 28

Translation

Three functions , , and . · : propositions → FOL formulas used for CIC0 terms of type Prop. · : types → guards used for CIC0 terms of type Type. · : all CIC0 → FOL terms

http://cl-informatik.uibk.ac.at/cek/coqhammer/

10/16

SLIDE 29

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 30

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 31

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 32

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x)

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 33

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x) · The function encodes Coq terms as FOL terms.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 34

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x) · The function encodes Coq terms as FOL terms.

· Γ (ts) is equal to:

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 35

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x) · The function encodes Coq terms as FOL terms.

· Γ (ts) is equal to:

· ǫ if Γ ⊢ ts : α : Prop,

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 36

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x) · The function encodes Coq terms as FOL terms.

· Γ (ts) is equal to:

· ǫ if Γ ⊢ ts : α : Prop, · Γ (t) if Γ ⊢ s : α : Prop,

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 37

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x) · The function encodes Coq terms as FOL terms.

· Γ (ts) is equal to:

· ǫ if Γ ⊢ ts : α : Prop, · Γ (t) if Γ ⊢ s : α : Prop, · Γ (t)Γ (s) otherwise.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 38

Translation

· The function encodes propositions as FOL formulas and is used for terms of Coq having type Prop.

· If Γ ⊢ t : Prop then Γ (Πx : t.s) = Γ (t) → Γ,x:t(s). · If Γ ⊢ t : Prop then Γ (Πx : t.s) = ∀x.Γ (t, x) → Γ,x:t(s).

· The function encodes types as guards and is used for terms of Coq which have type Type. For instance, for a (closed) type τ = Πx : α.β(x) we have (τ, f ) = ∀x.(α, x) → (β(x), f x) · The function encodes Coq terms as FOL terms.

· Γ (ts) is equal to:

· ǫ if Γ ⊢ ts : α : Prop, · Γ (t) if Γ ⊢ s : α : Prop, · Γ (t)Γ (s) otherwise.

· Γ (λ x : t.s) = F y where s does not start with a lambda-abstraction any more, F is a fresh constant, y = FV(λ x : t.s) and ∀ y.Γ (∀ x : t.F y x = s) is a new axiom.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

11/16

SLIDE 39

ATP invocation

· We use Vampire, E prover, and Z3.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

12/16

SLIDE 40

ATP invocation

· We use Vampire, E prover, and Z3. · The provers may be run in parallel with different numbers of premises and premise selection methods.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

12/16

SLIDE 41

Proof reconstruction

· Use dependencies from a successful ATP run.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

13/16

SLIDE 42

Proof reconstruction

· Use dependencies from a successful ATP run. · Do automatic proof search using different versions of our tactics (implemented in Ltac), with a fixed time limit for each.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

13/16

SLIDE 43

Proof reconstruction

· Use dependencies from a successful ATP run. · Do automatic proof search using different versions of our tactics (implemented in Ltac), with a fixed time limit for each. · 85% of proofs reconstructed.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

13/16

SLIDE 44

Overall hammer evaluation

All statements from the Coq standard libary ATP success 50% · ATPs used: E, Z3, Vampire with 30 seconds time limit Overall success 40.8% · 8 threads with different lemma selection, premises, provers, reconstruction

http://cl-informatik.uibk.ac.at/cek/coqhammer/

14/16

SLIDE 45

Conclusion

· Proof length already close to that of Isabelle/HOL.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

15/16

SLIDE 46

Conclusion

· Proof length already close to that of Isabelle/HOL. · Improvements needed for dependent types and boolean reflection.

http://cl-informatik.uibk.ac.at/cek/coqhammer/

15/16

SLIDE 47

Download

https://github.com/lukaszcz/coqhammer http://cl-informatik.uibk.ac.at/cek/coqhammer/

http://cl-informatik.uibk.ac.at/cek/coqhammer/

16/16