Towards Machine Learning Induction for Isabelle/HOL This work was - PowerPoint PPT Presentation

git clone https://github.com/data61/PSL goal apply (induct xs) apply (induct) apply (induct xs rule: Demo.sep.induct) Dynamic ( Induct ) Auto apply (auto) apply (auto) apply (auto) done IsSolved

git clone https://github.com/data61/PSL Try_Hard: the default strategy strategy Try_Hard = Ors [Thens [Subgoal, Basic], Thens [DInductTac, Auto_Solve], strategy Basic = Thens [DCaseTac, Auto_Solve], Ors [ Thens [Subgoal, Advanced], Auto_Solve, Thens [DCaseTac, Solve_Many], Blast_Solve, Thens [DInductTac, Solve_Many] ] FF_Solve, Thens [IntroClasses, Auto_Solve], Thens [Transfer, Auto_Solve], 16 percentage point performance Thens [Normalization, IsSolved], improvement compared to sledgehammer Thens [DInduct, Auto_Solve], Thens [Hammer, IsSolved], Thens [DCases, Auto_Solve], but the search space explodes Thens [DCoinduction, Auto_Solve], Thens [Auto, RepeatN(Hammer), IsSolved], Thens [DAuto, IsSolved]] PaMpeR: Proof Method Recommendation

preparation phase How does PaMpeR work? recommendation phase

preparation phase large proof corpora AFP and standard library How does PaMpeR work? recommendation phase proof ? state proof engineer

preparation phase large proof corpora AFP and standard library How does Archive of Formal Proofs (https://www.isa-afp.org) PaMpeR work? recommendation phase proof ? state proof engineer

preparation phase large proof corpora AFP and standard library How does PaMpeR work? recommendation phase proof ? state proof engineer

full feature extractor preparation phase large proof corpora 6021 CPU hours AFP and standard library 108 assertions How does PaMpeR work? recommendation phase proof ? state proof engineer

database ( 425334 data points ) full feature extractor preparation phase large proof corpora :: ( tactic_name, [ bool ] ) 6021 CPU hours AFP and standard library 108 assertions How does PaMpeR work? recommendation phase proof ? state proof engineer

database ( 425334 data points ) full feature extractor preparation phase large proof corpora :: ( tactic_name, [ bool ] ) preprocess 6021 CPU hours AFP and standard library 108 assertions How does decision tree construction PaMpeR work? recommendation phase proof ? state proof engineer

database ( 425334 data points ) full feature extractor preparation phase large proof corpora :: ( tactic_name, [ bool ] ) preprocess 6021 CPU hours AFP and standard library 108 assertions How does decision tree construction PaMpeR work? fast feature extractor recommendation phase proof ? state proof engineer

database ( 425334 data points ) full feature extractor preparation phase large proof corpora :: ( tactic_name, [ bool ] ) preprocess 6021 CPU hours AFP and standard library 108 assertions How does decision tree construction PaMpeR work? fast feature extractor recommendation phase feature vector proof ? state proof engineer

database ( 425334 data points ) full feature extractor preparation phase large proof corpora :: ( tactic_name, [ bool ] ) preprocess 6021 CPU hours AFP and standard library 108 assertions How does decision tree construction PaMpeR work? fast feature extractor recommendation phase feature vector proof ? state lookup proof engineer proof method recommendation

database ( 425334 data points ) full feature extractor preparation phase large proof corpora :: ( tactic_name, [ bool ] ) preprocess 6021 CPU hours AFP and standard library ? 108 assertions How does decision tree construction PaMpeR work? fast feature extractor recommendation phase feature vector proof ? state lookup proof engineer proof method recommendation

git clone https://github.com/data61/PSL AITP2018 review ? anonymous reviewer

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! ? anonymous reviewer

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? anonymous reviewer

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? anonymous reviewer

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? New users of Isabelle are facing many challenges from - writing their first definitions, - stating suitable theorem statements, and - anonymous producing properly structured proofs. reviewer

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? New users of Isabelle are facing many challenges from - writing their first definitions, - stating suitable theorem statements, and - anonymous producing properly structured proofs. reviewer Proof methods are merely the bits at the bottom of that.

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? New users of Isabelle are facing many challenges from - writing their first definitions, - stating suitable theorem statements, and - anonymous producing properly structured proofs. reviewer Proof methods are merely the bits at the bottom of that. I was writing how to prove not how to specify!

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? New users of Isabelle are facing many challenges from - writing their first definitions, - stating suitable theorem statements, and - anonymous producing properly structured proofs. reviewer Proof methods are merely the bits at the bottom of that. I was writing how to prove not how to specify!

git clone https://github.com/data61/PSL AITP2018 review Proof Method Recommendation, PaMpeR! I have doubts about various approaches proposed in the paper. ? New users of Isabelle are facing many challenges from - writing their first definitions, - stating suitable theorem statements, and - anonymous producing properly structured proofs. reviewer Proof methods are merely the bits at the bottom of that. I was writing how to prove not how to specify! Proof Goal Transformer, PGT!

git clone https://github.com/data61/PSL PSL with PGT PGT

git clone https://github.com/data61/PSL PSL with PGT proof goal sub-optimal for proof automation context PGT strategy PGT

git clone https://github.com/data61/PSL PSL with PGT proof goal sub-optimal proof goal context for proof automation tactic / sub-tool context PGT strategy PGT

git clone https://github.com/data61/PSL PSL with PGT proof goal sub-optimal proof goal context for proof automation tactic / sub-tool context PGT strategy PGT proved theorem / subgoals / message

git clone https://github.com/data61/PSL PSL with PGT proof goal sub-optimal proof goal context for proof automation tactic / sub-tool context PGT strategy PGT proof for the original goal, proved theorem / and auxiliary lemma subgoals / message optimal for proof automation

git clone https://github.com/data61/PSL PSL with PGT proof goal sub-optimal proof goal context for proof automation tactic / sub-tool context PGT strategy PGT proof for the original goal, proved theorem / and auxiliary lemma subgoals / message DEMO! optimal for proof automation

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

git clone https://github.com/data61/PSL goal Conjecture Fastforce Quickcheck DInd DInd

PSL can find how to apply induction for easy problems. git clone https://github.com/data61/PSL Success story

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. git clone https://github.com/data61/PSL Success story

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas. git clone https://github.com/data61/PSL Success story

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas. git clone https://github.com/data61/PSL Success story CADE2017

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas. git clone https://github.com/data61/PSL Success story CADE2017 ASE2018

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas. git clone https://github.com/data61/PSL Success story CADE2017 ASE2018 CICM2018 (best system award)

PSL can find how to apply induction for easy problems. PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas. git clone https://github.com/data61/PSL Too good to be true?

PSL can find how to apply PaMpeR recommends which proof methods to use. PGT produces useful auxiliary lemmas. induction for easy problems. git clone https://github.com/data61/PSL Too good to be true? h c r a e s f o o r p a s e t e l p m o c L S P f i y l n o a s e t e l p m o c T G P h t i w L S P f i y l n o h c r a e s f o o r p

PSL can find how to apply PaMpeR recommends which PGT produces useful auxiliary lemmas. induction for easy problems. proof methods to use. git clone https://github.com/data61/PSL Too good to be true? h c r a e s f o o r p a s e t e l p m o c L S P f i y l n o d n e m m o c e r t o n s e o d R e p M a P t u b s d o h t e m f o o r p r o f s t n e m u g r a a s e t e l p m o c T G P h t i w L S P f i y l n o h c r a e s f o o r p

PSL can find how to apply PaMpeR recommends which PGT produces useful auxiliary lemmas. induction for easy problems. proof methods to use. git clone https://github.com/data61/PSL Too good to be true? h c r a e s f o o r p a s e t e l p m o c L S P f i y l n o d n e m m o c e r t o n s e o d R e p M a P t u b s d o h t e m f o o Recommend how to r p r o f s t n e m u g r a apply induction without completing a proof. a s e t e l p m o c T G P h t i w L S P f i y l n o h c r a e s f o o r p

PSL can find how to apply PaMpeR recommends which PGT produces useful auxiliary lemmas. induction for easy problems. proof methods to use. git clone https://github.com/data61/PSL Too good to be true? h c r a e s f o o r p a s e t e l p m o c L S P f i y l n o d n e m m o c e r t o n s e o d R e p M a P t u b s d o h t e m f o o Recommend how to r p r o f s t n e m u g r a apply induction without completing a proof. a s e t e l p m o c T G P h t i w L S P f i y MeLoId: Machine l n o h c r a e s f o o r p Learning Induction

preparation phase active mining [ ( apply(induct x arbitrary: y), used ), ( apply(induct y arbitrary: x), not ), lemma “foo x y = bar x y” ( apply(induct arbitrary: y), used ), apply(induct x arbitrary: y) ( apply(induct x rule: bar.induct), not ),… ] large proof corpora [ ( [1,0,0,1,…1], used ), ( [0,1,0,1,…1], not ), full feature ( [1,1,0,0,…1], used ), extractor ( [0,1,0,0,…1], not ), … ] AFP and standard library decision tree about 40 assertions construction How does written in ML MeLoId work? recommendation phase fast feature extractor [ [1,1,0,1,…1], Dynamic [ apply(induct s), [0,0,0,1,…1], (Induct) lemma “f s t ==> g s u” apply(induct t), [1,1,1,0,…1], apply(induct u), [1,1,0,1,…1], … ] apply(induct s t arbitrary: u), … ] proof ? state lookup proof [ (0.3, apply(induct s t arbitrary: u)) engineer (0.2, apply(induct s t)), (0.15, apply(induct t arbitrary: u)), (0.11, apply(induct u)), … ]

preparation phase active mining [ ( apply(induct x arbitrary: y), used ), ( apply(induct y arbitrary: x), not ), lemma “foo x y = bar x y” ( apply(induct arbitrary: y), used ), apply(induct x arbitrary: y) ( apply(induct x rule: bar.induct), not ),… ] large proof corpora [ ( [1,0,0,1,…1], used ), ( [0,1,0,1,…1], not ), full feature ( [1,1,0,0,…1], used ), extractor ( [0,1,0,0,…1], not ), … ] AFP and standard library decision tree about 40 assertions construction How does written in ML MeLoId work? Writing useful assertions in ML is very tricky. => Domain specific language for writing assertions! recommendation phase fast feature extractor [ [1,1,0,1,…1], Dynamic [ apply(induct s), [0,0,0,1,…1], (Induct) lemma “f s t ==> g s u” apply(induct t), [1,1,1,0,…1], apply(induct u), [1,1,0,1,…1], … ] apply(induct s t arbitrary: u), … ] proof ? state lookup proof [ (0.3, apply(induct s t arbitrary: u)) engineer (0.2, apply(induct s t)), (0.15, apply(induct t arbitrary: u)), (0.11, apply(induct u)), … ]

preparation phase active mining [ ( apply(induct x arbitrary: y), used ), ( apply(induct y arbitrary: x), not ), lemma “foo x y = bar x y” ( apply(induct arbitrary: y), used ), apply(induct x arbitrary: y) ( apply(induct x rule: bar.induct), not ),… ] large proof corpora [ ( [1,0,0,1,…1], used ), ( [0,1,0,1,…1], not ), full feature ( [1,1,0,0,…1], used ), extractor ( [0,1,0,0,…1], not ), … ] AFP and standard library decision tree about 40 assertions construction How does written in ML MeLoId work? Writing useful assertions in ML is very tricky. => Domain specific language for writing assertions! recommendation phase fast feature extractor WIP! [ [1,1,0,1,…1], Dynamic [ apply(induct s), [0,0,0,1,…1], (Induct) lemma “f s t ==> g s u” apply(induct t), [1,1,1,0,…1], apply(induct u), [1,1,0,1,…1], … ] apply(induct s t arbitrary: u), … ] proof ? state lookup proof [ (0.3, apply(induct s t arbitrary: u)) engineer (0.2, apply(induct s t)), (0.15, apply(induct t arbitrary: u)), (0.11, apply(induct u)), … ]

git clone https://github.com/data61/PSL Thank you!

git clone https://github.com/data61/PSL Thank you! Leave a star at GitHub for PSL!

git clone https://github.com/data61/PSL Thank you! Leave a star at GitHub for PSL! Let’s write a review paper “AITP deserves High-Performance Computing, Too!”

git clone https://github.com/data61/PSL Thank you! Leave a star at GitHub for PSL! Let’s write a review paper “AITP deserves High-Performance Computing, Too!” PaMpeR’s feature extractor?

git clone https://github.com/data61/PSL Time

git clone https://github.com/data61/PSL 1986~ Isabelle Time

git clone https://github.com/data61/PSL 1986~ Isabelle Time 2004~ AFP

git clone https://github.com/data61/PSL 2017~ PaMpeR 1986~ Isabelle Time 2004~ AFP