Introduction to Inductive Logic Programming Stephen Muggleton - PowerPoint PPT Presentation

Introduction to Inductive Logic Programming Stephen Muggleton Department of Computing Imperial College, London

Motivation Logic Program [Kowalski, 1980] Inductive Logic Programming [Muggleton, 1991] Machine Learn arbitrary programs State-of-the-art ILP systems lacked Predicate Invention and Recursion [Muggleton et al, 2011]

Family relations (Dyadic) Family tree Target Theory Bob father ( ted, bob ) ← Jill Ted Jane Alice father ( ted, jane ) ← Bill parent ( X, Y ) ← mother ( X, Y ) Megan Jake Matilda Sam parent ( X, Y ) ← father ( X, Y ) Liz Harry John ancestor ( X, Y ) ← parent ( X, Y ) Mary Jo Susan ancestor ( X, Y ) ← parent ( X, Z ) , Andy ancestor ( Z, Y )

Generalised Meta-Interpreter prove ([] , BK, BK ) . prove ([ Atom | As ] , BK, BK H ) : − metarule ( Name, MetaSub, ( Atom :- Body ) , Order ) , Order, save subst ( metasub ( Name, MetaSub ) , BK, BK C ) , prove ( Body, BK C, BK Cs ) , prove ( As, BK Cs, BK H ) .

Metarules Name Meta-Rule Order P ( X, Y ) ← Instance True Base P ( x, y ) ← Q ( x, y ) P ≻ Q Chain P ( x, y ) ← Q ( x, z ) , R ( z, y ) P ≻ Q, P ≻ R TailRec P ( x, y ) ← Q ( x, z ) , P ( z, y ) P ≻ Q, x ≻ z ≻ y

Minimising sets of Metarules [ILP 2014] Set of Metarules Reduced Set P ( X, Y ) ← Q ( X, Y ) P ( X, Y ) ← Q ( Y, X ) P ( X, Y ) ← Q ( Y, X ) P ( X, Y ) ← Q ( X, Y ) , R ( Y, X ) P ( X, Y ) ← Q ( X, Y ) , R ( Y, Z ) P ( X, Y ) ← Q ( X, Y ) , R ( Z, Y ) P ( X, Y ) ← Q ( X, Z ) , R ( Z, Y ) P ( X, Y ) ← Q ( X, Z ) , R ( Z, Y ) .. P ( X, Y ) ← Q ( Z, Y ) , R ( Z, X )

Metagol implementation • Andrew Cropper’s YAP implementation - https://github.com/metagol/metagol . • Hank Conn’s Web interface - https://github.com/metagol/metagol web interface . • Live web-interface - http://c4778cab.ngrok.io/metagol/index.php

Vision applications Staircase Regular Geometric ILP 2013 ILP 2015 stair(X,Y) :- a(X,Y). stair(X,Y) :- a(X,Z), stair(Z,Y) . a(X,Y) :- vertical(X,Z), horizontal(Z,Y). Learned in 0.08s on laptop from single image. Note Predicate invention and recursion .

Scientific images [ILP 2017] Moon and Protist data North 12 11 1 10 2 West East 9 3 4 8 7 5 6 South Expt 1+2 - LogVis (1-shot) vs Stats Classifier (30-shot) LogVis Moon - 2 primitives, 5 invented, 88% vs 78%

Robotic applications L 2 L 1 a) b) c) Building a Stable Wall Learning Efficient Strategies IJCAI 2013 IJCAI 2015 T T C T C Initial state Final state IJCAI 2016 Abstraction and Invention

Language applications Formal grammars [MLJ 2014] Dependent string transformations [ECAI 2014] Dependent Learning Independent Learning Size Bound Time Out 17 4 9 5 9 5 3 13 11 3 1 6 7 8 12 4 5 7 8 4 12 13 11 6 1 10 15 3 17 10 2 2 15 2 14 16 14 16 1

Chain of programs from dependent learning f 03 (A,B) :- f 12 1 (A,C), f 12 (C,B). f 12 (A,B) :- f 12 1 (A,C), f 12 2 (C,B). f 12 1 (A,B) :- f 12 2 (A,C), skip 1 (C,B). f 12 2 (A,B) :- f 12 3 (A,C), write 1 (C,B,’.’). f 12 3 (A,B) :- copy 1 (A,C), f 17 1 (C,B). f 17 (A,B) :- f 17 1 (A,C), f 15 (C,B). f 17 1 (A,B) :- f 15 1 (A,C), f 17 1 (C,B). f 17 1 (A,B) :- skipalphanum (A,B). f 15 (A,B) :- f 15 1 (A,C), f 16 (C,B). f 15 1 (A,B) :- skipalphanum (A,C), skip 1 (C,B). f 16 (A,B) :- copyalphanum (A,C), skiprest (C,B).

Other applications Learning proof tactics [ILP 2015] Learning data transformations [ILP 2015]

Bayesian Meta-Interpretive Learning 0.1 0.1 Clauses 0.1 .. delta(Q0,0,Q0) delta(Q0,0,Q1) delta(Q2,1,Q2) 0.15 0.15 .. delta(Q0,0,Q0),delta(Q0,1,Q1) delta(Q0,0,Q0),accept(Q0) Finite 0.1 0.1 State 0.1 0 .. 0 q0 q0 q1 1 q2 Acceptors 0.15 0.15 (FSAs) .. 1 0 q0 q1 0 q0

Related work Predicate Invention. Early ILP [Muggleton and Buntine, 1988; Rouveirol and Puget, 1989; Stahl 1992] Abductive Predicate Invention. Propositional Meta-level abduction [Inoue et al., 2010] Meta-Interpretive Learning. Learning regular and context-free grammars [Muggleton et al, 2013] Higher-order Logic Learning. Without background knowledge [Feng and Muggleton, 1992; Lloyd 2003] Higher-order Datalog. HO-Progol learning [Pahlavi and Muggleton, 2012]

Conclusions and Challenges • New form of Declarative Machine Learning [De Raedt, 2012] • H 2 2 is tractable and Turing-complete fragment of High-order Logic • Knuth-Bendix style ordering guarantees termination of queries • Beyond classification learning - strategy learning Challenges • Generalise beyond Dyadic logic • Deal with classification noise • Active learning • Efficient problem decomposition • Meaningful invented names and types

Bibliography • A. Cropper, S.H. Muggleton. Learning efficient logical robot strategies involving composable objects. IJCAI 2015. • A. Cropper and S.H. Muggleton. Learning higher-order logic programs through abstraction and invention. IJCAI 2016. • W-Z Dai, S.H. Muggleton, J.Wen, A. Tamaddoni-Nezhad, Z-H Zhou. Logical vision: One-shot meta-interpretive learning from real images. ILP 2017. • S.H. Muggleton, D. Lin, A. Tamaddoni-Nezhad. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited. Machine Learning, 2015. • D. Lin, E. Dechter, K. Ellis, J.B. Tenenbaum, S.H. Muggleton. Bias reformulation for one-shot function induction. ECAI 2014.