logic is everywhere The Core Method la l ogica est a por todas - PowerPoint PPT Presentation

Hikmat har Jaga Hai The First Order CORE Method logika je svuda � � ��� � � �� ���� � Steffen H¨ olldobler � � �� Mantık her yerde International Center for Computational Logic logika je vˇ sude Technische Universit¨ at Dresden la logica ` e dappertutto Germany logic is everywhere ◮ The Core Method la l´ ogica est´ a por todas partes ◮ Logic Programs ��� � �� ��� ◮ Mapping Interpretations to Real Numbers � ��� � � �� ◮ Approximation of Interpretations Logik ist ¨ uberall ◮ Constructive Approaches Logika ada di mana-mana ◮ Implementation Logica este peste tot ◮ Open Problems ��� � ���� ��� �������� �� � �� � a l´ ogica est´ a em toda parte la logique est partout Steffen H¨ olldobler The First Order CORE Method 1

The CORE Method ◮ Relate logic programs and connectionist systems. ◮ Embed interpretations into (vectors of) real numbers. ◮ Hence, obtain an embedded version of the T P -operator. ◮ Construct a network computing one application of f P . ◮ Add recurrent connections from output to input layer. ◮ Compute (or approximate) the least fixed point of T P . embedding writable trainable Symbolic Connectionist System System extraction readable Steffen H¨ olldobler The First Order CORE Method 2

Logic Programs ◮ A logic program P over a first-order language L is a finite set of clauses A ← L 1 ∧ . . . ∧ L n , where A is an atom, L i are literals and n ≥ 0 . ◮ T L is the set of all ground terms over L . ◮ B L is the set of all ground atoms over L called Herbrand base. ◮ A Herbrand interpretation I is a mapping B L → {⊤ , ⊥} . ◮ 2 B L is the set of all Herbrand interpretations. ◮ g P is the set of all ground instances of clauses in P . P : 2 B L → 2 B L : ◮ Immediate consequence operator T P ( I ) = { A | there is a clause A ← L 1 ∧ . . . ∧ L n ∈ g P T such that I | = L 1 ∧ . . . ∧ L n } . ◮ I is a supported model iff T P ( I ) = I . Steffen H¨ olldobler 3 The First Order CORE Method

Two Examples ◮ Natural numbers n 0 % 0 is a natural number. nsX ← nX % The successor sX is a natural % number if X is a natural number. ◮ Even and odd numbers e 0 % 0 is an even number. esX ← oX % The successor of an odd X is even. oX ← ¬ eX % If X is not even then it is odd. ⊲ Herbrand base B L = { e 0 , es 0 , . . . , o 0 , os 0 , . . . } ⊲ Some interpretations { es 2 m 0 | m ≥ 1 } I 1 ∩ I 2 = ∅ I 1 = I 3 = { os 2 m +1 0 | m ≥ 0 } I 1 ∪ I 3 I 2 = I 4 = Steffen H¨ olldobler The First Order CORE Method 4

The Immediate Consequence Operator P ( I ) = { A | there is a clause A ← L 1 ∧ . . . ∧ L n ∈ g P ◮ T such that I | = L 1 ∧ . . . ∧ L n } . ◮ Natural Numbers { n 0 , nsX ← nX } ∅ �→ { n 0 } { n 0 } �→ { n 0 , ns 0 } { n 0 , ns 0 } �→ { n 0 , ns 0 , nss 0 } �→ B L B L ◮ Even and odd numbers { e 0 , esX ← oX, oX ← ¬ eX } ∅ �→ { e 0 , oX | X ∈ T L } { oX | X ∈ T L } �→ { e 0 , esX, oX | X ∈ T L } { es 2 m 0 | n ≥ 0 } { e 0 , os 2 m +1 0 | m ≥ 0 } �→ { os 2 m +1 0 | n ≥ 0 } { e 0 , es 2 m 0 | m ≥ 0 } �→ �→ { e 0 , esX | X ∈ T L } B L Steffen H¨ olldobler 5 The First Order CORE Method

The Initial Approach ◮ H¨ olldobler, Kalinke, St¨ orr 1999: Can the core method be extended to first-order logic programs? ◮ Problem ⊲ Given a logic program P over a first order language L P : 2 B L → 2 B L . together with T ⊲ B L is countably infinite. ⊲ The method used to relate propositional logic and connectionist systems is not applicable. ⊲ How can the gap between the discrete, symbolic setting of logic and the continuous, real valued setting of connectionist networks be closed? Steffen H¨ olldobler 6 The First Order CORE Method

The Goal ◮ Find rep : 2 B L → R and f P : R → R such that the following conditions hold. P ( I ) = I ′ f P ( rep ( I )) = rep ( I ′ ) . implies ⊲ T f P ( x ) = x ′ P ( rep − 1 ( x )) = rep − 1 ( x ′ ) . implies T ◮ f P is a sound and complete encoding of T P . ◮ P is a contraction on 2 B L iff f P is a contraction on R . ⊲ T ◮ The contraction property and fixed points are preserved. ◮ ⊲ f P is continuous on R . ◮ A connectionst network approximating f P is known to exist. ◮ ◮ f P and, hence, T P can be trained by backpropagation ◮ and related training methods. Steffen H¨ olldobler 7 The First Order CORE Method

Level Mappings ◮ Let P be a program over a first order language L . ◮ A level mapping for P is a function l : B L → N . ⊲ We define l ( ¬ A ) = l ( A ) . ◮ Examples ⊲ Natural Numbers { n 0 , nsX ← nX } l ( ns m 0) = m + 1 ⊲ Even and odd numbers { e 0 , esX ← oX, oX ← ¬ eX } l ( es m 0) = 2 n + 1 , l ( os m 0) = 2 m + 2 Steffen H¨ olldobler 8 The First Order CORE Method

Acyclic Logic Programs ◮ We can associate a metric d L with L and a level mapping l as follows. Let I, J ∈ 2 B L :  0 if I = J d L ( I, J ) = 2 − n if n is the smallest level on which I and J differ. ◮ Proposition (Fitting 1994) (2 B L , d L ) is a complete metric space. ◮ P is said to be acyclic wrt a level mapping l , if for every A ← L 1 ∧ . . . ∧ L n ∈ g P we find l ( A ) > l ( L i ) for all i . ◮ P is said to be acyclic if P is acylic wrt some level mapping. ⊲ Both running examples are acyclic. ◮ Proposition Let P be an acyclic logic program wrt l and d L the metric associated P is a contraction on (2 B L , d L ) . with L and l , then T Steffen H¨ olldobler 9 The First Order CORE Method

Mapping Interpretations to Real Numbers ◮ Let l be a bijective level mapping. ◮ Let rep be defined as X 4 − l ( A ) . rep ( I ) = A ∈ I ◮ Example = { e 0 , o 0 , es 0 , os 0 , ess 0 , . . . } B L rep ( { e 0 } ) = 0 . 1 0 0 0 0 4 = 0 . 25 10 rep ( { e 0 , es 0 , ess 0 } ) = 0 . 1 0 1 0 1 4 ≈ 0 . 27 10 Steffen H¨ olldobler 10 The First Order CORE Method

The Set of all Embedded Interpretations ◮ Let E = { rep ( I ) | I ∈ 2 B L } ◮ Bader 2003 E is the attractor of an iterated function system. ✲ ✲ ✲ ✲ ◮ Proposition rep is a bijection between 2 B L and E . We have a sound and complete encoding of interpretations. ◮ Proposition E is compact. Steffen H¨ olldobler The First Order CORE Method 11

Mapping Immediate Consequence Operators to Functions on the Reals P ( rep − 1 ( r ))) . ◮ We define f P : E → E : r �→ rep ( T T P I ′ I ✲ rep rep ❄ ❄ r ′ r ✲ f P We have a sound and complete encoding of T P . ◮ Proposition Let P be an acylic program wrt a bijective level mapping. f P is a contraction on E . Contraction property and fixed points are preserved. Steffen H¨ olldobler The First Order CORE Method 12

Approximating Continuous Functions ◮ Corollary f P is continuous. ◮ Recall Funahashi’s theorem: ⊲ Let K ⊆ R n be compact. Every continuous function f : K → R can be uniformly approximated by input-output functions of 3-layer feed forward networks. ◮ Theorem f P can be uniformly approximated by input-output functions of 3-layer feed-forward networks. P can be approximated as well by applying rep − 1 . ⊲ T Connectionist network approximating immediate consequence operator exists. Steffen H¨ olldobler 13 The First Order CORE Method

An Example ◮ Consider P = { n 0 , nsX ← nX } and let l ( ns m 0) = m + 1 . ⊲ P is acyclic wrt l , l is bijective, rep ( B L ) = 1 3 . 4 − l ( n 0) + P nX ∈ I 4 − l ( nsX ) ⊲ f P ( rep ( I )) = 4 − l ( n 0) + P nX ∈ I 4 − ( l ( nX ))+1) = 1+ rep ( I ) = . 4 ◮ Approximation of f P to accuracy ε yields » 1 + x − ε, 1 + x – f ( x ) ∈ + ε . 4 4 ◮ Starting with some x and iterating f yields in the limit a value » 1 − 4 ε , 1 + 4 ε – r ∈ . 3 3 ◮ Select r ∈ E with minimal distance to r . ◮ Applying rep − 1 to r we find ns m 0 ∈ rep − 1 ( r ) if m < − log 4 ε − 1 . Steffen H¨ olldobler The First Order CORE Method 14

Approximation of Interpretations ◮ Let P be a logic program over a first order language L and l a level mapping. ◮ An interpretation I approximates an interpretation J to a degree n ∈ N if for all atoms A ∈ B L with l ( A ) < n we find I ( A ) = ⊤ iff J ( A ) = ⊤ . ⊲ I approximates J to a degree n iff d L ( I, J ) ≤ 2 − n . Steffen H¨ olldobler 15 The First Order CORE Method

Approximation of Supported Models ◮ Given an acyclic logic program P with bijective level mapping. P be the immediate consequence operator associated with P and ◮ Let T M P the least supported model of P . ◮ We can approximate T P by a 3-layer feed-forward network. ◮ We can turn this network into a recurrent one. Does the recurrent network approximate the supported model of P ? ◮ Theorem For an arbitrary m ∈ N there exists a recursive network with sigmoidal activation function for the hidden layer units and linear activation functions for the input and output layer units computing a function f P such that there exists an n 0 ∈ N such that for all n ≥ n 0 and for all x ∈ [ − 1 , 1] we find n d L ( rep − 1 ( f P ( x )) , M P ) ≤ 2 − m . Steffen H¨ olldobler 16 The First Order CORE Method