Logic Tensor Networks Luciano Serafini Fondazione Bruno Kessler - PowerPoint PPT Presentation

Logic Tensor Networks Luciano Serafini Fondazione Bruno Kessler AITP 2017 joint work with Artur d’Avila Garces - City Univ. London and Ivan Donadello, FBK Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 1 / 30

The SRL Mindmap . . . AI Perception NLP Statistical Relational Learning Planning is a subdiscipline of artificial intelligence that is concerned Learning with domain models that exhibit KRR both uncertainty and complex relational structure . SRL Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 2 / 30

Hybrid domains We are interested in Statistical Relational Learning over hybrid domains , i.e., domains that are characterized by the presence of structured data (categorical/semantic); continuous data (continuous features); Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 3 / 30

Hybrid domains Example (SRL domain) hp years dollar dollar 34 15342 10000 130.00 age income price engine power person car Kurt Car2 owns date since 2/2/95 madeBy livesIn company FCA Rome town locatedIn area km 2 Detroit town 53.72 Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 4 / 30

Tasks in Statistical Relational Learning Object Classification: Example (SRL domain) Predicting the type of an hp years dollar dollar object based on its relations 34 15342 10000 130.00 and attributes; Reletion detenction: age income price engine power Predicting if two objects are connected by a relation, based person car Kurt Car2 owns date on types and attributes of the since 2/2/95 madeBy participating objects; livesIn Regression: predicting the company FCA Rome town (distribution of) values of the attributies of an object, (a locatedIn area pair of related objects) based on the types and relations of km 2 Detroit town 53.72 the object(s) involved. Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 5 / 30

Real-world uncertain, structured and hybrid domains Robotics: a robot’s location is a continuous values while the the types of the objects it encounters can be described by discrete set of classes Semantic Image Interpretation: The visual features of a bounding box of a picture are con- tinuous values, while the types of objects con- tained in a bounding box and the relations be- tween them are taken from a discrete set Natural Language Processing: The distri- butional semantics provide a vectorial (numer- ical) representation of the meaning of words, while WordNet associates to each word a set of synsets and a set of relations with other words which are finite and discrete Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 6 / 30

Language - to specify knowledge about models Two sorted first order language: (abstract sort and numeric sort) Abstract constant symbols ( Ann , Bob , Cole ); Abstract function symbols (fatherOf(x)); Abstract relation symbols (Person(x), Town(x), LivesIn(x,y); Numeric function symbols (age(x),area(y), livingInSince(x,y) Symbols for real numbers (1 , 0, π , . . . ); Symbols for real functions x + y , √ x , . . . ); Symbols for real relations ( x = y , x < y ). COLOR CODE: denotes objects and relations of the domain structure; denotes attributes and relations between attributes of the numeric part of the domain. Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 7 / 30

Domain description and queries Example (Domain descritpion:) company(A), company(B), worksFor(Alice,A), worksFor(Ann,A), worksFor(Bob,B),worksFor(Bill,B); friends(Alice,Ann), friends(Bob,Bill), ¬ friends(Ann,Bill) Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 8 / 30

Domain description and queries Example (Domain descritpion:) company(A), company(B), worksFor(Alice,A), worksFor(Ann,A), worksFor(Bob,B),worksFor(Bill,B); friends(Alice,Ann), friends(Bob,Bill), ¬ friends(Ann,Bill) salary ( Alice ) = 10 . 000, salary ( Ann ) ≤ 12 . 000, salary ( Bob ) = 30 . 000, salary ( Bill ) ≥ 27 . 000, 9 . 000 ≤ Salary ( Chris ) ≤ 11 . 000 Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 8 / 30

Domain description and queries Example (Domain descritpion:) company(A), company(B), worksFor(Alice,A), worksFor(Ann,A), worksFor(Bob,B),worksFor(Bill,B); friends(Alice,Ann), friends(Bob,Bill), ¬ friends(Ann,Bill) salary ( Alice ) = 10 . 000, salary ( Ann ) ≤ 12 . 000, salary ( Bob ) = 30 . 000, salary ( Bill ) ≥ 27 . 000, 9 . 000 ≤ Salary ( Chris ) ≤ 11 . 000 ∀ x . worksFor ( x , A ) ↔ ¬ worksFor ( x , B ) ∀ xy . friends ( x , y ) ↔ friends ( y , x ) ∀ xy , worksFor ( x , y ) → salary ( x ) > 3 . 000 ∀ x ∃ y . friends ( x , y ) Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 8 / 30

Domain description and queries Example (Domain descritpion:) Example (Queries) company(A), company(B), ? worksfor ( Chris , B ) worksFor(Alice,A), worksFor(Ann,A), ? ?x: friends ( Chris , ? x ) worksFor(Bob,B),worksFor(Bill,B); ? ? salary ( Bill ) friends(Alice,Ann), friends(Bob,Bill), ¬ friends(Ann,Bill) ? ? salary ( x ) : x = friendOf ( Ann ) salary ( Alice ) = 10 . 000, ? ? worksfor ( x , z ) ∧ worksfor ( z , z ) → salary ( Ann ) ≤ 12 . 000, friends ( x , y ) salary ( Bob ) = 30 . 000, salary ( Bill ) ≥ 27 . 000, ? ? salary ( x ) > 15 . 000 → 9 . 000 ≤ Salary ( Chris ) ≤ 11 . 000 worksfor ( x , A ) ∀ x . worksFor ( x , A ) ↔ ¬ worksFor ( x , B ) ∀ xy . friends ( x , y ) ↔ friends ( y , x ) ∀ xy , worksFor ( x , y ) → salary ( x ) > 3 . 000 ∀ x ∃ y . friends ( x , y ) Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 8 / 30

Fuzzy semantics for LTN Let L contains the set r 1 , . . . , r n unary real functions (like age, salary, . . . ) Fuzzy Semantics An interpretation G of L , called grounding, is a real function: G ( c ) ∈ R n for every constant c ; → R n for every m -ary abstract function f ; G ( f ) ∈ R n · m − G ( P ) ∈ R n · m − → [0 , 1] for every m -ary abstract predic symbol P ; Given a grounding G the semantics of closed terms and atomic formulas is defined as follows: G ( f ( t 1 , . . . , t m )) = G ( f )( G ( t 1 ) , . . . , G ( t m )) G ( P ( t 1 , . . . , t m )) = G ( P )( G ( t 1 ) , . . . , G ( t m )) Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 9 / 30

Grounding as parametrized neural network = Logic Tensor Network (LTN) Grounding of constant symbols: Real vectors G ( c ) ∈ R n For every i G i ( c ) = r i ( c ) if r i ( c ) is known, otherwise G i ( c ) is a parameter of the LTN. Grounding of functional symbols: Two layer feed-forward neural network with m · n imput nodes and n output nodes. G ( f )( v ) = M f σ ( N f v ) M f ∈ R mn × n and N f ∈ R mn × mn are parameters of the LTN; Grounding of predicate symbols: Tensor quadratic network � � �� v ⊺ W [1: k ] G ( P )( v ) = σ u ⊺ P tanh v + V P v + b P P w P ∈ R k × mn × mn , V P ∈ R k × mn , b P ∈ R k , and u P ∈ R k are parameters of the LTN. Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 10 / 30

Grounding as parametrized neural network = Logic Tensor Network (LTN) Grounding of real functions are the real functions themselves. For instance: G (+)( v , u ) = v + u Grounding of real relations are the real relations themselves. For instance: � 1 if v = u G (=)( v , u ) = 0 Otherwise or some soft version v · u G (=)( v , u ) = || v || || u || Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 11 / 30

Domain description and queries Example (Domain descritpion:) Example (Queries) company(A), company(B), ? worksfor ( Chris , B ) worksFor(Alice,A), worksFor(Ann,A), ? ?x: friends ( Chris , ? x ) worksFor(Bob,B),worksFor(Bill,B); ? ? salary ( Bill ) friends(Alice,Ann), friends(Bob,Bill), ¬ friends(Ann,Bill) ? ? salary ( x ) : x = friendOf ( Ann ) salary ( Alice ) = 10 . 000, salary ( Ann ) ≤ 12 . 000, salary ( Bob ) = 30 . 000, salary ( Bill ) ≥ 27 . 000, Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 12 / 30

Domain description and queries Example (Domain descritpion:) Example (Queries) company(A), company(B), ? worksfor ( Chris , B ) worksFor(Alice,A), worksFor(Ann,A), ? ?x: friends ( Chris , ? x ) worksFor(Bob,B),worksFor(Bill,B); ? ? salary ( Bill ) friends(Alice,Ann), friends(Bob,Bill), ¬ friends(Ann,Bill) ? ? salary ( x ) : x = friendOf ( Ann ) salary ( Alice ) = 10 . 000, ? ? worksfor ( x , z ) ∧ worksfor ( z , z ) → salary ( Ann ) ≤ 12 . 000, friends ( x , y ) salary ( Bob ) = 30 . 000, salary ( Bill ) ≥ 27 . 000, ? ? salary ( x ) > 15 . 000 → 9 . 000 ≤ Salary ( Chris ) ≤ 11 . 000 worksfor ( x , A ) ∀ x . worksFor ( x , A ) ↔ ¬ worksFor ( x , B ) ∀ xy . friends ( x , y ) ↔ friends ( y , x ) ∀ xy , worksFor ( x , y ) → salary ( x ) > 3 . 000 ∀ x ∃ y . friends ( x , y ) Luciano Serafini (Fondazione Bruno Kessler) Logic Tensor Networks AITP 2017 12 / 30