Learning Representations of Relational Data Sebastijan Dumani - PowerPoint PPT Presentation

Learning Representations of Relational Data Sebastijan Dumančić DTAI, CS Department, KU Leuven September 6, ILP 2017

1 – Outline 2/30 1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

1 – Representation matters 3/30 Learning relational latent features – Dumančić, Blockeel

1 – Finding good features 4/30 Deep learning - finding good features autonomously by gradually building complexity Learning relational latent features – Dumančić, Blockeel

1 – Focus on sensory data 5/30 Learning relational latent features – Dumančić, Blockeel

1 – Relational deep learning? 6/30 What about relational data? Learning relational latent features – Dumančić, Blockeel

1 – Relational deep learning? 7/30 Learning relational latent features – Dumančić, Blockeel

2 – Outline 8/30 1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

2 – Vector spaces in knowledge graphs 9/30 Learning relational latent features – Dumančić, Blockeel

2 – Vector spaces in knowledge graphs 10/30 Learning representation = learning vectors Learning relational latent features – Dumančić, Blockeel

2 – Vector spaces in knowledge graphs 11/30 wasBornIn(barack,honolulu). � � [ honolulu ] T ≈ 1 [ barack ] wasBornIn wasBornIn(barack,nairobi). � � [ nairobi ] T ≈ 0 [ barack ] wasBornIn Learning relational latent features – Dumančić, Blockeel

2 – Vector spaces in knowledge graphs 12/30 efficient uninterpretable latent spaces good performance on KB huge amounts of data completion tasks problems with unseen entities does not integrate in (statistical) relational learning Learning relational latent features – Dumančić, Blockeel

3 – Outline 13/30 1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

3 – Desirable features 14/30 Learning relational latent features – Dumančić, Blockeel

3 – Learning features with k-means 15/30 [Coates, Lee and NG, AISTATS 2011] Learning relational latent features – Dumančić, Blockeel

3 – Lifting the pipeline 16/30 Questions: What to cluster? How to cluster? Architecture? Learning relational latent features – Dumančić, Blockeel

3 – Lifting the pipeline 17/30 What to cluster? cluster vertices and relationships! For each type/domain of vertices in data Learning relational latent features – Dumančić, Blockeel

3 – Lifting the pipeline 18/30 How to cluster them? Unsupervised approach - which similarity is useful? (features, proximity, struc,...) Cluster with a diverse set of similarities Learning relational latent features – Dumančić, Blockeel

3 – Lifting the pipeline 19/30 How to choose the architecture ? a predicate for each latent feature Rely on clustering selection to choose a good clustering Learning relational latent features – Dumančić, Blockeel

4 – Outline 20/30 1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity 21/30 How similar are ProfA and ProfB ? Relational clustering over neighbourhood trees [Dumančić & Blockeel, MLJ 2017] Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity – Neighbourhood trees 22/30 Neighbourhood trees summarize the neighbourhood of an instance/example data neighbourhood tree Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity – Neighbourhood trees 22/30 Neighbourhood trees summarize the neighbourhood of an instance/example data neighbourhood tree similarity of instances = similarity of their neighbourhood trees Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity – similarity interpretation 23/30 Decompose neighbourhood trees into semantic parts Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity – similarity interpretation 23/30 Decompose neighbourhood trees into semantic parts similarity = linear combination of similarities of individual semantic parts Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity – comparing semantic parts 24/30 Decompose NT is multisets of: attribute edge labels vertex identities per level and vertex type Multiset of edge labels (level 1): { (Advised,2), (Advised,2), (TaughtBy,2) } Compare two multisets, A and B with χ 2 distance ( f A ( x ) − f B ( x )) 2 χ 2 ( A, B ) = � f A ( x ) + f B ( x ) x ∈ A ∪ B Learning relational latent features – Dumančić, Blockeel

4 – Relational similarity – hyperedge similarity 25/30 (Hyper)edge similarity – reduction to similarities of vertices Merging 1 Combination 2 Learning relational latent features – Dumančić, Blockeel

5 – Outline 26/30 1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

5 – Experiments and results 27/30 Datasets: Setup: IMDB 5-fold cross validation UWCSE learning features of train data Mutagenesis mapping test data to the obtained clusters Hepatitis learn TILDE models on Terrorist attacks latent/original representations WebKB Question: Does learning in relational latent spaces benefits leaning compared to learning in the original space? lower model complexity increased performance How does it compared to MRC [Kok & Domingos, ICML 07] Learning relational latent features – Dumančić, Blockeel

5 – Experiments and results 28/30 Models learned on latent representations are substantially simpler Models learned on latent representations often perform better exception: relationship info not useful Learning relational latent features – Dumančić, Blockeel

6 – Outline 29/30 1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

6 – Auto-encoding logic programs 30/30 Logic programs a computational framework for encoder and decoder Input: mother(anna,dirk). female(anna). father(tom,dirk). male(tom). Encoder: latent1(X,Y) :- mother(X,Y);father(X,Y). latent2(X) :- female(X). Latent rep.: latent1(anna,dirk). latent1(tom,dirk). latent2(anna). Decoder: mother(X,Y) :- latent1(X,Y),latent2(X). female(X) :- latent2(X). father(X,Y) :- latent1(X,Y),not(latent2(X)). male(X) :- not(latent2(X)). Output: mother(anna,dirk). female(anna). father(tom,dirk). male(tom). Learning relational latent features – Dumančić, Blockeel