[PPT] - Learning Representations of Relational Data Sebastijan Dumani PowerPoint Presentation

SLIDE 1

Learning Representations

f Relational Data

Sebastijan Dumančić DTAI, CS Department, KU Leuven September 6, ILP 2017

SLIDE 2

1 – Outline

2/30

1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

SLIDE 3

1 – Representation matters

3/30

Learning relational latent features – Dumančić, Blockeel

SLIDE 4

1 – Finding good features

4/30 Deep learning - finding good features autonomously by gradually building complexity

Learning relational latent features – Dumančić, Blockeel

SLIDE 5

1 – Focus on sensory data

5/30

Learning relational latent features – Dumančić, Blockeel

SLIDE 6

1 – Relational deep learning?

6/30

What about relational data?

Learning relational latent features – Dumančić, Blockeel

SLIDE 7

1 – Relational deep learning?

7/30

Learning relational latent features – Dumančić, Blockeel

SLIDE 8

2 – Outline

8/30

1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

SLIDE 9

2 – Vector spaces in knowledge graphs

9/30

Learning relational latent features – Dumančić, Blockeel

SLIDE 10

2 – Vector spaces in knowledge graphs

10/30

Learning representation = learning vectors

Learning relational latent features – Dumančić, Blockeel

SLIDE 11

2 – Vector spaces in knowledge graphs

11/30 wasBornIn(barack,honolulu).

[barack]

wasBornIn
[honolulu]T ≈ 1

wasBornIn(barack,nairobi).

[barack]

wasBornIn
[nairobi]T ≈ 0

Learning relational latent features – Dumančić, Blockeel

SLIDE 12

2 – Vector spaces in knowledge graphs

12/30 efficient good performance on KB completion tasks uninterpretable latent spaces huge amounts of data problems with unseen entities does not integrate in (statistical) relational learning

Learning relational latent features – Dumančić, Blockeel

SLIDE 13

3 – Outline

13/30

1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

SLIDE 14

3 – Desirable features

14/30

Learning relational latent features – Dumančić, Blockeel

SLIDE 15

3 – Learning features with k-means

15/30

[Coates, Lee and NG, AISTATS 2011]

Learning relational latent features – Dumančić, Blockeel

SLIDE 16

3 – Lifting the pipeline

16/30

Questions: What to cluster? How to cluster? Architecture?

Learning relational latent features – Dumančić, Blockeel

SLIDE 17

3 – Lifting the pipeline

17/30

What to cluster?

cluster vertices and relationships! For each type/domain of vertices in data

Learning relational latent features – Dumančić, Blockeel

SLIDE 18

3 – Lifting the pipeline

18/30

How to cluster them?

Unsupervised approach - which similarity is useful? (features, proximity, struc,...)

Cluster with a diverse set of similarities

Learning relational latent features – Dumančić, Blockeel

SLIDE 19

3 – Lifting the pipeline

19/30

How to choose the architecture?

a predicate for each latent feature

Rely on clustering selection to choose a good clustering

Learning relational latent features – Dumančić, Blockeel

SLIDE 20

4 – Outline

20/30

1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

SLIDE 21

4 – Relational similarity

21/30 How similar are ProfA and ProfB?

Relational clustering over neighbourhood trees [Dumančić & Blockeel, MLJ 2017]

Learning relational latent features – Dumančić, Blockeel

SLIDE 22

4 – Relational similarity – Neighbourhood trees

22/30 Neighbourhood trees summarize the neighbourhood of an instance/example data neighbourhood tree

Learning relational latent features – Dumančić, Blockeel

SLIDE 23

4 – Relational similarity – Neighbourhood trees

22/30 Neighbourhood trees summarize the neighbourhood of an instance/example data neighbourhood tree similarity of instances = similarity of their neighbourhood trees

Learning relational latent features – Dumančić, Blockeel

SLIDE 24

4 – Relational similarity – similarity interpretation

23/30 Decompose neighbourhood trees into semantic parts

Learning relational latent features – Dumančić, Blockeel

SLIDE 25

4 – Relational similarity – similarity interpretation

23/30 Decompose neighbourhood trees into semantic parts similarity = linear combination of similarities of individual semantic parts

Learning relational latent features – Dumančić, Blockeel

SLIDE 26

4 – Relational similarity – comparing semantic parts

24/30

Decompose NT is multisets of: attribute edge labels vertex identities per level and vertex type

Multiset of edge labels (level 1): { (Advised,2), (Advised,2), (TaughtBy,2) } Compare two multisets, A and B with χ2 distance χ2(A, B) =

x∈A∪B

(fA(x) − fB(x))2 fA(x) + fB(x)

Learning relational latent features – Dumančić, Blockeel

SLIDE 27

4 – Relational similarity – hyperedge similarity

25/30 (Hyper)edge similarity – reduction to similarities of vertices

1

Merging

2

Combination

Learning relational latent features – Dumančić, Blockeel

SLIDE 28

5 – Outline

26/30

1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

SLIDE 29

5 – Experiments and results

27/30 Datasets: IMDB UWCSE Mutagenesis Hepatitis Terrorist attacks WebKB Setup: 5-fold cross validation learning features of train data mapping test data to the

btained clusters

learn TILDE models on latent/original representations Question: Does learning in relational latent spaces benefits leaning compared to learning in the original space?

lower model complexity increased performance

How does it compared to MRC [Kok & Domingos, ICML 07]

Learning relational latent features – Dumančić, Blockeel

SLIDE 30

5 – Experiments and results

28/30 Models learned on latent representations are substantially simpler Models learned on latent representations often perform better exception: relationship info not useful

Learning relational latent features – Dumančić, Blockeel

SLIDE 31

6 – Outline

29/30

1 Introduction 2 Where are we now? 3 What can we do better? 4 Similarity of relational objects 5 Experiments and results 6 Auto-encoding logic programs Learning relational latent features – Dumančić, Blockeel

SLIDE 32

6 – Auto-encoding logic programs

30/30 Logic programs a computational framework for encoder and decoder

Input: mother(anna,dirk). female(anna). father(tom,dirk). male(tom). Encoder: latent1(X,Y) :- mother(X,Y);father(X,Y). latent2(X) :- female(X). Latent rep.: latent1(anna,dirk). latent1(tom,dirk). latent2(anna). Decoder: mother(X,Y) :- latent1(X,Y),latent2(X). female(X) :- latent2(X). father(X,Y) :- latent1(X,Y),not(latent2(X)). male(X) :- not(latent2(X)). Output: mother(anna,dirk). female(anna). father(tom,dirk). male(tom).

Learning relational latent features – Dumančić, Blockeel