Knowledge Graph Embedding for Mining Cultural Heritage Data Nada - - PowerPoint PPT Presentation

▶
knowledge graph embedding for mining cultural heritage
SMART_READER_LITE
LIVE PREVIEW

Knowledge Graph Embedding for Mining Cultural Heritage Data Nada - - PowerPoint PPT Presentation

Jan 04, 2024 371 likes •737 views

Knowledge Graph Embedding for Mining Cultural Heritage Data Nada Mimouni and Jean-Claude Moissinac Telecom ParisTech Institut Mines Telecom January 24 th , 2019 DIG - LTCI Knowledge Graph Embedding for Mining Cultural Heritage Data 1 / 34

slide-1
SLIDE 1

Knowledge Graph Embedding for Mining Cultural Heritage Data

Nada Mimouni and Jean-Claude Moissinac – Telecom ParisTech

Institut Mines Telecom

January 24th, 2019 DIG - LTCI

1 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-2
SLIDE 2

Project Data Method Experiments Conclusion

Outline

1

Project presentation

2

Data

3

Knowledge Graph Embedding Entities extraction Context graph Graph walks and kernel Neural language model Using the model

4

Experiments and preliminary results Entity similarity and relatedness Entity matching

5

Conclusion

2 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-3
SLIDE 3

Project Data Method Experiments Conclusion

Outline

1

Project presentation

2

Data

3

Knowledge Graph Embedding Entities extraction Context graph Graph walks and kernel Neural language model Using the model

4

Experiments and preliminary results Entity similarity and relatedness Entity matching

5

Conclusion

3 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-4
SLIDE 4

Project Data Method Experiments Conclusion

Project presentation

4 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-5
SLIDE 5

Project Data Method Experiments Conclusion

Project presentation

5 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-6
SLIDE 6

Project Data Method Experiments Conclusion

Outline

1

Project presentation

2

Data

3

Knowledge Graph Embedding Entities extraction Context graph Graph walks and kernel Neural language model Using the model

4

Experiments and preliminary results Entity similarity and relatedness Entity matching

5

Conclusion

6 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-7
SLIDE 7

Project Data Method Experiments Conclusion

Data

Gather data from institutions:

Collect data respecting privacy Adopt homogeneous representations to make the data comparable Choose a model able to represent links between data

Rely on external data:

DataTourism, tourist office data on places and events OpenAgenda, and other event calendar Joconde database, and other cultural data General knowledge bases: DBPedia, Wikidata, ... Geographical knowledge bases: geonames, data on data.gouv.fr ...

7 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-8
SLIDE 8

Project Data Method Experiments Conclusion

A simple example of links generation

8 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-9
SLIDE 9

Project Data Method Experiments Conclusion

Objectives

Questions:

How to collect, integrate and enrich this complex and large amount of data? How to mine such type of data to extract useful information?

Hypothesis:

Integrate external data source to enhance the quality of the original data; Limit the analysis to a specified context help boosting performance.

9 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-10
SLIDE 10

Project Data Method Experiments Conclusion

Approach

Represent instances as a set of n-dimensional numerical feature vectors Use representation with different ML tasks

1

Adapt neural language model : Word2vec

2

Transform RDF graph into sequences of entities and relations (sentences)

3

Train the model and generate entity vectors + Conserve the information in the original graph + Semantically similar/related entities have close vectors in the embedded space + Generate a reusable model, that could be enriched with new entities

10 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-11
SLIDE 11

Project Data Method Experiments Conclusion

Outline

1

Project presentation

2

Data

3

Knowledge Graph Embedding Entities extraction Context graph Graph walks and kernel Neural language model Using the model

4

Experiments and preliminary results Entity similarity and relatedness Entity matching

5

Conclusion

11 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-12
SLIDE 12

Project Data Method Experiments Conclusion

Knowledge graph embedding process

Generate walks random tf-idf Input Data CMN Paris Musée Extract entities Build context graph black-list kernel Train neural language model V1

. . . . . . . . . ...

Vn

. . .

6 V2 V3 Entities feature vectors Similarity / Relatedness Link prediction KG completion Recommandation Community detection 1 2 3 5 4 7 12 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-13
SLIDE 13

Project Data Method Experiments Conclusion

Extract entities (2)

Identify entities’ URIs from input data

1

URI exist: read and identify URI from data files

2

URI ! exist: use entity name to build URI (dbpedia, frdbpedia, wikidata)

13 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-14
SLIDE 14

Project Data Method Experiments Conclusion

Build context graph (3)

For each entity URI:

Build context from a generalized data source, ’around’ the entity Data source: e.g. DBpedia ’around’: get neighbours in the graph within α hops

Consider the undirected graph α = 1 or 2

Define a black-list to ignore predicates and objects:

very general, e.g. <http://www.w3.org/2002/07/owl#Thing> non-informative, e.g. <http://fr.dbpedia.org/resource/Mod` ele:P.> noisy, e.g. <http://www.w3.org/2000/01/rdf-schema#comment>

14 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-15
SLIDE 15

Project Data Method Experiments Conclusion

Merge context graphs (3)

ey e13 e12 e14 e8 e4 e7 e3 e5 e6 e13 e12 e14 Context graph of entity ey ex e7 e8 e4 e5 e6 e3 e1 e2 e9 e10 Context graph of entity ex Global context graph ex ey e7 e8 e4 e5 e6 e3 e1 e2 e9 e10 15 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-16
SLIDE 16

Project Data Method Experiments Conclusion

Generate walks (4)

Generate walks random tf-idf Input Data CMN Paris Musée Extract entities Build context graph black-list kernel Train neural language model V1

. . . . . . . . . ...

Vn

. . .

6 V2 V3 Entities feature vectors Similarity / Relatedness Link prediction KG completion Recommandation Community detection 1 2 3 5 4 7 16 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-17
SLIDE 17

Project Data Method Experiments Conclusion

Random walk (4)

Intuition: all neighbours are equally important for an entity

Specify walk parameters

nb-walks: number of walks (example: 500 walk) depth: number of hops in the graph (2, 4, 8) example: d=4 ⇒ e → p1 → e1 → p2 → e2

Specify the list of entities (all entities in the global context graph / a predefined list) For each entity:

1

get a random list of direct neighbours

2

calculate the corresponding number of walks for each neighbour

3

recursively..

Adjust the number of walks according to specific cases:

if (nb-neighbours < nb-walks) : divide, get the entire part of the division, sum-up the rest and add it to a randomly selected neighbour if (nb-neighbours == 0) : transfer its nb-walks to another randomly selected neighbour

17 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-18
SLIDE 18

Project Data Method Experiments Conclusion

Tf-Idf graph walk (4)

Intuition: Some neighbours are more important for an entity. Prioritize important neighbours by weighting their predicates.

Calculate tf-idf weights for predicates tf: evaluate the importance of a predicate p for an entity e

to(p, e) = number of p occurrences for entity e tp(e) = number of predicates associated with e tf(p, e) = to(p, e)/tp(e)

idf: evaluate the importance of a predicate p on the whole graph

D = number of entities in the graph d(p) = number of entities using predicate p idf(p) = log(D/d(p))

tfidf(p, e) = tf(p, e) ∗ idf(p)

18 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-19
SLIDE 19

Project Data Method Experiments Conclusion

Black-list walk (4)

Intuition: some predicates are noisy (less important) for an entity

Put weights on predicates:

predicate in the black-list: weight = 0 (to ignore)

  • ther predicate: weight = 1 (to consider in the walk)

Example: {http://dbpedia.org/ontology/wikiPageWikiLink}

19 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-20
SLIDE 20

Project Data Method Experiments Conclusion

Weisfeiler-Lehman kernel (4)

Intuition: Weisfeiler-Lehman subtree RDF graph kernels capture (richer) information of an entire subtree in a single node.

de Vries, Gerben K. D., ”A Fast Approximation of the Weisfeiler-Lehman Graph Kernel for RDF Data”, ECML PKDD 2013. 20 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-21
SLIDE 21

Project Data Method Experiments Conclusion

Weisfeiler-Lehman kernel (4)

For each iteration, for each entity in the graph, get random walks of depth d After 1 iteration, graph G sequences: 1− > 6− > 11; 1− > 6− > 11− > 13; 1− > 6− > 11− > 10; ... 4− > 11− > 6; 4− > 11− > 13; 4− > 11− > 10; 4− > 11− > 10− > 8; ...

Ristoski, Paulheim, ”RDF2Vec: RDF Graph Embeddings for Data Mining”, ISWC 2016. 21 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-22
SLIDE 22

Project Data Method Experiments Conclusion

Neural language model (5,6)

Word2vec A two-layer neural net that processes text Input: a text corpus (sentences) Output: a set of vectors (feature vectors for words in that corpus) Create neural embeddings for any group of discrete and co-occurring states → RDF data

22 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-23
SLIDE 23

Project Data Method Experiments Conclusion

Neural language model (5,6)

+ Similar words cluster together in the embedding space + Operations on vectors:

Madrid - Spain = Beijing - China Madrid - Spain + China = Beijing

Mikolov,Tomas et al., ”Distributed Representations of Words and Phrases and their Compositionality”, NIPS 2013. 23 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-24
SLIDE 24

Project Data Method Experiments Conclusion

Using the model (7)

N-dimensional numerical vector representation of entities Ve = (v1, v2, ..., vi, ..., vn)

V1

. . . . . . . . . ...

Vn

. . .

6 V2 V3 Entities feature vectors Similarity / Relatedness Link prediction KG completion Recommandation Community detection 7 24 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-25
SLIDE 25

Project Data Method Experiments Conclusion

Using the model (7)

N-dimensional numerical vector representation of entities Ve = (v1, v2, ..., vi, ..., vn)

24 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-26
SLIDE 26

Project Data Method Experiments Conclusion

Outline

1

Project presentation

2

Data

3

Knowledge Graph Embedding Entities extraction Context graph Graph walks and kernel Neural language model Using the model

4

Experiments and preliminary results Entity similarity and relatedness Entity matching

5

Conclusion

25 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-27
SLIDE 27

Project Data Method Experiments Conclusion

Entity similarity and relatedness

The outcome of our first experiments:

1

We discover hidden interesting information (non-trivial things) → can lead to new knowledge (facts)

2

We find rather trivial things, but nothing wrong

Task Try to understand and interpret the semantic relation behind the similarity/relatedness measure returned by the model

For strong similarities between two entities:

find the shortest path between them the shortest random walks used with these entities and that connect them

This path could give a form of ”explanation” of their connection

26 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-28
SLIDE 28

Project Data Method Experiments Conclusion

Similarity examples

Get top similar entities to ’dbr:Abbaye-de-Charroux’

1

’dbr:Benest’ (0.9997649192810059)

in fact, ’Benest’ is close to ’Abbaye-de-Charroux’ a ’general’ direct link exist: ’dbo:wikiPageWikiLink’ could be found by another walk : ’long’ and ’lat’ specify this general type of link : e.g. ’Benest − > is-close-to − > Abbaye-de-Charroux’

2

’dbr:Abbaye-Saint-Sauveur-de-Charroux’ (0.9994720816612244)

a ’general’ direct link exist: ’dbo:wikiPageRedirects’ ’Abbaye-Saint-Sauveur-de-Charroux − > same-as − > Abbaye-de-Charroux’

3

’dbr:Baudri-de-Bourgueil’ (0.9998940825462341)

Charroux is a Benedictine abbey Baudri is a religious of the order of benediction that has greatly changed the monastic practice A non-trivial link to analyse...

27 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-29
SLIDE 29

Project Data Method Experiments Conclusion

Similarity examples

An event about ’Beethoven’ is organised in ’Mus´ ee Bourdelle’ → Transpose this event to other museums ? ???? is to ’Mus´ ee Balzac’ what ’Beethoven’ is to ’Mus´ ee Bourdelle’

1

’Romeo-Void’, 0.8273534774780273

2

’Era-(musical-project)’, 0.8242164850234985,

3

’Spectrum-(band)’, 0.8137580156326294,

4

’Oladad’, 0.8116711378097534,

5

’Time-Crash-(band)’, 0.8114833831787109,

6

’Ellegarden’, 0.810987114906311,

7

’John-Mayer-Trio’, 0.8100252151489258,

8

’Motion-Trio’, 0.8094779253005981,

9

’The-Bala-Brothers’, 0.8068113923072815

28 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-30
SLIDE 30

Project Data Method Experiments Conclusion

Entity matching: Joconde data

Joconde: ≈ 600000 artworks, ≈ 10000 techniques, ≈ 1000 places (museums...), ≈ 60000 creators

29 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-31
SLIDE 31

Project Data Method Experiments Conclusion

Entity matching: idea

30 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-32
SLIDE 32

Project Data Method Experiments Conclusion

Entity matching: idea

* Some artworks are in several domains or relatives to several techniques

Manual linking is a starting point to enrich the links between data from Joconde and our Context Graph Hypothesis: helps to get better walks between entities in Joconde and the Context Graph

31 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-33
SLIDE 33

Project Data Method Experiments Conclusion

Outline

1

Project presentation

2

Data

3

Knowledge Graph Embedding Entities extraction Context graph Graph walks and kernel Neural language model Using the model

4

Experiments and preliminary results Entity similarity and relatedness Entity matching

5

Conclusion

32 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-34
SLIDE 34

Project Data Method Experiments Conclusion

Conclusion

Integrate cultural data from different heterogeneous sources Use an adaptation of a neural language model for entity embedding Build numerical model that can serve to calculate similarities The output of the model can be used with different ML tasks

33 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data

slide-35
SLIDE 35

Project Data Method Experiments Conclusion

Thank you

34 / 34 Knowledge Graph Embedding for Mining Cultural Heritage Data