Rule Learning from Knowledge Graphs Guided by Embedding Models Vinh - - PowerPoint PPT Presentation

▶
rule learning from knowledge graphs guided by embedding
SMART_READER_LITE
LIVE PREVIEW

Rule Learning from Knowledge Graphs Guided by Embedding Models Vinh - - PowerPoint PPT Presentation

Nov 25, 2022 377 likes •702 views

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion Rule Learning from Knowledge Graphs Guided by Embedding Models Vinh Thinh Ho 1 , Daria Stepanova 1 , Mohamed Gad-Elrab 1 , Evgeny Kharlamov 2 , Gerhard Weikum 1 1

slide-1
SLIDE 1

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Learning from Knowledge Graphs Guided by Embedding Models

Vinh Thinh Ho1, Daria Stepanova1, Mohamed Gad-Elrab1, Evgeny Kharlamov2, Gerhard Weikum1

1Max Planck Institute for Informatics, Saarbr¨

ucken, Germany

2University of Oxford, Oxford, United Kingdom

ISWC 2018

1 / 13

slide-2
SLIDE 2

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Learning from KGs

1 / 13

slide-3
SLIDE 3

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Learning from KGs

Confidence, e.g., WARMER [Goethals and den Bussche, 2002] CWA: whatever is missing is false conf(r) = | | | | + | | =2 4 r : livesIn(X, Y) ← isMarriedTo(Z, X), livesIn(Z, Y)

1 / 13

slide-4
SLIDE 4

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Learning from KGs

Confidence, e.g., WARMER [Goethals and den Bussche, 2002] CWA: whatever is missing is false conf(r) = | | | | + | | =2 4 r : livesIn(X, Y) ← isMarriedTo(Z, X), livesIn(Z, Y)

1 / 13

slide-5
SLIDE 5

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Learning from KGs

PCA confidence AMIE [Gal´ arraga et al., 2015] PCA: Since Alice has a living place already, all others are incorrect.

Brad Ann

isMarriedTo

John Kate

isMarriedTo hasBrother

Berlin Chicago Alice

isMarriedTo

Bob

livesIn

Clara

isMarriedTo

Dave Researcher

livesIn IsA IsA

Amsterdam

livesIn livesIn livesIn livesIn livesIn

confPCA(r) = | | | | + | | =2 3 r : livesIn(X, Y) ← isMarriedTo(Z, X), livesIn(Z, Y)

1 / 13

slide-6
SLIDE 6

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Learning from KGs

Exception-enriched rules: [ISWC 2016, ILP 2016]

Brad Ann

isMarriedTo

John Kate

isMarriedTo hasBrother

Berlin Chicago Alice

isMarriedTo

Bob

livesIn

Clara

isMarriedTo

Dave Researcher

livesIn IsA IsA

Amsterdam

livesIn livesIn livesIn livesIn livesIn

conf(r) = confPCA(r) = | | | | + | | =1 r : livesIn(X, Y) ← isMarriedTo(Z, X), livesIn(Z, Y), not isA(X, researcher)

1 / 13

slide-7
SLIDE 7

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Absurd Rules due to Data Incompleteness

Problem: rules learned from highly incomplete KGs might be absurd..

Brad ITech

worksAt

John OpenSys

worksAt

Berlin Chicago ITService

worksAt

Bob Clara

worksAt

SoftComp

IsA

ItCompany

IsA livesIn

  • fficeIn
  • fficeIn

livesIn

  • fficeIn
  • fficeIn

conf(r) = confPCA(r) = 1

livesIn(X, Y) ← worksAt(X, Z), officeIn(Z, Y), not isA(Z, itCompany)

2 / 13

slide-8
SLIDE 8

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Ideal KG

µ(r, Gi): measure quality of the rule r on Gi

KG Ideal KG

 i

3 / 13

slide-9
SLIDE 9

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Ideal KG

µ(r, Gi): measure quality of the rule r on Gi, but Gi is unknown

KG Ideal KG

 i

3 / 13

slide-10
SLIDE 10

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Probabilistic Reconstruction of Ideal KG

µ(r, Gi

p): measure quality of r on Gi

p KG probabilistic reconstruction of

 i i

p

3 / 13

slide-11
SLIDE 11

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Hybrid Rule Measure

µ(r, Gi

p) = (1 − λ) × µ1(r, G) + λ × µ2(r, Gi p)

KG probabilistic reconstruction of

 i i

p

3 / 13

slide-12
SLIDE 12

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Hybrid Rule Measure

µ(r, Gi

p) = (1 − λ) × µ1(r, G) + λ × µ2(r, Gi p)

  • λ ∈ [0..1] :

λ ∈ [0..1] : λ ∈ [0..1] : weighting factor

  • µ1 :

µ1 : µ1 : descriptive quality of rule r over the available KG G

  • confidence
  • PCA confidence
  • µ2 :

µ2 : µ2 : predictive quality of r relying on Gi

p (probabilistic

reconstruction of the ideal KG Gi)

3 / 13

slide-13
SLIDE 13

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

KG Embeddings

  • Popular approach to KG completion, which proved to be effective
  • Relies on translation of entities and relations into vector spaces

colleagueOf Jane Bob Mat colleagueOf colleagueOf livesIn Berlin livesIn SFO colleagueOf livesIn Patti Jane Bob Mat SFO Berlin livesIn Text Bob is a developer in ITService, CA score(<Bob livesIn SFO>) = 0.8 score(<Bob livesIn Berlin>) = 0.4

...

Patti Mat working as a developer in ITService, CA Bob and Mat have successfully completed a project initiated by the SFO department of ITService SFO is a cultural and commercial center of CA

TransE [Bordes et al., 2013], SSP [Xiao et al., 2017], TEKE [Wang and Li, 2016]

4 / 13

slide-14
SLIDE 14

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Our Approach

5 / 13

slide-15
SLIDE 15

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Construction

  • Clause exploration from general to specific
  • all first-order clauses: [Shapiro, 1991]

livesIn(X, Y) ←

add atom

livesIn(bob, Y) ←

unify variable to constant

livesIn(X, Y) ← livesIn(U, V)

unify variables

livesIn(X, X) ←

6 / 13

slide-16
SLIDE 16

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Construction

  • Clause exploration from general to specific
  • closed rules: AMIE [Gal´

arraga et al., 2015] livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

livesIn(X, Y) ←

add dangling atom

livesIn(X, Y) ← isA(X, researcher)

add instantiated atom

livesIn(X, Y) ← marriedTo(X, Z)

add closing atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

6 / 13

slide-17
SLIDE 17

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Construction

  • Clause exploration from general to specific
  • closed rules: AMIE [Gal´

arraga et al., 2015] livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

livesIn(X, Y) ←

add dangling atom

livesIn(X, Y) ← isA(X, researcher)

add instantiated atom

livesIn(X, Y) ← marriedTo(X, Z)

add closing atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

6 / 13

slide-18
SLIDE 18

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Construction

  • Clause exploration from general to specific
  • closed rules: AMIE [Gal´

arraga et al., 2015] livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

livesIn(X, Y) ←

add dangling atom

livesIn(X, Y) ← isA(X, researcher)

add instantiated atom

livesIn(X, Y) ← marriedTo(X, Z)

add closing atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

6 / 13

slide-19
SLIDE 19

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Construction

  • Clause exploration from general to specific
  • This work: closed and safe rules with negation

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y), not isA(X, researcher)

livesIn(X, Y) ←

add dangling atom

livesIn(X, Y) ← isA(X, researcher)

add instantiated atom

livesIn(X, Y) ← marriedTo(X, Z)

add closing atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

add negated instantiated atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y), not isA(X, researcher) livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y), not moved(X, Y)

add negated atom

6 / 13

slide-20
SLIDE 20

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Construction

  • Clause exploration from general to specific
  • This work: closed and safe rules with negation

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y), not isA(X, researcher)

livesIn(X, Y) ←

add dangling atom

livesIn(X, Y) ← isA(X, researcher)

add instantiated atom

livesIn(X, Y) ← marriedTo(X, Z)

add closing atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

add negated instantiated atom

livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y), not isA(X, researcher) livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y), not moved(X, Y)

add negated atom

6 / 13

slide-21
SLIDE 21

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Rule Prunning

livesIn(X, Y) ← worksAt(X, Z),

  • fficeIn(Z, Y)

livesIn(X, Y) ← worksAt(X, Z) livesIn(X, Y) ← livesIn(X, Y) ← marriedTo(X, Z) livesIn(Z, Y) livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y) not researcher(X)

... ... ...

Prune rule search space relying on

  • novel hybrid embedding-based rule measure

7 / 13

slide-22
SLIDE 22

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Embedding-based Rule Quality

  • Estimate average quality of predictions made by a given rule r

µ2(r, Gi

p) =

1

|predictions(r, G)|

  • fact∈predictions(r,G)

Gi

p(fact)

  • Rely on truthfulness of predictions made by r based on the

probabilistic reconstruction Gi

p of Gi

8 / 13

slide-23
SLIDE 23

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Embedding-based Rule Quality

  • Estimate average quality of predictions made by a given rule r

µ2(r, Gi

p) =

1

|predictions(r, G)|

  • fact∈predictions(r,G)

Gi

p(fact)

  • Rely on truthfulness of predictions made by r based on the

probabilistic reconstruction Gi

p of Gi

Example: livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y)

  • Rule predictions: livesIn(mat, monterey),livesIn(dave, chicago)

µ2(r, Gi

p)=

Gi

p(< mat livesIn monterey >)+Gi p(< dave livesIn chicago >)

2

8 / 13

slide-24
SLIDE 24

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Embedding-based Rule Quality

  • Estimate average quality of predictions made by a given rule r

µ2(r, Gi

p) =

1

|predictions(r, G)|

  • fact∈predictions(r,G)

Gi

p(fact)

  • Rely on truthfulness of predictions made by r based on the

probabilistic reconstruction Gi

p of Gi

Example: livesIn(X, Y) ← marriedTo(X, Z), livesIn(Z, Y),not isA(X, surfer)

  • Rule predictions: ✭✭✭✭✭✭✭✭✭✭✭

✭ livesIn(mat, monterey),livesIn(dave, chicago) µ2(r, Gi

p)=

Gi

p(< dave livesIn chicago >)

1

  • µ2(r, Gi

p) goes down for noisy exceptions

8 / 13

slide-25
SLIDE 25

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Evaluation Setup

  • Datasets:
  • FB15K: 592K facts, 15K entities and 1345 relations
  • Wiki44K: 250K facts, 44K entities and 100 relations
  • Training graph G: remove 20% from the available KG
  • Embedding models Gi

p:

  • TransE [Bordes et al., 2013], HolE [Nickel et al., 2016]
  • With text: SSP [Xiao et al., 2017]
  • Goals:
  • Evaluate effectiveness of our hybrid rule measure

µ(r, Gi

p) = (1 − λ) × µ1(r, G) + λ × µ2(r, Gi p)

  • Compare against state-of-the-art rule learning systems

9 / 13

slide-26
SLIDE 26

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Evaluation of Hybrid Rule Measure

λ

top_5 top_10 top_20 top_50 top_100 top_200

0.7 0.75 0.8 0.85 0.9 0.95 1 0.0 0.2 0.4 0.6 0.8 1.0

  • Avg. prec.

λ

(a) Conf-HolE

0.7 0.75 0.8 0.85 0.9 0.95 1 0.0 0.2 0.4 0.6 0.8 1.0

  • Avg. prec.

λ

(b) Conf-SSP

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.0 0.2 0.4 0.6 0.8 1.0

  • Avg. prec.

λ

(c) PCA-SSP

Precision of top-k rules ranked using variations of µ on FB15K.

10 / 13

slide-27
SLIDE 27

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Evaluation of Hybrid Rule Measure

λ

top_5 top_10 top_20 top_50 top_100 top_200

0.7 0.75 0.8 0.85 0.9 0.95 1 0.0 0.2 0.4 0.6 0.8 1.0

  • Avg. prec.

λ

(a) Conf-HolE

0.7 0.75 0.8 0.85 0.9 0.95 1 0.0 0.2 0.4 0.6 0.8 1.0

  • Avg. prec.

λ

(b) Conf-SSP

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.0 0.2 0.4 0.6 0.8 1.0

  • Avg. prec.

λ

(c) PCA-SSP

Precision of top-k rules ranked using variations of µ on FB15K.

  • Positive impact of embeddings in all cases for λ = 0.3
  • Note: in (c) comparison to AMIE [Gal´

arraga et al., 2015] (λ = 0)

10 / 13

slide-28
SLIDE 28

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Example Rules

Examples of rules learned from Wikidata

Script writers stay the same throughout a sequel, but not for TV series r1 : scriptwriterOf(X, Y) ← precededBy(Y, Z), scriptwriterOf(X, Z), not isA(Z, tvSeries) Nobles are typically married to nobles, but not in the case of Chinese dynasties r2 : nobleFamily(X, Y)←spouse(X, Z), nobleFamily(Z, Y), not isA(Y,chineseDynasty)

11 / 13

slide-29
SLIDE 29

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Related Work

Inductive logic programming Data mining Rule Learning guided by embedding models

[Goethals et al, 2002, Galárraga et al, 2015, ISWC 2016, ILP 2017]

Relational association rule learning

[Muggleton et al, 1990]

Neural-based rule learning

[Yang et al, 2017, Evans et al, 2018] [Mannila, 1996] [Goethals et al, 2011, Dzyuba et al, 2017]

Interactive pattern mining Exploiting cardinality info

[ISWC 2017] [De Raedt et al, 2014]

Learning from interpretations

12 / 13

slide-30
SLIDE 30

Motivation Our Approach Rule Construction Rule Prunning Evaluation Conclusion

Conclusion

  • Summary:
  • Framework for learning rules from KGs with external sources
  • Hybrid embedding-based rule quality measure
  • Experimental evaluation on real-world KGs
  • Approach is orthogonal to a concrete embedding used
  • Outlook:
  • Other rule types, e.g., with existentials in the head or constraints
  • Plug-in portfolio of embeddings
  • Mimic framework of exact learning [Angluin, 1987] by establishing

complex queries to embeddings

13 / 13

slide-31
SLIDE 31

References I

Dana Angluin. Queries and concept learning. Machine Learning, 2(4):319–342, 1987. Antoine Bordes, Nicolas Usunier, Alberto Garc´ ıa-Dur´ an, Jason Weston, and Oksana Yakhnenko. Translating Embeddings for Modeling Multi-relational Data. In Proceedings of NIPS, pages 2787–2795, 2013. Luis Gal´ arraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. Fast rule mining in ontological knowledge bases with AMIE+. VLDB J., 24(6):707–730, 2015. Bart Goethals and Jan Van den Bussche. Relational association rules: Getting warmer. In PDD, 2002. Maximilian Nickel, Lorenzo Rosasco, and Tomaso A. Poggio. Holographic embeddings of knowledge graphs. In AAAI, 2016. Ehud Y. Shapiro. Inductive inference of theories from facts. In Computational Logic - Essays in Honor of Alan Robinson, pages 199–254, 1991. Zhigang Wang and Juan-Zi Li. Text-enhanced representation learning for knowledge graph. In IJCAI, 2016. Han Xiao, Minlie Huang, Lian Meng, and Xiaoyan Zhu. SSP: semantic space projection for knowledge graph embedding with text descriptions. In AAAI, 2017.