Plausible reasoning based on qualitative entity embeddings Steven - - PowerPoint PPT Presentation

Plausible reasoning based on qualitative entity embeddings

School of Computer Science & Informatics Cardiff University, Cardiff, UK SchockaertS1@cardiff.ac.uk http://users.cs.cf.ac.uk/S.Schockaert

Steven Schockaert

(joint work with Shoaib Jameel)

Plausible reasoning Qualitative representations of meaning Learning conceptual spaces Geometric representations of meaning

Category based induction

Chardonnay contains polyphenols Most wines contain polyphenols Malbec contains polyphenols

Category based induction

Chardonnay contains polyphenols Most wines contain polyphenols Malbec contains polyphenols Pinot Blanc contains polyphenols Most wines contain polyphenols Pinot Gris contains polyphenols

?

Category based induction

Chardonnay contains polyphenols Most wines contain polyphenols Malbec contains polyphenols Champagne contains polyphenols Most wines contain polyphenols Port wine contains polyphenols Pinot Blanc contains polyphenols Most wines contain polyphenols Pinot Gris contains polyphenols

? ?

Interpolation

Ice cream shops in Wales are required to display food hygiene ratings Restaurants in Wales are required to display food hygiene ratings Sandwich shops in Wales are required to display food hygiene ratings

Similarity based reasoning

Restaurants in Wales are required to display food hygiene ratings Sandwich shops in Wales are required to display food hygiene ratings

?

A fortiori inference

University staff are not permitted to travel in first class University staff are not permitted to travel in business class

A fortiori inference

University staff are not permitted to travel in business class University staff are not permitted to travel in first class First class is more expensive than business class The problem with business class is that it is expensive

domain theory domain theory domain theory domain theory domain theory Completed domain theory unstructured data domain theory

Interpretable machine learning models Ontology based data access Recognising textual entailment

Motivation

Plausible reasoning Qualitative representations of meaning Learning conceptual spaces Geometric representations of meaning

Word embeddings

chardonnay wine

(Mikolov 2013, Pennington 2014)

Word embeddings

Paris London France UK

(Mikolov 2013, Pennington 2014)

Knowledge graph embeddings

d i r e c t e d b y d i r e c t e d b y lives in lives in Jurassic park Trainspotting Steven Spielberg Danny Boyle USA UK

(Bordes 2013, Wang 2014, Zhong 2015)

Conceptual spaces

dangerous large

rattlesnake black widow spider lion black bear sheep cardinal spider wild boar

mammal spider vertebrate scary

(Gärdenfors 2000)

Plausible reasoning Qualitative representations of meaning Learning conceptual spaces Geometric representations of meaning

Learning conceptual spaces

textual description

• f entities

Betweenness

aliens between star trek and cloverfield cast away between titanic and into the wild lord of the rings between harry potter and troy mission impossible between the rock and skyfall star wars between lord of the rings and star trek troy between braveheart and thor wall-e between monsters inc and 2001: a space odyssey good will hunting between dead poets society and rain man unbreakable between sin city and the sixth sense scarface between sin city and the godfather forest gump between million dollar baby and stand by me shrek 2 between wedding crashers and the lion king

Betweenness

abbey between castle and chapel bistro between restaurant and tea room butcher shop between marketplace and slaughterhouse conservatory between greenhouse and playhouse duplex between detached house and triplex flower shop between garden center and gift shop grocery store between convenience store and farmers market manor between castle and mansion house rice paddy between bamboo forest and cropland sushi restaurant between Japanese restaurant and tapas restaurant veterinarian between animal shelter and emergency room wine shop between gourmet shop and liquor store

Direction towards more “violent” films

films whose associated text contains the word “violent” films whose associated text does not contain the word “violent”

Learning interpretable directions

Learning interpretable directions

measure.

Foursquare GeoNames OpenCYC Algorithm n Acc. F1 Acc. F1 Acc. F1 Col 0.947 0.717 0.881 0.401 0.956 0.383 BtwA 0.949 0.717 0.883 0.395 0.956 0.373 BtwB 0.943 0.617 0.881 0.349 0.954 0.295 AnalogA 0.921 0.636 0.822 0.330 0.933 0.375 AnalogB 0.940 0.707 0.853 0.347 0.945 0.382 AnalogC 0.925 0.686 0.859 0.411 0.942 0.391 FOIL0 0.926 0.564 0.876 0.201 0.950 0.267 FOIL1 50 0.925 0.596 0.860 0.272 0.943 0.329 FOIL2 0.926 0.627 0.861 0.285 0.946 0.335 FOIL3 0.928 0.594 0.876 0.300 0.949 0.268 1-NN 0.939 0.710 0.853 0.357 0.945 0.380 C4.5MDS 0.925 0.534 0.849 0.178 0.941 0.245 C4.5dir 0.918 0.382 0.849 0.374 0.939 0.262 SVMMDS 0.932 0.656 0.859 0.343 0.912 0.328 SVMBoW 0.913 0.358 0.874 0.172 0.946 0.205

Commonsense classifiers

interpolation analogical classifiers a fortiori inference

Entity embeddings with conceptual subspaces

Entity embeddings with conceptual subspaces

Ranking Induction Analogy ρ MAP P@5 MRR Acc. Skip-gram 0.155 0.176 0.356 0.505 0.184 CBOW 0.159 0.182 0.350 0.500 0.213 RESCAL 0.081 0.020 0.189 0.423 0.371 TransE 0.110 0.060 0.200 0.451 0.382 TransH 0.142 0.072 0.210 0.415 0.382 TransR 0.100 0.102 0.302 0.489 0.378 CTransR 0.122 0.132 0.323 0.499 0.402 pTransEanch 0.099 0.101 0.301 0.488 0.476 pTransEart 0.202 0.218 0.475 0.751 0.512 pTransEfull 0.213 0.224 0.490 0.756 0.532 EECSfull 0.319 0.231 0.609 0.883 0.591 EECSno rel 0.301 0.229 0.588 0.868 0.552 EECSno type 0.266 0.225 0.585 0.854 0.549 EECSno NN 0.258 0.220 0.581 0.843 0.545 EECStext 0.254 0.218 0.579 0.831 0.540 EECStype-comb 0.312 0.231 0.601 0.883 0.595 EECStype-dist 0.295 0.231 0.585 0.858 0.550 EECSrel-dim 0.309 0.225 0.585 0.859 0.551 EECSrel-dist 0.299 0.225 0.585 0.855 0.549

Plausible reasoning Qualitative representations of meaning Learning conceptual spaces Geometric representations of meaning

Limitations of learned semantic spaces

dangerous large

rattlesnake black widow lion black bear sheep cardinal spider wild boar

mammal spider vertebrate scary mammal spider vertebrate scary

Qualitative semantic representations

venue restaurant shop

italian restaurant ice cream shop dessert restaurant pizzeria

Easy to use in a variety of KR and NLP tasks Can be automatically learned/refined from text collections Is-a relationships often not sufficient Often many ways to structure the terms of a given domain Often too shallow to allow for reliable inductive inferences Easy to understand/correct/extend by human domain experts

One-dimensional representations

A B C D E

d1 d2

A ↓ d

1

C ↓ d

1

D ↓ d

1

D↓d2 B↓d2

Directions

formal cheap trendy healthy

ice cream shop Michelin star restaurant Betweenness, analogy and adjacency can only be approximately represented Compact and easy-to-understand representation Possible to encode (and efficient to reason with) incomplete information (e.g. from text) Similarity estimates possibly less accurate

Directions

Modelling vagueness/typicality effects

formal cheap trendy healthy

Michelin star restaurant

Directions

formal cheap trendy healthy

Modelling context effects

Region connection calculus

a DC b disconnected

a b

a EC b externally connected

a b

a PO b partially overlapping

a b

a EQ b equal

a b

a TPP b tangential proper part

a b

b TPP-1 a a NTPP b non-tangential PP

a b

b NTPP-1 a

Betweenness

A B C D E A⋈B

Conclusions

Plausible reasoning, based on semantic background knowledge, can be used to fill in the gaps or resolve inconsistencies in imperfect knowledge bases By deriving qualitative representations from these vector spaces we can:

• Model relationships between concepts more precisely
• Improve the representations based based on relation extraction

methods or by interacting with experts

• Evaluate semantic relationships in a context-dependent manner

This semantic background knowledge can be obtained by learning vector-space representations, where:

• Entities correspond to points
• Concepts correspond to regions
• Relative features correspond to directions

References

Qualitative reasoning about conceptual spaces Learning conceptual spaces from data Plausible reasoning based on qualitative conceptual space representations

Steven Schockaert, Sanjiang Li. Realizing RCC8 networks using convex regions. Artificial Intelligence, 2015 Steven Schockaert, Sanjiang Li. Combining RCC5 relations with betweenness information. IJCAI 2013 Steven Schockaert, Jae Hee Lee. Qualitative reasoning about directions in semantic spaces. IJCAI 2015 Joaquín Derrac, Steven Schockaert. Inducing semantic relations from conceptual spaces: a data- driven approach to plausible reasoning. Artificial Intelligence, 2015 Steven Schockaert, Henri Prade. Interpolative and extrapolative reasoning in propositional theories using qualitative knowledge about conceptual spaces. Artificial Intelligence, 2013 Steven Schockaert, Henri Prade. Solving conflicts in information merging by a flexible interpretation

• f atomic propositions. Artificial Intelligence, 2011

Shoaib Jameel, Steven Schockaert. Entity embeddings with conceptual sub-spaces. ECAI, 2016