Semant ic Knowledge f or Semant ic Knowledge f or Text ual Ent - - PDF document

1

Semant ic Knowledge f or Semant ic Knowledge f or Text ual Ent ailment Text ual Ent ailment

Bernardo Magnini

J oint work wit h: Elena Cabrio, Milen Kouylekov, Mat t eo Negri FBK-I rst Trent o, I t aly

NSF Symposium on Semant ic Knowledge Discovery, Organizat ion and Use November, 14 and 15, 2008 New York Universit y

Out line Out line

Text ual Ent ailment Applied TE: A TE engine f or Quest ion Answering Open issue: int eract ions and dependencies of t he linguist ic phenomena wit h respect t o ent ailment . Proposal: a general f ramework, f lexible enough t o allow t he combinat ion of specialized ent ailment engines.

2

Typical Application Inference: Entailment

Overture’s acquisition by Yahoo Yahoo bought Overture

Question Expected answer form

Who bought Overture? >> X bought Overture

text hypothesized answer

entails

• Similar for IE: X acquire Y
• Similar for “semantic” IR: t: Overture was bought for …
• Summarization (multi-document) – identify redundant info
• MT evaluation (and recent ideas for MT)
• Educational applications

TE tutorial at ACL 2007, Dagan, Roth, Zanzotto

“Almost certain” Entailments

t: The technological triumph known as GPS … was incubated in the mind of Ivan Getting. h: Ivan Getting invented the GPS.

TE tutorial at ACL 2007, Dagan, Roth, Zanzotto

3

Applied Textual Entailment

• A directional relation between two text fragments:

Text (t) and Hypothesis (h): t entails h (t⇒h) if humans reading t will infer that h is most likely true

Operational (applied) definition:

Human gold standard - as in NLP applications Assuming common background knowledge –

which is indeed expected from applications

TE tutorial at ACL 2007, Dagan, Roth, Zanzotto

Probabilistic Interpretation

Definition:

• t probabilistically entails h if:

– P(h is true | t) > P(h is true)

• t increases the likelihood of h being true
• ≡ Positive PMI – t provides information on h’s truth
• P(h is true | t ): entailment confidence

– The relevant entailment score for applications – In practice: “most likely” entailment expected

TE tutorial at ACL 2007, Dagan, Roth, Zanzotto

4

The Role of Knowledge

• For textual entailment to hold we require:

– text AND knowledge ⇒ h but – knowledge should not entail h alone

• Systems are not supposed to validate h’s truth

regardless of t (e.g. by searching h on the web)

t: The technological triumph known as GPS … was incubated in the mind of Ivan Getting. h: Ivan Getting invented the GPS.

TE tutorial at ACL 2007, Dagan, Roth, Zanzotto

Entailment Entailment-

• Based Approach in

Based Approach in Qallme Qallme

HasDirector[MOVIE, PERSON]

Ontology

HasAddress[CINEMA, ADDRESS]

5

Entailment Engine for QA Entailment Engine for QA

P1: What is the telephone number of Cinema:X? P2: Who is the director of Movie:X? P3: What is the ticket price of Cinema:X? P4: Give me the address of Cinema:X. …

Relational Textual Patterns

HasDirector[MOVIE, PERSON]

Ontology

HasAddress[CINEMA, ADDRESS] Q: “Where is cinema Astra located?”

P1: What is the telephone number of Cinema:X? P2: Who is the director of Movie:X? P3: What is the ticket price of Cinema:X? P4: Give me the address of Cinema:X. …

Relational Textual Patterns

HasDirector[MOVIE, PERSON]

Ontology

HasAddress[CINEMA, ADDRESS]

Entailment Engine for QA Entailment Engine for QA

6

Q: “Where is cinema Astra located?”

P1: What is the telephone number of Cinema:X? P2: Who is the director of Movie:X? P3: What is the ticket price of Cinema:X? P4: Give me the address of Cinema:X. …

Entailment engine Relational Textual Patterns

Q ⇒ {P?, P?, …}

QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.

entails HasDirector[MOVIE, PERSON]

Ontology

HasAddress[CINEMA, ADDRESS]

Entailment Engine for QA Entailment Engine for QA

Q: “Where is cinema Astra located?”

Entailment engine

QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.

P4: Give me the address of Cinema:X.

Q ⇒ P4

entails

P1: What is the telephone number of Cinema:X? P2: Who is the director of Movie:X? P3: What is the ticket price of Cinema:X? P4: Give me the address of Cinema:X. …

Relational Textual Patterns

HasDirector[MOVIE, PERSON]

Ontology

HasAddress[CINEMA, ADDRESS]

Entailment Engine for QA Entailment Engine for QA

7

Q: “Where is cinema Astra located?”

Entailment engine

SELECT ?street WHERE {?cinema tourism:name “X”. ?cinema tourism:hasPostalAddress . ?addressr tourism:street ?street}

A: Corso Buonarroti, 16 - Trento

QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture.

P4: Give me the address of Cinema:X.

Q ⇒ P4

entails

P1: What is the telephone number of Cinema:X? P2: Who is the director of Movie:X? P3: What is the ticket price of Cinema:X? P4: Give me the address of Cinema:X. …

Relational Textual Patterns

HasDirector[MOVIE, PERSON]

Ontology

HasAddress[CINEMA, ADDRESS]

Entailment Engine for QA Entailment Engine for QA

Semant ic Dependencies Semant ic Dependencies

Multiple linguistic aspects are relevant f or entailment:

<pair id=“400” entailment=“ENTAILMENT” task=“QA”> <t>The polygraph came along in 1921, invented by John A. Larson, a University of California medical student working with help from a police official. The device

• stensibly detects when a person is lying by monitoring

and recording certain body changes affected by a person’s emotional condition. </t> <h>The polygraph is a device that ostensibly detects when a person is not telling the truth</h></pair>

ANTONYMS NEGATION

8

Provide a modular approach t hrough which evaluat e progresses on single aspect s of ent ailment , using specialized t raining and t est dat aset . Devise a general f ramework, based on t he dist ance bet ween T and H, f lexible enough t o allow t he combinat ion of single ent ailment engines. I nvest igat e t he int eract ions and t he dependencies

• f t he dif f erent linguist ic phenomena wit h respect

t o ent ailment .

Semant ic Dependencies Semant ic Dependencies TE engines combinat ions TE engines combinat ions

Dif f erent independent entailment engines, each of which able t o deal wit h an aspect of t he language variabilit y (e.g. negat ion, modals). Output of the whole system: sum of the edit distances produced by each module (alt hough t he dif f erent linguist ic phenomena can be dependent on one anot her in dif f er ent and complex ways)

9

TE Engine-x

The ling. phenomenon is present in t he pair it is relevant it is NOT relevant D=score D=0 Expect ed behavior of each single TE engine: The ling. phenomenon is not present in t he pair. D=0

Dist ance Dist ance-

• Based TE Engine

Based TE Engine Dist ance Dist ance-

• Based TE Engine

Based TE Engine

Det ermines t he best (less cost ly) sequence of edit

• perat ions t hat allow t o

t ransf orm T int o H:

• Linear dist ance
• Tree Edit Dist ance

Det ermines t he cost of t he t hree edit operat ions (insert ion, delet ion, subst it ut ion) Each r ule has a probabilit y represent ing t he degree of conf idence of t he rule. Rules can be at dif f erent levels (e.g. lexical, synt act ic)

10

A TE Engine f or Negat ion A TE Engine f or Negat ion

NEGATI ON, processed f ocusing on direct licensors of negat ion such as overt negat ive markers (not , n’t ), negat ive quant if iers (no, not hing), st rong negat ive adverbs (never ); ANTONYMS.

Linear Distance Algorithm (Levenshtein distance calculated on tokens) Cost schema, e.g.: <rule name=“deletion_not"> <left><syntax><token><text>n

• t</text></token></syntax></l

eft> <score>20</score>

<t>Giles Chichester's position was viewed as untenable partly because he had been given the job of a sleazebuster by Mr Cameron to ensure the integrity of Tory MEP expenses. He is not the leader of the Tory MEPs.</t> <h>Giles Chichester is the leader of the Tories MEPs.</h>

dneg

<pair id="107" entailment="CONTRADICTION" task="IR">

A TE f or Negat ion at RTE4 A TE f or Negat ion at RTE4

1st run ACC 0.54

• Avg. Pr.

0.4946

EDI TSneg 1000 pairs 164 p. 836 p. presence of NPIs D=0 438 TP + 398 FP 46 p. 116 p. relevant non-relevant D=max D=max 102 TN 62 FN

11

LEXI CAL SI MI LARI TI ES (WordNet )

Linear Distance Algorithm (Levenshtein distance calculated on tokens) Cost schema, e.g.: <rule name="entail"> <!‐‐ The function erules.entails returns a probability that A entails

• B. ‐‐>

<left><syntax><token><text>$VAR{A}</text></t

• ken></syntax></left>

<right><syntax><token><text>$VAR{B}</text></ token></syntax></right> <score>(* SUBSTITUTION (‐ 1 (erules.entails A B)))</score></rule> Entailment rules: generated exploiting the WordNet similarity between T and H <t>South Korea has lifted a five‐year ban on beef imports from the US, despite growing public protests prompted by fears of mad cow disease.</t> <h>South Korea removes a US beef ban.</h>

dlex

<pair id="34" entailment="ENTAILMENT" task="IR">

A TE Engine f or Lexical Similarity A TE Engine f or Lexical Similarity

Conclusion and Fut ure work Conclusion and Fut ure work

Semant ic Knowledge plays a crucial role in TE Applied TE: A TE engine f or Quest ion Answering Dependencies among phenomena need invest igat ion A general f ramework, f lexible enough t o allow t he combinat ion of specialized ent ailment engines. Ext end t he model adding specialized engines, f ocusing on dif f erent linguist ic phenomena (e.g. modals, act ive/ passive synt act ic const ruct ion)