Tricks for Statistical Semantic Tricks for Statistical Semantic - - PDF document

tricks for statistical semantic tricks for statistical
SMART_READER_LITE
LIVE PREVIEW

Tricks for Statistical Semantic Tricks for Statistical Semantic - - PDF document

Tricks for Statistical Semantic Tricks for Statistical Semantic Knowledge Discovery: Knowledge Discovery: A Selectionally Selectionally Restricted Sample Restricted Sample A Marti A. Hearst Marti A. Hearst UC Berkeley UC Berkeley


slide-1
SLIDE 1

1

Tricks for Statistical Semantic Tricks for Statistical Semantic Knowledge Discovery: Knowledge Discovery:

A A Selectionally Selectionally Restricted Sample Restricted Sample

Marti A. Hearst Marti A. Hearst UC Berkeley UC Berkeley

Marti Hearst, NYU Semantics ‘08

Statistical Approaches Statistical Approaches

► ►An alternative to hand

An alternative to hand-

  • coded meaning.

coded meaning.

► ►Solve sub

Solve sub-

  • problems first.

problems first.

e.g., Acquiring Semantic Relations e.g., Acquiring Semantic Relations

slide-2
SLIDE 2

2

Marti Hearst, NYU Semantics ‘08

Tricks I Like Tricks I Like

Lots o’ Text Unambiguous Cues Rewrite and Verify

Marti Hearst, NYU Semantics ‘08

Trick: Lots o Trick: Lots o’ ’ Text Text

► ►Idea: words in the same syntactic context are

Idea: words in the same syntactic context are semantically related. semantically related.

  • Hindle

Hindle, ACL , ACL’ ’90, 90, “ “Noun classification from predicate Noun classification from predicate-

  • argument structure.

argument structure.” ”

slide-3
SLIDE 3

3

Marti Hearst, NYU Semantics ‘08

Trick: Lots o Trick: Lots o’ ’ Text Text

► ►Idea: words in the same syntactic context are

Idea: words in the same syntactic context are semantically related. semantically related.

  • Nakov

Nakov & Hearst, ACL/HLT & Hearst, ACL/HLT’ ’08 08 “ “Solving Relational Similarity Problems Using the Web as a Corpus Solving Relational Similarity Problems Using the Web as a Corpus” ”

Marti Hearst, NYU Semantics ‘08

Trick: Lots o Trick: Lots o’ ’ Text Text

► ►Idea: bigger is better than smarter!

Idea: bigger is better than smarter!

  • Banko

Banko & Brill ACL & Brill ACL’ ’01: 01: “ “Scaling to Very, Very Large Corpora for Natural Scaling to Very, Very Large Corpora for Natural Language Disambiguation Language Disambiguation” ”

slide-4
SLIDE 4

4

Marti Hearst, NYU Semantics ‘08

Trick: Lots o Trick: Lots o’ ’ Text Text

► ►Idea: apply web

Idea: apply web-

  • scale n

scale n-

  • grams to every

grams to every problem imaginable. problem imaginable.

  • Lapata

Lapata & Keller, HLT/NACCL & Keller, HLT/NACCL ‘ ‘04 04: : “ “Web as a Baseline: Evaluating Web as a Baseline: Evaluating the Performance of Unsupervised Web the Performance of Unsupervised Web-

  • Based Models for a Range

Based Models for a Range

  • f NLP Tasks
  • f NLP Tasks”

MT candidate selection Article suggestion Noun compound interpretation Noun compound bracketing Adjective ordering > supervised = supervised

Marti Hearst, NYU Semantics ‘08

Limitation Limitation

► ►Sometimes counts alone are too ambiguous.

Sometimes counts alone are too ambiguous.

Solution Solution

► ►Bootstrap from

Bootstrap from unambiguous unambiguous contexts. contexts.

slide-5
SLIDE 5

5

Marti Hearst, NYU Semantics ‘08

Trick: Use Unambiguous Context Trick: Use Unambiguous Context

► ►…

… to build statistics for ambiguous contexts. to build statistics for ambiguous contexts.

  • Hindle

Hindle & & Rooth Rooth, ACL , ACL ’ ’91 91“ “Structural Ambiguity and Lexical Relations Structural Ambiguity and Lexical Relations” ”

Example: PP attachment I eat spaghetti with sauce. Bootstrap from unambiguous contexts: Spaghetti with sauce is delicious. I eat with a fork.

Marti Hearst, NYU Semantics ‘08

Trick: Use Unambiguous Context Trick: Use Unambiguous Context

► ► …

… to identify semantic relations ( to identify semantic relations (lexico lexico-

  • syntactic contexts)

syntactic contexts)

  • Hearst, COLING

Hearst, COLING ’ ’92, 92, “

“Automatic Acquisition of Hyponyms from Large Text

Automatic Acquisition of Hyponyms from Large Text Corpora Corpora” ”

Example: Hyponym I dentification

slide-6
SLIDE 6

6

Combine Tricks 1 and 2 Combine Tricks 1 and 2

Marti Hearst, NYU Semantics ‘08

Trick: Use Unambiguous Contexts + Trick: Use Unambiguous Contexts + Lot Lot’ ’s O s O’ ’ Text Text

► ►Combine

Combine lexico lexico-

  • syntactic patterns with

syntactic patterns with

  • ccurrence counts.
  • ccurrence counts.
  • Kozareva

Kozareva, , Riloff Riloff, , Hovy Hovy, HLT , HLT-

  • ACL

ACL’ ’08.

  • 08. “

“Semantic Class learning form the Web with Semantic Class learning form the Web with Hyponym Pattern Linkage Graphs Hyponym Pattern Linkage Graphs” ”. .

slide-7
SLIDE 7

7

Marti Hearst, NYU Semantics ‘08

Trick: Use Unambiguous Contexts + Trick: Use Unambiguous Contexts + Lot Lot’ ’s O s O’ ’ Text Text

► ►Combine (usually) unambiguous surface

Combine (usually) unambiguous surface patterns with occurrence counts. patterns with occurrence counts.

  • Nakov

Nakov & Hearst, HLT/EMNLP & Hearst, HLT/EMNLP’ ’05 05 “ “Using the Web as an Implicit Training Using the Web as an Implicit Training Set: Application to Structural Ambiguity Resolution Set: Application to Structural Ambiguity Resolution” ”. .

Left dash Left dash cell cell-

  • cycle analysis

cycle analysis

left

left Possessive marker Possessive marker brain brain’ ’s stem cell s stem cell

right

right Parentheses Parentheses growth factor (beta) growth factor (beta)

left

left Punctuation Punctuation heath care, provider heath care, provider

left

left Abbreviation Abbreviation tum tum. . necr.(TN necr.(TN) factor ) factor

right

right Concatenation Concatenation heathcare heathcare reform reform

left

left

Marti Hearst, NYU Semantics ‘08

Trick: Use Unambiguous Contexts + Trick: Use Unambiguous Contexts + Lot Lot’ ’s O s O’ ’ Text Text

► ►Identify a

Identify a “ “protagonist protagonist” ” in each text to learn in each text to learn narrative structure narrative structure

  • Chambers &

Chambers & Jurafsky Jurafsky, ACL , ACL’ ’08 08 “ “Unsupervised Learning of Narrative Event Chains Unsupervised Learning of Narrative Event Chains” ”. .

slide-8
SLIDE 8

8

Trick 3: Trick 3: Rewrite & Verify Rewrite & Verify

Marti Hearst, NYU Semantics ‘08

Trick: Rewrite & Verify Trick: Rewrite & Verify

► ► Check if alternatives exist in text

Check if alternatives exist in text

  • Nakov

Nakov & Hearst, HLT/EMNLP & Hearst, HLT/EMNLP’ ’05 05 “ “Using the Web as an Implicit Training Set: Application to Using the Web as an Implicit Training Set: Application to Structural Ambiguity Resolution Structural Ambiguity Resolution” ”. .

  • Example: NP bracketing

Example: NP bracketing

  • Prepositional

Prepositional

► ► stem cells

stem cells in

in the

the brain brain

  • right

right

► ► stem cells

stem cells from

from the

the brain brain

right

right

► ► cells

cells from

from the

the brain brain stem stem

left

left

  • Verbal

Verbal

► ► virus

virus causing

causing human immunodeficiency

human immunodeficiency

left

left

► ► pain

pain associated with

associated with arthritis migraine

arthritis migraine

  • left

left

  • Copula

Copula

► ► office building

  • ffice building that is

that is a

a skyscraper skyscraper

right

right

slide-9
SLIDE 9

9

Towards New Approaches Towards New Approaches to Semantic Analysis to Semantic Analysis

Marti Hearst, NYU Semantics ‘08

Ideas Ideas

► ►Inducing Semantic Grammars

Inducing Semantic Grammars

  • Boggess

Boggess, , Agarwal Agarwal, & Davis, AAAI , & Davis, AAAI ’ ’91, 91, “ “Disambiguation of Prepositional Disambiguation of Prepositional Phrases in Automatically Phrases in Automatically Labelled Labelled Technical Text Technical Text” ”

slide-10
SLIDE 10

10

Marti Hearst, NYU Semantics ‘08

Ideas Ideas

► ►Use Cognitive Linguistics

Use Cognitive Linguistics

  • Hearst,

Hearst, ’ ’90, 90,’ ’92, 92, “ “Direction Direction-

  • Based Text Interpretation

Based Text Interpretation” ”. .

  • Talmy

Talmy’ ’s s Force Dynamics + Reddy Force Dynamics + Reddy’ ’s Conduit Metaphor s Conduit Metaphor

  • Path Model

Path Model

  • Solves: Was the person in favor of or opposed to the idea

Solves: Was the person in favor of or opposed to the idea

Marti Hearst, NYU Semantics ‘08

Using Cognitive Linguistics Using Cognitive Linguistics

► ►Talmy

Talmy’ ’s s Theory of Force Dynamics Theory of Force Dynamics

  • Talmy

Talmy, , “ “Force Dynamics in Language and Thought, Force Dynamics in Language and Thought,” ” in in Parasession on Causatives and Agentivity, Chicago Linguistic Society 1985.

  • Describes how the interaction of agents with respect to force is

Describes how the interaction of agents with respect to force is lexically lexically and grammatically expressed. and grammatically expressed.

  • Posits two opposing entities: Agonist and Antagonist.

Posits two opposing entities: Agonist and Antagonist.

  • Each entity expresses an intrinsic force: towards rest or motion

Each entity expresses an intrinsic force: towards rest or motion. .

  • The balance of the strengths of the entities determines the outc

The balance of the strengths of the entities determines the outcome of the

  • me of the

event. event.

► ► Grammatical expression includes using a

Grammatical expression includes using a claused claused headed by headed by “ “despite despite” ” to express a weaker to express a weaker antagonist. antagonist.

slide-11
SLIDE 11

11

Marti Hearst, NYU Semantics ‘08

Using Cognitive Linguistics Using Cognitive Linguistics

► ►Reddy

Reddy’ ’s Conduit Metaphor s Conduit Metaphor

  • Reddy,

Reddy, “ “The Conduit Metaphor The Conduit Metaphor – – A Case of Frame Conflict in Our Language about Language, A Case of Frame Conflict in Our Language about Language,” ” in in Metaphor and Thought, Ortony (Ed), Cambridge University Press, 1979.

  • A thought is schematized as an object which is placed by the spe

A thought is schematized as an object which is placed by the speaker into aker into a container that is sent along a conduit. a container that is sent along a conduit.

  • The receiver at the other end is the listener, who removes the o

The receiver at the other end is the listener, who removes the objectified bjectified thought from the container and thus possesses it. thought from the container and thus possesses it.

  • Inferences that apply to conduits can be applied to communicatio

Inferences that apply to conduits can be applied to communication. n.

► ► “

“Your meaning did not come through. Your meaning did not come through.” ”

► ► “

“I can I can’ ’t put this thought into words. t put this thought into words.” ”

► ► “

“She is sending you some kind of message with that remark. She is sending you some kind of message with that remark.” ”

Marti Hearst, NYU Semantics ‘08

Using Cognitive Linguistics Using Cognitive Linguistics

► ►Combine into the Path Model

Combine into the Path Model

  • Hearst,

Hearst, “ “Direction Direction-

  • based Text Interpretation as an Information Access Refinement,

based Text Interpretation as an Information Access Refinement,” ” in in Text- based Intelligent Systems, Jacobs (Ed), Lawrence Erlbaum Associates, 1992.

  • If an agent favors an entity or event, that agent can be said to

If an agent favors an entity or event, that agent can be said to desire the desire the existence or existence or “ “well well-

  • being

being” ” of that entity, and vice

  • f that entity, and vice-
  • versa.

versa.

  • Thus if an agent favors an entity

Thus if an agent favors an entity’ ’s triumph in a force s triumph in a force-

  • dynamic interaction,

dynamic interaction, then the agent favors that entity or event. then the agent favors that entity or event.

  • But: force dynamics does not have the expressive power for a seq

But: force dynamics does not have the expressive power for a sequence. uence.

Instead of focusing on the relative strength of two interacting

entities, the model should represent what happens to a single entity through the course of its encounters with other entities.

  • Thus the entity can be schematized as if it were moving along a path

toward some destination or goal.

slide-12
SLIDE 12

12

Marti Hearst, NYU Semantics ‘08

Using Cognitive Linguistics Using Cognitive Linguistics

► ►The Path Model

The Path Model

  • Hearst,

Hearst, “ “Direction Direction-

  • based Text Interpretation as an Information Access Refinement,

based Text Interpretation as an Information Access Refinement,” ” in in Text- based Intelligent Systems, Jacobs (Ed), Lawrence Erlbaum Associates, 1992.

Marti Hearst, NYU Semantics ‘08

Using Cognitive Linguistics Using Cognitive Linguistics

► ►The Path Model

The Path Model

  • Hearst,

Hearst, “ “Direction Direction-

  • based Text Interpretation as an Information Access Refinement,

based Text Interpretation as an Information Access Refinement,” ” in in Text- based Intelligent Systems, Jacobs (Ed), Lawrence Erlbaum Associates, 1992.

slide-13
SLIDE 13

13

Marti Hearst, NYU Semantics ‘08

Using Cognitive Linguistics Using Cognitive Linguistics

► ►The Path Model

The Path Model

  • Hearst,

Hearst, “ “Direction Direction-

  • based Text Interpretation as an Information Access Refinement,

based Text Interpretation as an Information Access Refinement,” ” in in Text- based Intelligent Systems, Jacobs (Ed), Lawrence Erlbaum Associates, 1992.

Marti Hearst, NYU Semantics ‘08

Summary Summary

► ►Statistical approaches to semantics help

Statistical approaches to semantics help solve sub solve sub-

  • problems in clever ways.

problems in clever ways.

  • Use lots of text

Use lots of text

  • Find unambiguous cues

Find unambiguous cues

  • Rewrite and Verify

Rewrite and Verify ► ► It

It’ ’s time to think about more radical alternatives s time to think about more radical alternatives

  • Statistical semantic grammars

Statistical semantic grammars

  • Cognitive linguistics for powerful generalizations.

Cognitive linguistics for powerful generalizations.