Statistical Models for Frame-Semantic Parsing Dipanjan Das * Google - PowerPoint PPT Presentation

LTH Frame-Semantic Parser Johansson and Nugues (2007) Frame Identification Argument Filtering Argument Labeling 47

Argument Filtering S TORE Bengal ’s massive stock of food was reduced to nothing massive stock

Argument Filtering S TORE Bengal ’s massive stock of food was reduced to nothing Potential Arguments of food Bengal ’s massive stock Bengal food Bengal ’s massive to nothing massive to of ’s massive stock

Argument Filtering S TORE Bengal ’s massive stock of food was reduced to nothing Potential Arguments Binary SVM Classification of food Bengal ’s massive stock Bengal food Bengal ’s massive to nothing massive to of ’s massive stock

LTH Frame-Semantic Parser Johansson and Nugues (2007) Frame Identification Argument Filtering Argument Labeling 50

Argument Filtering S TORE Bengal ’s massive stock of food was reduced to nothing Multiclass SVM Classification massive stock

Argument Filtering S TORE Bengal ’s massive stock of food was reduced to nothing Potential Arguments Multiclass SVM Classification of food Bengal ’s massive stock Bengal food Bengal ’s massive to nothing massive to of ’s massive stock

Argument Filtering S TORE Bengal ’s massive stock of food was reduced to nothing Potential Arguments Multiclass SVM Classification of food Bengal ’s massive stock Bengal Possessor Resource food Bengal ’s massive to nothing massive ø Descriptor to of ’s massive stock

LTH Frame-Semantic Parser Johansson and Nugues (2007) F-Measure 70.0 61.3 52.5 44.4 43.8 35.0 LTH Full Frame-Semantic Structure Prediction

SEMAFOR Das, Chen, Martins, Schneider, Smith (2014) Frame Identification Argument Identification 54

SEMAFOR Das, Chen, Martins, Schneider, Smith (2014) Frame Identification Argument Identification 55

SEMAFOR: Frame Identification N X A N ADP N V V ADP N Bengal ’s massive stock of food was reduced to nothing

SEMAFOR: Frame Identification N X A N ADP N V V ADP N Bengal ’s massive stock of food was reduced to nothing

SEMAFOR: Frame Identification Logistic regression with a latent variable

SEMAFOR: Frame Identification Logistic regression with a latent variable

SEMAFOR: Frame Identification Logistic regression with a latent variable Predicates evoking a frame in supervised data, e.g. cargo.N, inventory.N, reserve.N, stockpile.N, store.N, supply.N evoke S TORE

SEMAFOR: Frame Identification S TORE stock.N stockpile.N N X A N ADP N V V ADP N to nothing Bengal ’s massive stock of food was reduced

SEMAFOR: Frame Identification S TORE stock.N LexSem = { synonym } stockpile.N N X A N ADP N V V ADP N to nothing Bengal ’s massive stock of food was reduced

SEMAFOR: Frame Identification S TORE stock.N LexSem = { synonym } stockpile.N N X A N ADP N V V ADP N to nothing Bengal ’s massive stock of food was reduced S TORE If stockpile.N synonym LexSem

SEMAFOR: Frame Identification S TORE stock.N LexSem = { synonym } stockpile.N comes from WordNet ! N X A N ADP N V V ADP N to nothing Bengal ’s massive stock of food was reduced S TORE If stockpile.N synonym LexSem

Datasets Benchmark Dataset New Data (SemEval 2007) (FrameNet 1.5, 2010) 665 frames 877 frames 720 role labels 1068 role labels 8.4K unique predicate types 9.3K unique predicate types Training set: Training set: 2.2K sentences 3.3K sentences 11.2K predicate tokens 19.6K predicate tokens Test set: Test set: 120 sentences 2420 sentences 1. 1K predicate tokens 4.5K predicate tokens

Datasets Benchmark Dataset New Data (SemEval 2007) (FrameNet 1.5, 2010) 665 frames 877 frames 720 role labels 1068 role labels 8.4K unique predicate types 9.3K unique predicate types Training set: Training set: 2.2K sentences 3.3K sentences 11.2K predicate tokens 19.6K predicate tokens Test set: Test set: 120 sentences 2420 sentences 1. 1K predicate tokens 4.5K predicate tokens

SEMAFOR: Frame Identification Results Benchmark New Data F-Measure 95.0 83.8 72.5 61 61.3 57.3 50.0 LTH SEMAFOR log-linear

SEMAFOR: Frame Identification Results Benchmark New Data F-Measure 95.0 83.8 72.5 61 61.3 57.3 50.0 LTH SEMAFOR log-linear auto predicates

SEMAFOR: Frame Identification Results Benchmark New Data Accuracy F-Measure 95.0 95.0 83.0 83.8 83.8 72.5 72.5 61 61.3 61.3 57.3 50.0 50.0 LTH SEMAFOR SEMAFOR log-linear log-linear auto predicates gold predicates

SEMAFOR: Frame Identification Frame Identification Accuracy Accuracy 95.0 95.0 83.0 76.3 76.3 57.5 57.5 38.8 38.8 23.1 20.0 20.0 All Predicates Unknown Predicates

SEMAFOR: Handling Unknown Predicates Knowledge of only 9,263 predicates in supervised data

SEMAFOR: Handling Unknown Predicates Knowledge of only 9,263 predicates in supervised data However, English has lot more potential predicates (~65,000 in newswire English)

SEMAFOR: Handling Unknown Predicates Knowledge of only 9,263 predicates in supervised data However, English has lot more potential predicates (~65,000 in newswire English) Lexicon expansion using graph-based semi-supervised learning

How can label propagation help?

Example Graph Seed predicates

Example Graph Unseen predicates Seed predicates

Example Graph Unseen predicates Seed predicates Graph Propagation

Example Graph Unseen predicates Seed predicates Graph Propagation

Example Graph Continues till convergence... Unseen predicates Seed predicates Graph Propagation

SEMAFOR: Unknown Predicates Frame Identification Accuracy 70.0 56.3 42.7 42.5 28.8 23.1 18.9 15.0 Supervised Self-Training Graph-Based

SEMAFOR Das, Chen, Martins, Schneider, Smith (2014) Frame Identification Argument Identification 81

SEMAFOR: Argument Identification S TORE Bengal ’s massive stock of food was reduced to nothing

SEMAFOR: Argument Identification S TORE Bengal ’s massive stock of food was reduced to nothing Possessor Resource Descriptor Use Supply

SEMAFOR: Argument Identification S TORE Bengal ’s stock Bengal massive stock Possessor of food Resource food Descriptor massive Bengal ’s massive Use massive stock Supply ø

SEMAFOR: Argument Identification S TORE Bengal ’s stock Bengal massive stock Possessor of food Resource food Descriptor massive Bengal ’s massive Use massive stock Supply ø

SEMAFOR: Argument Identification S TORE Bengal ’s stock Bengal massive stock Possessor Violates overlap of food Resource constraints food Descriptor massive Bengal ’s massive Use massive stock Supply ø

SEMAFOR: Argument Identification Other types of structural constraints Mutual P LACING Agent exclusion Cause constraint Goal Theme archive.V, Area arrange.V, bag.V, Time bestow.V bin.V

SEMAFOR: Argument Identification Other types of structural constraints Mutual P LACING Agent exclusion Cause constraint Goal Theme archive.V, Area arrange.V, bag.V, Time bestow.V bin.V If an agent places something, there cannot be a cause role in the sentence