Introduction Prediction and Grammar Predictive Parsing Evaluation Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh http://homepages.inf.ed.ac.uk/keller/ October 30, 2013 Frank Keller ANLP Guest Lecture 1
Introduction Prediction and Grammar Predictive Parsing Evaluation Introduction 1 Incrementality Prediction Prediction and Grammar 2 Conceptual Issues Formalism Comparison with TAG Modeling Prediction Predictive Parsing 3 Treebank Conversion and Lexicon Induction Parsing Algorithm and Probability Model Linking Theory Evaluation 4 Parsing Performance Cognitive Plausibility Frank Keller ANLP Guest Lecture 2
Introduction Prediction and Grammar Incrementality Predictive Parsing Prediction Evaluation Introduction 1 Incrementality Prediction Prediction and Grammar 2 Conceptual Issues Formalism Comparison with TAG Modeling Prediction Predictive Parsing 3 Treebank Conversion and Lexicon Induction Parsing Algorithm and Probability Model Linking Theory Evaluation 4 Parsing Performance Cognitive Plausibility Frank Keller ANLP Guest Lecture 3
Introduction Prediction and Grammar Incrementality Predictive Parsing Prediction Evaluation Incrementality Text and speech are perceived serially. Human language processing is adapted to this: sentence comprehension proceeds incrementally: the interpretation of a sentence is built word by word; each new word is integrated as fully as possible into a representation of the sentence thus far; processing effort depends on the properties of the word and its relationship to the preceding context. Frank Keller ANLP Guest Lecture 4
Introduction Prediction and Grammar Incrementality Predictive Parsing Prediction Evaluation Structural Prediction Staub and Clifton (2006) show that the sentence processor can also make structural predictions: (1) Peter read either a book or an essay in the school magazine. (2) Peter read a book or an essay in the school magazine. The presence of either leads to shorter reading times on or and on the NP that follows it (eye-tracking study). The word either makes it possible to anticipate an upcoming NP conjunction (rather than VP conjunction). Frank Keller ANLP Guest Lecture 5
Introduction Prediction and Grammar Incrementality Predictive Parsing Prediction Evaluation Semantic Prediction Visual world paradigm: image and speech presented synchronously; eye-movements reflect listeners’ interpretation of input; they can also indicate predictions about upcoming input. Altmann and Kamide (1999) use this paradigm to provided evidence for semantic prediction. They presented sentences such as: (3) a. The boy will eat . . . b. The boy will move . . . together with a scene that contained one edible but several movable objects. Frank Keller ANLP Guest Lecture 6
Introduction Prediction and Grammar Incrementality Predictive Parsing Prediction Evaluation Semantic Prediction When participants heard eat , they looked more at the cake. Evidence for prediction induced by semantic restrictions of the verb. Frank Keller ANLP Guest Lecture 7
Introduction Prediction and Grammar Incrementality Predictive Parsing Prediction Evaluation Granularity of Prediction What is the granularity of prediction? We saw predictions can be triggered by: specific collocations ( either . . . or ); semantic restrictions of a lexical item. Recent evidence also points to lexically specific syntactic prediction (Arai and Keller 2013): the subcategorization frame of a verb is predicted; verb morphology (e.g., tense information) is used for prediction. Frank Keller ANLP Guest Lecture 8
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Introduction 1 Incrementality Prediction Prediction and Grammar 2 Conceptual Issues Formalism Comparison with TAG Modeling Prediction Predictive Parsing 3 Treebank Conversion and Lexicon Induction Parsing Algorithm and Probability Model Linking Theory Evaluation 4 Parsing Performance Cognitive Plausibility Frank Keller ANLP Guest Lecture 9
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Conceptual Issues Challenge: develop a model of prediction in sentence processing that accounts for these experimental results. Assumptions: structures are built incrementally (word by word); partial structures do not contain unconnected nodes; upcoming syntactic material is predicted. Evidence for connectedness: Sturt and Lombardo (2005). Existing incremental parsers don’t build fully connected structures. Approach: devise a grammar formalism that supports incrementality and connectedness; prediction then follows (Demberg et al. 2013). Frank Keller ANLP Guest Lecture 10
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Implementing Prediction Experimental results inform the model regarding the granularity of prediction. The model predicts: lexical items when they are syntactically required (e.g., either . . . or , pick . . . up ); syntactic structure when required by subcat frames; syntactic structure when required by connectedness. Frank Keller ANLP Guest Lecture 11
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Demberg et al. (2013) propose Psycholinguistically Motivated TAG (PLTAG), a variant of tree-adjoining grammar: in standard TAG, the lexicon consists of initial trees and auxiliary trees (both are lexicalized); we add unlexicalized predictive trees to achieve connectivity; the standard TAG operations are substitution and adjunction; we add verification to verify predictive trees; we use TAG’s extended domain of locality for lexical prediction. PLTAG supports parsing with incremental, fully connected structures. Frank Keller ANLP Guest Lecture 12
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Lexicon: Standard TAG lexicon Predictive lexicon (PLTAG) Operations: Substitution Adjunction Verification (PLTAG) Frank Keller ANLP Guest Lecture 13
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Lexicon: Example Standard TAG lexicon NP S Initial Tree: Predictive lexicon Peter NP ↓ VP (PLTAG) sleeps Operations: VP Auxiliary Tree: Substitution AP VP* Adjunction Verification (PLTAG) often Frank Keller ANLP Guest Lecture 13
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Lexicon: Example Standard TAG lexicon NP substitutes into S Predictive lexicon Peter NP ↓ VP (PLTAG) sleeps Operations: resulting in S Substitution NP VP Adjunction Peter sleeps Verification (PLTAG) Frank Keller ANLP Guest Lecture 13
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Example Lexicon: VP adjoins to S Standard TAG lexicon AP VP* Predictive lexicon NP VP (PLTAG) often Peter sleeps resulting in S Operations: Substitution NP VP Adjunction Peter AP VP Verification (PLTAG) often sleeps Frank Keller ANLP Guest Lecture 13
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Lexicon: Standard TAG lexicon Predictive lexicon Example (PLTAG) Prediction Tree: S k NP k ↓ VP k Operations: k Substitution Index k marks predicted node. Adjunction Verification (PLTAG) Frank Keller ANLP Guest Lecture 13
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Formalism Example S 1 is verified by S Lexicon: NP ↓ VP Standard TAG lexicon NP 1 VP 1 Predictive lexicon sleeps AP VP 1 Peter (PLTAG) often Operations: resulting in S Substitution NP VP Adjunction Peter AP VP Verification (PLTAG) often sleeps All nodes indexed with k have to be verified. Frank Keller ANLP Guest Lecture 13
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Comparison with and TAG TAG derivations are not always incremental. Example S S S subst NP VP adj VP VP NP ↓ NP Peter AP VP sleeps Peter sleeps often sleeps Frank Keller ANLP Guest Lecture 14
Introduction Conceptual Issues Prediction and Grammar Formalism Predictive Parsing Comparison with TAG Evaluation Modeling Prediction Comparison with and TAG PLTAG derivation are always incremental and fully connected. Example S S 1 S 1 NP 1 subst 1 NP VP adj NP VP verif 1 1 NP VP 1 Peter Peter AP VP Peter AP VP 1 Peter often often sleeps Frank Keller ANLP Guest Lecture 14
Recommend
More recommend
