Computational Linguistics: Syntax I Raffaella Bernardi e-mail: - PowerPoint PPT Presentation

Computational Linguistics: Syntax I Raffaella Bernardi e-mail: raffaella.bernardi@unitn.it Contents First Last Prev Next ◭

Contents 1 SCHEDULE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 Reminder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Formal Languages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Regular Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3 Today . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5 Dependency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 6 Long-distance Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 6.1 Relative Pronouns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6.2 Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 7 Sentence Structures: English . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 8 Formal Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 9 Syntax Recognizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 9.0.1 Pumping Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 9.1 NLs are not RL: Example I . . . . . . . . . . . . . . . . . . . . . . . . . . 21 9.2 NLs are not RL: Example II . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Contents First Last Prev Next ◭

10 FSA for syntactic analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 11 Formal Grammar: Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 12 Formal Grammars: Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 12.1 Derivations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 12.2 Formal Languages and FG . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 12.3 FG and Regular Languages . . . . . . . . . . . . . . . . . . . . . . . . . . 29 12.4 FSA and RG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 13 Context Free Grammars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 14 CFG : Formal Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 14.1 CFG : More derivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 14.2 CFG : Language Generated. . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 15 FG and Natural Language: parse trees . . . . . . . . . . . . . . . . . . . . . . . 35 16 FG for Natural Languages: Lexicon vs. Grammatical Rules . . . . . 37 17 PSG : English Toy Fragment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 18 English Toy Fragment: Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 19 English Toy Fragment: Phrase Structure Trees . . . . . . . . . . . . . . . . 41 20 Summing up (I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 21 Summing up (II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 22 Generative Power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Contents First Last Prev Next ◭

23 Hierarchy of Grammars and Languages. . . . . . . . . . . . . . . . . . . . . . . 45 24 Chomsky Hierarchy of Languages. . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 25 Dissenting Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 25.1 Are NL Context Free (CF)? . . . . . . . . . . . . . . . . . . . . . . . . . . 49 25.2 Nested and Crossing Dependencies . . . . . . . . . . . . . . . . . . . . 51 25.3 English & Copy Language. . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 25.4 Cross-serial dependencies in Dutch . . . . . . . . . . . . . . . . . . . . 53 25.5 Cross-serial dependencies Swiss German . . . . . . . . . . . . . . . 54 26 Where does NL fit? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 27 Mildly Context-sensitive Languages (MSC) . . . . . . . . . . . . . . . . . . . 56 28 Where do the different Formal Grammars stand? . . . . . . . . . . . . . . 57 29 Complexity Issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 29.1 Input length. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 29.2 Complexity of a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 29.3 Complexity w.r.t. Chomsky Hierarchy . . . . . . . . . . . . . . . . . 61 30 Human Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 31 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Contents First Last Prev Next ◭

1. SCHEDULE I’ve checked with the admin and the MSc coordinator, we can move the class from Mondays to Fridays: 13:30-16:30 (starting at 13:30) Any objections? Contents First Last Prev Next ◭

2. Reminder Main issues of last lecture: ◮ Different levels of Natural Language 1. Phonology 2. Morphology 3. Syntax 4. Semantics 5. Discourse ◮ Linguistically motivated computational models. For any topic: 1. Linguistic Theory 2. Formal Analysis 3. Implementation Contents First Last Prev Next ◭

2.1. Example ◮ Linguistic Theories 1. Morphology: Stems vs. Affixes; Inflectional and derivational forms. ◮ Natural Language as Formal Language 1. Morphology can be formalized by means of Regular Languages and as such modeled by FSA. ◮ Implementation 1. Use some programming language to computationally deal with the Lin- guistic Theory modeled by the Formal Languages. Contents First Last Prev Next ◭

2.2. Formal Languages ◮ A formal language is a set of strings. E.g. { a, b, c } , { the, a, student, students } . ◮ Strings are by definition finite in length. ◮ The language accepted (or recognized) by an FSA is the set of all strings it recognizes when used in recognition mode. ◮ The language generated by an FSA is the set of all strings it can generate when used in generation mode. ◮ The language accepted and the language generated by an FSA are exactly the same. ◮ FSA recognize/generate Regular Languages. Contents First Last Prev Next ◭

2.3. Regular Language V ∗ denotes the set of all strings formed over the alphabet V . A ∗ denotes the set of all strings obtained by concatenating strings in A in all possible ways. Given an alphabet V , 1. {} is a regular language 2. For any string x ∈ V ∗ , { x } is a regular language. 3. If A and B are regular languages, so is A ∪ B . 4. If A and B are regular languages, so is AB . 5. If A is a regular language, so is A ∗ . 6. Nothing else is a regular language. Examples For example, let V = { a, b, c } . Are { aab } and { cc } RL? Yes, by 2. since aab and cc are members of V ∗ . { aab, cc } ? Yes, by 3, union. { aabcc } ? Contents First Last Prev Next ◭

Yes, by 4, concatenation. Likewise, by 5 { aab } ∗ { cc } ∗ are regular languages. Contents First Last Prev Next ◭

3. Today FSA don’t have memory: (they cannot recognize/generate a n b n ) Contents First Last Prev Next ◭

4. Syntax ◮ Syntax : “setting out things together”, in our case things are words. The main question addressed here is “ How do words compose together to form a grammatical sentence ( s ) (or fragments of it)? ” ◮ Constituents : Groups of categories may form a single unit or phrase called constituent. The main phrases are noun phrases ( np ), verb phrases ( vp ), prepo- sitional phrases ( pp ). Noun phrases for instance are: “she”; “Michael”; “Rajeev Gor´ e”; “the house”; “a young two-year child”. Tests like substitution help decide whether words form constituents. Another possible test is coordination. Contents First Last Prev Next ◭

5. Dependency Dependency : Categories are interdependent, for example Ryanair services [Pescara] np Ryanair flies [to Pescara] pp *Ryanair services [to Pescara] pp *Ryanair flies [Pescara] np the verbs services and flies determine which category can/must be juxtaposed. If their constraints are not satisfied the structure is ungrammatical. Contents First Last Prev Next ◭

6. Long-distance Dependencies Interdependent constituents need not be juxtaposed, but may form long-distance dependencies, manifested by gaps ◮ What cities does Ryanair service [ . . . ]? The constituent what cities depends on the verb service, but it is at the front of the sentence rather than at the object position. Such distance can be large, ◮ Which flight do you want me to book [ . . . ]? ◮ Which flight do you want me to have the travel agent book [ . . . ]? ◮ Which flight do you want me to have the travel agent nearby my office book [ . . . ]? Contents First Last Prev Next ◭