Complexity and Character of Human Languages The Faculty of Language - PowerPoint PPT Presentation

Human Language Complexity Human Language Complexity Linear Indexed Grammars Linear Indexed Grammars Complexity vs. Difficulty Complexity vs. Difficulty 1 Human Language Complexity Chomsky Hierarchy Complexity and Character of Human Languages The Faculty of Language Informatics 2A: Lecture 28 Strong and Weak Adequacy Excluding Complexity Classes Bonnie Webber 2 Linear Indexed Grammars School of Informatics University of Edinburgh 3 Complexity vs. Difficulty bonnie@inf.ed.ac.uk 27 November 2009 Reading: J&M. Chapter 16.3–16.4; Hauser, Chomsky, and Fitch (2002) Informatics 2A: Lecture 28 Complexity and Character of Human Languages 1 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 2 Chomsky Hierarchy Chomsky Hierarchy Human Language Complexity Human Language Complexity The Faculty of Language The Faculty of Language Linear Indexed Grammars Linear Indexed Grammars Strong and Weak Adequacy Strong and Weak Adequacy Complexity vs. Difficulty Complexity vs. Difficulty Excluding Complexity Classes Excluding Complexity Classes Review Obvious Questions At the top of the Chomsky Hierarchy are: recursive languages, where a Turing machine (TM) can halt on deciding whether a string is or isn’t in the language; Where do human languages fit within this complexity hierarchy? recursively enumerable languages, where a TM can only halt Do all human languages fit in the same place? Or are some with a decision as to whether a string is in the language. languages more complex than others? non-recursively enumerable languages that exceed the power Does the sense that one language is harder than another of even a TM to decide membership of an arbitrary string. correspond to a difference in their complexity? non recursively enumerable languages What features of human languages seem to make them harder or recursively enumerable languages easier? recursive languages context-sensitive languages context-free languages regular languages Informatics 2A: Lecture 28 Complexity and Character of Human Languages 3 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 4

Chomsky Hierarchy Chomsky Hierarchy Human Language Complexity Human Language Complexity The Faculty of Language The Faculty of Language Linear Indexed Grammars Linear Indexed Grammars Strong and Weak Adequacy Strong and Weak Adequacy Complexity vs. Difficulty Complexity vs. Difficulty Excluding Complexity Classes Excluding Complexity Classes The Faculty of Language The Faculty of Language According to The Faculty of Language (Hauser, Chomsky, and Narrowly, the language faculty is an abstract computational Fitch 2002), “language faculty” has a broad sense and a narrow system, one part of which (narrow syntax) generates sense. Broadly, it includes: representations internal to the mind/brain and maps them to: sensory-motor system for producing and perceiving linguistic communication; sensory-motor interface through the phonological or gestural spoken language: vocal track, auditory system system; sign language: gestural system, visual system conceptual-intentional system through the semantic (and written language: writing system, visual or tactile system pragmatic) systems. conceptual-intentional system, which establishes who to communicate with and what to communicate about, and Questions about the formal complexity of human language in involves general or a particular human language are about the computational power of syntax, as represented by a grammar generating mental states and attributing them to others; acquiring conceptual representations that are non-linguistic; that’s adequate for it. referring to entities and events. Informatics 2A: Lecture 28 Complexity and Character of Human Languages 5 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 6 Chomsky Hierarchy Chomsky Hierarchy Human Language Complexity Human Language Complexity The Faculty of Language The Faculty of Language Linear Indexed Grammars Linear Indexed Grammars Strong and Weak Adequacy Strong and Weak Adequacy Complexity vs. Difficulty Complexity vs. Difficulty Excluding Complexity Classes Excluding Complexity Classes Strong and Weak Adequacy Excluding Complexity Classes When we ask whether a grammar of a particular complexity is It seems unlikely that natural language belongs to one of the adequate for a NL, we distinguish two senses of adequacy: complexity classes at the top of the Chomsky hierarchy. A grammar is strongly adequate for a language if it: Why? If one believes that a child acquires a grammar when s/he generates all and only the strings of the language; learns a language (or several, if s/he is brought up multi-lingual), assigns them the “right” structures — ones that support a then the more complex the language class, the more difficult it correct representation of meaning. would be to acquire the grammar. A grammar is weakly adequate if it generates all and only the We are looking for an upper bound to the formal complexity of NL. strings of a language but assigns them “wrong” structures. But we also want to account for the fact that people seem to Adequacy(strong/weak) relates a language and a grammar. process language in near-linear time: Parsing even CFGs is of order n 3 for sentences of length n. Equivalency(strong/weak) relates two grammars. Informatics 2A: Lecture 28 Complexity and Character of Human Languages 7 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 8

Chomsky Hierarchy Human Language Complexity Human Language Complexity The Faculty of Language Linear Indexed Grammars Linear Indexed Grammars Strong and Weak Adequacy Complexity vs. Difficulty Complexity vs. Difficulty Excluding Complexity Classes Context-Free Languages Linear Indexed Grammars We saw in Lecture 25 that a sub-language of Swiss German and a A linear indexed grammar (LIG) is more powerful than a CFG, but sub-language of Dutch are both isomorphic to wa n b m xc n d m y , much less powerful than an arbitrary CSG – one of the “mildly CS” which is not context-free (Shieber, 1985). languages. omdat Wim 1 Jan 2 Henk 3 de kinderen 4 zag 1 helpen 2 leren 3 zwemmen 4 . . . because Wim 1 saw 1 Jan 2 help 2 Henk 3 teach 3 the children 4 to swim 4 context-sensitive languages “mildly context-sensitive” languages The thing to notice is these strings involve crossing dependencies : indexed languages linear-indexed languages | zag | depends on | Wim | , and | helpen | depends on | Jan | , etc. context-free languages So we need a new class of grammar to describe these constructions regular languages in Swiss German, Dutch, and every other language in which they occur. Informatics 2A: Lecture 28 Complexity and Character of Human Languages 9 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 10 Human Language Complexity Human Language Complexity Linear Indexed Grammars Linear Indexed Grammars Complexity vs. Difficulty Complexity vs. Difficulty Indexed Grammars Derivation in an Indexed Grammar An indexed grammar has three disjoint sets of symbols: terminals, S → D f g = � A → Aa | B → Bb | C → Cc � non-terminals and indices. D → D g f = � A → a | B → b | C → c � D → ABC An index is a stack of symbols that can be passed from the LHS of a rule to its RHS, allowing counting and recording what rules were S applied in what order. S → D f pushes an f onto the index on D D → D g pushes a g onto the index on D D → ABC passes the index on D to A, B and C g = � A → Aa | B → Bb | C → Cc � pops g from an index f = � A → a | B → b | C → c � pops f from an index Informatics 2A: Lecture 28 Complexity and Character of Human Languages 11 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 12

Human Language Complexity Human Language Complexity Linear Indexed Grammars Linear Indexed Grammars Complexity vs. Difficulty Complexity vs. Difficulty Derivation in an Indexed Grammar Derivation in an Indexed Grammar S → D f g = � A → Aa | B → Bb | C → Cc � S → D f g = � A → Aa | B → Bb | C → Cc � D → D g f = � A → a | B → b | C → c � D → D g f = � A → a | B → b | C → c � D → ABC D → ABC S S D f D f D gf Informatics 2A: Lecture 28 Complexity and Character of Human Languages 13 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 14 Human Language Complexity Human Language Complexity Linear Indexed Grammars Linear Indexed Grammars Complexity vs. Difficulty Complexity vs. Difficulty Derivation in an Indexed Grammar Derivation in an Indexed Grammar S → D f g = � A → Aa | B → Bb | C → Cc � S → D f g = � A → Aa | B → Bb | C → Cc � D → D g f = � A → a | B → b | C → c � D → D g f = � A → a | B → b | C → c � D → ABC D → ABC S S D f D f D gf D gf A gf B gf C gf A gf B gf C gf A f a Informatics 2A: Lecture 28 Complexity and Character of Human Languages 15 Informatics 2A: Lecture 28 Complexity and Character of Human Languages 16