Cognitive Complexity of Linguistic Patterns Artificial Grammar - PDF document

AGL Workshop 1 Cognitive Complexity of Linguistic Patterns Artificial Grammar Learning Workshop Max Planck Institute for Psycholinguistics 23–24 November 2010 Slide 1 James Rogers Dept. of Computer Science Earlham College jrogers@cs.earlham.edu http://cs.earlham.edu/~jrogers/slides/agl.ho.pdf This work completed, in part, while in residence at the Radcliffe Institute for Advanced Study Joint work with: • Geoffrey K. Pullum School of Philosophy, Psychology and Language Sciences University of Edinburgh • Marc D. Hauser Depts. of Psychology, Organismic & Evolutionary Biology and Biological Anthropology Slide 2 Harvard University • Jeffery Heinz Dept. of Linguistics and Cognitive Science University of Delaware • Gil Bailey, Matt Edlefsen, Magaret Fero, Molly Visscher, David Wellcome, Aaron Weeden and Sean Wibel Dept. of Computer Science Earlham College

AGL Workshop 2 Cognitive Complexity of Simple Patterns Sequences of ‘ A ’s and ‘ B ’s which end in ‘ B ’: S 0 − → AS 0 , S 0 − → BS 0 , S 0 − → B B A B ( A + B ) ∗ B A Slide 3 Sequences of ‘ A ’s and ‘ B ’s which contain an odd number of ‘ B ’s: S 0 − → AS 0 , S 0 − → BS 1 , S 0 − → B, S 1 − → AS 1 , S 1 − → BS 0 , S 1 − → A A A B ( A ∗ BA ∗ BA ∗ ) ∗ A ∗ BA ∗ B Finite State Automata and Regular Grammars A A B A B B A A A B A A State N Y Slide 4 A A B S 0 S 1 B S 0 − → AS 0 , S 0 − → BS 1 , S 0 − → B, S 1 − → AS 1 , S 1 − → BS 0 , S 1 − → A

AGL Workshop 3 Some More Simple Patterns Sequences of ‘ A ’s and ‘ B ’s which contain at least one ‘ B ’: S 0 − → AS 0 , S 0 − → BS 1 , S 0 − → B, S 1 − → AS 1 , S 1 − → BS 1 , S 1 − → A, S 1 − → B A, B A B A ∗ B ( A + B ) ∗ Slide 5 Sequences of ‘ A ’s and ‘ B ’s which contain exactly one ‘ B ’: S 0 − → AS 0 , S 0 − → BS 1 , S 0 − → B, S 1 − → AS 1 , S 1 − → A A, B A A B B A ∗ BA ∗ Cognitive Complexity from First Principles What kinds of distinctions does a cognitive mechanism need to be sensitive to (attend to) in order to classify an event with respect to a pattern? Slide 6 Reasoning about patterns • What objects/entities/things are we reasoning about? • What relationships between them are we reasoning with?

AGL Workshop 4 Some Assumptions about (Proto-)Linguistic Behaviors • Perceive/process/generate linear sequence of (sub)events Slide 7 • Can model as strings—linear sequence of abstract symbols – Positions—Discrete linear order (initial segment of N ). – Labeled with alphabet of events Partitioned into subsets, each the set of positions at which a particular event occurs. Dual characterizations of complexity classes Computational classes • Characterized by abstract computational mechanisms • Equivalence between mechanisms • Means to determine structural properties of stringsets Slide 8 Descriptive classes • Characterized by the nature of information about the properties of strings that determine membership • Independent of mechanisms for recognition • Subsume wide range of types of patterns

AGL Workshop 5 Local and Piecewise Hierarchies Reg MSO SF FO LTT Slide 9 LT PT Propositional SL SP Restricted Prop. Fin Local (+1) Piecewise ( < ) Stringset inference experiments { A, B } ∗ AABBB ∅ ∅ ABABAB AABBBA A m B n 2 | ( m + n ) A m B n A n B n AAABBB | w | A = | w | B I F Slide 10 A n B n n ≤ 3 AABBBB I = { A n B n | n ≥ 1 } F = { AAABBB } D =?

AGL Workshop 6 Formal Issues for AGL Experiments Design • Identifying relevant classes of patterns • Finding minimal pairs of stringsets • Finding sets of stimuli that distinguish those stringsets Slide 11 Interpretation • Identifying the class of patterns subject has generalized to • Inferring the properties of the cognitive mechanism involved – properties common to all mechanisms capable of identifying that class of patterns Inferences from AGL experiments Subject successfully generalizes a pattern in a given complexity class: • Mechanism is sensitive to features characteristic of class. • Does not imply that subject can generalize every pattern in that class. – Other processing factors may interfere. Slide 12 Subject consistently fails to generalize patterns in a given class: • Suggests mechanism is not sensitive to features characteristic of class. • Inability to generalize may be due to interfering factors. – Complexity of patterns properly in class may exceed other limitations of processing.

AGL Workshop 7 Assumptions • Inferred set is not arbitrary Slide 13 • Principle determining membership is structural • Inference exhibits some sort of minimality Yawelmani Yokuts (Kissberth’73) ⋆ CCC Σ ∗ CCC Σ ∗ V Σ ∗ C C C Slide 14 V C CC CCC V V CCV CV V CV CCV CV CV CCV CCV V V CV CCV CV V CV CCC ⋆ V CV CV CCV CCV V V CV

AGL Workshop 8 Adjacency—Substrings Definition 1 ( k -Factor) v is a factor of w if w = uvx for some u, v ∈ Σ ∗ . v is a k -factor of w if it is a factor of w and | v | = k .  { v ∈ Σ k | ( ∃ u, x ∈ Σ ∗ )[ w = uvx ] } if | w | ≥ k, F k ( w ) def  = { w } otherwise .  Slide 15 F k ( w ) is the set of length k substrings (contiguous) of w (or just w itself if length of w < k ). AB ABAB F 2 ( ABABAB ) = { AB, BA } F 7 ( ABABAB ) = { ABABAB } Strictly Local Stringsets—SL Strictly k -Local Definitions G ⊆ F k ( { ⋊ } · Σ ∗ · { ⋉ } ) = G def w | ⇐ ⇒ F k ( ⋊ · w · ⋉ ) ⊆ G L ( G ) def = { w | w | = G} A stringset L is Strictly k -Local iff membership depends Slide 16 solely on the k -factors that are permitted. G ( AB ) n = { ⋊ A, AB, BA, B ⋉ } * AB ABAB ABBAB ⋊ ⋉ ⋊ ⋉ Membership in an SL k stringset depends only on the individual k -factors which actually occur in the string.

AGL Workshop 9 Scanners a b a b a b a b a a b a b a b a b a · · · · · · b k k D Q G : · · · ∈ Slide 17 φ · · · a a · · · b b · · · a b · · · k Recognizing an SL k stringset requires only remembering the k most recently encountered symbols. Character of Strictly k -Local Sets Theorem (Suffix Substitution Closure): A stringset L is strictly k -local iff whenever there is a string x of length k − 1 and strings w , y , v , and z , such that k − 1 Slide 18 ���� w · x · y ∈ L · · ∈ L v x z then it will also be the case that · · ∈ L w x z

AGL Workshop 10 Examples of Suffix Substitution The dog · likes · the biscuit ∈ L Alice · likes · Bob ∈ L The dog · likes · Bob ∈ L Slide 19 But: The dog · likes · the biscuit ∈ L Bob, Alice · likes · ∈ L ε ⋆ The dog · likes · �∈ L ε SL Hierarchy Definition 2 ( SL ) A stringset is Strictly k -Local if it is definable with an SL k definition. A stringset is Strictly Local (in SL) if it is SL k for some k . Slide 20 Theorem 1 (SL-Hierarchy) SL 2 � SL 3 � · · · � SL i � SL i +1 � · · · � SL Every Finite stringset is SL k for some k : Fin ⊆ SL. There is no k for which SL k includes all Finite languages.

AGL Workshop 11 Alawa 0 ⋊ σ σ σσ ⋉ ´ σ ´ σ ⋊ ´ σ σ ⋉ 3 σ ⋆ ⋊ σ σ ⋉ Slide 21 1 ´ σ G Alawa = { ⋊ σσ, ⋊ σ ´ σ, ⋊ ´ σσ, σσσ, σσ ´ σ, σ ´ σσ, σ 4 ⋊ ´ σ ⋉ , ´ σσ ⋉ } σ 2 Some syllable must get primary stress k − 1 0 σ � �� � ⋊ σ 1 σ 0 · · · σ 0 σ 2 ⋉ ´ k − 1 Slide 22 � �� � ´ ⋊ ´ σ 0 · · · σ 0 σ σ 2 σ 1 ⋉ k − 1 σ , ´ σ � �� � σ 0 · · · σ 0 ⋆ ⋊ σ 1 σ 1 ⋉ 1

AGL Workshop 12 Cognitive interpretation of SL • Any cognitive mechanism that can distinguish member strings from non-members of an SL k stringset must be sensitive, at least, to the length k blocks of events that occur in the presentation of the string. Slide 23 • If the strings are presented as sequences of events in time, then this corresponds to being sensitive, at each point in the string, to the immediately prior sequence of k − 1 events. • Any cognitive mechanism that is sensitive only to the length k blocks of events in the presentation of a string will be able to recognize only SL k stringsets. Strictly Local Stress Patterns Heinz’s Stress Pattern Database (ca. 2007)—109 patterns 9 are SL 2 Abun West, Afrikans, . . . Cambodian,. . . Maranungku 44 are SL 3 Alawa, Arabic (Bani-Hassan),. . . Slide 24 24 are SL 4 Arabic (Cairene),. . . 3 are SL 5 Asheninca, Bhojpuri, Hindi (Fairbanks) 1 is SL 6 Icua Tupi 28 are not SL Amele, Bhojpuri (Shukla Tiwari), Ara- bic Classical, Hindi (Keldar), Yidin,. . . 72% are SL, all k ≤ 6. 49% are SL 3 .