Tracking lexical garden-path resolution with MEG: Phonological - - PowerPoint PPT Presentation

Laura Gwilliams | NYU | Sept 2nd

Tracking lexical garden-path resolution with MEG: Phonological commitment and sensitivity to subphonemic detail are independent

Laura Gwilliams, Tal Linzen, Kyriaki Neophytou, David Poeppel & Alec Marantz

Amlap | Bilbao | September 2nd 2016

1

Laura Gwilliams | NYU | Sept 2nd

Challenge of Speech Comprehension

• Speech is an inherently noisy and ambiguous signal
• To fluently derive meaning, listeners must integrate top-

down contextual information to guide their interpretation

• Top-down input occurring after an acoustic signal can be

integrated to affect the perception of earlier sounds

(Connine et al., 1991; Samuel, 1981; Szostak & Pitt, 2013; Warren & Sherman, 1974)

2

Laura Gwilliams | NYU | Sept 2nd

0.00 0.25 0.50 0.75 1.00 1 2 3 4 5 6 7 8 9 10 11

Continuum % /p/ responses

Categorical Perception

Challenge of Speech Comprehension

3

? a r ə k ? a r ə k i: t eɪ d p b

“Point of Disambiguation” (POD) (this is a parakeet) (this is a barricade)

Laura Gwilliams | NYU | Sept 2nd

Today’s Questions

• How does the auditory system respond to

phonological ambiguity?

• What mechanism(s) uphold subsequent context

integration to resolve bottom-up ambiguity?

• Is there a time-limit on how late subsequent

context can be received?

4

Laura Gwilliams | NYU | Sept 2nd

Technique: MEG

5

Laura Gwilliams | NYU | Sept 2nd

/p/ /b/

0.00 0.25 0.50 0.75 1.00 25 50 75 100

Continuum Selection Proportion Ambiguity T

Selection ~ Continuum

Design & Materials

6

? a r ə k a r ə k i: t eɪ d 65 x

(see also McMurray et al., 2009)

Laura Gwilliams | NYU | Sept 2nd

Experiment 1: Design & Materials

7

50 100 25 75 /pa/ /ba/

• VOT (31 pairs) {p-b, t-d, k-g} and PoA (22 pairs) {t-k, p-t}

Laura Gwilliams | NYU | Sept 2nd

parakeet 50 100 25 75 barakeet

Experiment 2: Design & Materials

8

50 100 25 75 /pa/ /ba/

Laura Gwilliams | NYU | Sept 2nd

Experiment 2: Design & Materials

9

parakeet 100 50 50 100 25 75 75 25 barricade parricade barakeet

• Point of Disambiguation (PoD) ranged 3-8 phonemes / 150-750 ms

Laura Gwilliams | NYU | Sept 2nd

Procedure & Analysis

10

parakeet 100 50 50 100 25 75 75 25 barricade parricade barakeet

• Point of Disambiguation (PoD) ranged 3-8 phonemes / 150-750 ms

Data Collection & Preprocessing

• 25 native English participants in each experiment
• Low pass filter online at 200 Hz
• MNE-Python used for preprocessing and source localisation

Analysis

• Multiple regression spatio-temporal cluster test (Eelbrain)
• (for more information see Gwilliams, Lewis & Marantz, 2016 - NeuroImage)
• Corrected for multiple comparisons (Maris & Oostenveld, 2007)

Laura Gwilliams | NYU | Sept 2nd

Today’s Questions

How does the auditory system respond to phonological ambiguity?

11

Laura Gwilliams | NYU | Sept 2nd

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

Ambiguity at Onset

12

Experiment 1: Syllables Experiment 2: Words

• Time-window: 0-200 ms after syllable/word onset
• Region: Heschl’s Gyrus and superior temporal gyrus bilaterally

Laura Gwilliams | NYU | Sept 2nd

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

Ambiguity at Onset

13

0.0 0.2 0.4 0.6 25 50 75 100

Continuum dSPM

0.0 0.2 0.4 25 50 75 100

Continuum dSPM

PoA: 40:80 ms VOT: 40:80 ms

0.0 0.2 0.4 0.6 25 50 75 100

Continuum dSPM

PoA, Word Onset: 40−60 ms 0.0 0.2 0.4 0.6 25 50 75 100

Continuum dSPM

VOT, Word Onset: 80−100 ms

PoA: 40:60 ms VOT: 80:100 ms

Experiment 1: Syllables Experiment 2: Words

Laura Gwilliams | NYU | Sept 2nd

Ambiguity at Onset

14

−0.3 −0.2 −0.1 0.0 Bilabial Labio− dental Velar

dSPM

PoA, Word Onset: 100−130 ms −0.4 −0.3 −0.2 −0.1 0.0 Voiced Voiceless

dSPM

VOT, Word Onset: 100−130 ms −1.0 −0.5 0.0 Bilabial Labio− dental Velar

dSPM

PoA, Syllable Onset: 100−130 ms −0.9 −0.6 −0.3 0.0 Voiced Voiceless

dSPM

VOT, Syllable Onset: 100−130 ms

PoA: 100:130 ms VOT: 100:130 ms PoA: 100:130 ms VOT: 100:130 ms

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

Experiment 1: Syllables Experiment 2: Words

Laura Gwilliams | NYU | Sept 2nd

Today’s Questions

What mechanism(s) uphold subsequent context integration to resolve bottom-up ambiguity?

15

Laura Gwilliams | NYU | Sept 2nd

Ambiguity at POD

16

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

Left Heschls Gyrus: POD Onset

• Time-window: 0-200 ms after POD onset
• Region: Heschl’s Gyrus and superior temporal gyrus bilaterally

Laura Gwilliams | NYU | Sept 2nd

Ambiguity at POD

17

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

Left Heschls Gyrus: POD Onset

PoA VOT

−0.15 −0.10 −0.05 0.00 50 100 50 100

Continuum dSPM

Late Disambiguation (> 450ms)

−1.00 −0.75 −0.50 −0.25 0.00 25 50 75 100

PoA, POD Onset: Heschl's Gyrus: 50−100 ms −0.9 −0.6 −0.3 0.0 25 50 75 100

VOT, POD Onset: Heschl's Gyrus: 50−100 ms

PoA: 40:80 ms VOT: 40:80 ms

Laura Gwilliams | NYU | Sept 2nd

Ambiguity at POD

18

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

Left Heschls Gyrus: POD Onset

Onset phoneme is “heard again” at disambiguation

−0.15 −0.10 −0.05 0.00 Bilabial Labio− dental Velar

dSPM

PoA, Syllable Onset: 100−130 ms −0.15 −0.10 −0.05 0.00 Voiced Voiceless

dSPM

VOT, Syllable Onset: 100−130 ms

PoA: 40:80 ms VOT: 40:80 ms

−1.00 −0.75 −0.50 −0.25 0.00 25 50 75 100

PoA, POD Onset: Heschl's Gyrus: 50−100 ms −0.9 −0.6 −0.3 0.0 25 50 75 100

VOT, POD Onset: Heschl's Gyrus: 50−100 ms

PoA: 40:80 ms VOT: 40:80 ms

Laura Gwilliams | NYU | Sept 2nd

Today’s Questions

Is there a time-limit on how late subsequent context can be received?

19

Laura Gwilliams | NYU | Sept 2nd

Commitment Before POD

20

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

pan paranoid past parakeet barricade barrister band back ballet

/pa/ cohort /ba/ cohort

paddle

(25% /b/) (75% /p/)

Laura Gwilliams | NYU | Sept 2nd

Commitment Before POD

21

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

• Time-window: 0-400 ms

after POD onset

• Region: Heschl’s Gyrus,

superior and middle temporal gyrus bilaterally

pan paranoid past parakeet

/pa/ cohort

paddle

(75% /p/)

Laura Gwilliams | NYU | Sept 2nd

0.850 0.875 0.900 0.925 0.950 0.975 Early Word Early Non−Word Late Word Late Non−Word

dSPM

Lexicality*Latency

**

200:230 ms

Commitment Before POD

22

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/

pan paranoid past parakeet

/pa/ cohort

paddle

(75% /p/)

Laura Gwilliams | NYU | Sept 2nd

Conclusion

• Sensitivity to phoneme ambiguity ~50 ms

after onset in primary auditory cortex

• Subphonemic detail is maintained over long

time-scales (+700 ms) and re-evoked at point of disambiguation

• Phonological commitment resolves ~450 ms

after phoneme onset in superior temporal gyrus

23

**

Laura Gwilliams | NYU | Sept 2nd

Take Home Message

24

p b a r ə k i: t p b p

commit to most likely phonological interpretation ~450 ms maintain subphonemic detail across long time-scales

With big thanks to:

• My supervisors, Alec Marantz & David Poeppel
• Everyone in the Neuroscience of Language Lab and Poeppel Lab

Funding: G1001 Abu Dhabi Institute

laura.gwilliams@nyu.edu @GwilliamsL

Eskerrik Asko!

laura.gwilliams@nyu.edu @GwilliamsL

Laura Gwilliams | NYU | Sept 2nd

Discussion

27

• 50 ms is very early — what information is present in the input that

could determine phonological category?

pb td kg pt tk

0e+00 1e+10 2e+10 3e+10 4e+10 0e+00 5e+09 1e+10 0.0e+00 2.5e+09 5.0e+09 7.5e+09 1.0e+10 0.0e+00 2.5e+09 5.0e+09 7.5e+09 0e+00 2e+09 4e+09 6e+09 2000 4000 6000 2000 4000 6000 2000 4000 6000 2000 4000 6000 2000 4000 6000

Frequency (Hz) Power

Contin 6000 Continuum 25 50 75 100

p - b t - d k - g p - t t - k

20 ms

Laura Gwilliams | NYU | Sept 2nd

Experiment 1: Design & Materials

30

• Re-sampled the continuum to match perceptual categorisation

0.00 0.25 0.50 0.75 1.00 25 50 75 100

Continuum Selection Proportion Ambiguity Type

place VOT

Selection ~ Continuum

0.00 0.25 0.50 0.75 1.00 1 2 3 4 5 6 7 8 9 10 11

Continuum Selection Proportion

Selection ~ Continuum

Laura Gwilliams | NYU | Sept 2nd

No Ambiguity Effect in Right Hemisphere

31

Time (ms) dSPM

Right Heschls Gyrus: Syllable Onset

Time (ms) dSPM

Right Heschls Gyrus: Word Onset

/p/ /a/ /r/ /ə/ /k/ /i:/ /t/