O e Overview & Aim e & Investigating common properties of - - PowerPoint PPT Presentation

Overview & Aim O e e &

Investigating common properties of across-channel i i ti d h ti processing in time, space and speech perception

Shuji Mori

K h U i it Kyushu University

JSPS Kiban A Grant Kickoff Workshop August 2, 2013

Across-channel processing Across channel processing

Channel – Independent information pathway

200Hz 400Hz 800Hz 1600Hz 25Hz 50Hz 100Hz 200Hz

Auditory processing across channels

‘frequency’ channels

Auditory processing across channels

– Time: between-frequency gap detection S b t d t ti – Space: between-ear gap detection – Speech: voiced-voiceless categorical boundary

Psychophysics Lab. Department of Informatics Faculty of ISEE

1

Between-frequency gap detection Between frequency gap detection

Gap detection

Measurement of auditory temporal resolution (Moore & Glasberg, 1988; Plomp, 1964, etc.) Traditional, ‘within-frequency’ task

y equency

No gap 5 ms

Fre Time

Gap thresholds below 5 msec (Scharf & Buus, 1986)

Psychophysics Lab. Department of Informatics Faculty of ISEE

2

Between-frequency gap detection Between frequency gap detection

Between-frequency task

10

ency

No gap 5 80 10 ms

Frequ

5 ms 80 ms

Time

Psychophysics Lab. Department of Informatics Faculty of ISEE

3

(Phillips et al., 1997, JASA, pp.3697, Fig.2)

Between-ear gap detection Between ear gap detection

de

R

Ear/Sid

L R

( ) Time Leading Trailing marker Dependence on leading marker duration (Phillips et al., 1997, 1998) Headphones Speakers

shold (ms) Gap thres

(Adapted from Phillips et al., (Adapted from Phillips et al.,

Psychophysics Lab. Department of Informatics Faculty of ISEE

4

Leading marker duration (ms)

p , 1997, JASA, pp.3702, Fig.5) p , 1998, JASA, pp.2066, Fig.2)

Between-ear gap detection Between ear gap detection

Similarity to between-frequency thresholds (Phillips et al., 1997) 1997)

(Adapted from Phillips et al

Headphones Speakers

Leading marker duration (ms)

Phillips et al., 1997, JASA, pp.3698, Fig.3)

shold (ms) Gap thres

(Adapted from Phillips et al., (Adapted from Phillips et al.,

Psychophysics Lab. Department of Informatics Faculty of ISEE

5

Leading marker duration (ms)

p , 1997, JASA, pp.3702, Fig.5) p , 1998, JASA, pp.2066, Fig.2)

Voiced-voiceless categorical boundary Voiced voiceless categorical boundary

Phoneme contrasts of stop consonants /b/ - /p/, /d/ - /t/, /g/ - /k/ Voice onset time (V0T) as perceptual cues (Liberman et l 1957 Li k & Ab 1964) al., 1957; Lisker & Abramson, 1964) /ba/

100

/ba/

75

/pa/ /ba/

percentage

Consonant

/pa/

25 50 ntification p

Time VOT Vowel Consonant

19 30 41 52 63 74 85 Iden Psychophysics Lab. Department of Informatics Faculty of ISEE

6

Time

VOT (msec)

(Based on Oyama, 2013, pp.11, Fig. 6)

Voiced-voiceless categorical boundary Voiced voiceless categorical boundary

Similarity to between-frequency thresholds (Phillips et al., 1997 Elango an & St art 2008) 1997; Elangovan & Stuart, 2008)

cy requenc Fr Time

Psychophysics Lab. Department of Informatics Faculty of ISEE

7

(Adapted from Elangovan & Stuart, 2008, Ear & Hearing, pp.769, Fig.6)

Research questions Research questions

• 1. Common properties in time, space, and speech

Does same processing mediate those three domains of auditory perception? What really is across-channel processing?

• 2. Brain mechanism

Where in auditory pathway does across-channel processing take place?

• 3. Possible operation in other domains

Vision Perception of geminate consonants

Psychophysics Lab. Department of Informatics Faculty of ISEE

8

JSPS Kiban A Grant JSPS Kiban A Grant

‘Investigating common properties of across-channel processing in time, space and speech perception’

 FY 2013 to 2017  FY 2013 to 2017  Shuji Mori (chief), Kaoru Sekiyama, Keiji Iramina, Nobuyuki Hirose Willy Wong Makiko Sadakata Nobuyuki Hirose, Willy Wong, Makiko Sadakata, Takako Mitsudo

Psychophysics Lab. Department of Informatics Faculty of ISEE

9

Common properties Common properties

Oyama (2013) Correlations of individual gap thresholds and /b/ - /p/ boundaries (cf. Elangovan & Stuart, 2008)

• Between-frequency
• Between-ear
• /ba/ /pa/ identification

ency

• /ba/-/pa/ identification

21 native Japanese listeners

Frequ

21 native Japanese listeners

Time

Psychophysics Lab. Department of Informatics Faculty of ISEE

10

Common properties Common properties

Between-frequency and -ear positively correlated

20 30 40 50 shold (msec) 20 30 40 50 shold (msec) 20 30 40 50 shold (msec) 10 20 10 20 30 40 50 Gap Thres Gap Threshold (msec) 2000Hz 10 20 10 20 30 40 50 Gap Thres Gap Threshold (msec) 1000Hz 10 20 10 20 30 40 50 Gap Thres Gap Threshold (msec) 4000Hz

/b/-/p/ boundary correlated with between-frequency, but not with between ear not with between-ear

30 40 ry (msec) 30 40 ry (msec) 10 20 10 20 30 40 50 VOT Boundar 2000Hz 10 20 10 20 30 40 50 VOT Boundar Between-ear

Psychophysics Lab. Department of Informatics Faculty of ISEE

11

Gap Threshold (msec) Gap Threshold (msec)

Across-channel processing Across channel processing

Relative timing (Phillips, 1999)

Monitoring offset of leading and onset of trailing marker Reflects central computation

Discontinuity detection

Performed peripherally Performed peripherally

Psychophysics Lab. Department of Informatics Faculty of ISEE

12

Across-channel processing Across channel processing

Distinction between two operations?

(Phillips et al., 1997, JASA, pp.3697, Fig.2)

Why takes longer for larger separation?

Psychophysics Lab. Department of Informatics Faculty of ISEE

13

Across-channel processing Across channel processing

Attention shift

Attention dwell time (Fitzgibbons et al., 1974) Imprecise time-stamping in unattended channel g (Phillips et al., 1997) Attention shifts fast to a cued frequency (Scharf et al., 2007) Attention disruption does NOT explain prolonged gap thresholds

Psychophysics Lab. Department of Informatics Faculty of ISEE

14

Psychometric function Psychometric function

Within-frequency

s 1.0

(Florentine et al., 1990)

tion of esponses 0.5

Between-frequency

Proport ‘Yes’ re 0.5

Between-frequency Attention disrupt. Prediction

1 10 100 Gap duration (msec)

Internal noise

Psychophysics Lab. Department of Informatics Faculty of ISEE

15

Brain mechanism Brain mechanism

MEG MEG ABR

(Coren et al., 1994, Sensation & perception, pp.204, Fig.6-17)

Psychophysics Lab. Department of Informatics Faculty of ISEE

16

Other domains Other domains

Perceptual channel

Frequency, ear/side, phoneme (vowels, consonants), …

Vision

Spatial frequency, color, orientation, … Temporal gap detection task (Yeshurun, 2004) p g p ( , )

Gap Time

Psychophysics Lab. Department of Informatics Faculty of ISEE

17

Other domains Other domains

Geminate consonants

/kk/, /tt/, /ss/, … Stop geminates contain silent intervals of over 100 msec g

y equency Fre Time Time

Largely cognitive (Sadakata & Sekiyama, 2011) but …

Psychophysics Lab. Department of Informatics Faculty of ISEE

18

Next … Next …

Psychophysical theory of speech perception

Speech is special (Liberman & Mattingly,1985; Fowler, 1986) Natural psychophysical boundary (Kuhl & Miller, 1978) VOT boundary identical for humans and nonhumans All phonetic contrasts are psychophysical in nature

Identifying psychophysical correlates of phonetic contrasts

Psychophysics Lab. Department of Informatics Faculty of ISEE

19

Ultimately … Ultimately …

Psychophysics of language

Psychophysics of Japanese reading (with K. Sekiyama,

• W. Teramoto)

Combining psychophysical studies of speech and reading

Psychophysics Lab. Department of Informatics Faculty of ISEE

20

Thank you for your attention y y

Psychophysics Lab. Department of Informatics Faculty of ISEE

21