Mandarin and English Zhihua Xia, Rivka Levitan, Julia Hirschberg - PowerPoint PPT Presentation

Prosodic Entrainment in Mandarin and English Zhihua Xia, Rivka Levitan, Julia Hirschberg

Entrainment 2

Entrainment 3

Entrainment 4

Entrainment 5

Entrainment 6

Entrainment 7

Motivation • Prevalence of entrainment – Lexical, syntactic, stylistic – Phonetic – Intensity, pitch, speaking rate • Liking, positive social characteristics – Liking, smoother interaction – Task score – Positive interactions between married couples • Lack of cross-linguistic comparisons 8

Columbia Games Corpus 9

Columbia Games Corpus • 12 spontaneous task-oriented sessions • 13 participants – 11 participated in two sessions with different partners. • 9h 8m of recorded dialogue • Orthographically transcribed, word-aligned, intonationally transcribed, annotated for turn-taking behaviors and other phenomena • Acoustic features automatically extracted using Praat 10

Tongji Games Corpus • 115 spontaneous task-oriented sessions • 70 pairs of speakers (40 female, 30 male) • University students with a National Mandarin Test Certificate level 2, grade A or above • Elicited using two games: Picture Ordering (role imbalance), Picture Classifying (cooperative) • 12 hours of recorded dialogue 11

Units of analysis • Inter-pausal unit (IPU) Pause-free segment of speech from a single speaker. speech <silence> speech <silence> speech • Turn Sequence of speech from one speaker without intervening speech from the other speaker. • Session Complete interaction between two subjects. 12

Units of analysis • Inter-pausal unit (IPU) Pause-free segment of speech from a single speaker. speech <silence> speech <silence> speech IPU IPU IPU • Turn Sequence of speech from one speaker without intervening speech from the other speaker. • Session Complete interaction between two subjects. 13

Features • • Intensity mean F0 max • • Intensity max F0 min • • Intensity min Speaking rate • F0 mean 14

Acoustic-prosodic entrainment • Global or local? • Exact or relative? • Convergent or constant? 15

Acoustic-prosodic entrainment • Global or local? • Exact or relative? • Convergent or constant? 16

Acoustic-prosodic entrainment • Global or local? • Exact or relative? • Convergent or constant? 17

Global entrainment 18

Global entrainment 19

Global entrainment 20

Global entrainment 21

Global entrainment • Partner difference 22

Global entrainment • Other difference 23

Global entrainment • Self difference 24

Global entrainment • Partner difference 25

Local entrainment IPU IPU t i+1 t i IPU t j IPU t k IPU t l 26

Local entrainment IPU IPU t i+1 t i IPU IPU t i+2 t i+1 IPU IPU t i+3 t i+2 IPU IPU t i+4 t i+3 27

Three aspects of entrainment: • Proximity : similarity between interlocutors, relative to baseline. • Synchrony : turn by turn relative similarity • Convergence : increase in similarity over time 28

Acoustic-prosodic entrainment Chinese • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ✓ Intensity mean 0.63 ✓ ✓ Intensity max 0.55 (0.47) Intensity min 0.31 -- 0.09 F0 mean 0.66 (0.28) 0.22 ✓ F0 max 0.61 (0.18) 0.24 F0 min 0.63 -- 0.19 ✓ ✓ Speaking rate (0.15) 29

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 30

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 31

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 32

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 33

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 34

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 35

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 36

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 37

Acoustic-prosodic entrainment Chinese (SAE) • Global or local? • Exact or relative? • Convergent or constant? Feature Global Local Synchrony Global Local proximity proximity convergence convergence ✓ ( ✓ ) ✓ ( ✓ ) 0.63 ( ✓ ) Intensity mean (0.50) (0.03) ✓ ( ✓ ) ✓ ( ✓ ) 0.55 Intensity max (0.47) (0.02) Intensity min -- -- 0.31 -- -- 0.09 -- ( ✓ ) 0.66 ( ✓ ) F0 mean (0.28) 0.22 (0.06) ✓ ( ✓ ) ( ✓ ) 0.61 F0 max (0.18) 0.24 (0.05) F0 min -- -- 0.63 -- -- 0.19 -- ✓ ( ✓ ) ✓ ( ✓ ) Speaking rate (0.15) (0.03) 38