Quantifying prosodic boundary strength using functional data - - PowerPoint PPT Presentation

Quantifying prosodic boundary strength using functional data analysis of articulatory movement

Benjamin Parrell, Sungbok Lee, & Dani Byrd Department of Linguistics, University of Southern California

Work supported by NIH 2nd Pan-American/Iberian Meeting on Acoustics November 2010, Cancun, Mexico

Effects of Prosodic Boundaries

• Prosodic boundaries have been shown to have

effects on both the duration and magnitude of articulatory gestures

▫ boundary adjacent lengthening: gestures near a boundary are longer (e.g. Byrd et al. 2000, Byrd et al. 2006) ▫ “initial strengthening”: gestures near a boundary have a larger magnitude (e.g. Fougeron & Keating 1997, Cho 2005)

2 Parrell, Lee, & Byrd, ASA 2010, Cancun

• These effects are gradient

▫ Gestures at higher level boundaries (e.g. a phrase boundary) show more slowing and greater magnitude than those at lower level boundaries (e.g. a word boundary)

• Is it possible to measure the degree of these

effects as a measure of boundary strength?

▫ Could give insight into how many boundaries a language/speaker of a language has, and how they are different from one another

3 Parrell, Lee, & Byrd, ASA 2010, Cancun

Past Analyses of Boundary Effects

• Quantifying differences in strength of prosodic

boundary is difficult

▫ what to measure? ▫ where to define boundary?

• Rely on drastic data reduction

▫ measurements at only a few points in time

4 Parrell, Lee, & Byrd, ASA 2010, Cancun

5 Parrell, Lee, & Byrd, ASA 2010, Cancun

acoustic articulatory

tongue ¡(p ¡

[t a t a # t a t a ] [t a t a # t a t a ]

Standard Approach

Functional Data Analysis (FDA)

• FDA analyzes entire, continuous trajectories and

captures non-linear warping in time and space

(Ramsey et al. 2005)

• FDA has been used on time functions of speech

articulators to examine both gestural coordination and boundary effects

▫ Reveals slowing of articulator movement in the presence of a phrase boundary as the speech stream approaches and recedes from the phrase edge (Koening et al 2008, Lee et al. 2006)

6 Parrell, Lee, & Byrd, ASA 2010, Cancun

• Recent work has shown that using FDA on

synthetic articulatory data can both distinguish and quantify different types of prosodic boundaries

(Parrell et al. 2010) ▫ Prosodic boundaries were instantiated using π-gesture model of Byrd & Saltzman 2000.

• FDA alignment was conducted on spatial

position measurements of the synthesized trajectories

7 Parrell, Lee, & Byrd, ASA 2010, Cancun

Purpose of the study

• Goal: To determine the ability of FDA analysis

to capture boundary strength differences

• Explore how choice of kinematic variable—e.g.

position vs. velocity—affects FDA measurement

• Can FDA capture differences in boundary

strength in non-synthetic articulatory speech data?

8 Parrell, Lee, & Byrd, ASA 2010, Cancun

Materials

• Stimuli (taken from Byrd & Saltzman 1998):

▫ Target sequence: [mamə (#) mimi] ▫ Four prosodic conditions:

Boundary Sentence none Poppa-Pikt and Momma-Mimi tapped Cody. list Poppa, Pikt, Momma, Mimi, and Bibi tapped Cody. vocative Quick Momma, Mimi tapped Cody. utterance Poppa picked Momma. Mimi tapped Cody.

 4 new subjects  Lip Aperture measured using electromagnetic articulometry (EMA)  Position and velocity curves used for FDA

9 Parrell, Lee, & Byrd, ASA 2010, Cancun

FDA Method

Raw trajectories of all repetitions of each boundary condition

duration: max. closure for [m] in [mama] to max. closure for m in [mimi] example: vocative condition for subject TC

10 Parrell, Lee, & Byrd, ASA 2010, Cancun

First, signals are linearly time-normalized to 300 equally-spaced samples

11 Parrell, Lee, & Byrd, ASA 2010, Cancun

12 Parrell, Lee, & Byrd, ASA 2010, Cancun

[m a m əӚ m i m]

Then, time-normalized trajectories are aligned to kinematic landmarks in a control signal using FDA non-linear time warping

control = mean of time-normalized no-boundary tokens

The time-deformation function: local advancing or local slowing of the system time of a test signal with respect to clock time of the control signal

13 Parrell, Lee, & Byrd, ASA 2010, Cancun

FDA Deformation Function

The area under the curve of the time-deformation function calculated as a measure of the strength

• f the boundary (Deformation Index)

14 Parrell, Lee, & Byrd, ASA 2010, Cancun

Deformation Index

Deformation Index= area under curve

• Deformation Index distinguishes between at least

utterance and no-boundary for all subjects

• Other differences among boundaries were also

found, with different patterns across subjects

• mirrors results in previous work of Byrd & Saltzman 1998
• FDA results from velocity curves match those for

position curves

Results

15 Parrell, Lee, & Byrd, ASA 2010, Cancun

16 Parrell, Lee, & Byrd, ASA 2010, Cancun

Position Velocity

F(3,36) = 61.6, p < 0.0001 none < list < voc., utt. (p < 0.05) F(3,36) = 114.8, p < 0.0001 none < list < voc., utt. (p < 0.05)

Subject TA Deformation Index Results

17 Parrell, Lee, & Byrd, ASA 2010, Cancun

Position Velocity

F(3,34) = 71.4, p < 0.0001 none, list < voc. < utt. (p < 0.05) F(3,34) = 174.0, p < 0.0001 none < list < voc. < utt. (p < 0.05)

Subject TB

18 Parrell, Lee, & Byrd, ASA 2010, Cancun

Position Velocity

F(3,36) = 54.4, p < 0.0001 none, list, voc. < utt. (p < 0.05) F(3,36) = 42.5, p < 0.0001 none, list, voc. < utt. (p < 0.05)

Subject TC

19 Parrell, Lee, & Byrd, ASA 2010, Cancun

Position Velocity

F(3,36) = 5.9, p < 0.005 none < list, utt. (p < 0.05) F(3,36) = 2.9, p < 0.05 none < utt. (p < 0.05)

Subject TD

Conclusions

• Deformation Index, a measurement derived

from applying FDA to kinematic articulatory data, can detect and quantify differences in the strength of prosodic boundaries in experimental articulatory data

• Using either position or velocity curves gives

similar results

▫ Though there are a few slight differences for some subjects

20 Parrell, Lee, & Byrd, ASA 2010, Cancun

Conclusions

21 Parrell, Lee, & Byrd, ASA 2010, Cancun

• Deformation

Index results compare well with piecewise analysis

• f similar

sentences with a different set of subjects in Byrd & Saltzman 1998

Advantages of Deformation Index

• Captures effects of boundary on entire kinematic

trajectory, not just a few points in time

• Can capture effects that occur in non-boundary-

adjacent syllables

▫ Given correct stimuli design

• Provides a simple quantification of boundary

strength

▫ past work has relied on a mutiplicity of derived variables in a single dataset to provide a holistic picture of boundary-adjacent lengthening

22 Parrell, Lee, & Byrd, ASA 2010, Cancun

References

Byrd, D. and E. Saltzman. (1998) Intragestural dynamics of multiple prosodic boundaries. Journal of Phonetics 26, 173-199. Byrd, D., Kaun, A., Narayanan, S., & Saltzman, E. (2000). Phrasal signatures in articulation In M. B. Broe & J. B. Pierrehumbert (Eds.), Papers in laboratory phonology V. (pp. 70-87). Cambridge: Cambridge University Press. Byrd, D. and E. Saltzman. (2003) The elastic phrase: Modeling the dynamics of boundary-adjacent lengthening. Journal of Phonetics 31(2), 149-180. Byrd, D., J. Krivokapic, and S. Lee. (2006) How far, how long: On the temporal scope of phrase boundary effects. Journal of the Acoustical Society of America, 120, 1589-1599. Cho, T. (2005). Prosodic strengthening and featural enhancement: Evidence from acoustic and articulatory realizations of /a,i/ in English Journal of the Acoustical Society of America, 117(6), 3867-3878.

23 Parrell, Lee, & Byrd, ASA 2010, Cancun

Fougeron, C., & P.A. Keating. (1997) Articulatory strengthening at edges of prosodic domains. The Journal of the Acoustical Society of America, 101(6), 3728-40. Koenig, L.L., J.C. Lucero, and E. Perlman (2008) Speech production variability in fricatives of children and adults: Results of functional data analysis. Journal of the Acoustical Society of America 124, 3158. Parrell, B., S. Lee & D. Byrd (2010). Evaluation of juncture strength using articulatory synthesis of prosodic gestures and Functional Data Analysis. Paper presented at Speech Prosody V, Chicago, Illinois, May 2010. Lee, S., D. Byrd, and J. Krivokapic. (2006) Functional data analysis of prosodic effects on articulatory timing. Journal of the Acoustical Society of America 119, 1666-1671. Ramsay, J.O., and B.W. Silverman. (2005) Functional Data Analysis, 2nd ed. Springer-Verlag, New York.

24 Parrell, Lee, & Byrd, ASA 2010, Cancun