[PPT] - Large-scale analysis of Spanish /s/-lenition using audiobooks Neville PowerPoint Presentation

SLIDE 1

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Large-scale analysis of Spanish /s/-lenition using audiobooks

Neville Ryant1 and Mark Liberman2

Linguistic Data Consortium, USA

1nryant@gmail.com, 2markyliberman@gmail.com

September 5, 2016

SLIDE 2

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

SLIDE 3

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

SLIDE 4

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

SLIDE 5

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

SLIDE 6

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

5

Future directions

SLIDE 7

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

5

Future directions

SLIDE 8

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Spanish /s/-lenition

Definition Spanish /s/-lenition is the weakening of /s/ in syllable-final position to one of the following variants: [s] [h] [z] (before a voiced stop) deletion

SLIDE 9

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Example: [s] (Venezuelan)

SLIDE 10

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Example: [h] (Chilean)

SLIDE 11

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Example: [z] (Mexican)

SLIDE 12

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Example: Deletion (Venezuelan)

SLIDE 13

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Prevalence

sC s#C s#{sil} s#V1 s#V0 Argentina 88 89 22 7 6 Chile 93 96 36 10 22 Cuba 97 98 39 53 90 Dominican Republic 91 98 65 49 85 El Salvador 45 90 14 37 72 Honduras 41 89 28 15 39 Nicaragua 87 98 64 72 93 Panama 95 95 66 53 80 Paraguay 86 98 17 53 85 Puerto Rico 94 96 54 55 84 Venezuela 95 49 98 41 89

Percent aspiration/deletion in conversational speech (Lipski, 1983; Lipski, 1985)

SLIDE 14

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Prior work

Overview One of the most widely-studied sociolinguistic variables Wide range of conditioning factors (phonetic and sociolinguistic) examined Conditioning factors Sex Age Social class Speech style Phonetic context Grammatical category Lexical frequency ...

SLIDE 15

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Prior work: Limitations

Methodological Transcription is inherently subjective Transcriber error and biases In reality /s/-lenition is a gradient process → any partitioning of the space of outcomes is wrong Logistical Segmentation and measurement typically manual → expensive and slow Datasets typically number only in hundreds or thousands of measurement But, cross product of linguistic factors of interest is large and requires correspondingly large datasets of tens or even hundreds of thousands of

bservations

SLIDE 16

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Scaling via audiobooks

Pro Easy to obtain

LibriVox (free) Audible (pay)

Large-scale: single book may yield > 40,000

bservations

Doesn’t require expensive transcriptions → cheap segmentations Con Currently, number of distinct speakers is limited → very in-depth data for single speakers Limited to single speech style: read speech

SLIDE 17

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Audiobooks provide scale: LibriVox

Works Hours Works (2015) Hours (2015) English 8,516 50,591 914 5,170 German 482 2,805 35 197 Dutch 180 2,100 9 102 French 163 1,057 5 33 Spanish 103 638 15 112

SLIDE 18

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

LibriVox

Works Hours Works (2015) Hours (2015) English 8,516 50,591 914 5,170 German 482 2,805 35 197 Dutch 180 2,100 9 102 French 163 1,057 5 33 Spanish 103 638 15 112

SLIDE 19

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

5

Future directions

SLIDE 20

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Corpora

Overview Seven audiobooks encompassing four varieties of Spanish Eight speakers Audio from LibriVox or Audible Peninsular Spanish Los Pazos de Ulloa by Emilia Pardo Baz´ an Historietas Nacionales by Pedro Antonio de Alarc´

n y Ariza

El 19 de Marzo y el 2 de Mayo by Benito P´ erez Gals´

SLIDE 21

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Corpora

Argentinian Cien A˜ nos de Soledad by Gabriel Garc´ ıa M´ arquez La Isla Del Tesoro by Robert Louis Stevenson (translated by Manuel Caballero) Chilean La Casa de los Esperitus by Isabel Allende Mexican interior Angelina by Rafael Delgado

SLIDE 22

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Corpora

Hours Speakers Words /s/ Peninsular 24.24 3 204,448 85,645 Chilean 16.98 2 165,620 66,726 Argentinian 28.72 2 226,489 88,246 Mexican 10.15 1 92,811 37,493 TOTAL 80.09 8 689,368 278,110

SLIDE 23

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Alignment

Acoustic features 13 MFCCs + deltas + delta-deltas Per-utterance cesptral mean-variance normalization 10 ms step, 25 ms analysis window 11-frame context window HMM topology speech: 3-state Bakis non-speech: 5-state w/ skips boundary: 1-state

MFCCs (5-1-5 Frames) 512 Rectified Linear Units 512 Rectified Linear Units 512 Rectified Linear Units 512 Rectified Linear Units SoftMax

SLIDE 24

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Alignment

Training All turns from West Point-Heroico corpus of Mexican Spanish CALLHOME Spanish pronunciation dictionary OOV pronunciations generated via grapheme-to-phoneme transducer trained on CALLHOME Vowel stress differentiated

MFCCs (5-1-5 Frames) 512 Rectified Linear Units 512 Rectified Linear Units 512 Rectified Linear Units 512 Rectified Linear Units SoftMax

SLIDE 25

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

5

Future directions

SLIDE 26

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Acoustic measurements

Approach Extract acoustic features every 5 ms Average across frames within each /s/ segment Compare segment-level averages across phonetic contexts Differences across contexts expected to be more pronounced in dialects known for being leniting (Argentinian and Chilean) Contexts word-final before pause word-final before vowel before voiced stop before nasal before voiceless stop

SLIDE 27

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Acoustic measurements

Spectral centroid Center of mass of decibel power spectrum, viewed as density Excluded frequencies below 1 kHz POV Derived from output of Kadi pitch tracker Random spot checks done to verify sanity on these materials Duration Duration of /s/ segment (seconds) Derived from forced alignment boundaries of /s/

SLIDE 28

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Expected patterns

Spectral centroid Highest in environments encouraging retention (word-final before vowel

r pause)

Lowest in environments typically associated with weakening; in particular, before voiced consonant POV Lowest in environments encouraging retention (word-final before vowel

r pause)

Highest in environments typically associated with weakening to [z] (before voiced stop or nasal)

SLIDE 29

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Expected patterns

Duration Highest before pause and, to lesser degree, word-final before vowel Lowest in environments typically associated with weakening to [z] (before voiced stop or nasal)

SLIDE 30

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Spectral Centroid

SLIDE 31

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Probability-of-voicing

SLIDE 32

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Duration

SLIDE 33

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Conclusion

Known dialect variation in /s/ lenition shows up in audiobooks as expected All forms of lenition (length changes, intrusion of voicing, weakening of frication) show up to some degree in all varieties Quasi-categorical differences seem in most cases to be the endpoint of a gradient process

But re-lexicalization of some forms may occur

SLIDE 34

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

5

Future directions

SLIDE 35

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Discovering relevant phonetic dimensions

Traditional approach Extract some number of hand-engineered acoustic features Compare each feature across conditions Potentially, extract large complex of features and perform dimensionality reduction Alternate approach Learn mapping to phonetically relevant dimensions from data itself Perform dimensionality reduction on this feature space Should focus attention on subspace most relevant to phonetic differentiation

SLIDE 36

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Discovering relevant phonetic dimensions

Feature extraction 13 MFCCs Per-speaker CMVN 5 ms step 25 ms analysis window Mahalanobis distances For each of 28 phone classes, compute:

Full covariance matrix Mean

For each frame in the corpus, for each phone class, compute:

Mahalanobis distance to centroid of that class

SLIDE 37

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Discovering relevant phonetic dimensions

Dimensionality reduction Center distances matrix Perform singular value decomposition (SVD) Keep highest two or three singular vectors

SLIDE 38

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Spain 1

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 39

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Spain 2

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 40

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Spain 3

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 41

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Mexico

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 42

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Chile

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 43

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Argentina 1

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 44

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Argentina 2

1: s # [aeiou] 2: s [ptk] 3: s [bdg]

SLIDE 45

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Conclusion

Known dialect variation apparent:

Gross differences in shape for Spain/Mexico vs Chile/Argentina Pre-pausal and pre-voiceless stop /s/ pattern alike in non-leniting dialects, but not in leniting dialects Pre-pausal /s/ and pre-voiced stop /s/ strongly separated in all dialects

Speaker variation also apparent Suggests potential value in such derived features

SLIDE 46

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Outline

1

Introduction

2

Audiobooks

3

Acoustic measurements

4

Nontraditional acoustic measurement

5

Future directions

SLIDE 47

Introduction Audiobooks Acoustic measurements Nontraditional acoustic measurement Future directions

Future directions

Expand preliminary corpus to include more speakers and dialects (in progress) Leverage scale pattern of variation in individual tokens with particular reference to predictors other than phonetic context Investigate lexicalization of new forms (e.g. with deleted /s/) Explore alternate derived feature representations: