LINGUISTIC TIMBRE: FORMANTS YU / LAMONT FEBRUARY 27 & MARCH 1, - PowerPoint PPT Presentation

LINGUIST 197M, SPRING 2018. CLASS 6.1-6.2 LINGUISTIC TIMBRE: FORMANTS YU / LAMONT FEBRUARY 27 & MARCH 1, 2018

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 2 PARSING WORDS FROM CONTINUOUS SPEECH

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 3 VOWEL MERGERS IN SPECTROGRAMS

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 4 RATE OF PHONEMES PER SECOND

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 5 SPEECH INFORMATION RATE ARTICLE http://citeseerx.ist.psu.edu/viewdoc/download? doi=10.1.1.433.3226&rep=rep1&type=pdf

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 6 BRIEF ARTICLE SUMMARY: BLOG POST http://rosettaproject.org/blog/02012/mar/1/language-speed-vs-density/

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 7 HOW MANY PHONEMES A LANGUAGE USES 
 Phoible: http://phoible.org/ UPSID: http://web.phonetik.uni-frankfurt.de/upsid_info.html LAPSyD: http://www.lapsyd.ddl.ish-lyon.cnrs.fr/lapsyd/index.php

8 THE VOCAL TRACT AND TIMBRE: VOWELS

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 9 MRI VIDEOS OF VOWEL PRODUCTIONS! front close unrounded vowel back open unrounded vowel http://sail.usc.edu/span/rtmri_ipa/pk_2015.html

LINGUIST 197M, SPRING 2018. CLASS 5.2 10 REAL-TIME OSCILLOSCOPE http://academo.org/demos/virtual-oscilloscope/

LINGUIST 197M, SPRING 2018. CLASS 5.2 11 REAL-TIME SPECTRUM ANALYZER II http://musiclab.chromeexperiments.com/Spectrogram

LINGUIST 197M, SPRING 2018. CLASS 5.2 12 REAL-TIME SPECTRUM ANALYZER http://academo.org/demos/spectrum-analyzer/

13 WAVEFORMS, SPECTRA, AND SPECTROGRAMS

LINGUIST 197M, SPRING 2018. CLASS 5.1 14 SYNTHESIZE A VOCALIC SOUND…

LINGUIST 197M, SPRING 2018. CLASS 5.1 15 REVIEW: FIND PERIOD FROM THE WAVEFORM What are the x- and y-axes in the waveform? What’s that blue line?

LINGUIST 197M, SPRING 2018. CLASS 5.1 16 REVIEW: TAKE SPECTRAL SLICE (SPECTRUM) Make sure you select a window of at least 5 cycles of the waveform to take the spectral slice over!

LINGUIST 197M, SPRING 2018. CLASS 5.1 17 REVIEW: UNDERSTANDING THE SPECTRUM Make sure to zoom in on the spectrum! What are the axes? How far apart are two adjacent spectral peaks and why?

LINGUIST 197M, SPRING 2018. CLASS 5.1 18 NEW: THE SPECTROGRAM!

LINGUIST 197M, SPRING 2018. CLASS 5.1 19 NEW: THE SPECTROGRAM!

20 FROM THE SPECTRUM TO THE SPECTROGRAM

0.5 f = 1 Hz f = 1 Hz -1.0 0.0 0.5 1.0 1.5 2.0 t 0.5 f = 2 Hz f = 2 Hz -1.0 0.0 0.5 1.0 1.5 2.0 t 0.5 f = 3 Hz f = 3 Hz -1.0 0.0 0.5 1.0 1.5 2.0 t 0.5 f = 4 Hz f = 4 Hz -1.0 0.0 0.5 1.0 1.5 2.0 t

LINGUIST 197M, SPRING 2018. CLASS 5.1 22 SPECTRUM OF COMPLEX WAVEFORM Amplitude (dB) 20 Amplitude (dB) 15 wave.complex 10 5 1 0 0.0 0.5 1.0 1.5 2.0 1 2 3 4 t Time (s) Frequency (Hz)

LINGUIST 197M, SPRING 2018. CLASS 5.1 23 SPECTRUM OF COMPLEX WAVEFORM Amplitude (dB) 20 Amplitude (dB) 15 Waveform: x-axis is time wave.complex 10 Spectrum: x-axis is frequency 5 1 0 0.0 0.5 1.0 1.5 2.0 1 2 3 4 t Time (s) Frequency (Hz)

LINGUIST 197M, SPRING 2018. CLASS 5.1 24 FROM SPECTRUM TO SPECTROGRAM: STEP 1 Amplitude (dB) Rotate clockwise Amplitude (dB) 3 4 1 2 4 1 2 3 4 3 Freq (Hz) 2 1 1 2 3 4 Freq (Hz)

LINGUIST 197M, SPRING 2018. CLASS 5.1 25 FROM SPECTRUM TO SPECTROGRAM: STEP 2 Amplitude (dB) Frequency (Hz) 2 3 4 1 1 2 3 4 1 2 3 4 Freq (Hz) Flip vertically 1 2 3 4 Amplitude (dB)

LINGUIST 197M, SPRING 2018. CLASS 5.1 26 FROM SPECTRUM TO SPECTROGRAM: STEP 3 Encode amplitude Frequency (Hz) levels (the height/ 1 2 3 4 length of the blue spikes) in grayscale* 1 2 3 4 1 2 3 4 *or with heat map Amplitude (dB)

LINGUIST 197M, SPRING 2018. CLASS 5.1 27 FROM SPECTRUM TO SPECTROGRAM: STEP 4 Encode amplitudes in grayscale… Frequency (Hz) 1 2 3 4 Frequency (Hz) 1 2 3 4 1 2 3 4 …at time t 1 Amplitude (dB)

LINGUIST 197M, SPRING 2018. CLASS 5.1 28 FROM SPECTRUM TO SPECTROGRAM: STEP 5 Line up spectral slices taken over time… Frequency (Hz) 1 2 3 4 t 1 t 2 t 3 t 4 t 5 t 6

LINGUIST 197M, SPRING 2018. CLASS 5.1 29 FROM SPECTRUM TO SPECTROGRAM: FINIS …and you have a spectrogram 5000 Frequency (Hz) 0 0.8772 1.128 Time (s)

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 30 WIDE VS. NARROW-BAND SPECTROGRAMS Wide: high temporal resolution, low frequency resolution Narrow: low temporal resolution, high frequency resolution NARROW WIDE

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 31 NARROW-BAND SPECTROGRAM Narrow: low temporal resolution, high frequency resolution

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 32 WIDE-BAND SPECTROGRAM Wide: high temporal resolution, low frequency resolution

33 SOURCE-FILTER THEORY

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 34 SOURCE-FILTER THEORY ▸ Input = (Voice) source ▸ Result: harmonics of voice source ▸ Filter = Vocal tract ▸ Result: harmonic amplitudes get modulated ▸ Output = Speech ▸ Result: combined effects of source and filter

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 35 VOCAL TRACT From glottis to lips! Ladefoged and Johnson (2010), p. 5

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 36 SOURCE-FILTER THEORY: A DIAGRAM Vocal tract http://www.phon.ucl.ac.uk/courses/plin/plin2108/week5.php

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 37 DIFFERENT SPEECH SOUNDS HAVE DIFFERENT VOCAL TRACT CONFIGURATIONS Different vocal tract configurations Same source http://www.phon.ucl.ac.uk/courses/plin/plin2108/week5.php

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 38 SOURCE-FILTER THEORY: A SCHEMATIC source filter output waveform spectrum Fucci and Lass

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 39 SOURCE-FILTER THEORY: A SCHEMATIC source filter output waveform spectrum Fucci and Lass

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 40 OUTPUT SPECTRUM •Output = Speech •Result: combined effects of source and filter 1st harmonic is f0 Frequency of nth harmonic = n * f0 40

41 THE SOURCE: THE VIBRATING LARYNX

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 42 SOURCE-FILTER THEORY: A SCHEMATIC source filter output waveform spectrum Fucci and Lass

43 AMPLITUDE OF HARMONICS IN SOURCE ▸ Can get by recording at the larynx or inverse filtering of speech ▸ Amplitudes of source harmonics generally decrease as frequency goes up (about 3 dB fall per octave) ▸ Rate of decrease depends on phonation quality (creaky, breathy, etc.)

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 44 ELECTROGLOTTOGRAPHY: RECORDING AT THE LARYNX http://www.linguistics.ucla.edu/faciliti/facilities/physiology/EGG.htm

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 45 ELECTROGLOTTOGRAPHY: RECORDING AT THE LARYNX Glottal Glottal waveform spectrum Vocal tract Vocal tract transfer shape for [i] function Filtered Filtered waveform: [i] spectrum: [i] http://sail.usc.edu/~lgoldste/General_Phonetics/Source_Filter_Demo/index.html

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 46 ELECTROGLOTTOGRAPHY: RECORDING AT THE LARYNX Mic http://sail.usc.edu/~lgoldste/General_Phonetics/Source_Filter_Demo/index.html

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 47 INVERSE FILTERED SPECTRUM EXAMPLE

48 THE FILTER: THE VOCAL TRACT

49 THE VOCAL TRACT AS A FILTER ▸ Configuration of vocal tract acts on amplitude of harmonics from voice source ▸ No new harmonics are added nor or their frequencies changed! ▸ Some harmonics get stronger, some get weaker ▸ Particular vocal tract configuration has particular resonance frequencies (formants); if these are close to the frequencies of some harmonics, those harmonics get strengthened

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 50 SYMPATHETIC VIBRATION: TUNING FORKS https://youtu.be/zWKiWaiM3Pw Other fun resonance videos at: http://blog.prosig.com/2011/09/20/5-videos-that-explain- resonance/

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 51 EXAMPLE: RESONANCE B resonates to A if B’s vibrations make A vibrate too. Effect: B has more energy than it did The closer B’s natural frequency is to A’s the stronger the vibrations of B 51

52 RESONANCES FOR DIFFERENT VOCAL TRACT CONFIGURATIONS ▸ Resonances depend on size and shape of airway ▸ Can be approximated as multitube models, with connected Helmholtz resonators ▸ Helmholtz resonances are the formants ▸ See article by Sandberg on The Acoustics of the Singing Voice for further reading http://faculty.washington.edu/losterho/Sundberg.pdf

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 53 MRI VIDEOS OF VOWEL PRODUCTIONS! front close unrounded vowel back open unrounded vowel http://sail.usc.edu/span/rtmri_ipa/pk_2015.html

LINGUIST 197M, SPRING 2018. CLASS 6.1, 6.2 54 HELMHOLTZ RESONATORS AND FORMANTS http://www.exploratorium.edu/exhibits/vocal_vowels/vocal_vowels.html duck call sound source: vibrating reed

55 SOME POINTS OF CONFUSION