accent reclassification and speech recognition of
play

Accent reclassification and speech recognition of Afrikaans, Black - PowerPoint PPT Presentation

Nov 15, 2022 •265 likes •710 views

Accent reclassification and speech recognition of Afrikaans, Black and White South African English Herman Kamper and Thomas Niesler Digital Signal Processing Laboratory Department of Electrical and Electronic Engineering Stellenbosch University

  1. Accent reclassification and speech recognition of Afrikaans, Black and White South African English Herman Kamper and Thomas Niesler Digital Signal Processing Laboratory Department of Electrical and Electronic Engineering Stellenbosch University UNIVERSITEIT • STELLENBOSCH • UNIVERSITY • your knowledge partner jou kennisvennoot

  2. Introduction Accented English is highly prevalent in South Africa We consider three accents of South African English: ◮ Afrikaans English (AE) ◮ Black South African English (BE) ◮ White South African English (EE) For multi-accent speech recognition, accent labels must be assigned to training set utterances These are assigned by human annotators based on a speaker’s mother-tongue or ethnicity and might not necessarily be optimal for modelling purposes We consider the unsupervised reclassification of training set accent labels H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 2 / 14

  3. Oracle and parallel recognition of AE and EE H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 3 / 14

  4. Oracle and parallel recognition of AE and EE Oracle: Separate accent-specific recognisers for each accent AE AE Hypothesised speech recogniser transcription EE EE Hypothesised speech recogniser transcription H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 3 / 14

  5. Oracle and parallel recognition of AE and EE Oracle: Separate accent-specific recognisers for each accent AE AE Hypothesised speech recogniser transcription EE EE Hypothesised speech recogniser transcription Parallel: Two accent-specific recognisers operating in parallel Select output with highest likelihood AE recogniser AE & EE Hypothesised speech transcription EE recogniser H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 3 / 14

  6. Accent misclassifications Select output with highest likelihood AE recogniser AE Hypothesised speech transcription EE recogniser Correctly identified: The matching recogniser is selected H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 4 / 14

  7. Accent misclassifications Select output with highest likelihood AE recogniser AE Hypothesised speech transcription EE recogniser Misclassification: A recogniser from another accent is selected H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 4 / 14

  8. Oracle and parallel recognition of AE and EE Oracle: Separate accent-specific recognisers for each accent AE AE Hypothesised speech recogniser transcription EE EE Hypothesised speech recogniser transcription Parallel: Two accent-specific recognisers operating in parallel Select output with highest likelihood AE recogniser AE & EE Hypothesised speech transcription EE recogniser H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 5 / 14

  9. Oracle and parallel recognition of AE and EE Oracle: Separate accent-specific recognisers for each accent AE AE Hypothesised speech recogniser transcription EE EE Hypothesised speech recogniser transcription Parallel: Two accent-specific recognisers operating in parallel Select output with highest likelihood AE recogniser AE & EE Hypothesised speech transcription EE recogniser Small improvements of parallel over oracle for AE+EE H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 5 / 14

  10. Accent reclassification Conclusions from oracle vs. parallel recognition Misclassifications do not always lead to deteriorated accuracies The accent labels assigned to training/test utterances might not be the most appropriate H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 6 / 14

  11. Accent reclassification Conclusions from oracle vs. parallel recognition Misclassifications do not always lead to deteriorated accuracies The accent labels assigned to training/test utterances might not be the most appropriate Propose accent reclassification Use first-pass acoustic models trained on the originally labelled data to reclassify the accent of training set utterances and then retrain the acoustic models H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 6 / 14

  12. Accent reclassification Conclusions from oracle vs. parallel recognition Misclassifications do not always lead to deteriorated accuracies The accent labels assigned to training/test utterances might not be the most appropriate Propose accent reclassification Use first-pass acoustic models trained on the originally labelled data to reclassify the accent of training set utterances and then retrain the acoustic models: AE+EE: relatively similar accents BE+EE: relatively dissimilar accents H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 6 / 14

  13. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  14. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  15. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  16. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  17. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  18. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  19. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  20. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  21. Accent reclassification Transcriptions with original accent labels Train Create transcrip- accent-specific tions with new HMMs accent labels Reclassified accent labels Use HMMs to No Last iteration? reclassify training set Yes Reclassified Multi-accent speech accent-specific recognition HMMs H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 7 / 14

  22. Speech databases African Speech Technology (AST) databases: ◮ Afrikaans English (AE) database ◮ Black South African English (BE) database ◮ White South African English (EE) database Training set: approximately 6 hours of speech in each accent Test set: approximately 24 minutes of speech from 20 speakers in each accent Development set: used to optimise recognition parameters H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 8 / 14

  23. Experimental setup Setup of systems Word recognition of continuous telephone speech Trained 8-mixture cross-word triphone HMMs Parameterisation: MFCCs, 1 st and 2 nd order derivatives, per-utterance CMN Accent-independent language models and pronunciation dictionaries H. Kamper (Stellenbosch University) Reclassification of SAE accents PRASA 2011 9 / 14

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

8-Speech Recognition Speech Recognition Concepts Speech Recognition Approaches

8-Speech Recognition Speech Recognition Concepts Speech Recognition Approaches

8-Speech Recognition Speech Recognition Concepts Speech Recognition Approaches Recognition Theories Bayse Rule Simple Language Model P(A|W) Network Types 1 7-Speech Recognition (Cont d) HMM Calculating Approaches

1.08k views • 74 slides
HMMS and Speech HMMS and Speech HMMS and Speech Recognition Recognition Recognition Presented

HMMS and Speech HMMS and Speech HMMS and Speech Recognition Recognition Recognition Presented

HMMS and Speech HMMS and Speech HMMS and Speech Recognition Recognition Recognition Presented by Jen-Wei Kuo Reference 1. X. Huang et. al., Spoken Language Processing, Chapter 8 2. Daniel Jurafsky and James H. Martin, Speech and Language

1.05k views • 65 slides
Problem Statement To design an automatic speech recognition system that gives best recognition

Problem Statement To design an automatic speech recognition system that gives best recognition

Problem Statement To design an automatic speech recognition system that gives best recognition results for both male and female speakers. both male and female speakers. Parameters effecting ASR performance 1. Gender 2. Accent 3. Age 3. Age

416 views • 8 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 25: Speech

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 25: Speech

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 25: Speech synthesis (Concluding lecture) Instructor: Preethi Jyothi Nov 6, 2017 Recall: SPSS framework O Speech Speech Train Parameter

273 views • 26 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 23: Speech

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 23: Speech

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 23: Speech Synthesis (Part I) Instructor: Preethi Jyothi Oct 30, 2017 T ext- T o- S peech Systems Storied History Von Kempelens speaking machine (1791)

290 views • 8 slides
Cyborg speech: Deep multilingual speech synthesis for generating segmental foreign accent with

Cyborg speech: Deep multilingual speech synthesis for generating segmental foreign accent with

Cyborg speech: Deep multilingual speech synthesis for generating segmental foreign accent with natural prosody Gustav Eje Henter 1 , Jaime Lorenzo-Trueba 1 , Xin Wang 1 , Mariko Kondo 2 , Junichi Yamagishi 1,3 gustav@nii.ac.jp, jyamagis@nii.ac.jp

1.22k views • 58 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 1: Introduction

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 1: Introduction

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 1: Introduction to Statistical Speech Recognition Instructor: Preethi Jyothi July 24, 2017 Course Specifics Pre-requisites Ideal Background: Completed one of

732 views • 44 slides
Speech recognition Brief history Technology Computer Literacy 1 Lecture 22 How does

Speech recognition Brief history Technology Computer Literacy 1 Lecture 22 How does

Topics Definition of speech recognition Speech recognition Brief history Technology Computer Literacy 1 Lecture 22 How does speech recognition work 10/11/2008 Speaker recognition Problems of speech and speaker recognition

325 views • 6 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 12: Acoustic

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 12: Acoustic

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 12: Acoustic Feature Extraction for ASR Instructor: Preethi Jyothi Feb 13, 2017 Speech Signal Analysis Generate A frame discrete samples Need to

441 views • 26 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 9: Brief

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 9: Brief

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 9: Brief Introduction to Neural Networks Instructor: Preethi Jyothi Feb 2, 2017 Final Project Landscape Tabla bol transcription Voice-based music Sanskrit

409 views • 17 slides
Review reclassification criteria changes in SY 2018-19 Share preliminary reclassification

Review reclassification criteria changes in SY 2018-19 Share preliminary reclassification

Review reclassification criteria changes in SY 2018-19 Share preliminary reclassification outcomes for SY 2018-19 Present and seek approval of standard reclassification criteria for SY 2019-20 SAN FRANCISCO UNIFIED SCHOOL DISTRICT

351 views • 10 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 4: WFSTs in ASR

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 4: WFSTs in ASR

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 4: WFSTs in ASR + Basics of Speech Production Instructor: Preethi Jyothi Lecture 4 Qv iz-1 Postmortem Common Mistakes: Correct Incorrect Output

589 views • 25 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 4: WFST

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 4: WFST

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 4: WFST algorithms contd. + WFSTs in ASR Instructor: Preethi Jyothi August 3, 2017 Qv iz-1 Postmortem Common Mistakes: Correct Incorrect Missing

432 views • 24 slides
Combining Speech and Speaker Recognition - A Joint Modeling Approach Hang Su Supervised by:

Combining Speech and Speaker Recognition - A Joint Modeling Approach Hang Su Supervised by:

Introduction and Motivation Backgrounds on Speech and Speaker Recognition Connecting Speech and Speaker Recognition Joint Modeling of Speech and Speaker Conclusion and Future Work Combining Speech and Speaker Recognition - A Joint Modeling

729 views • 72 slides
Speech Recognition Speech Recognition Berlin Chen,

Speech Recognition Speech Recognition Berlin Chen,

Speech Recognition Speech Recognition Berlin Chen, berlin@csie.ntnu.edu.tw http://berlin.csie.ntnu.edu.tw Course Contents Both the theoretical and practical issues for spoken language processing will be considered

827 views • 37 slides
Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 11: Recurrent

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 11: Recurrent

Automatic Speech Recognition (CS753) Automatic Speech Recognition (CS753) Lecture 11: Recurrent Neural Network (RNN) Models for ASR Instructor: Preethi Jyothi Feb 9, 2017 Recap: Hybrid DNN-HMM Systems Triphone state labels (DNN

651 views • 20 slides
Accent Location 53.5

Accent Location 53.5

Accent Location 53.5

411 views • 5 slides
Motivating Dramatic Interactions Stefan Rank 1 Paolo Petta 1 , 2 1 Austrian Research Institute for

Motivating Dramatic Interactions Stefan Rank 1 Paolo Petta 1 , 2 1 Austrian Research Institute for

1 Agents that want and like @ AISB2005 Convention, University of Hertfordshire 2005/04/14 Motivating Dramatic Interactions Stefan Rank 1 Paolo Petta 1 , 2 1 Austrian Research Institute for Artificial Intelligence 2 Department of Medical

590 views • 25 slides
Career Development Workshops for Women in Physics Shobhana Narasimhan Jawaharlal Nehru Centre

Career Development Workshops for Women in Physics Shobhana Narasimhan Jawaharlal Nehru Centre

Career Development Workshops for Women in Physics Shobhana Narasimhan Jawaharlal Nehru Centre for Advanced Scientific Research Bangalore, India I entered Harvard University as a PhD student in physics in 1985 The graduate classes at Harvard

784 views • 44 slides
10-701 Machine Learning Decision trees Optional additional reading: Mitchell Chapter 3 Types of

10-701 Machine Learning Decision trees Optional additional reading: Mitchell Chapter 3 Types of

10-701 Machine Learning Decision trees Optional additional reading: Mitchell Chapter 3 Types of classifiers We can divide the large variety of classification approaches into roughly two main types 1. Instance based classifiers - Use

788 views • 40 slides
Challenges in Digital Sanskrit Lexicography The Example of Vedic Accent Felix Rau (University of

Challenges in Digital Sanskrit Lexicography The Example of Vedic Accent Felix Rau (University of

Challenges in Digital Sanskrit Lexicography The Example of Vedic Accent Felix Rau (University of Cologne) Cologne Sanskrit Dictionary Vedic Accent a t a a t t t i t r d a d u v n u s a ( Kiparsky 1973) Vedic

406 views • 18 slides
Prosody in Dialog Predicting Accent based on Information Structure and context Defining

Prosody in Dialog Predicting Accent based on Information Structure and context Defining

Prosody in Dialog Predicting Accent based on Information Structure and context Defining Prosodic Features in English Pitch Accent Local Maxima or Minima in f0 o Variable depending on discourse o Lexical Stress Stress results in

501 views • 13 slides
Testing 221 238 197 223 171 213 Manoj Nambiar, Tata Blue 50% Tata Blue 25% Purple 50 %

Testing 221 238 197 223 171 213 Manoj Nambiar, Tata Blue 50% Tata Blue 25% Purple 50 %

Title and Content 109 255 131 0 85 214 207 255 56 99 165 73 246 255 155 190 28 42 Dark 1 Light 1 Dark 2 Light 2 Accent 1 Accent 2 185 151 193 255 255 236 175 75 187 221 255 137 164 7 0 62 255 29 Accent 3

329 views • 30 slides
Information Structure Annotation and Secondary Accents Arndt Riester 1 Stefan Baumann 2 1

Information Structure Annotation and Secondary Accents Arndt Riester 1 Stefan Baumann 2 1

Information Structure Annotation and Secondary Accents Arndt Riester 1 Stefan Baumann 2 1 Institute for Natural Language Processing University of Stuttgart 2 IfL Phonetics University of Cologne DGfS Workshop Beyond Semantics 24.2.2011 Riester

939 views • 56 slides

More recommend