Tim Bunnell Center for Pediatric Auditory & Speech Sciences - - PowerPoint PPT Presentation

Tim Bunnell

Center for Pediatric Auditory & Speech Sciences Nemours/Alfred I. duPont Hospital for Children Wilmington, DE

Nemours Children’s Health System

• Clinical

– Hospitals in Wilmington & Orlando – Clinics & satellites in DE, FL, PA, NJ, MD, GA – Over 700 physicians – Over 30 pediatric specialties – Over 1.2 million encounters per year – Around 300,000 unique patients seen per year

• Research

– Almost $9M in NIH funding for 2014 – 11 Research Centers – ~40 Laboratories – Focus on Neurodevelopmental & Musculoskeletal disease, Diabetes & Obesity, Asthma & Cystic Fibrosis, Cancer, Applied Genomics, and Healthcare Delivery Science.

CPASS

• Four Labs + Bioinformatics

– Auditory Physiology & Psycho-acoustics – Head, Morlet – Balance & Vestibular Disorders – Head, O’Reilly – Craniofacial Outcomes Research – Head, Vallino – Speech Research Lab – Head, Bunnell

• Acoustic Phonetics
• Speech Perception/Production
• Clinical Speech Technology

– Applications involving speech recognition & synthesis technology

– Bioinformatics – Head, Bunnell

Clinical Speech Technology

• Utterance Verification /

Classification

– Acoustic Phenotyping – Used as functional hearing evaluation – Auditory/verbal therapy – Used as objective speech intelligibility measure

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

Human vs Machine CSIM Scoring

ASR−based CSIM Score Human−Based CSIM Score * * Hearing Status NH CI Age 36 42 48 72

nor s02 s11 s07 s13 s03 s16 s01 s10 s06 s12 s17 s18 s04 s09 s05 s15 s08 s14 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Cluster Dendrogram

Subject Height

Speech Synthesis for Assistive Technology

• Problem – Speech Generating devices are:

– Limited in choice of voices – Impersonal – Lacking expressiveness

• Existing Solutions

– Voice banking – Voice conversion/creation – Parametric synthesis to modulate prosodic/expressive features

ModelTalker TTS System

Speech Files XML Control File

This is a demonstration of the ModelTalker Speech Synthesis System.

Dictionary & Linguistic Rules Unit Select. Database MTVC

Feature Extraction HMM Training Data Pruning DB Construction

Child Adult2 Child2

Going forward for personal voices…

• User Needs

– More Expressive!

• Lacking in Unit Selection without massive amounts of data
• Not modeled well in statistical parametric synthesis

– More Natural

• Issue particularly for parametric synthesis
• Fewer ‘glitches’ in unit selection

– Lower Barriers to creation

• Fewer hours of recording
• Improved morphing
• Research Needs

– Phonetics/Phonology

• Improve acoustic models of emotion and expressiveness
• Improve models of the time-varying structure of speech (!)

– Engineering

• Improvements in signal processing to model voice/vocal-tract interaction

– Capture the features of an individual’s speech in a small number of dimensions than can be manipulated in expressively useful ways.