Dat Data- a-Dri Drive ven Spe n Speech ech Synt nthe hesis - - PowerPoint PPT Presentation

▶
dat data a dri drive ven spe n speech ech synt nthe hesis
SMART_READER_LITE
LIVE PREVIEW

Dat Data- a-Dri Drive ven Spe n Speech ech Synt nthe hesis - - PowerPoint PPT Presentation

Nov 14, 2022 125 likes •494 views

Seminar on Language Technology Dat Data- a-Dri Drive ven Spe n Speech ech Synt nthe hesis Konstantin Tretjakov kt@ut.ee 11.12.07 Speech Synthesis Computers are getting smarter all the time. Scientists tell us that soon they will

slide-1
SLIDE 1

Dat Data- a-Dri Drive ven Spe n Speech ech Synt nthe hesis

Konstantin Tretjakov kt@ut.ee Seminar on Language Technology 11.12.07

slide-2
SLIDE 2

Speech Synthesis

“Computers are getting smarter all the

  • time. Scientists tell us that soon they

will be able to talk with us. (By “they”, I mean computers. I doubt scientists will ever be able to talk to us.)

  • Dave Barry
slide-3
SLIDE 3

Speech Synthesis in year 1791

slide-4
SLIDE 4

Speech Synthesis in year 1835

  • J. Faber

“Euphonia”

http://www.ling.su.se/staff/hartmut/kemplne.htm

slide-5
SLIDE 5

Speech Synthesis in year 1937

Riesz Model

http://www.ling.su.se/staff/hartmut/kemplne.htm

slide-6
SLIDE 6

Speech Synthesis in year 1939

H.Dudley “VODER”

http://www.ling.su.se/staff/hartmut/kemplne.htm

slide-7
SLIDE 7

Speech Synthesis in year 1939

H.Dudley “VODER”

http://www.ling.su.se/staff/hartmut/kemplne.htm

slide-8
SLIDE 8

Speech Synthesis in year 1953

Gunnar Fant's “OVE” (Orator Verbis Electris)

http://www.ling.su.se/staff/hartmut/kemplne.htm

Formant Synthesizer for vowels

slide-9
SLIDE 9

Formant Synthesis

slide-10
SLIDE 10

http://www.geofex.com/Article_Folders/wahpedl/voicewah.htm

slide-11
SLIDE 11

Modern Speech Synthesis

  • 1968 - First full TTS (Umeda et al.)
  • 1977 – Diphone concat. (J. Olive)
  • 1979 – MITTalk (Allen et al)
  • 1984 – DECTalk (Klatt, DEC)
  • 1995 – Eurovocs
  • 200? - IBM
slide-12
SLIDE 12

Modern Speech Synthesis

  • 1968 - First full TTS (Umeda et al.)
  • 1977 – Diphone concat. (J. Olive)
  • 1979 – MITTalk (Allen et al)
  • 1984 – DECTalk (Klatt, DEC)
  • 1995 – Eurovocs
  • 200? - IBM

Data-driven Rule-based

slide-13
SLIDE 13

Outline

  • History of Speech Synthesis
  • Text-To-Speech System Architecture
slide-14
SLIDE 14

Text-to-Speech System

Text Text Analysi Analysis

  • Text normalization
  • PoS tagging
  • Homonym disambiguation

Phoneti

  • netic

c analys nalysis

  • Dictionary Lookup
  • Grapheme-to-Phoneme

Pros rosod

  • dic A

ic Ana nalys lysis is

  • Boundary placement
  • Pitch accent assignment
  • Duration computation

Wa Wavefor

  • rm

Synth ynthes esis is

http://www.stanford.edu/class/linguist236/

slide-15
SLIDE 15

Text-to-Speech System

Text Text Analysi Analysis

  • Text normalization
  • PoS tagging
  • Homonym disambiguation

Phoneti

  • netic

c analys nalysis

  • Dictionary Lookup
  • Grapheme-to-Phoneme

Pros rosod

  • dic A

ic Ana nalys lysis is

  • Boundary placement
  • Pitch accent assignment
  • Duration computation

Wa Wavefor

  • rm

Synth ynthes esis is

Data-driven?

slide-16
SLIDE 16

1) Text Normalization

  • He stole $100 million from the bank.
  • It's 13 St. Andrews St.
  • The home page is http://www.ut.ee.

Method:

  • Split to tokens.
  • Map tokens to words.
  • Identify types for words.
slide-17
SLIDE 17

2) Phonetic Analysis

  • My latest project is to learn how to

better project my voice.

  • On May 5 1996, the university bought

1996 computers.

  • Yesterday it rained 3 in. Take 1 out, then

put 3 in.

slide-18
SLIDE 18

2) Phonetic Analysis

  • How to pronounce a word?

– Look in the dictionary!

  • But what about unknown words and names?
  • Complex languages: German/French/Turkish

– Letter to sound rules

  • .. also neural networks (NETTalk)
  • .. pr. by analogy (PRONOUNCE)
  • .. case-based (MBRTalk)
  • ... and muc

uch more. more later

slide-19
SLIDE 19

3) Prosodic Analysis

  • Prosody: phrases, accents, F0 contour,

duration

  • The Tilt Intonation Model

e.g. Trees

slide-20
SLIDE 20

4) Waveform synthesis

  • Articulatory synthesis (a-la VODER)
  • Formant (a-la OVE)
  • Concatenative synthesis

– Domain-specific (“talking clock”, “weather”) – Diphones (PSOLA, MBROLA) – Unit selection

slide-21
SLIDE 21

4) Waveform synthesis

  • Domain-specific synthesis is easy:

#!/bin/bash hours=`date +"%-l"` mins=`date +"%-M"` ampm=`date +"%-P"` play $hours.wav play $mins.wav play $ampm.wav

slide-22
SLIDE 22

4) Waveform synthesis

  • Diphone synthesis

– Use diphones: middle of one phone to middle

  • f next.

– Just a bit of DSP to connect diphones.

  • PSOLA
  • MBROLA
slide-23
SLIDE 23

4) Waveform synthesis

  • Unit selection

– Use the entire speech corpus as the acoustic

inventory.

– Select at runtime the longest available

string of phonetic segments.

– Minimize number of concatenations. – Reduce DSP.

slide-24
SLIDE 24

Text-to-Speech System

Text Text Analysi Analysis

  • Text normalization
  • PoS tagging
  • Homonym disambiguation

Phoneti

  • netic

c analys nalysis

  • Dictionary Lookup
  • Grapheme-to-Phoneme

Pros rosod

  • dic A

ic Ana nalys lysis is

  • Boundary placement
  • Pitch accent assignment
  • Duration computation

Wa Wavefor

  • rm

Synth ynthes esis is

Data-driven?

slide-25
SLIDE 25

Text-to-Speech System

Text Text Analysi Analysis

  • Text normalization
  • PoS tagging
  • Homonym disambiguation

Phoneti

  • netic

c analys nalysis

  • Dictionary Lookup
  • Grapheme-to-Phoneme

Pros rosod

  • dic A

ic Ana nalys lysis is

  • Boundary placement
  • Pitch accent assignment
  • Duration computation

Wa Wavefor

  • rm

Synth ynthes esis is

Data-driven?

slide-26
SLIDE 26

Outline

  • History of Speech Synthesis
  • Text-To-Speech System Architecture
  • Grapheme-to-Phoneme transcription
slide-27
SLIDE 27

GTP transcription

  • Lexicon:

– “cepstra” -> (k eh p)' (s t r aa) – What about unknown words? – Commercial systems have 3-part system:

  • Big dictionary
  • Special code for names/acronyms/etc
  • Mach

Machine-learned ine-learned let letter ter-to-soun

  • -sound

(LTS) (LTS) syst system em for other unknown words

slide-28
SLIDE 28

Learning LTS rules

  • Induce LTS from a dictionary of the

language (Black et al. 1998)

  • Two steps:

– Alignment – Decision tree-based rule-induction

slide-29
SLIDE 29

Alignment

  • Letters: c h e c k e d
  • Phones: ch _ eh _ k _ t
  • Black et al. propose 2 methods:

– Expectation-Maximization – Estimate p(letter | phone) from

valid alignments, take best.

  • Devil in the details
slide-30
SLIDE 30

Decision trees for LTS

  • Now that aligned data is available, train a

decision tree:

– ###chek -> ch – checked -> _

  • 92-96% letter acc. (58-75% word acc.)

for English

slide-31
SLIDE 31

GTP transcription

  • Decision-tree based (Black et al.)
  • ANN-based (NETTalk, Sejnowski et al.)
  • Pronunciation-by-Analogy (Damper et al.)
  • Memory-based (MBRTalk, Stanfill)
  • Transducer-based (I. Bulyko)
  • Non-segmental (A. Cohen)
slide-32
SLIDE 32

GTP transcription

  • Decision-tree based (Black et al.)
  • ANN-based (NETTalk, Sejnowski et al.)
  • Pronunciation-by-Analogy (Damper et al.)
  • Memory-based (MBRTalk, Stanfill)
  • Transducer-based (I. Bulyko)
  • Non-segmental (A. Cohen)
slide-33
SLIDE 33

Outline

  • History of Speech Synthesis
  • Text-To-Speech System Architecture
  • Grapheme-to-Phoneme transcription
  • Conclusion
slide-34
SLIDE 34

Text-to-Speech System

Text Text Analysi Analysis

  • Text normalization
  • PoS tagging
  • Homonym disambiguation

Phoneti

  • netic

c analys nalysis

  • Dictionary Lookup
  • Grapheme-to-Phoneme

Pros rosod

  • dic A

ic Ana nalys lysis is

  • Boundary placement
  • Pitch accent assignment
  • Duration computation

Wa Wavefor

  • rm

Synth ynthes esis is

http://www.stanford.edu/class/linguist236/

slide-35
SLIDE 35

???