Speech Synthesis and Perception with Envelope Cue B ACKGROUND I - - PowerPoint PPT Presentation

Speech Synthesis and Perception with Envelope Cue

Signals and Systems

目录

BACKGROUND IMPLEMENTATION RESULTS DISCUSSION IMPROVEMENT

B ACKGROUND | PART1

History - Artificial Cochlea

1748

• First extra-auricular electric simulation

1905

• Invention of an electrical stimulating system

1930

• Electrode placed in the acoustic nerve produced

a copy of the speech waveform.

1961

• The first true cochlea implant was implanted by

the American otologist William Bill House

1984

• FDA allowed them to be implanted in adults.

2000

• The implants are approved for infants over 12

months old.

I MPLEMENTATION | PART2

Figure 1. The operation of a four-channel cochlear implant. Reprinted from "Introduction to cochlear implants," by P . C. Loizou, 1999, IEEE Engineering in Medicine and Biology Magazine, vol. 18, no. 1.

band = 8; order = 4

synthesize.m-modulation

• rder = 4

synthesize.m-8 band pass filters

SNR=-5

add_ssn.m

GUI.m

R ESULTS D ISCUSSION | PART3

Task1 Variation in Channel Number

Butter Filters: Order = 4 𝑔

𝑑𝑣𝑢𝑝𝑔𝑔 = 50𝐼𝑨

• N=1
• N=2
• N=4
• N=8
• N=16
• N=20
• N=32

Why N is limited?

• Instability of filters
• Interference between electrodes
• Continuous interleaved sampling

Task2 Variation in Cut-off Frequency

Set the number

• f bands N=4.

Implement tone- vocoder by changing the LPF cut-off frequency .

Describe how the LPF cut-off frequency affects the intelligibility

• f synthesized sentence.

Task2 Results and Conclusion

• 𝑔

𝑑𝑣𝑢𝑝𝑔𝑔 = 20Hz

• 𝑔

𝑑𝑣𝑢𝑝𝑔𝑔 = 50Hz

• 𝑔

𝑑𝑣𝑢𝑝𝑔𝑔 = 100Hz

• 𝑔

𝑑𝑣𝑢𝑝𝑔𝑔 = 400Hz

N=4

Task3 Noise & Variation in Band Number

Generate a noisy signal at SNR

• 5 dB

Set LPF cut-off frequency to 50 Hz Implement tone-vocoder by changing the number of bands

Describe how the number of bands affects the intelligibility of synthesized sentence, and compare findings with those obtained in task 1

Task3 Results and Conclusion

• N=2
• N=4
• N=6
• N=8
• N=16

Task4 Noise & Variation in Cut-off Frequency

Generate a noisy signal at SNR -5 dB Set the number

• f bands

to N=6 Implement tone-vocoder by changing the LPF cut-off frequency Describe how the LPF cut-off frequency affects the intelligibility

• f synthesized

sentence

Task4 Noise & Variation in Cut-off Frequency

• Synthesized speech is likely to lose its tone.
• Chinese: tonal; English: non-tonal

Processed：

English & Chinese Comparison

English & Chinese Comparison

Reprinted from "电子耳蜗言语处理策略的频谱特征研究." by 陈又圣, et al. (2017) 生物医学工程学杂志 34(5): 760-766.

How about music?

I MPROVEMENT | PART4

Noise Reduction

• S. V. Vaseghi, Advanced Digital Signal Processing and Noise Reduction. 2008.

Noise Reduction using Wiener filters

• Original
• Noisy
• Noise Reduced
• Synthesized (noisy)
• Synthesized (noise reduced)

Reference：

[1]

• A. Mudry and M. Mills, "The early history of the cochlear implant: a retrospective," (in eng), JAMA

Otolaryngol Head Neck Surg, vol. 139, no. 5, pp. 446-53, May 2013. [2]

• R. V. Shannon, F. G. Zeng, V. Kamath, J. Wygonski, and M. Ekelid, "Speech recognition with primarily

temporal cues," (in eng), Science, vol. 270, no. 5234, pp. 303-4, Oct 13 1995. [3]

• S. V. Vaseghi, Advanced Digital Signal Processing and Noise Reduction. Wiley, 2008.

[4] Chen, F., et al. (2015). "Evaluation of noise reduction methods for sentence recognition by mandarin- speaking cochlear implant listeners." Ear and hearing 36(1): 61-71. [5]

陈又圣, et al. (2017). "电子耳蜗言语处理策略的频谱特征研究." 生物医学工程学杂志 34(5): 760-766.

[6]

龚树生, and 郝瑾, “国产人工耳蜗,任重道远,” 中国医学文摘: 耳鼻咽喉科学, vol. 28, no. 5, pp. 231-236, 2013.

感谢观看 | THANK YOU