Machine Learning and Signal Processing Technologies for Assistive - - PowerPoint PPT Presentation

▶
machine learning and signal processing technologies for
SMART_READER_LITE
LIVE PREVIEW

Machine Learning and Signal Processing Technologies for Assistive - - PowerPoint PPT Presentation

Feb 28, 2024 165 likes •436 views

Machine Learning and Signal Processing Technologies for Assistive Hearing Devices Dr. Yu Tsao , Associate Research Fellow Research Center for IT Innovation, Academia Sinica 2018/5/15 Assistive Technology: Assistive Listening and Speaking

slide-1
SLIDE 1

Machine Learning and Signal Processing Technologies for Assistive Hearing Devices

  • Dr. Yu Tsao 曹昱, Associate Research Fellow

Research Center for IT Innovation, Academia Sinica

2018/5/15

slide-2
SLIDE 2

Assistive Technology: Assistive Listening and Speaking Devices (by Dr. Yu Tsao)

2

FM system Hearing aids Cochlear implants

  • Conventional FM systems
  • Smarthear
  • Two modes of Smarthear
  • Current issues
  • Noise reduction (NR)
  • Amplification scheme

(AS)

  • Signal processing of CI
  • Current issues
  • Noise reduction (NR)
slide-3
SLIDE 3

Assistive Technology: Assistive Listening and Speaking Devices (by Dr. Yu Tsao)

3

FM system Hearing aids Cochlear implants

  • Conventional FM systems
  • Smarthear
  • Two modes of Smarthear
  • Current issues
  • Noise reduction (NR)
  • Amplification scheme

(AS)

  • Signal processing of CI
  • Current issues
  • Noise reduction (NR)
slide-4
SLIDE 4

Conventional FM System

➢ Speaker and listeners must wear the additional systems ➢ Stigma and inconvenience ➢ High cost: ~ 80,000 NTD（microphone+ transmitter+ receiver） Receiver plus an HA for individuals with more severe hearing loss A direct receiver for individuals with minor hearing loss Speaker wears microphone and transmitter Smarthear: smartphone-based FM system

slide-5
SLIDE 5

5

SmartHear

slide-6
SLIDE 6

6

SmartHear 2.0

  • 1. 2,500K supported by 科技部
  • 2. 中央社新聞、大愛電視台、中視新聞專訪
  • 3. 臺灣輔具暨長期照護大展2017 ATLife、聽覺毛細

胞至大腦皮層研討會、榮民總醫院聽覺部演講、 輕度聽損輔具研討會 2016

  • 3. IEEE System Journal, IEEE Access, 中華民國專利
  • 4. SmartHear 2.0 (Demo Session)

Hearing test Speech recognition test

slide-7
SLIDE 7

Assistive Technology: Assistive Listening and Speaking Devices (by Dr. Yu Tsao)

7

FM system Hearing aids Cochlear implants

  • Conventional FM systems
  • Smarthear
  • Two modes of Smarthear
  • Current issues
  • Noise reduction (NR)
  • Amplification scheme

(AS)

  • Signal processing of CI
  • Current issues
  • Noise reduction (NR)
slide-8
SLIDE 8

8

Hearing loss Normal hearing

Audiograms

Hearing Aids

Only enlarge the volume is not enough!

slide-9
SLIDE 9

A Critical Issue of HA

  • Kochkin investigated the users’ satisfaction of

hearing instrument in United States that the first prevalent requirements was :

➢ 95% of hearing aid user seek the improvement to listen

speech in noise.

  • The hearing-impaired person needs a better

signal-to-noise ratio (SNR) than that of a normal- hearing person.

  • We proposed to use the DL-based noise reduction

(NR) methods in digital hearing aids.

9

slide-10
SLIDE 10

DL in Speech Signal Processing

10

➢ X. Lu, Y. Tsao, S. Matsuda and C. Hori, “Speech Enhancement based

  • n Deep Denoising Autoencoder,”

Interspeech 2013.

slide-11
SLIDE 11

Denoising

Clean speech Noise: 2baby Crying Noise: Siren 2baby Crying Original Noisy MMSE (Trandtional-1) KLT (Trandtional-2) DDAE Siren Original Noisy MMSE (Trandtional-1) KLT (Trandtional-2) DDAE

slide-12
SLIDE 12

Clean speech 0.4s Reverberation Original Noisy DDAE(overall) DDAE(0.3s) IDEL 0.7s Reverberation Original Noisy DDAE(overall) DDAE(0.6s) IDEL 1.0s Reverberation Original Noisy DDAE(overall) DDAE(0.9s) IDEL

De-reverbaration

slide-13
SLIDE 13

Evaluation Results (Subject Tests)

13

Clinical trial: 8 CI subjects.

PDNN

slide-14
SLIDE 14

Assistive Technology: Assistive Listening and Speaking Devices (by Dr. Yu Tsao)

14

FM system Hearing aids Cochlear implants

  • Conventional FM systems
  • Smarthear
  • Two modes of Smarthear
  • Current issues
  • Noise reduction (NR)
  • Amplification scheme

(AS)

  • Signal processing of CI
  • Current issues
  • Noise reduction (NR)
slide-15
SLIDE 15

15

  • Surgically implanted device that electrically

stimulates surviving auditory nerve fibers to provide sound for those with severe hearing loss.

  • Over 200,000 users worldwide.
  • FDA approved in 1985, now approved for children as

young as 12 months.

Cochlear Implant (CI)

slide-16
SLIDE 16

CI Device

16

Transmitter coil Speech processor:

  • 1. Microphone.
  • 2. DSP chip.
  • 3. Battery
  • 4. Others…
slide-17
SLIDE 17

Speech Processor Emitte r Receiver with stimulating reference electrode and

CI Device

slide-18
SLIDE 18

1961 Traveling wave theory

Von Békésy, Georg (1960). Experiments in

  • hearing. Ed. Ernest Glen
  • Wever. Vol. 8. New

York: McGraw-Hill.

CI Device

slide-19
SLIDE 19

Bandpass and Envelope Extraction

(A) Waveform of the word “human” spoken by a native American speaker. (B) Spectrogram of the same word. (C) Green lines: Output of a set of six bandpass filters in response to the same word. The filter spacing and bandwidth in this example are two- thirds of an octave.

slide-20
SLIDE 20

A Critical Issue of CI

  • The tremendous progress of CI technologies in the past three

decades has enabled many CI users to enjoy high level of speech understanding in quiet.

  • For most CI users, however, the performance of speech

understanding in noise still remains challenging.

  • Deep learning based speech enhancement (SE) for CI.

➢F. Chen, Y. Hu, and M. Yuan, “Evaluation of Noise Reduction Methods for Sentence Recognition by Mandarin-Speaking Cochlear Implant Listeners,” Ear and hearing, vol. 36, no. 1, pp. 61-71, 2015. 20

slide-21
SLIDE 21

Signal Processing of CI

21

Speech processor

Microphone Transmitter skin Receiver Electric array

BPF 1 RECT.

LPF

COMP. BPF 2 RECT.

LPF

COMP. BPF n RECT.

LPF

COMP. E 1 Microphone Band-pass filter Envelope detection Compression Pulse generation E 2 E n Electrodes

Artificial Neural Network Speech Enhancement

slide-22
SLIDE 22

DDAE on CI

22

DDAE

slide-23
SLIDE 23

23

Clean 2T Noise 0dB MMSE DDAE

Vocoded Speech

slide-24
SLIDE 24

24

Clean Clean Babble Noise 0dB MMSE DDAE

Normal Speech

slide-25
SLIDE 25

Evaluation Results (Simulations and Subject Tests)

25

Objective evaluation (NCM) Clinical trial: 9 CI subjects. Vocoder results: 10 normal hearing subjects.

➢ Y.-H. Lai, F. Chen, S.-S. Wang, X. Lu, Y. Tsao, and C.-H. Lee, "A Deep Denoising Autoencoder Approach to Improving the Intelligibility of Vocoded Speech in Cochlear Implant Simulation," IEEE Transactions on Biomedical Engineering. ➢ Y.-H. Lai, Y. Tsao, X. Lu, F. Chen, Y.-T. Su, K.-C. Chen, Y.-H. Chen, L.-C. Chen, P.-H. Li, and C.-H. Lee, "Deep Learning based Noise Reduction Approach to Improve Speech Intelligibility for Cochlear Implant Recipients,” Ear and Hearing.

slide-26
SLIDE 26

Assistive Technology: Assistive Listening and Speaking Devices (by Dr. Yu Tsao)

26

FM system Hearing aids Cochlear implants

  • Conventional FM systems
  • Smarthear
  • Two modes of Smarthear
  • Current issues
  • Noise reduction (NR)
  • Amplification scheme

(AS)

  • Signal processing of CI
  • Current issues
  • Noise reduction (NR)