Noise Reduction in Robot Audition Tanja Flemming University of - - PowerPoint PPT Presentation

▶
noise reduction in robot audition
SMART_READER_LITE
LIVE PREVIEW

Noise Reduction in Robot Audition Tanja Flemming University of - - PowerPoint PPT Presentation

Oct 06, 2023 423 likes •705 views

MIN Faculty Department of Informatics Noise Reduction in Robot Audition Tanja Flemming University of Hamburg Faculty of Mathematics, Informatics and Natural Sciences Department of Informatics Technical Aspects of Multimodal Systems 16.

slide-1
SLIDE 1

MIN Faculty Department of Informatics

Noise Reduction in Robot Audition

Tanja Flemming

University of Hamburg Faculty of Mathematics, Informatics and Natural Sciences Department of Informatics Technical Aspects of Multimodal Systems

  • 16. December 2019
  • T. Flemming – Noise Reduction in Robot Audition

1 / 27

slide-2
SLIDE 2

Outline

Introduction Approaches Evaluation Conclusion

  • 1. Introduction

Motivation Basics

  • 2. Approaches

Dictionary based Matrix Factorization

  • 3. Evaluation
  • 4. Conclusion
  • T. Flemming – Noise Reduction in Robot Audition

2 / 27

slide-3
SLIDE 3

What is Robot Audition?

Introduction Approaches Evaluation Conclusion

[MHP19]

  • T. Flemming – Noise Reduction in Robot Audition

3 / 27

slide-4
SLIDE 4

Use Cases

Introduction Approaches Evaluation Conclusion

[WC16] [MYM+18] [PRS+14] [NAO]

  • T. Flemming – Noise Reduction in Robot Audition

4 / 27

slide-5
SLIDE 5

Main Challenges in Robot Audition

Introduction Approaches Evaluation Conclusion

◮ Real-time processing ◮ Robustness against noise

◮ Background noise ◮ Reverberation ◮ Ego noise

  • T. Flemming – Noise Reduction in Robot Audition

5 / 27

slide-6
SLIDE 6

Signal Processing Basics

Introduction Approaches Evaluation Conclusion

1D signal characterization: ◮ Amplitude ◮ Phase ◮ Frequency spatial spectral

  • T. Flemming – Noise Reduction in Robot Audition

6 / 27

slide-7
SLIDE 7

Local Analysis

Introduction Approaches Evaluation Conclusion

[SDK16]

  • T. Flemming – Noise Reduction in Robot Audition

7 / 27

slide-8
SLIDE 8

Typical Sound Field

Introduction Approaches Evaluation Conclusion

[SLK18]

  • T. Flemming – Noise Reduction in Robot Audition

8 / 27

slide-9
SLIDE 9

Presented Approaches

Introduction Approaches Evaluation Conclusion

◮ Dictionary based:

Ego-Noise Reduction Using a Motor Data-Guided Multichannel Dictionary Alexander Schmidt1, Antoine Deleforge2 and Walter Kellermann1 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

◮ Matrix Factorization:

Multichannel Nonnegative Matrix Factorization for Ego-Noise Suppression Thomas Haubner1, Alexander Schmidt1 and Walter Kellermann1 2018, Speech Communication; 13th ITG-Symposium

1Friedrich-Alexander University, Erlangen-Nürnberg 2INRIA center of Rennes, France

  • T. Flemming – Noise Reduction in Robot Audition

9 / 27

slide-10
SLIDE 10

Strategy - Dictionary based Approach

Introduction Approaches Evaluation Conclusion

◮ Capture characteristics of ego noise ◮ Save prototype signals (atoms) in dictionaries ◮ Associate motor data to atoms ◮ Noise removal by subtracting atoms

  • T. Flemming – Noise Reduction in Robot Audition

10 / 27

slide-11
SLIDE 11

Approach: Starting Point

Introduction Approaches Evaluation Conclusion

Adapted from [SDK16]

  • T. Flemming – Noise Reduction in Robot Audition

11 / 27

slide-12
SLIDE 12

Motor Data - Atom Association

Introduction Approaches Evaluation Conclusion

[SDK16]

  • T. Flemming – Noise Reduction in Robot Audition

12 / 27

slide-13
SLIDE 13

Adaption

Introduction Approaches Evaluation Conclusion

Adapted from [SDK16]

  • T. Flemming – Noise Reduction in Robot Audition

13 / 27

slide-14
SLIDE 14

Presented Approaches

Introduction Approaches Evaluation Conclusion

◮ Dictionary based:

Ego-Noise Reduction Using a Motor Data-Guided Multichannel Dictionary Alexander Schmidt1, Antoine Deleforge2 and Walter Kellermann1 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

◮ Matrix Factorization:

Multichannel Nonnegative Matrix Factorization for Ego-Noise Suppression Thomas Haubner1, Alexander Schmidt1 and Walter Kellermann1 2018, Speech Communication; 13th ITG-Symposium

1Friedrich-Alexander University, Erlangen-Nürnberg 2INRIA center of Rennes, France

  • T. Flemming – Noise Reduction in Robot Audition

14 / 27

slide-15
SLIDE 15

Strategy - Matrix Factorization Approach (MNMF)

Introduction Approaches Evaluation Conclusion

◮ Goal: Separate target source from noise ◮ Approximate signal with basis and activation matrices ◮ Minimize difference between original and approximated signal ◮ Assign bases to noise or speech ◮ Reconstruct speech signal

[MIM+17]

  • T. Flemming – Noise Reduction in Robot Audition

15 / 27

slide-16
SLIDE 16

General MNMF Approach

Introduction Approaches Evaluation Conclusion

[SKAU13]

blue: single-channel NMF red: multichannel NMF

  • T. Flemming – Noise Reduction in Robot Audition

16 / 27

slide-17
SLIDE 17

Adaption

Introduction Approaches Evaluation Conclusion

  • 1. Learn ego noise model
  • 2. On input signal:

2.1 Add bases and transfer matrices to model 2.2 Minimize difference to real signal 2.3 Assign bases to noise resp. speech 2.4 Reconstruct speech signal

[HSK18]

  • T. Flemming – Noise Reduction in Robot Audition

17 / 27

slide-18
SLIDE 18

Evaluation Experiment

Introduction Approaches Evaluation Conclusion

Adapted from [HSK18]

  • T. Flemming – Noise Reduction in Robot Audition

18 / 27

slide-19
SLIDE 19

Results

Introduction Approaches Evaluation Conclusion

Dictonary based

[SDK16]

Matrix Factorization

[HSK18]

SIR: Signal-to-Inference-Ratio SDR: Signal-to-Distortion-Ratio

  • T. Flemming – Noise Reduction in Robot Audition

19 / 27

slide-20
SLIDE 20

Conclusion

Introduction Approaches Evaluation Conclusion

Dictionary based ◮ Good noise suppression ◮ Fast execution on input signal ◮ Complex training is needed Matrix Factorization ◮ Stronger noise suppression ◮ Minimization for every incoming signal required ◮ Complex training is needed

  • T. Flemming – Noise Reduction in Robot Audition

20 / 27

slide-21
SLIDE 21

Thank you for your attention. Do you have any questions?

  • T. Flemming – Noise Reduction in Robot Audition

21 / 27

slide-22
SLIDE 22

References

Introduction Approaches Evaluation Conclusion

[HSK18]

  • T. Haubner, A. Schmidt, and W. Kellermann,

Multichannel nonnegative matrix factorization for ego-noise suppression, Speech Communication; 13th ITG-Symposium, Oct 2018, pp. 1–5. [MHP19] Mauricio Matamoros, Karin Harbusch, and Dietrich Paulus, From commands to goal-based dialogs: A roadmap to achieve natural language interaction in robocup@home, RoboCup 2018: Robot World Cup XXII (Cham) (Dirk Holz, Katie Genter, Maarouf Saad, and Oskar von Stryk, eds.), Springer International Publishing, 2019, pp. 217–229.

  • T. Flemming – Noise Reduction in Robot Audition

22 / 27

slide-23
SLIDE 23

References (cont.)

Introduction Approaches Evaluation Conclusion

[MIM+17] Narumi Mae, Masaru Ishimura, Shoji Makino, Daichi Kitamura, Nobutaka Ono, Takeshi Yamada, and Hiroshi Saruwatari, Ego noise reduction for hose-shaped rescue robot combining independent low-rank matrix analysis and multichannel noise cancellation, Latent Variable Analysis and Signal Separation (Cham) (Petr Tichavský, Massoud Babaie-Zadeh, Olivier J.J. Michel, and Nadège Thirion-Moreau, eds.), Springer International Publishing, 2017, pp. 141–151.

  • T. Flemming – Noise Reduction in Robot Audition

23 / 27

slide-24
SLIDE 24

References (cont.)

Introduction Approaches Evaluation Conclusion

[MYM+18] Narumi Mae, Koei Yamaoka, Y Mitsui, Mitsuo Matsumoto, Shoji Makino, Daichi Kitamura, Nobutaka Ono, T Yamada, and Hiroshi Saruwatari, Ego noise reduction and sound localization adapted to human ears using hose-shaped rescue robot, Proc. International Workshop on Nonlinear Circuits, Communications and Signal Processing, 2018,

  • pp. 371–374.

[NAO] Picture of NAO robot from SoftBank Robotics, Accessed: 19.12.2019.

  • T. Flemming – Noise Reduction in Robot Audition

24 / 27

slide-25
SLIDE 25

References (cont.)

Introduction Approaches Evaluation Conclusion

[PRS+14]

  • S. Park, J. Rho, M. Shin, D. K. Han, and H. Ko,

Acoustic feature extraction for robust event recognition on cleaning robot platform, 2014 IEEE International Conference on Consumer Electronics (ICCE), Jan 2014, pp. 145–146. [SDK16]

  • A. Schmidt, A. Deleforge, and W. Kellermann,

Ego-noise reduction using a motor data-guided multichannel dictionary, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct 2016, pp. 1281–1286.

  • T. Flemming – Noise Reduction in Robot Audition

25 / 27

slide-26
SLIDE 26

References (cont.)

Introduction Approaches Evaluation Conclusion

[SKAU13]

  • H. Sawada, H. Kameoka, S. Araki, and N. Ueda,

Multichannel extensions of non-negative matrix factorization with complex-valued data, IEEE Transactions on Audio, Speech, and Language Processing 21 (2013), no. 5, 971–982. [SLK18]

  • A. Schmidt, H. W. Löllmann, and W. Kellermann, A

novel ego-noise suppression algorithm for acoustic signal enhancement in autonomous systems, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), April 2018,

  • pp. 6583–6587.
  • T. Flemming – Noise Reduction in Robot Audition

26 / 27

slide-27
SLIDE 27

References (cont.)

Introduction Approaches Evaluation Conclusion

[WC16]

  • L. Wang and A. Cavallaro, Ear in the sky: Ego-noise

reduction for auditory micro aerial vehicles, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Aug 2016,

  • pp. 152–158.
  • T. Flemming – Noise Reduction in Robot Audition

27 / 27