absolute sound energy Aly Magassouba, Nancy Bertin and Francois - PowerPoint PPT Presentation

Audio-based robot control from inter-channel level difference and absolute sound energy Aly Magassouba, Nancy Bertin and Francois Chaumette IROS 2016 Presented by Youngki Kwon

Motivation ● Goal : Positioning the robot with respect to the sound source at given distance and orientation. 2

Motivation ● Goal : Positioning the robot with respect to the sound source at given distance and orientation. 3

Motivation ● Goal : Positioning the robot with respect to the sound source at given distance and orientation. How? Audio-based robot control 4

Audio-based robot control ● Real-time robot control to use sound! ● Aural Servo ● Techniques which uses feedback information extracted from a aural sensor to control the motion of a robot 5

Geometric configuration ● Consider that the sound source is in front side of the robot ● 𝑌 𝑡 : omnidirectional sound source ● 𝑁 1 , 𝑁 2 : microphones ● 𝑁 : midpoints of microphones ● 𝑆 : center of robot 𝑟 ( 𝑣, 𝑥 ): control input (2-DOF) ● ሶ ● 𝑣: velocity along 𝑧 𝑆 ● 𝑥: angular velocity around 𝑨 𝑆 6

ሶ ሶ ሶ ሶ ሶ General framework ● The aim is to minimize an error 𝑓 𝑢 𝑓 𝑢 = 𝑡 𝑢 − 𝑡 ∗ 𝑟 + 𝐾 𝑡 Micro phone 𝑟 = −𝜇෢ + 𝑓 𝑡 = 𝑀 𝑡 𝑤, 𝑡 = 𝐾 𝑡 ሶ 𝑟 = 𝑀 𝑡 𝐾 𝑠 ሶ 𝑟, 𝐾 𝑡 𝑡 𝑢 : acoustic feature (e.g. ILD, Sound energy) 𝑡 ∗ : desired acoustic feature + : Pseudo-inverse of interaction matrix 𝐾 𝑡 𝑟 : control input 7

ILD based Aural Servo ● Inter-channel Level Difference (ILD) ● Ratio of two amounts of energy received by each microphones ● ILD 𝜍 is approximated as 𝜍 > 1 𝜍 = 1 𝜍 < 1 8

ILD based Aural Servo ● Approximating position of sound source 𝑌 𝑡 ● Condition : 9

ሶ ILD based Aural Servo ● Inter-channel Level Difference (ILD) ● Ratio of two amounts of energy received by each microphones 2 𝑚 2 ● ILD 𝜍 is approximated as ൗ 2 𝑚 1 𝑓 𝑢 = 𝜍 𝑢 − 𝜍 ∗ 𝜍 ∗ 𝑟 + 𝐾 𝜍 𝜍 Micro phone 10

ILD accuracy limitation ● When robot is far from sound source 2 = 𝑚 1 2 + (2𝑦 𝑡 ) 2 ● 𝑚 2 ● 𝑚 → ∞, 𝑚 1 ≈ 𝑚 2 , 𝜍 → 1 𝒀 𝒕 ● ILD 𝜍 is not significant anymore 𝒎 𝟑 𝒎 𝟐 𝒎 𝒚 𝒕 𝑵 𝟐 𝑵 𝟑 𝑵 11

ILD accuracy limitation ● When reverberation time is high ● Each microphones 𝑁 𝑗 perceive an additional energy from virtual sound sources ● Make error 𝑓 biased 12

ILD accuracy limitation Accuracy is high only near by the sound source 13

+ Absolute sound energy ● Control the distance to the sound source by setting a desired energy level ● Reverberation has a minor effect when robot is nearby the sound source 14

ሶ ILD + Absolute sound energy ● The ILD constrains the orientation of the microphones while the distance is constrained by the energy level 𝑓 𝑢 = (𝜍, 𝐹 𝑁 ) 𝑢 − (𝜍, 𝐹 𝑁 ) ∗ (𝜍, 𝐹 𝑁 ) ∗ 𝑟 + 𝐾 𝜍𝐹 𝜍, 𝐹 𝑁 Micro phone 15

Experiment ● Pioneer 3DX + Two omnidirectional microphones ● Three experiment scenarios ● Typical positioning tasks ● Long range navigation ● Cooperative application 16

Typical positioning tasks ● 𝑆𝑈 60 ≈ 580𝑛𝑡, 𝑇𝑂𝑆 = 20𝑒𝐶, static sound source ● Desired acoustic feature : 𝑚 ∗ = 80𝑑𝑛, 𝜍 ∗ = 1 17

Typical positioning tasks ● 𝑆𝑈 60 ≈ 580𝑛𝑡, 𝑇𝑂𝑆 = 20𝑒𝐶, dynamic sound source ● Desired acoustic feature : 𝑚 ∗ = 80𝑑𝑛, 𝜍 ∗ = 1 18

Long range navigation 19

Cooperative application ● UAV led the unicycle ground robot by the sound from the propellers 20

Summary ● Audio-based robot control ● Techniques which uses feedback information extracted from a aural sensor to control the motion of a robot ● Two acoustic features ● Inter-channel Level Difference (ILD) ● Finding direction of source ● Sound Energy ● Control distance to source 21