Daedalus: Summer 2017 Internship Project Final Report Ashish - - PowerPoint PPT Presentation

Ashish Tondwalkar

August 3, 2017

Daedalus: Summer 2017 Internship Project Final Report

Who am I?

Senior - TJHSST Class of 2018 Hobbyist, Chess player, Game Enthusiast

• Snr. Research Plan: Locate an object's position using Sound

Interning @ USC-ISI under Dr. Andrew Schmidt to research Microphone Array Beamforming for Sound Source Localization - Project DAEDALUS

Daedalus

Overview: Research methods to improve accuracy of localization Possible applications:

• Military:
• Locating long range sharp shooters
• Inform squads of firefight locations

Possible areas to research accuracy:

• Microphone Distance
• Array Shape
• Microphone Count

Setup

4

• Pynq - Z1 Board
• Jupyter to run python
• OLED Display to output status & data
• Microphones in triangle configuration
• Servo placed in the middle to provide direction
• Speaker to calibrate microphone recordings

PYNQ OLED MIC0 MIC1 MIC2 Speaker Servo

Project Plan

5

• 1. Time Sound thresholds
• 2. Record Small intervals of sound from microphones
• 3. Convert data to manipulatable format
• 4. Filter Noise by Butterworth
• 5. Calibrate Recordings using speaker sound
• 6. Cross Correlation for Time Phase Delay
• 7. Direction: Plug Time Differences into Matrix Equation &

retrieve unit vector components

• 8. Distance: Plug Time Differences into Algebraic Equation &

retrieve estimated circle radius

• 9. Output Vector Data through display and servos

DirecEon

6

Given:

• Position Vectors of Microphones
• Sound time delay for each Microphone

Unknown:

• Direction of Sound

DirecEon

7

Distance

8

Curviness of arc estimated by registering time delay of microphone input

ImplementaEon

9

ImplementaEon

10

Issues:

• Eq Ax = b requires A to be square and

Invertible

• Cannot Implement more than 1+D Microphones
• Solving using SciPy augmented matrix yields incorrect answer; does not

like floats On average solution requires less than 1 ms to calculate

SimulaEng SoluEon

11

Reconfiguring Board

12

• Change System Block Design for more microphones, removing unnecessary

blocks, reassigning port inputs

• Made mistakes in top level source file -> Vivado

taking 30 minutes every time to tell me I made a mistake…

• Messed up address assignment which forced

me to redownload original source files and redo necessary changes Issues:

• Mic Input is very sharp; requires load sounds
• IOB issues?
• Mics are bad?
• Pull Down in constraints file?
• Misinterpretation of Design?

ReformaMng & BuOerworth

• PDM -> PCM
• ~30 seconds per 1 second clip
• Bu>erworth centers audio
• ~10 seconds per 1 second clip
• ~2 Minutes to reformat 3 recordings of 1 second

13

AlternaEves

• Using raw data from audio buffer
• 192000 Hz sampling rate!
• Manipulate FFT Array to filter audio
• iFFT and continue to GCC

14

RAW DATA FFT iFFT

Parallel Recording

• Did not use hardware to record in sync
• Python Threading to record in parallel; not in sync

Calibration Solution:

• Record speaker in center of mic array initially, then make sound
• Realign recording using GCC of speaker recording segment
• GCC modified recordings

15

Speaker initially played to detect

• ffset of mic

recordings Segment of Recording to detect target’s sound

Threading

16

Speaker Placement

• Microphones located at 3 vertices of a square
• Speaker Placed in center of square
• Speaker Pointed upward for even sound propagation

17

Hardware RestricEon

• Microphones not very sensitive, cannot pickup conversational

speech

• Microphones not omnidirectional

18

NOT ALIGNED! Speaker playing twice Calibration Testing Calibration without all microphones pointing directly at speaker

Hardware RestricEon

• Audio Segments of speaker played twice are better aligned

when microphones are pointed toward the speaker

• Microphones not omnidirecEonal & not very sensiEve through

causes inaccuracies in posiEon calculaEons

19

Calibration

CalibraEon

20

GCC Time Difference Shifting Recording Arrays

Servo Output

• PWM signal
• Duty Cycle Determines Position
• Smaller Period -> larger range for duty cycle -> better

precision

• Period of 2 ms
• Angle: [0, π/2] ~ Duty: (21%, 55%)

21

OLED Output

• Status of the Program
• Displays which Phase program is in
• Target Position Output

22

Conclusion

23

Project Conclusion:

• Proof of concept completed
• Demonstration of Project available
• Able to indicate direction with fair accuracy
• Final Program ~ 400 Lines of Python
• Total Notebook Code ~ 1800 Lines of Python

Areas for further improvement:

• Implementing simultaneous recording through hardware
• Higher quality omnidirectional microphones
• Find a method to improve distance estimation
• Research accuracy improvement

Conclusion

24

Some of what I have gained from the internship:

• Learned Python and Jupyter
• Major experience in using an FPGA
• Far better understanding of Verilog and Vivado
• Beginner level research experience
• Interest in pursuing Research over Industry
• Had Fun

Senior Research

• Larger budget will allow for better microphones
• More time will allow me to implement hardware enabled

recording Build Up from Internship Project

• Sound bar mimicry
• Time-delay Speaker Beam forming to produce stereo

sound at target location

25

Sources

26

• W. Jiang, Z. Cai, M. Luo, and Z. Yu. A Simple Microphone Array For Source

Direction and Distance Estimation.

• J. Valin, F. Michaud, J. Rouat, and D. Letourneau. Robust Sound Source

Localization Using a Microphone Array on a Mobile Robot.

• N. Kwok, J. Buchholz, G. Fang, and J. Gal. Sound Source Localization: Microphone

Array Design and Evolutionary Estimation.

• M. Omologo, and P. Svaizer. Use of the Crosspower-Spectrum Phase in Acoustic

Event Location. Special Thanks to:

• Dr. Schmidt

Other RCG Members