[PPT] - Gradient Flow Source Separation and Localization Gert Cauwenberghs PowerPoint Presentation

SLIDE 1

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Source Separation and Localization

Gert Cauwenberghs

Johns Hopkins University gert@jhu.edu 520.776 Learning on Silicon

http://bach.ece.jhu.edu/gert/courses/776

SLIDE 2

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Source Separation and Localization

Introduction

– Spatial diversity in array signal processing – Directional hearing at sub-wavelength scale

Broadband Localization and Separation

– From delays to temporal derivatives –

Gradient Flow

– Equivalent static linear ICA problem – Multipath extension and convolutive ICA

Performance Analysis

– Scaling properties – Cramer-Rao bounds – Differential sensitivity

Bearing Estimation

– Micropower mixed-signal VLSI implementation – Experimental GradFlow/ ASU acoustic bearing estimation

Independent Component Analysis

– Micropower mixed-signal VLSI implementation – Experimental acoustic source separation

Hearing Aid Implications

SLIDE 3

G. Cauwenberghs

520.776 Learning on Silicon

Blind Separation and Beamforming Localization

Modeling

– Source signals propagate as traveling waves – Spatially diverse sensor array receives linear mixtures of time- delayed sources – The time delays determine the direction coordinates of the waves relative to the sensor geometry

Methods

– Super-resolution techniques estimate the time delays in the spectral domain, assuming narrowband sources – J

int estimation of multiple broadband sources and their time

delays is possible in an extended ICA framework, but requires non-convex optimization leading to unpredictable performance

SLIDE 4

G. Cauwenberghs

520.776 Learning on Silicon

Biomechanics of Tympanal Directional Hearing

– Parasitoid fly localizes sound- emitting target (cricket) by a beamforming acoustic sensor

f dimensions a factor 100

smaller than the wavelength. – Tympanal beamforming organ senses acoustic pressure gradient, rather than time delays, in the incoming wave

Robert, D., Miles, R.N. and Hoy, R.R., “Tympanal hearing in the sarcophagid parasitoid fly Emblemasoma sp.: the biomechanics of directional hearing,” J. Experimental Biology, v. 202, pp. 1865- 1876, 1999.

SLIDE 5

G. Cauwenberghs

520.776 Learning on Silicon

Directional Selectivity in Hearing Aids

www.oticon.com

Two microphones allow for one null angle in directionality

pattern

Adaptive beamforming allows to steer the null to noise source
Presence of multiple noise sources requires source

localization and separation with multiple microphones

SLIDE 6

G. Cauwenberghs

520.776 Learning on Silicon

Wave Propagation

Traveling wave (e.g., acoustic, sonar, RF, …) in free space: In the far field limit:

)) ( ( ) ( ) , ( r r r τ + = t s A t S u r r r ⋅ = ≡

c

A

1

) ( 1 ) ( τ

r u

τ(r)

source sensor

SLIDE 7

G. Cauwenberghs

520.776 Learning on Silicon

Temporal Series Expansion

K & & & + + + = + ) ( ) ( ) ( ) ( ) ( )) ( (

2 2 1

t s t s t s t s r r r τ τ τ

delay 0th-order 1st-order 2nd-order 3rd-order 4th-order

– Reduces the problem of identifying time delayed source mixtures to that of separating static mixtures of the time-differentiated sources – Implies sub- wavelength geometry of the sensor array

SLIDE 8

G. Cauwenberghs

520.776 Learning on Silicon

Spatial Sensing

Sensor distribution:

e.g., for a planar sensor geometry:

– sensor array:

p, q discrete

– distributed sensor: p, q continuous

Source delays:

with: the direction coordinates of source relative to sensor geometry

2 1

r r r q p

pq

+ =

r u

τ(r)

sensor array

r1 r2 p q

sensor

2 1

τ τ τ q p

pq

+ = u r u r ⋅ = ⋅ =

2 1 2 1 1 1 c c

τ τ

SLIDE 9

G. Cauwenberghs

520.776 Learning on Silicon

Wave Flow: Spatial and Temporal Gradients

Linear flow:

Sensor signals: Gradients:

Higher-order flow:

K & + + + = + = ) ( ) ( ) ( ) ( ) (

2 1

t s q p t s t s t x

pq pq

τ τ τ

) ( ) ( ) (

2 01 1 10 00

| |

|

t s q x t s p x t s x

q p pq q p pq q p pq

& & τ ξ τ ξ ξ = ∂ ∂ = = ∂ ∂ = = =

= = = = = =

) ( 1

2 1

t s & ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ = τ τ ) ( ) ( ) (

) (

|

t s x

j i j i pq j i + +

= ∂ = τ τ ξ

2 1

q p

q p j i ij = =

∂ ∂

SLIDE 10

G. Cauwenberghs

520.776 Learning on Silicon

Miniature Sensor Arrays

Finite-difference gradient approximation on a grid:

e.g., planar array of 4 sensors:

) ( ) ( ) ( ) ( ) ( ) (

2 01 1 , 1 , 2 1 1 10 , 1 , 1 2 1 00 1 , 1 , , 1 , 1 4 1

t s x x t s x x t s x x x x & & τ ξ τ ξ ξ ≈ ≈ − ≈ ≈ − ≈ ≈ + + +

− − − −

ξ00 ξ10

1cm

SLIDE 11

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Localization

s(t)

τ2 τ1

) ( τ + t s

τ1 τ2 t

+

+ + + + +

) (t s & τ

[ ]

) ( ) ( ) (

2 01 1 10 00

t s t s t s

dt d

& & & & τ ξ τ ξ ξ ≈ ≈ ≈

τ2 τ1

00

ξ &

ξ01 ξ10

t

Gradient flow bearing resolution is fundamentally independent of

aperture

Resolution is determined by sensitivity of gradient acquisition

– Mechanical differential coupling (Miles et al.) – Optical differential coupling (Degertekin) – Analog VLSI differential coupling

SLIDE 12

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Localization and Separation

s(t)

τl2 τl1

) ( τ + t s

τl

1

τl

2

+

+ + + + +

) (t s & τ

[ ]

∑ ∑ ∑

≈ ≈ ≈

l l l l l l l l

& & & & ) ( ) ( ) (

2 01 1 10 00

t s t s t s

dt d

τ ξ τ ξ ξ

τl2 τl1

00

ξ &

ξ01 ξ10

t t

Gradient flow bearing resolution is fundamentally independent of

aperture

Resolution is determined by sensitivity of gradient acquisition

– Mechanical differential coupling (Miles et al.) – Optical differential coupling (Degertekin) – Analog VLSI differential coupling

Multiple target tracking with independent component analysis (ICA)

SLIDE 13

G. Cauwenberghs

520.776 Learning on Silicon

Separation and Localization

Source mixtures are observed with additive sensor noise: Gradient flow reduces to a static (noisy) mixture problem: solved by means of linear static ICA

) ( ) ( ) (

1

t n t s t x

pq L pq pq

+ + =∑

= l l l

τ

n s A x + = ↓ ↓ ↓ ↓ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ + ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡ = ⎥ ⎥ ⎥ ⎦ ⎤ ⎢ ⎢ ⎢ ⎣ ⎡

01 10 00 1 2 1 2 1 1 1 01 10 00

) ( ) ( 1 1 ν ν ν τ τ τ τ ξ ξ ξ & & M & L L L & t s t s

L L L

direction vectors sources

(time-differentiated)

bservations

(gradients)

noise

(gradients)

SLIDE 14

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Acoustic Separation

Outdoors Environment

– 4 microphones within 5 mm radius – 2 male speakers at 0.5 m, lawn surrounded by buildings at 30 m

1cm

SLIDE 15

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Acoustic Separation

Indoors Environment

– 4 microphones within 5 mm radius – 2 male speakers at 0.5 m, reverberant room of dimensions 3, 4 and 8 m

1cm

SLIDE 16

G. Cauwenberghs

520.776 Learning on Silicon

Multipath Wave Propagation

Multipath convolutive wave expansion: In the far field:

)) , , ( ( ) , , ( ) , ( θ τ θ θ θ u r u r u r + − = ∫∫ t s A d d t S u r u r u r u u r ⋅ = ≡ ≡

c

A A

1

) , ( ) , , ( ) , ( ) , , ( τ θ τ θ θ

r u

τ(r,u)

source sensor

θ

path time lag path direction

SLIDE 17

G. Cauwenberghs

520.776 Learning on Silicon

Multipath Gradient Flow Separation and Localization

Gradient Flow, uniformly sampled above the Nyquist rate: yields a mixing model of general convolutive form: with moments of multipath distributions over sensor geometry:

] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [

01 2 01 10 1 10 00 00

i j i s j i i j i s j i i j i s j i

j j j

ν τ ξ ν τ ξ ν α ξ + − ≈ + − ≈ + − ≈

∑ ∑ ∑ ∑ ∑ ∑

l l l l l l l l l

& & & & &

] [ ] [ ] [ ] [ i j i j i

j

n s A x + − ⋅ =∑

∫ ∫ ∫ ∫ ∫ ∫

− − = − − = − − =

= = =

u u u

u r u u u r u u u u

s s s s s s

T n T n T n T n T n T n

A d d j A d d j A d d j

) ( ) ( 2 2 ) ( ) ( 1 1 ) ( ) (

2 1 2 1 2 1 2 1 2 1 2 1

) , ( ) , ( ] [ ) , ( ) , ( ] [ ) , ( ] [

θ θ θ

τ θ θ τ τ θ θ τ θ θ α

l l l l l l

r u

τ(r,u)

sensor array

r1 r2 p q

sensor

u r⋅ = c

1

SLIDE 18

G. Cauwenberghs

520.776 Learning on Silicon

Scaling Properties

Order k, dimension m: Maximum separable number of sources Lmax:

h ij L h j i h n j i h ij

t s

... 1 ) ... ( 2 1 ...

) ( ) ...( ) ( ) ( ν τ τ τ ξ + ≈∑

= + + + l l l l l

}

m ≤ k

n s A x + ⋅ = k m

1 2 3 1 1 1 1 1 1 2 3 4 2 1 3 6 10 3 1 4 10 20

– Assumes full-rank A with linearly independent mixture combinations – Depends on the geometry of the source direction vectors relative to the array – More sources can be separated in the overcomplete case by using prior information on the sources

SLIDE 19

G. Cauwenberghs

520.776 Learning on Silicon

Noise Characteristics

Mixing model: Signal and bearing estimates: Error covariance:

– Angular directions of the sources (matrix A), besides sensor noise, affect the error variance of the estimated sources. – Determinant of square matrix A measures the volume (area) spanned by the direction vectors. When direction vectors are co- planar (co-linear), error variance becomes singular. – For two sources in the plane with angular separation ∆θ, the error variance scales as 1/ sin2(∆θ).

n s A x + ⋅ = n A s n A s A A x A s

1 1 1 1 − − − −

+ ≈ + ⋅ = = ˆ ˆ ˆ ˆ

bias

}

variance = e

T T T

E E ) ]( [ ] [

1 1

A nn A ee

− −

=

SLIDE 20

G. Cauwenberghs

520.776 Learning on Silicon

Cramer-Rao Lower Bounds on Bearing Estimation

J 1 ≥ ∆θ

Fisher information

⎥ ⎦ ⎤ ⎢ ⎣ ⎡ ∂ ∂ ∂ ∂ − = θ θ θ θ ) ( ) ( E L L J

S

Signal power

N Ambient noise power E Acquisition error power

Aperture

λ π λ π ωτ D r a = = = | | 2

2a2sin2θ S N+E a2S (a2N +E)

1 2

∫

+ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + =

2 2 2

/ 2 1 / 2 a E N S a E N S df T J

Gradient Flow:

01 01 10 10

sin cos ν θ τ ξ ν θ τ ξ + = + = s s & &

ξ10 θ ξ01 D

∫

+ + ⎟ ⎠ ⎞ ⎜ ⎝ ⎛ + = E N S E N S a df T J 2 1 2 sin

2

θ

(Friedlander, 1984)

2 2 1 1

) cos ( ) cos ( n t s x n t s x + − = + + = θ τ θ τ

x1 x2 θ D

Time Delay:

SLIDE 21

G. Cauwenberghs

520.776 Learning on Silicon

Cramer-Rao Lower Bounds on Bearing Estimation

– Conventional:

time delayed

source

uncorrelated

noise

– Gradient:

spatial gradients

(ξ10 and ξ01)

ambient noise is

highly correlated

mechanical or

electrical coupling enhances differential spatial sensitivity

– Further refinements:

non-Gaussian

source statistics

non-stationary

source dynamics

SLIDE 22

G. Cauwenberghs

520.776 Learning on Silicon

Differential Sensitivity

– Cramer-Rao bound on angular precision is fundamentally independent of aperture. – The sensor and acquisition design challenge is to resolve small signal gradients amidst a large common- mode signal pedestal. – Differential coupling eliminates the common mode component and boosts the differential sensitivity by a factor C, the ratio of differential to common mode signal amplitude range. – Signal to acquisition error power ratio S/E is effectively enhanced by the differential coupling factor C. – Mechanical (sensor) and electrical (amplifier) differential coupling can be combined to yield large gain C > 1,000.

D Aperture Signal and Interference Common mode range Differential range

C E N a S a E N a S a

coupling diff

+ ⎯ ⎯ ⎯ ⎯ → ⎯ +

2 2 . 2 2

1cm 1in

SLIDE 23

G. Cauwenberghs

520.776 Learning on Silicon

Adaptive Common-Mode Suppression

Systematic common-mode error in finite-difference gradients:

due to gain mismatch across sensors in the array: can be eliminated using second order statistics only:

Adaptive LMS calibration:

∑ ∑ ∑ ∑ ∑

+ ≈ + ≈ − + ≈ + ≈ − ≈ ≈ + + +

− − − − l l l l l l l l l l l l

& & ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (

2 2 00 2 01 1 , 1 , 2 1 1 1 00 1 10 , 1 , 1 2 1 00 1 , 1 , , 1 , 1 4 1

t s t s x x t s t s x x t s x x x x ε τ ξ ε ξ ε τ ξ ε ξ ξ

01 10 00

ˆ ˆ ˆ ξ ξ ξ ⎩ ⎨ ⎧ = = ⇒ ∀ = ] [ E ] [ E , , )] ( ) ( [ E

01 00 10 00

ξ ξ ξ ξ m t s t s

m

l &l

00 2 00 01 00 01 01 00 2 00 10 00 10 10

ˆ ] ˆ [ E ] ˆ ˆ [ E ˆ ˆ ] ˆ [ E ] ˆ ˆ [ E ˆ ξ ξ ξ ξ ξ ξ ξ ξ ξ ξ ξ ξ − ≈ − ≈

SLIDE 24

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow DSP Frontend

Milutin Stanacevic

– 4 miniature microphones

Knowles IM-3246
100mV/Pa sensitivity (w/ internal preamp)
100Hz-8kHz audio range
0.2mW each

– 2 stereo audio ∆−Σ ADCs

Cirrus Logic CS5333A
24bit, 96kHz
11mW active, 0.2mW standby

– Low-power DSP backend

Texas Instruments TMS320C5204
100MIPS peak
0.32mW/MIPS

– Benchmark, and prototyping testbed, for micropower VLSI miniaturized integration

1in

SLIDE 25

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow System Diagram

.

Digital estimated delays Average, temporal derivative and estimated spatial gradients Spatial gradients with suppressed common-mode Analog inputs

x10 x-10 x01 x0-1

Least Mean Squares (LMS) digital adaptation

– Common mode offset correction for increased sensitivity in the analog differentials – 3-D bearing direction cosines

Analog in, digital out (A/D smart sensor)

SLIDE 26

G. Cauwenberghs

520.776 Learning on Silicon

CDS Differential Sensing

Switched-capacitor, discrete-time analog signal processing – Correlated Double Sampling (CDS)

Offset cancellation

and 1/f noise reduction

– Fully differential

Clock and supply

feedthrough rejection

+ + + + + + + +

[ ]

dt d

+

SLIDE 27

G. Cauwenberghs

520.776 Learning on Silicon

Spatial Gradient Acquisition

] [ ] 2 1 [ ] [

10 10 10

n x n x n − − =

− −

ξ

^

] [ ] 2 1 [ ] [

10 10 10

n x n x n

− +

− − = ξ

^

φ1 φ2 φ1e

Uncompensated spatial finite difference computation Multiplying DAC for common- mode compensation T-cell attenuates

utput swing

SLIDE 28

G. Cauwenberghs

520.776 Learning on Silicon

Mixed-Signal LMS Adaptation

]) [ ] [ ( ] [ ] [

00 1 00 1 10 10

n n n n e

−

−

+

+

+ +

+ − = ξ τ ξ τ ξ

+ −

− − =

1 1

1 2 τ τ

n

]) [ ] [ sgn( ]) [ ] [ sgn( ] [ ] 1 [

00 00 10 10 1 1

n n n e n e n n

−

+
−

+ + +

− − + = + ξ ξ τ τ ]) [ ] [ ( ] [ ] [

00 1 00 1 10 10

n n n n e

−

+

+

−

− −

+ − = ξ τ ξ τ ξ

Two stages

– Common mode compensation – Delay parameter estimation

Sign-sign LMS differential on-line adaptation rule

– Delay parameter estimation :

Digital storage and update of parameter estimates

– 12-bit counter – 8-bit multiplying DAC to construct LMS error signal

SLIDE 29

G. Cauwenberghs

520.776 Learning on Silicon

Gradient Flow Processor

Stanacevic and Cauwenberghs (2003)

LMS REGISTERS LMS REGISTERS MULTIPLYING DAC MULTIPLYING DAC

ξ00 ξ00

.

ξ10 ξ01 τ10 τ01

Digital LMS

adaptive 3-D bearing estimation

Analog

microphone inputs

Digital bearing
utputs
Analog gradient
utputs
8-bit effective

digital resolution

0.5µs at 240Hz

input

3mm x 3mm in

0.5µm 3M2P CMOS

32µW power

dissipation at 10 kHz clock

SLIDE 30

G. Cauwenberghs

520.776 Learning on Silicon

GradFlow Delay Estimation

Sinewave inputs and spatial gradient Digital output - estimated delays

200 Hz synthetic sine wave input

signals

2 kHz sampling frequency
Time delay varied from -400µs to

400µs in 2µs increments

sin(ω t) sin(ω t) sin(ω (t-τ)) sin(ω (t-τ))

SLIDE 31

G. Cauwenberghs

520.776 Learning on Silicon

GradFlow/ASU Localization

Acoustic Surveillance Unit

courtesy of Signal Systems Corporation

One directional source in open-field environment

Band-limited (20-300Hz) Gaussian signal presented through loudspeaker

16cm effective distance between microphones
18m distance between loudspeaker and sensor array

SLIDE 32

G. Cauwenberghs

520.776 Learning on Silicon

Independent Component Analysis

The task of blind source separation (BSS) is to separate and

recover independent sources from mixed sensor observations, where both the sources and mixing matrix are unknown.

A W

s(t)

M N N

x(t) y(t)

Source signals Sensor

bservations

Reconstructed source signals Mixing matrix Unmixing matrix

Independent component analysis (ICA) minimizes higher-order

statistical dependencies between reconstructed signals to estimate the unmixing matrix.

SLIDE 33

G. Cauwenberghs

520.776 Learning on Silicon

ICA System Diagram

System block diagram Cell functionality

Digitally reconfigurable ICA update rule

– Jutten-Herault – InfoMax – SOBI

Digital storage and update of weight coefficients

– 14-bit counter – 8-bit multiplying DAC to construct output signal

SLIDE 34

G. Cauwenberghs

520.776 Learning on Silicon

Example Mixed-Signal ICA Implementation

Implemented ICA update rule :

– Corresponds to the feed-forward version of the Jutten-Herault network. – Implements the ordinary gradient of the InfoMax cost function, multiplied by WT.

For source signals with Laplacian probability density, the

distribution optimal function f(y) is sign(y), implemented with a single comparator.

The linear term in the output signal in the update rule is

quantized to two bits.

) ) ( ( ] [ ] 1 [

T

y y f I n W n W − + = + µ

SLIDE 35

G. Cauwenberghs

520.776 Learning on Silicon

ICA VLSI Processor

Celik, Stanacevic and Cauwenberghs (2004)

3 inputs – sensor

signals or gradient flow signals

3 outputs –

estimated sources

14-bit digital

estimates of unmixing coefficients

3mm x 3mm in

0.5µm CMOS

180µW power

consumption at 16kHz

W31 W32 W33 W21 W22 W23 W11 W12 W13

S/H OUTPUT BUFFERS ICA REGISTERS MULTIPLYING DAC

SLIDE 36

G. Cauwenberghs

520.776 Learning on Silicon

ICA Experimental Results

Two mixed speech signals

presented at 16kHz

InfoMax ICA implemented

in VLSI

30dB separation in this

case

SLIDE 37

G. Cauwenberghs

520.776 Learning on Silicon

Hearing Aid Implications

Gradient flow method provides estimates of three independent

acoustic sources along with the cosines of the angles of arrival.

Directional hearing aids amplify the signals in the front plane

and suppress the signals in the back plane of the microphone array.

Gradient flow offers more flexibility in choice of the signal that

will be amplified and presented to the listener. The signal can be chosen based on the direction of arrival with respect to microphone array or based on the power of the signal. The estimation of independent sources leads to adaptive suppression of number of noise sources independent of their angle of arrival.

SLIDE 38

G. Cauwenberghs

520.776 Learning on Silicon

Conclusions

Wave gradient “flow” converts the problem to that of static ICA,

with unmixing coefficients yielding the direction cosines of the sources.

The technique works for arrays of dimensions smaller than the

shortest wavelength in the sources.

Localization and separation performance is independent of

aperture, provided that differential sensitivity be large enough so that ambient interference noise dominates acquisition error noise.

High resolution delay estimation for source localization using

miniature sensor arrays and blind separation of artificially mixed signals with reconfigurable adaptation has been demonstrated.

System allows integration with sensor array for small, compact,

battery-operated “smart” sensor applications in surveillance and hearing aids.

SLIDE 39

G. Cauwenberghs

520.776 Learning on Silicon