Distributed localization of networked cameras Stanislav Funiak - - PowerPoint PPT Presentation

1

Distributed localization of networked cameras

Stanislav Funiak Carlos Guestrin

Carnegie Mellon University

Mark Paskin

Stanford University

Rahul Sukthankar

Intel Research

IPSN 2006 presentation, April 19, 2006

2

Distributed Localization of Cameras

Place wireless cameras around an environment Need to know locations Costly to measure locations

3

Distributed Localization of Cameras

If camera 1 sees person, then camera 2 sees person, learn about relative positions of cameras As person moves around, estimate positions of all cameras

4

Localization from pairwise distances

Ihler et al., IPSN 2004 Whitehouse, Culler, ACM WSNA 02 Pollefeys, IJCV 2004 Soatto, Perona, IEEE PAMI 1998

Montemerlo et al., AAAI 2002

Paskin, IJCAI 2003

Simultaneous localization and mapping Structure from motion Simultaneous calibration and tracking

Rahimi et al., CVPR 2004

Prior Work

5

Distributed Localization of Cameras

If camera 1 sees person, then camera 2 sees person, learn about relative positions of cameras As person moves around, estimate positions of all cameras

Want a solution:

• nline
• distributed
• represents

uncertainty about estimated locations e.g. for active control

6

Tracking with Kalman Filter: Estimation

prior distribution Observation model:

1 2

• bject location

camera poses

prior distribution over object location: uncertain

posterior distribution: more certain

• bservation likelihood

posterior distribution

posterior distribution: even more certain

image camera at known pose

previous

• bservations

7

Tracking with Kalman Filter: Prediction

t t+1

motion model Motion model: posterior distribution predicted distribution (prior at t+1)

8

Camera Localization: Estimation

d unknown camera pose

prior distribution

Start with wide prior on C Observe person at dist. d

– Camera could be anywhere in a ring

• bject

location

posterior distribution

• bservation likelihood

Posterior distribution in absolute parameters

camera angle

9

Kalman Filter uses a linear representation…

Exact non-Gaussian posterior Gaussian approximation Gaussian approximation

?

−4 −3 −2 −1 1 2 3 4 −4 −3 −2 −1 1 2 3 4 x y 1 2 3 4 5 6 7 8 9 10 11 12

Far from ground truth Overconfident

Exact posterior in absolute parameters

Problem structure lost with Gaussian approximation

ground truth estimate

10

Ring distribution in polar coordinates – Almost Gaussian!!!

Relative Over-Parameterization (ROP)

Intuition: a ring structure can be represented with polar coordinates Not enough: Camera does not view person head on Relative over-parameterization – position relative to location of person 1. Distance u, angle φ 2. Lateral displacement v 3. The center – the unknown location of object

u φ

(mx, my )

ROP v

φ u

• π

+π

11

Comparing parameterizations

true posterior: best Gaussian

• approx. in x,y,θ:

Standard parameterization u φ

(mx, my )

v ROP

best Gaussian

• approx. in ROP:

12

Test run on Tower Scenario

−4 −3 −2 −1 1 2 3 4 −4 −3 −2 −1 1 2 3 4 x y 1 2 3 4 5 6 7 8 9 10 11 12

standard parameterization

−4 −3 −2 −1 1 2 3 4 −4 −3 −2 −1 1 2 3 4 x y 1 2 3 4 5 6 7 8 9 10 11 12

ROP with further improvements (see paper)

13

Donuts and Bananas on real data – Network of 5 cameras

14

Distributed Localization of Cameras

Goal: each camera estimates the location of itself and the object

• ROP lets us use a single

Gaussian

• Challenges?

1 2 3 4 5 6 7 8

Want algorithm:

• efficient
• robust to message

loss, node loss

15

Motion model introduces dependencies

t t + 1 Estimation at t: Prediction: Estimation at t+1: Motion model introduces dependencies among distant cameras communication and computation inefficiency

16

1 2 3 4 5 6 7 8

Mt, C1, C2 Mt, C3, C4 Mt, C4, C5 Mt, C5, C6 Mt, C6, C7 Mt, C7, C8 Mt, C2, C3

Assumed density filtering

C1, C2 C2, C3 C3, C4 C4, C5 C5, C6 C6, C7 C7, C8

Intuition: only capture strong dependencies among cameras based on [Boyen Koller 1998]

Each clique contains Each clique contains:

• Camera and its neighbor
• Object location

17

• 1. Assign each clique to one
• r more nodes
• can give clique to > 1

node for robustness

• 2. The nodes build a

network junction tree [Paskin et al. 2005]

• build a routing tree
• ensure the flow of

information

Distributed Filtering: Initialization

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

Mt, C1, C2 Mt, C3, C4 Mt, C4, C5 Mt, C5, C6 Mt, C6, C7 Mt, C2, C3 Mt, C2, C3, C4 Mt, C7, C8 Mt, C7, C8 Mt, C4, C5 , C6 Mt , C6, C7, C8

18

• 1. Each node starts with

prior over its clique

• 2. Nodes make
• bservations
• 3. Nodes communicate

relevant likelihoods & priors neighbors

• 4. At convergence:

condition on all measurements made in the network

Distributed Filtering: Estimation

1 2 3 4 5 6 7 8 1 2 3 5 6 7 8

Mt, C1, C2 Mt, C5, C6 Mt, C6, C7 Mt, C2, C3 Mt, C2, C3, C4 Mt, C7, C8 Mt, C4, C5 , C6 Mt , C6, C7, C8 Instance of Robust Distributed Inference [Paskin Guestrin, UAI 2004]

19

weak indirect dependence

Prediction Revisited

motion model posterior distribution prediction

t t+1

How to implement the prediction step distributedly? How to prune weak dependencies?

strong direct dependencies

20

Distributed Filtering: Prediction

Want the best approximation (minimizing KL divergence):

• captures short-range dependencies
• drops long-range dependencies

Sufficient to compute the marginals over cliques

[Boyen, Koller 1998]

21

Summary of our approach

1 2 3 4 5 6 7 8

• 1. Each node maintains a

clique marginal

• 2. Nodes build communication

structure, network junction tree [Paskin et al. 2005]

• 3. Estimation: nodes condition
• n observations

[Paskin & Guestrin UAI 04]

• 4. Prediction:

best approximation computed locally

Mt, C1, C2 Mt, C2, C3 Mt, C3, C4 Mt, C4, C5 Mt, C5, C6 Mt, C6, C7 Mt, C7, C8 Mt, C7, C8

22

Results: 44 simulated side-facing cameras

23

Results: 44 simulated side-facing cameras

24

Network of 25 cameras at Intel Research Pittsburgh

25

Network of 25 cameras at Intel Research Pittsburgh

26

Results: Model Complexity vs. Accuracy

RMS error better

pruning all dependencies dependencies among neighbors keeping all dependencies (exact solution) 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

27

Comparison with Rahimi et al., CVPR 2004

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

RMS error better

pruning all dependencies dependencies among neighbors keeping all dependencies (exact solution) Rahimi et al. CVPR 2004

Our approach:

• distributed
• online
• estimates uncertainty

28

Results: Communication vs. Accuracy

RMS error better centralized solution 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 3 5 10 15 20

epochs per time step

29

Conclusion

• Accurate camera localization with only

a single Gaussian!!!

– ROP – parameterization accurately representing ring-like distributions – Effective technique for incorporating nonlinear

• bservations
• Distributed online algorithm for camera

localization that represents uncertainty

• Algorithm for distributed filtering for

general dynamic models

• Evaluated on network of 25 real cameras