Privacy Preserving Multi-target Tracking Anton Milan Stefan - - PowerPoint PPT Presentation

▶
privacy preserving multi target tracking
SMART_READER_LITE
LIVE PREVIEW

Privacy Preserving Multi-target Tracking Anton Milan Stefan - - PowerPoint PPT Presentation

Jul 18, 2023 246 likes •637 views

Privacy Preserving Multi-target Tracking Anton Milan Stefan Roth Konrad Schindler Mineichi Kudo Visual People Tracking Applications and Benefits CCTV: Increased safety Automated video analysis Crowd motion estimation

slide-1
SLIDE 1

Privacy Preserving Multi-target Tracking

Anton Milan Stefan Roth Konrad Schindler Mineichi Kudo

slide-2
SLIDE 2
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

2

Visual People Tracking

✔ CCTV: Increased safety ✔ Automated video analysis ✔ Crowd motion estimation ✔ Robotic navigation

Applications and Benefits

slide-3
SLIDE 3
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

3

Visual People Tracking

Drawback: Heavy intrusion of privacy

✔ CCTV: Increased safety ✔ Automated video analysis ✔ Crowd motion estimation ✔ Robotic navigation

Applications and Benefits

slide-4
SLIDE 4
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

4

Existing Alternatives

[Spindler et al., 2006] [Schiff et al., 2009] [Wickramasuriya et al., 2005]

slide-5
SLIDE 5
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

5

Existing Alternatives

[Spindler et al., 2006] [Schiff et al., 2009] Such systems may fail (or be switched off) [Wickramasuriya et al., 2005]

slide-6
SLIDE 6
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

6

Our Approach

  • A different sensor modality
  • Existing multi-target tracking techniques

Pyroelectric infrared sensors* ... ...mounted on a ceiling

*Also known as: Infrared motion sensors

slide-7
SLIDE 7
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

7

The Setup

43 nodes, ca. 3m stride. Total cost: ≈ $100 USD.

slide-8
SLIDE 8
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

8

Tracking with Infrared Sensors

[Hosokawa et al., 2009] [Luo et al., 2009]

  • Expensive sensor array

with Fresnel lenses

A mostly unexplored research area!

  • Limited state space
  • Ad hoc algorithm for data association
slide-9
SLIDE 9
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

9

Benefits

  • Individal identification impossible

– Respects privacy

  • Insensitive to lighting conditions
  • Low cost

Limitations

  • Indoor application only
  • Less flexible
slide-10
SLIDE 10
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

10

Main Challenges

  • Extremely low resolution (43 sensors)
  • A binary response at 2 Hz per sensor
  • No visual (appearance) information
  • Poor localization + sensor noise / delay
  • Activation by several people
  • Multiple measurements by one person
slide-11
SLIDE 11
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

11

Main Challenges

  • Extremely low resolution (43 sensors)
  • A binary response at 2 Hz per sensor
  • No visual (appearance) information
  • Poor localization + sensor noise / delay
  • Activation by several people
  • Multiple measurements by one person
slide-12
SLIDE 12
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

12

Main Challenges

  • Extremely low resolution (43 sensors)
  • A binary response at 2 Hz per sensor
  • No visual (appearance) information
  • Poor localization + sensor noise / delay
  • Activation by multiple people
  • Multiple measurements by one person
slide-13
SLIDE 13
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

13

Main Challenges

  • Extremely low resolution (43 sensors)
  • A binary response at 2 Hz per sensor
  • No visual (appearance) information
  • Poor localization + sensor noise / delay
  • Activation by several people
  • Multiple measurements by one person
slide-14
SLIDE 14
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

14

Continuous Energy Minimization

E X =EobsEdynEexc Eper Ereg

State vector: X,Y-locations of all targets at all frames

X ∈ℝd ,d≈2000

[Milan et al., PAMI 2014]

slide-15
SLIDE 15
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

15

Why Continuous Energy?

  • Continuous trajectories

– low sensor resolution not an issue

  • No implicit data association

– multiple assignments possible

  • Provides best results

– Measured by standard tracking metrics

slide-16
SLIDE 16
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

16

The Energy

E=Eobs+aEdyn+bEexc+cEper+dEreg

data physically-based priors regularizer

dynamics exclusion persistence parsimony

∑

i≠ j

∣ ∣X i−X j∣ ∣

−2

−∑

g

∣ ∣X i−D g∣ ∣

−2

∑

i

∣ ∣vi

t−vi t 1∣

∣

2

N+∑i 1/lengthi

∑

i 1exp1−b X i −1

slide-17
SLIDE 17
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

17

Data Term

lobe size

E

slide-18
SLIDE 18
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

18

Optimization

  • conjugate gradient descent for local optimization
  • discontinuous jumps to determine dimensionality (number of targets)

E(X) X Jump moves Conjugate gradient descent Merge – Split Grow – Shrink Add – Remove

slide-19
SLIDE 19
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

19

Experiments

Synthetic Data

  • Manual assignment of keyframes
  • Interpolation and sensor simulation
slide-20
SLIDE 20
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

20

Measurements

slide-21
SLIDE 21
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

21

Measurements

Time

slide-22
SLIDE 22
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

22

Ground Truth

Time

slide-23
SLIDE 23
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

23

Experiments

Synthetic Data

  • Manual assignment of keyframes
  • Interpolation and sensor simulation

GT Result

slide-24
SLIDE 24
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

24

Result

Time

slide-25
SLIDE 25
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

25

Result

slide-26
SLIDE 26
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

26

Real Data

  • Up to six people in

a large lab

  • Two cameras

(2 Hz)

  • Temporal

alignment

  • Annotation of key

frames (very approximate)

slide-27
SLIDE 27
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

27

Real Data

  • Up to six people in

a large lab

  • Two cameras

(2 Hz)

  • Temporal

alignment

  • Annotation of key

frames (very approximate)

slide-28
SLIDE 28
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

28

Results (real)

GT Result

slide-29
SLIDE 29
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

29

Results (real)

GT Result

slide-30
SLIDE 30
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

30

Other Approaches

[Berclaz et al., PAMI 2011] [Tao et al., Sensors 2012]

slide-31
SLIDE 31
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

31

Quantitative Results

Dataset Method MOTA [%] MOTP [%] ID sw #Targets (MAE) Ours 76.0 73.6 13 0.54 Ours 55.3 54.6 43 0.76 synthetic Real data MOTA = normalized error count MOTP = localization error (73% ≈ 35 cm)

slide-32
SLIDE 32
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

32

Quantitative Results

Dataset Method MOTA [%] MOTP [%] ID sw #Targets (MAE) Ours 76.0 73.6 13 0.54 Linear DA [1] 66.6 64.6 58 0.57 DP [2] 55.9 65.3 57 0.62 KSP [3] 75.5 67.5 6 1.52 Ours 55.3 54.6 43 0.76 Linear DA [1] 9.3 50.1 252 1.00 DP [2] 9.6 47.3 128 1.25 KSP [3] 31.1 48.3 48 1.52

[1] Tao et al., Sensors 2012 [2] Pirsiavsah et al., CVPR 2011 [3] Berclaz et al., PAMI 2014

synthetic Real data

slide-33
SLIDE 33
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

33

Advertisement

slide-34
SLIDE 34
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

34

Advertisement

  • 22 Sequences (old + new)
  • > 1300 Trajectories
  • > 100,000 Bounding boxes
  • Live online evaluation
slide-35
SLIDE 35
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

35

Advertisement

slide-36
SLIDE 36
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

36

Advertisement

http://motchallenge.net

slide-37
SLIDE 37
  • A. Milan et al. | Privacy-Preserving Multi-Target Tracking

37

Summary

  • A principled alternative to preserve privacy
  • Continuous energy with soft assignments
  • Still a very challenging problem
  • Data + Code online

http://research.milanton.net/irtracking/