Adaptive Background Mixture Models for Real-Time Tracking Chris - - PowerPoint PPT Presentation

β–Ά
adaptive background mixture
SMART_READER_LITE
LIVE PREVIEW

Adaptive Background Mixture Models for Real-Time Tracking Chris - - PowerPoint PPT Presentation

Mar 12, 2024 915 likes β€’1.16k views

Adaptive Background Mixture Models for Real-Time Tracking Chris Stauffer and W.E.L Grimson CVPR 1998 Brendan Morris http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Motivation Video monitoring and surveillance is a challenging task Must

slide-1
SLIDE 1

http://www.ee.unlv.edu/~b1morris/ecg782/

Adaptive Background Mixture Models for Real-Time Tracking

Chris Stauffer and W.E.L Grimson CVPR 1998 Brendan Morris

slide-2
SLIDE 2

Motivation

  • Video monitoring and surveillance is a

challenging task

  • Must deal with

β–« Cluttered areas, shadows, occlusions, lighting changes, moving elements in scene, slow moving

  • bjects, objects (dis)appear

2

slide-3
SLIDE 3

Standard Practice

  • Use of adaptive background model

β–« 𝐢 𝑦, 𝑧, 𝑒 = 1 βˆ’ 𝛽 𝐢 𝑦, 𝑧, 𝑒 βˆ’ 1 + 𝛽𝐽 𝑦, 𝑧, 𝑒

ο‚– 𝛽 – is the learning rate

  • Strengths: simple and effective of scenes with mostly

background and constantly moving objects

  • Other techniques try to model the background pixels

statistically but cannot deal with bimodal background

β–« Kalman filter to track pixel value and has automatic threshold β–« Gaussian distribution for each pixel used to classify as a background or not

3

slide-4
SLIDE 4

Standard Limitations

  • Weakness: Poor performance

for many slow moving objects, recovers slowly, and uses a single threshold for the entire scene

  • Example of a rainy day

β–« Pixel intensity values over 16 frames (rain occurs halfway through)

ο‚– 139,140,141,141,138,140,140,139 ,240,241,243,244,180,141,140,1 42

β–« Model as two different distributions

4 𝜈2 = 196.37, πœ€2 = 50.43 𝜈1 = 139.75, πœ€1 = 1.22

slide-5
SLIDE 5

Contributions

  • Develop a computationally efficient background

modeling technique

  • Pixel intensity distribution modeled using a

mixture of Gaussians

β–« Able to model arbitrary distributions (e.g. bimodal)

  • Designed an online approximation for

computationally efficient update of model

5

slide-6
SLIDE 6

Background Distribution

  • Single Gaussian distribution is

insufficient for real scenes

  • ver long periods

β–« Mean background assumes a single distribution with the threshold a variance parameter

  • Many scenarios with multiple

values for a pixel

6

Kyungnam Kim, Thanarat H. Chalidabhongse, David Harwood, Larry Davis, Real-time foreground–background segmentation using codebook model, Real-Time Imaging, Volume 11, Issue 3, June 2005, Pages 172-185

slide-7
SLIDE 7

Robust Background Subtraction

  • Should handle:

β–« Lighting changes

ο‚– Adaptive

β–« Repetitive motion from clutter

ο‚– Multimodal distribution

β–« Long term scene changes

ο‚– Multi-threshold 7 RG plots of a single pixel Differing threshold

  • ver time

Bimodal distribution

  • ver time
slide-8
SLIDE 8

Algorithm Overview

  • Pixel value is modeled as a mixture of adaptive

Gaussian distributions

β–« Why a mixture?

ο‚– Multiple surfaces appear in a pixel (mean background assumes a single pixel distribution)

β–« Why adaptive?

ο‚– Lighting conditions change

  • Gaussians are evaluated to determine which
  • nes are most likely to correspond to the

background

β–« Based on persistence and variance

  • Pixels that do not match the background

Gaussians are classified as foreground

8

slide-9
SLIDE 9

Online Mixture Model

  • History of a pixel is known up to current time 𝑒

β–« π‘Œ1, … , π‘Œπ‘’ = 𝐽 𝑦𝑝, 𝑧𝑝, 𝑗 : 1 ≀ 𝑗 ≀ 𝑒

  • Model the history as a mixture of 𝐿 Gaussian

distributions

β–« 𝑄 π‘Œπ‘’ = π‘₯𝑗,𝑒π’ͺ(π‘Œπ‘’|𝑣𝑗,𝑒, Σ𝑗,𝑒)

𝐿 𝑗=1

ο‚– π‘₯𝑗,𝑒 - prior probability (weight) of Gaussians 𝑗

β–« Able to represent arbitrary distributions

  • Gaussian distribution

β–« Univariate β–« Multivariate

9

slide-10
SLIDE 10

Mixture Model Example

  • For a grayscale image with 𝐿 = 5

β–« Pixel intensity distribution (over time) modeled with five Gaussians

10

slide-11
SLIDE 11

Model Adaption I

  • Online K-means approximation is used to

update the Gaussians

β–« Enables fast and efficient model parameter estimation

  • Each pixel is compared with its distribution

model

β–« New pixel π‘Œπ‘’+1 is compared with each of the existing 𝐿 Gaussians until a match is found β–« Match is defined as a pixel value within 2.5𝜏 standard deviations of a distribution

11

slide-12
SLIDE 12

Model Adaption II

  • Match found:
  • Update parameters

β–« πœˆπ‘—,𝑒+1 = 1 βˆ’ 𝜍 πœˆπ‘—,𝑒 + πœπ‘Œπ‘’+1 β–« πœπ‘—,𝑒+1

2

= 1 βˆ’ 𝜍 πœπ‘—,𝑒

2 + 𝜍 π‘Œπ‘’+1 βˆ’ πœˆπ‘—,𝑒 2

ο‚– 𝜍 = 𝛽π’ͺ Xt+1 πœˆπ‘—,𝑒, πœπ‘—,𝑒

2

ο‚– 𝛽 – is a learning rate

  • Update Gaussian weights

β–« π‘₯𝑗,𝑒+1 = 1 βˆ’ 𝛽 π‘₯𝑗,𝑒 + 𝛽 𝑁𝑗,𝑒+1

ο‚– 𝑁𝑗,𝑒+1 = 1 for matching Gaussian or 𝑁𝑗,𝑒+1 = 0 for all

  • thers

ο‚– Match increases weight

12

slide-13
SLIDE 13

Model Adaption III

  • No match found:
  • None of the 𝐿 Gaussians match pixel value π‘Œπ‘’+1

β–« Observed value not well explained by model

  • Replace the least probable distribution with a

new one

β–« Newly created distribution based on current value

ο‚– πœˆπ‘’+1 = π‘Œπ‘’+1 ο‚– Has high variance and low prior weight

β–« Least probable in the πœ•/𝜏 sense (to be explained)

13

slide-14
SLIDE 14

Background Model Estimation

  • A background pixel value should be consistent
  • Heuristic: Gaussians with the most supporting

evidence and least variance should correspond to the background

  • Gaussians are ordered by the value of πœ•/𝜏

β–« High support πœ• and smaller variance 𝜏 give larger value

  • First 𝐢 distributions are selected as the background

model

β–« 𝐢 = π‘π‘ π‘•π‘›π‘—π‘œπ‘( π‘₯𝑗 > π‘ˆ)

𝑐 𝑗=1

ο‚– π‘ˆ minimum portion of image expected to be background

14

slide-15
SLIDE 15

Background Estimation Example

  • After background estimation, red are the

background and black are foreground (not background)

15

slide-16
SLIDE 16

Results

  • Not much in paper, comparison from homework

16

slide-17
SLIDE 17

Discussion

  • Advantages

β–« Different threshold for each pixel β–« Pixel-wise thresholds adapt over time β–« Objects are allowed to become part of the background without destroying the existing background model β–« Provides fast recovery

  • Disadvantages

β–« Cannot handle sudden, drastic lighting changes β–« Must have good Gaussian initialization (median filtering) β–« There are a number of parameters to tune

17

slide-18
SLIDE 18

More Issues?

  • Shadows detection

β–« [Prati, Mikic, Trivedi, Cucchiara 2003]

  • Chen & Aggarwal: The likelihood of a pixel being

covered or uncovered is decided by the relative coordinates of optical flow vector vertices in its neighborhood.

  • Oliver et al.: β€œEigenbackgrounds" and its variations.
  • Seki et al.: Image variations at neighboring image

blocks have strong correlation.

18

slide-19
SLIDE 19

Simple Improvement

  • Incorporate both spatial and temporal

information into the background model

  • Adaptive background mixture model + 3D

connected component analysis [Goo et al.]

β–« 3rd dimension is time

19

slide-20
SLIDE 20

Summary

  • Simple background subtraction approaches such

as fame diff, mean, and median filtering are fast

β–« Constant thresholds make them ill-suited for challenging real-world problems

  • Adaptive background mixture model approach

can handle challenging situations

β–« Bimodal backgrounds, long-term scene changes, and repetitive motion

  • Improvements include upgrade the approach

with temporal information or using region- based techniques

20

slide-21
SLIDE 21

Thank You

  • Questions?

21 Background subtraction implementation using GMM at OpenCV

slide-22
SLIDE 22

References

  • Reading

β–« Stauffer, Chris; Grimson, W.E.L., "Adaptive background mixture models for real-time tracking," in Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on. , vol.2, no., pp.252 Vol. 2, 1999

β–« Kyungnam Kim, Thanarat H. Chalidabhongse, David Harwood, Larry Davis, Real-time foreground–background segmentation using codebook model, Real-Time Imaging, Volume 11, Issue 3, June 2005, Pages 172-185

  • Background Subtraction Datasets

β–« https://sites.google.com/site/backgroundsubtraction/ test-sequences

22