Video Segmentation for Video Segmentation for Surveillance - - PDF document

SLIDE 1

1

1

Video Segmentation for Video Segmentation for Surveillance Surveillance

• - A Transform Domain Approach

A Transform Domain Approach

Juhua Zhu

• Dept. of Electrical Engineering

April 23, 2005

2

2

Problem Formulation Problem Formulation

• Goal: To segment

moving objects in video

• Static camera
• Change detection

– What to compare to? – What change is of interest?

• Challenges

SLIDE 2

3

3

from [K. Toyama 99] -to be cont’d 4

4

from [K. Toyama 99]

SLIDE 3

5

5

Overview Overview

Input Block DCT Background model Initial segmentation AC and DC feature High-level processing Update background model training

6

6

Features Features

• Block-based DCT(e.g.8x8)
• DC and AC features

– –

• Local information is encoded, intensity and texture.
• Features insensitive to noise, small scene changes,

and light shadows

• Foreground objects usually lead to significant

changes in AC and DC features

∑ ∑

− = − =

⋅ + =

1 2 / 1 2 / 2 2

| ) , ( | ) (

N i N j AC

j i DCT j i f

) , ( DCT f

DC =

SLIDE 4

7

7

Background Blocks Background Blocks

• Features change slowly from frame to frame
• Single Exponential Smoothing
• Predict from and
• For background blocks close to
• Deviation of from has mean close to 0 and

variance

• AC

t AC AC t AC AC t

f f f ˆ ) 1 ( ˆ

1

α α − + ⋅ =

+ DC t DC DC t DC DC t

f f f ˆ ) 1 ( ˆ

1

α α − + ⋅ =

+

2 1 2 2

) 1 ( ) ˆ (

−

− + − =

t t t t

f f σ α α σ

1

ˆ

+ t

f

t

f ˆ

t

f

t

f

t

f ˆ

t

f ˆ

t

f

2

σ

8

8

Foreground Blocks Foreground Blocks

• Features significantly differ from prediction
• Initial Segmentation

– Declare a block as foreground if

• r

2nd term in thresholds due to small # of training frames

• If foreground, do not predict and update

AC t AC t AC t AC t

f f f ˆ | ˆ |

1

⋅ + ⋅ ≥ −

−

λ σ κ

DC t DC t DC t DC t

f f f ˆ | ˆ |

1

⋅ + ⋅ ≥ −

−

λ σ κ

2

σ

SLIDE 5

9

9

Selection of Selection of

• Initialization

– Minimize Sum of Squared Error

• Update

– Based on classification confidence and frame-wise correlation

     ⋅ = needed update fast ) ) 5 . ( , max( block background block foreground

max min min

α α α α

n t

α

10

10

High High-

• level Processing

level Processing

Re-model and/or re-segment Global Change? 8-connected Labeling Size Filtering (optional) Stationary blob? Filling (optional) Remove blob Increase No No Yes Yes Temporal Filtering

α

Increase α

SLIDE 6

11

11

Form blobs Form blobs

• Group eight-connected foreground

blocks as blob

• Tag all blobs in each frame
• Match each blob with blobs in previous

k frames using proximity

• Link matched blobs from frame to frame
• Apply temporal filtering to remove

temporally isolated blobs

12

12

Temporal Filtering Temporal Filtering

Temporally Isolated Remove!!

t

SLIDE 7

13

13

Blob Matching Blob Matching

Keep the blob or not? – continuous motion

X X X X O O X O O … …

t-k+1: t-k+2: t-k+3: t:

… …

14

14

Compare with Pixel Compare with Pixel-

• wise

wise MoG MoG

Frame 31 Frame 35 Frame 33 Frame 37

SLIDE 8

15

15

Experimental results Experimental results

• D. Waving water, strong

reflections, small objects, camouflage

• C. Flowing water, swaying

branches, occlusions, small

• bjects
• B. Severe global/local

illumination change, strong lighting, mirror-effect

• A. Strong moving

reflections

A. B. C. D.

16

16

Comparison Comparison

Color information Texture and intensity CLASSIC MOG PROPOSED METHOD Sensitive to noise and small scene changes Robust to noise and small scene changes Slow adaptation Fast adaptation to changes Modeled by Mixture of Gaussians Modeled by Single Gaussian RGB or YUV Grayscale Pixel-wise Block-based DCT

SLIDE 9

17

17

Summary Summary

• Fast (~40 fps)
• Robust to noise, small scene changes,

and illumination changes

• Can handle

– Moved background objects – Foreground aperture – Bootstrapping – Shadows

18

18

Q & A Q & A