Video Segmentation for Video Segmentation for Surveillance Surveillance -- A Transform Domain Approach A Transform Domain Approach -- Juhua Zhu Dept. of Electrical Engineering April 23, 2005 1 1 Problem Formulation Problem Formulation • Goal: To segment moving objects in video • Static camera • Change detection – What to compare to? – What change is of interest? • Challenges 2 2
from [K. Toyama 99] -to be cont’d 3 3 from [K. Toyama 99] 4 4
Overview Overview training Input Block DCT AC and DC feature Background model Initial segmentation High-level processing Update background 5 model 5 Features Features • Block-based DCT(e.g.8x8) • DC and AC features DC = ( 0 , 0 ) f DCT – − − N / 2 1 N / 2 1 ∑ ∑ = + ⋅ 2 2 AC ( ) | ( , ) | – f i j DCT i j = = 0 0 i j • Local information is encoded, intensity and texture. • Features insensitive to noise, small scene changes, and light shadows • Foreground objects usually lead to significant changes in AC and DC features 6 6
Background Blocks Background Blocks • Features change slowly from frame to frame • Single Exponential Smoothing ˆ ˆ f f f • Predict from and + t 1 t t ˆ ˆ = α ⋅ + − α AC AC AC AC AC ( 1 ) f f f + t 1 t t ˆ = α ⋅ + − α ˆ DC DC DC DC DC f f ( 1 ) f + 1 t t t ˆ • For background blocks close to f f t t ˆ • Deviation of from has mean close to 0 and f f t t σ 2 variance ˆ σ 2 = α − 2 + − α σ 2 • ( ) ( 1 ) f f − t t t t 1 7 7 Foreground Blocks Foreground Blocks • Features significantly differ from prediction • Initial Segmentation – Declare a block as foreground if ˆ ˆ − ≥ κ ⋅ σ + λ ⋅ DC DC DC DC | | f f f − 1 t t t t ˆ ˆ − ≥ κ ⋅ σ + λ ⋅ or AC AC AC AC | | f f f − 1 t t t t 2 nd term in thresholds due to small # of training frames σ 2 • If foreground, do not predict and update 8 8
α Selection of Selection of • Initialization – Minimize Sum of Squared Error • Update – Based on classification confidence and frame-wise correlation 0 foreground block α = α background block min t α ⋅ α n max( , ( 0 . 5 ) ) fast update needed min max 9 9 High- -level Processing level Processing High Re-model and/or Global α Yes Increase re-segment Change? No 8-connected Labeling Size Filtering (optional) Temporal Filtering Increase α Stationary Yes Remove blob blob? No Filling (optional) 10 10
Form blobs Form blobs • Group eight-connected foreground blocks as blob • Tag all blobs in each frame • Match each blob with blobs in previous k frames using proximity • Link matched blobs from frame to frame • Apply temporal filtering to remove temporally isolated blobs 11 11 Temporal Filtering Temporal Filtering Temporally Isolated Remove!! t 12 12
Blob Matching Blob Matching Keep the blob or not? – continuous motion X X X t-k+1: O O X t-k+2: O X t-k+3: … … … … O t: 13 13 Compare with Pixel- -wise wise MoG MoG Compare with Pixel Frame 31 Frame 33 Frame 35 Frame 37 14 14
Experimental results Experimental results A. B. A. Strong moving reflections B. Severe global/local illumination change, strong lighting, mirror-effect C. D. C. Flowing water, swaying branches, occlusions, small objects D. Waving water, strong reflections, small objects, camouflage 15 15 Comparison Comparison PROPOSED METHOD CLASSIC MOG Texture and intensity Color information Block-based DCT Pixel-wise Grayscale RGB or YUV Modeled by Single Modeled by Mixture of Gaussian Gaussians Fast adaptation to changes Slow adaptation Robust to noise and small Sensitive to noise and scene changes small scene changes 16 16
Summary Summary • Fast (~40 fps) • Robust to noise, small scene changes, and illumination changes • Can handle – Moved background objects – Foreground aperture – Bootstrapping – Shadows 17 17 Q & A Q & A 18 18
Recommend
More recommend
