[PPT] - NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT PowerPoint Presentation

SLIDE 1

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Giorgio Gemignani Lucia Maddalena Alfredo Petrosino

SLIDE 2

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Outline:

– Moving and Stopped Object detection. – Dual Background approach and experimental results. – Alghoritm parallelization on GPU.

SLIDE 3

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Stopped Object : temporally static

image regions indicating objects that do not constitute the original background, but were brought into the scene at a subsequent time.

Examples: abandoned luggage or

illegally parked vehicles.

SLIDE 4

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Dual Background Approach

Construct 2 separate models:

Long term model BL for scene background.
Short term model Bs for static background elements

Compare each sequence frame It with these models and calculate 2 foreground binary masks:

Long foreground mask FL containing stopped and moving objects.
Short foreground mask Fs containing moving objects.

FS FL

SLIDE 5

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Dual Background Approach

     ∨ − ∧ ∆ + =

− −

)) ( ) ( ( ) ) ( , max( )) ( ! ) ( ( ) ) ( , min( ) (

1 1

x F x F if k x E x F x F if t x E x E

S L t S L t t

τ

decay factor: determine how fast system should recognize that a stopped pixel has moved again.

For each pixel an evidence score is computed by applying the set of hypothesis on the foreground masks:

stationary threshold: minimum number of consecutive frames after which a pixel is classified as static.

k

SLIDE 6

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Neural Self Organizing Background Model

The background model constructed and maintained in SOBS

algorithm [Maddalena & Petrosino, TIP’08], here adopted for both the long-term and the short-term backgrounds, is based on a self organizing neural network organized as a 2-D flat grid of neurons.

SLIDE 7

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Neural Self Organizing Backgroung Model

1. For each pixel x, build a neuronal map consisting of n x n weight bt

i(x).

( ) ( )

2 i

n , 1, i I b 

= =

, x x

( ) ( )

( )

( ) ( )

( )

x x x , x

t i t n , 1, i t BM t

I , b d min I b d

2



=

( ) ( )

j j j j i i i i

v , s , h I , v , s , h I :

= =

x x

space colour HSV

2. At each subsequent time instant t, every pixel x of It is compared to current pixel weight vectors (bt

1(x), …, bt L(x)) to determine the weight vector bt BM(x) that best

matches it according to a metric d() :

( ) (

) ( )

( )

j j j j j j j i i i i i i i j i

v ), sin(h s v ), cos(h s v v ), sin(h s v ), cos(h s v I I d

− =

x , x

SLIDE 8

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Neural Self Organizing Backgroung Model

( ) ( ) ( ) ( ) ( ) ( )

z

y , y z y y z y y N I , α B , α 1 B

t t 1 t t t

∈ ∀ + − =

−

( ) ( )

z y z y

− =

G , α

t t

γ

( ) { } [ ] ( ) [ ]

1 , , α s.t. 0,1 β , G max γ

t t t

∈ ∈ − =

z y z y

t

β

3. Weight vectors are updated in a neighborhood of best matching neuron.

Updating the model Bt in a neighborhood Nz :

Bt

L is updated according to (1) in

a selective way, only if

( ) ( )

( )

ε <

x , x

t BM t

I b d

Bt

S is updated according to (1) in a

non selective way with

L t S t

γ γ > >

4. For the purpose of the double background approach to stopped object detection:

(1)

SLIDE 9

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Results

Pets2006

Computational compexity both in space and time is O(n2 x M x N) where n2 is the number of weight vectors used to model each pixel and N x M is the image size.

PVEasy

SLIDE 10

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Time Processing

Image Size Processing Time Frame Rate 720 x 480 431.68 ms 2.3 fps 960 x 720

862.94 ms

1.15 fps 1200 x 960

1430.30 ms

0.69 fps

Well…..but Real-time video surveillance applications require a frame rate of 24 fps.

SLIDE 11

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Parallel Implementation

M x N Pixels
thx x thy number of threads per block
G1 ( GS ,GL) grid of Blocks:
GS short-term background model .
GL long-term background model.
G2 generate the evidence Image

SLIDE 12

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Image Size CPU Time GPU Time Speed Up Frame Rate 720 x 480 431.68 ms 20.55 ms 21 x 49 fps 960 x 720 862.94 ms 40.96 ms 21 x 25 fps 1200 x 960 1430.30 ms 65.41 ms 22 x 16 fps

Serial Implementation: Intel Core i7 CPU at 3.3 GHz.
Parallel implementation: Tesla C1060 (30 multiprocessors).
Measurement of processing time for on-line phase.

Parallel Performance Evaluation

AB-Easy

SLIDE 13

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

OpenMp parallelization with N cores.

Thread 0 Thread i Thread N

SLIDE 14

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

N. of

Threads Processing Time Speed Up Efficiency Frame Rate 2 216.50 ms 1.99 x 0.99 4.61 fps 4 115.20 ms 3.74 x 0.93 8.68 fps 6 96.70 ms 4.46 x 0.74 10.34 fps 8 76.42 ms 5.64 x 0.70 13.08 fps 2 436.76 ms 1.97 x 0.98 2.28 fps 4 224.64 ms 3.84 x 0.96 4.45 fps 6 190.26 ms 4.53 x 0.75 5.25 fps 8 152.08 ms 5.67 x 0.70 6.57 fps 2 719.29 ms 1.98 x 0.99 1.39 fps 4 373.58 ms 3.82 x 0.95 2.67 fps 6 317.86 ms 4.49 x 0.74 3.14 fps 8 254.25 ms 5.62 x 0.70 3.9 fps

Image Size: 720 x 480 Gpu processing time: 20.55 ms

Seq. processing time: 431.68 ms

Image Size: 960 x 720 Gpu processing time: 40.96 ms

Seq. processing time: 862.94 ms

Image Size: 1200 x 960 Gpu processing time: 65.41 ms

Seq. processing time: 1430.3 ms

Test on Intel Core i7 CPU at 3.33 GHz

SLIDE 15

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

Parallel computing is an excellent solution for scientific research and unconventional hpc platforms as GPUs, due to their computing power and low cost represent an interesting alternative to conventional parallel shared memory architectures. Thank you………..

SLIDE 16

Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION

R.T. Collins et al., “A system for video surveillance and monitoring,” The Robotics

Institute, Carnegie Mellon University, Tech. Rep. CMU-RI-TR-00-12, 2000.

J.M. Ferryman (Ed.): Proceedings of the 9th IEEE International Workshop on PETS,

New York, June 18, 2006.

J.M. Ferryman (Ed.): Proceedings of the 10th IEEE International Workshop on

PETS, Rio de Janeiro, Brazil, October 14, 2007.

E. Herrero-Jaraba et al., Detected motion classification with a double-background

and a Neighborhood-based difference, Patt. Recogn. Lett. 24, 2079–2092, 2003.

L. Maddalena and A. Petrosino, A Self-Organizing Approach to Background

Subtraction for Visual Surveillance Applications, IEEE Transactions on Image Processing, vol. 17, no. 7, pp. July, 2008.

NVIDIA, CUDA guide, http://www.nvidia.com/object/cuda home new.html
F. Porikli,Y. Ivanov, T. Haga, Robust Abandoned Object Detection Using Dual

Foregrounds, EURASIP Journal on Advances in Signal Processing, 2008.

Proc. of Fourth IEEE International Conference on Advanced Video and Signal Based