Actom Sequence Models for Efficient Action Detection LEAR INRIA - - PowerPoint PPT Presentation

Actom Sequence Models for Efficient Action Detection

LEAR – INRIA Grenoble Adrien Gaidon Zaid Harchaoui Cordelia Schmid Presentation by Benoit Massé

Introduction

• Video : Big Data
• Automatisation ?

– Semantic analysis – Retrieval

Problem :

Find if and when a specific action happen

State of the art

• Training

– Define the action – Choose the features – Train

• Retrieval

– Classification – Detection

State of the art

• Training

– Define the action – Choose the features – Train

• Retrieval

– Classification – Detection

=> Spatio-temporal extent => HoG, HoF, SP interest Point => Bag-of-Feature => SVM, Bayesian Network => ?

Actoms

• Actom : short atomic action

Actoms

An actom has

– A location t – A radius r

Actom descriptors : Set of visual words

– Bag of Features applied on HoG, HoF, Harris Interest points... – Ponderated sum from t - r to t + r

Interest of Actoms

• An action is composed of several actoms

– New goal : find an ordered sequence of actoms – No temporal dependance inside an action

• Gap between actoms
• Overlap
• An action can be composed of very different parts

=> Classic methods compute the average

Actom Sequence Model (ASM)

One Action = One Actom Sequence

– The radius r i of actom i depends on its distance to the

closer other actoms : min(t i - t i-1, t i+1 - t i)

– ASM : concatenation of actoms words

(x11, …, x1k, x21, …, x2k, x31, …, x3k)

Classification

• Given a new ASM (x11, ... xnk), does it corresponds to

the trained action ? (for instance : « drinking »)

– Classic machine learning problem – Chosen solution : SVM – Including negative examples improves the classifier

Detection

• Given a video, find all the occurences of the trained
• action. (for instance : « drinking »)

For every 5 frames Set the current frame as the middle actom Generate candidates for other actoms Apply classification on the result End Delete non-maximal overlapping actions

Detection

Tricky step : Generating the other actoms We must estimate the distance between actoms

– Training : Build the multivariate distribution {t i+1 – t i }

Remove the outliers

– Estimation : Try all the possible combinations

(starting from the middle limit the error propagation)

Experiments

4 kind of actions

–

Drinking

–

Smoking

–

Open a door

–

Sit down

Criteria

–

OV20 (20 % Overlap)

–

OVAA (All Actoms Overlap)

State of the art Comparison

–

Bag of Features

–

Bag of Features with a grid

–

Other published methods

Results

Conclusion

ASM gives better result than state-of-the-art, using the same data sets. => Actoms are particularly adapted for representing the temporal structure of actions into videos

QUESTIONS ?