Inferring Temporal Compositions of Actions Using Probabilistic - - PowerPoint PPT Presentation

▶
inferring temporal compositions of actions using
SMART_READER_LITE
LIVE PREVIEW

Inferring Temporal Compositions of Actions Using Probabilistic - - PowerPoint PPT Presentation

Mar 28, 2023 267 likes •354 views

Australias National Science Agency Inferring Temporal Compositions of Actions Using Probabilistic Automata Rodrigo Santa Cruz 1,2 , Anoop Cherian 3 , Basura Fernando 4 , Dylan Campbell 2 , and Stephen Gould 2 1 The Australian e-Health Research

slide-1
SLIDE 1

Australia’s National Science Agency

Inferring Temporal Compositions of Actions Using Probabilistic Automata

Rodrigo Santa Cruz1,2, Anoop Cherian3, Basura Fernando4, Dylan Campbell2, and Stephen Gould2

1The Australian e-Health Research Centre, CSIRO, Brisbane, Australia 2Australian Centre for Robotic Vision (ACRV), Australian National University, Canberra, Australia 3Mitsubishi Electric Research Labs (MERL), Cambridge, MA 4A*AI, A*STAR Singapore

rodrigo.santacruz@csiro.au www.rfsantacruz.com Compositionality in Computer Vision - CVPR20 - June 15th 2020

slide-2
SLIDE 2

Compositional Action Recognition

The task of recognizing complex activities expressed as temporally-ordered compositions of simple and atomic actions in videos.

2

Corner kick Ball traveling Goal

slide-3
SLIDE 3

Problem Formulation

4

Describe complex activities by regular expressions of subset of primitive actions:

Primitives Alphabet Operators

Sequential Recursive Ex: “driving (ad) and talking on the phone (atc) or to someone (ats) repeatedly just after he got in the car (agc)” Then, our goal is to model a function f that assigns high values to a video v if it depicts the action pattern described by the regular expression r and low values otherwise. Action Patterns Alternative

One-or-more repetition

slide-4
SLIDE 4

Proposed Models

➔ Deterministic Inference (DFA based) ➔ Probabilistic Inference (PA based)

slide-5
SLIDE 5

Experiments - Activity Recognition - MultiTHUMOS

11

slide-6
SLIDE 6

Experiments - Activity Recognition - Charades

12

slide-7
SLIDE 7

Experiments - Qualitative Results

13

Primitives: holding a glass (hg), pouring water into the glass (pg), drinking from the glass (dg), running (r), cricket bowling (cb), and pole vault planting (pp).

slide-8
SLIDE 8

Australia’s National Science Agency

Inferring Temporal Compositions of Actions Using Probabilistic Automata

Rodrigo Santa Cruz1,2, Anoop Cherian3, Basura Fernando4, Dylan Campbell2, and Stephen Gould2

1The Australian e-Health Research Centre, CSIRO, Brisbane, Australia 2Australian Centre for Robotic Vision (ACRV), Australian National University, Canberra, Australia 3Mitsubishi Electric Research Labs (MERL), Cambridge, MA 4A*AI, A*STAR Singapore

rodrigo.santacruz@csiro.au www.rfsantacruz.com Compositionality in Computer Vision - CVPR20 - June 15th 2020