FOOTBALL EXAMPLE 1 COGNITIVE VISION INTRODUCTION BUILD A VISION - - PowerPoint PPT Presentation

School of Informatics, University of Edinburgh

COGNITIVE VISION INTRODUCTION BOB FISHER PHD IN 3D OBJECT RECOGNITION, LOTS OF GEOMETRIC BASED VISION NOW ALSO SOME COGNITIVE VISION DIRECTOR INSTITUTE OF PERCEPTION, ACTION AND BEHAVIOUR AT UNIV OF EDINBURGH ORGANISES CVONLINE: http://homepages.inf.ed.ac.uk/rbf/CVonline/

ECVision Summer School: 1 - Introduction Fisher slide 1 School of Informatics, University of Edinburgh

FOOTBALL EXAMPLE 1 BUILD A VISION SYSTEM TO ANALYZE THIS FOOTBALL SCENE: WHY IS THIS A COGNITIVE VISION SYSTEM? (INTILLE ET AL)

ECVision Summer School: 1 - Introduction Fisher slide 2 School of Informatics, University of Edinburgh

FOOTBALL EXAMPLE 2

THE VISION SYSTEM HAS TO:

• COPE WITH MULTIPLE ACTORS
• COPE WITH NOISY, ERRONEOUS, FRAGMENTED

TRACKING

• USE IDEAL MODELS MATCHED TO REAL ACTIONS:
• UNDERSTAND TEMPORAL RELATIONSHIPS
• USE PROBABILISTIC REASONING

84% CORRECT ON 25 EXAMPLES WITH 10 MODEL PLAYS

ECVision Summer School: 1 - Introduction Fisher slide 3 School of Informatics, University of Edinburgh

WHAT COGNITIVE VISION IS NOT

IMAGE PROCESSING: IMAGE-TO-IMAGE TRANSFORMS BASIC FEATURE EXTRACTION: EDGE DETECTION GEOMETRIC MODEL BUILDING STRUCTURE & MOTION OPTICAL FLOW SHAPE-FROM-X VIDEO GOOGLE FACE RECOGNITION PATTERN RECOGNITION . . . ALL LARGELY ONE-STEP OR DETERMINISTIC ALGORITHMS, POSSIBLY USED BY COG VIS

ECVision Summer School: 1 - Introduction Fisher slide 4

School of Informatics, University of Edinburgh

SOME COGNITIVE VISION APPLICATIONS

INTRUDER/ESCAPING PRISONER DETECTION ROAD TRAFFIC SURVEILLANCE CITY CENTER SURVEILLANCE CROWD SAFETY MONITORING PEOPLE DETECTION, LOCALIZATION, COUNTING LIP READING, EXPRESSION UNDERSTANDING GESTURE/HAND SIGN RECOGNITION SHOPPER ANALYSIS GENERIC OBJECT RECOGNITION SPORTS VIDEO ANNOTATION CONTEXT BASED IMAGE RETRIEVAL AERIAL AND GROUND BASED SCENE UNDERSTANDING VIDEO ARCHIVE SUMMARIZATION

ECVision Summer School: 1 - Introduction Fisher slide 5 School of Informatics, University of Edinburgh

WHAT IS CENTRAL TO COG VIS?

HYPOTHESIS-BASED: MULTIPLE, RANKED BY PROBABILITY, PRUNING OF COMBINATORIAL EXPLOSION HEURISTICS & INCOMPLETE KNOWLEDGE GENERIC CAPABILITIES: CASE-BASED/RULE-BASED RICH KNOWLEDGE BASED: HUMAN/WORLD/SITUATION TEMPORAL/SEQUENCE ANALYSIS REASONING; ALTERNATIVES, UNCERTAINTY, COPING WITH CHANGE

ECVision Summer School: 1 - Introduction Fisher slide 6 School of Informatics, University of Edinburgh

ECVISION DEFINITION

A COGNITIVE VISION SYSTEM CAN ACHIEVE THE FOUR LEVELS OF GENERIC VISUAL FUNCTIONALITY:

• DETECTION
• LOCALIZATION
• RECOGNITION
• UNDERSTANDING (ROLE, CONTEXT, PURPOSE)

AND EXHIBITS PURPOSIVE GOAL-DIRECTED BEHAVIOUR, IS ADAPTIVE TO UNFORESEEN CHANGES, AND CAN ANTICIPATE THE OCCURRENCE OF OBJECTS AND EVENTS.

ECVision Summer School: 1 - Introduction Fisher slide 7 School of Informatics, University of Edinburgh

ECVISION DEFINITION CONT.

THIS IS ACHIEVED THROUGH:

• LEARNING SEMANTIC KNOWLEDGE (FORM,

FUNCTION, & BEHAVIOURS)

• RETENTION OF KNOWLEDGE (ABOUT THE

COGNITIVE SYSTEM, ITS ENVIRONMENT, AND THE RELATIONSHIP WITH THE ENVIRONMENT)

• DELIBERATION ABOUT OBJECTS AND EVENTS,

INCLUDING THE COGNITIVE SYSTEM ITSELF.

ECVision Summer School: 1 - Introduction Fisher slide 8

School of Informatics, University of Edinburgh

COG VISION MODEL SYLLABUS

http://homepages.inf.ed.ac.uk/rbf/CCVO/cvsyldraft.htm TOP LEVEL OF HIERARCHY HERE

• 1. Knowledge Representation
• 1. Overview/Issues
• 2. Knowledge Representation Technologies
• 3. Applications/Case Studies
• 4. General Resources
• 2. Recognition, Categorization and Estimation
• 1. Overview/Issues
• 2. Recognition Technologies
• 3. Applications/Case Studies
• 4. General Resources
• 3. Reasoning about Structures and Events
• 1. Overview/Issues

ECVision Summer School: 1 - Introduction Fisher slide 9 School of Informatics, University of Edinburgh

• 2. Reasoning Technologies
• 3. Applications/Case Studies
• 4. General Resources
• 4. Model Learning
• 1. Overview/Issues
• 2. Learning Technologies
• 3. Applications/Case Studies
• 4. General Resources
• 5. Visual Process Control
• 1. Overview/Issues
• 2. Process Control Technologies
• 3. General Resources
• 6. Good example areas and Case Studies
• 1. Static Image Understanding
• 2. Image Sequence Understanding

ECVision Summer School: 1 - Introduction Fisher slide 10 School of Informatics, University of Edinburgh

MODEL SYLLABUS II

SAMPLE LINK:

• 1. Knowledge Representation
• 1. Overview/Issues
• 1. Style
• 3. Probabilistic

HAS ONLINE TUTORIAL MATERIALS: Internet resources: ***Introduction to Bayesian Reasoning*** (Maria Petrou) AND ANNOTATED BIBLIOGRAPHY: Publications: Multi-Object Tracking: Explicit Knowledge Representation and Implementation for Complexity Reduction, "***Spengler, M. and Schiele, B.***", Cognitive Vision

ECVision Summer School: 1 - Introduction Fisher slide 11 School of Informatics, University of Edinburgh

Workshop 2002, 2002. Many successful single-object tracking algorithms are formulated or may be reformulated as Bayesian inference

• problem. It is straight-forward to generalize the

Bayesian formulation to the problem of multi-object

• tracking. However, due to the increase in dimensionality

this formulation also opens Pandoras box in terms of exponential explosion of the computational complexity. In this paper we propose to constraint the computational complexity by exploiting and explicitly using prior knowledge at various levels of the Bayesian formulation

• f multi-object tracking. More specifically we discuss

the use of a knowledge hierarchy which makes explicit where and how to introduce available knowledge.

ECVision Summer School: 1 - Introduction Fisher slide 12

School of Informatics, University of Edinburgh

OVERVIEW OF MY LECTURES

• 1. DETECTING AND TRACKING MOVING

HUMANS

• 2. MAINTAINING PERSISTENCE WHEN

HUMANS TRAJECTORIES OVERLAP

• 3. IDENTIFYING SHORT-TERM ACTIONS
• 4. SYNTACTIC REPRESENTING AND

RECOGNIZING ACTION

• 5. PROBABILISTIC REPRESENTING AND

RECOGNIZING ACTION

ECVision Summer School: 1 - Introduction Fisher slide 13