Hopalong Casualty Capabilities and Limitations of Visual - - PowerPoint PPT Presentation
Introduction Analysis Methods Tracking Tools Conclusion Hopalong Casualty Capabilities and Limitations of Visual Surveillance Ingo L utkebohle Computational Perception Lab Applied Computer Science Group Bielefeld University 27.
Introduction Analysis Methods Tracking Tools Conclusion
Capabilities and Limitations of Visual Surveillance Ingo L¨ utkebohle
Computational Perception Lab Applied Computer Science Group Bielefeld University
Ingo L¨ utkebohle Hopalong Casualty 1
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Visual Motion Analysis
Goal: Compact description of motion. Various levels: body configuration motion path “operate on block” Application Areas Human-Computer Interaction Games (e.g., PS2 EyeToy) Motion Capture (for movies) Surveillance
Ingo L¨ utkebohle Hopalong Casualty 2
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Contents of the talk
1
Introduction Motivation and Overview Problem Sketch Surveillance
2
Analysis Methods Locating Humans
3
Tracking Interest Points Results Analysis
4
Tools Systems
5
Conclusion
Ingo L¨ utkebohle Hopalong Casualty 3
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Our scenario
Ingo L¨ utkebohle Hopalong Casualty 4
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Why this is difficult
Ambiguity Low Resolution Occlusion
Ingo L¨ utkebohle Hopalong Casualty 5
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
The Roadrunner problem
when you see it, it’s too late already Appearance is not enough
1 Take visual experience 2 Add world knowledge 3 Predict activity Ingo L¨ utkebohle Hopalong Casualty 6
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Human Visual Analysis
model-based vision resolves visual ambiguity learn from visual and motor experience
Ingo L¨ utkebohle Hopalong Casualty 7
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Human Visual Analysis
model-based vision resolves visual ambiguity learn from visual and motor experience
Ingo L¨ utkebohle Hopalong Casualty 7
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Human Visual Analysis
model-based vision resolves visual ambiguity learn from visual and motor experience
Ingo L¨ utkebohle Hopalong Casualty 7
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Human Visual Analysis
model-based vision resolves visual ambiguity learn from visual and motor experience
Ingo L¨ utkebohle Hopalong Casualty 7
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Surveillance Applications
Restricted Areas Little activity Presence detection Use cases:
Alarm trigger Forensic use
needs storage for weeks Public Areas Continuous activity Separation, classification use cases
deterrent investigative
needs storage for days
Ingo L¨ utkebohle Hopalong Casualty 8
Introduction Analysis Methods Tracking Tools Conclusion Motivation and Overview Problem Sketch Surveillance
Surveillance Specifics
Conditions low resolution low frame rate long stretches of nothing going on Goals Categorize behaviour Levels
1
regular vs. irregular
2
run - fight - chase
Ingo L¨ utkebohle Hopalong Casualty 9
Introduction Analysis Methods Tracking Tools Conclusion Locating Humans
Task Sketch
Computer View image: block of pixels (numbers) everything the same Goal Teach a computer to detect relevant image parts. Interpret it
Ingo L¨ utkebohle Hopalong Casualty 10
Introduction Analysis Methods Tracking Tools Conclusion Locating Humans
First Approach: Motion Detection
Look for large enough changes from one frame to the next. Pro easy and fast gets rid of static parts Cons purely intensity/color → homogenous parts acquire holes
Ingo L¨ utkebohle Hopalong Casualty 11
Introduction Analysis Methods Tracking Tools Conclusion Locating Humans
First Approach: Motion Detection
Look for large enough changes from one frame to the next. Pro easy and fast gets rid of static parts Cons purely intensity/color → homogenous parts acquire holes
Ingo L¨ utkebohle Hopalong Casualty 11
Introduction Analysis Methods Tracking Tools Conclusion Locating Humans
Prevent holes: Learn how background looks like
Reference Image Input Image Result Image Gotcha
Ingo L¨ utkebohle Hopalong Casualty 12
Introduction Analysis Methods Tracking Tools Conclusion Locating Humans
Prevent holes: Learn how background looks like
Reference Image Input Image Result Image Gotcha
Ingo L¨ utkebohle Hopalong Casualty 12
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Tracking to resolve ambiguities and overlap
Tracking Procedure
1 First frame: Find interest points 2 Compute unique description 3 Subsequent frames: Rediscover by
similarity proximity to expected location
Ingo L¨ utkebohle Hopalong Casualty 13
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Similarity: Color
color distribution can focus on hands & face large variation → silhouette as constraint rediscover by proximity → not robust
Ingo L¨ utkebohle Hopalong Casualty 14
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Similarity: Color
color distribution can focus on hands & face large variation → silhouette as constraint rediscover by proximity → not robust
Ingo L¨ utkebohle Hopalong Casualty 14
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Similarity: Appearance
“looks like” (face image) Look for best match Generalization: Collection of generic patches Very (sometimes too) specific Problems with rotation
Ingo L¨ utkebohle Hopalong Casualty 15
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Similarity: Model prediction
Estimate possible positions Look for best match How to start? Large views only
Ingo L¨ utkebohle Hopalong Casualty 16
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Tracking Results
Associated Postures Trajectories Summaries No intrinsic meaning Ambiguous
Ingo L¨ utkebohle Hopalong Casualty 17
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Machine Learning Approach
General Approach
1 Gather examples for training 2 Categorize as desired 3 Compare new images to examples 4 Assign most likely category
Challenges Appearance = function Duration varies Context matters What is a category anyway?
Ingo L¨ utkebohle Hopalong Casualty 18
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Posture
Idea: Some postures are unique Find these key postures Self-occlusion problematic Context big part of interpretation
Ingo L¨ utkebohle Hopalong Casualty 19
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Motion History Images
Inspired by human peripheral vision Compare to example images Only for large motions Requires sufficient resolution View-angle specific
Ingo L¨ utkebohle Hopalong Casualty 20
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Trajectories
Position (center of mass) Velocity, duration Low resolution OK Not much information left
Ingo L¨ utkebohle Hopalong Casualty 21
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Task Scripts: Recognizing abstract activities
Event Triples Capture context Fixed sample size Event types selected manually
Event Vocabulary
{ E, S, M, H }
Example Sequence
{EMHMS..}
Event n-Grams
{EMH, MHM, HMS, ... }
n-Gram Histograms Ingo L¨ utkebohle Hopalong Casualty 22
Introduction Analysis Methods Tracking Tools Conclusion Interest Points Results Analysis
Tracking Summary
State of the Art Tracking associates objects over time Fails relatively often (even in humans) Robust approaches yield little information No clear decision between relevant and irrelevant Results Hard problem for recognition State-of-the-art progresses fast Sequences not learned, yet
Ingo L¨ utkebohle Hopalong Casualty 23
Introduction Analysis Methods Tracking Tools Conclusion Systems
System Summary
Digitize Locate Segment Track Summarize Classify walking
Scope For more details on camera technology, see “Hacking CCTV”, right after this talk.
Ingo L¨ utkebohle Hopalong Casualty 24
Introduction Analysis Methods Tracking Tools Conclusion Systems
Cautious note on implementations
production software not available → use research implementations, where available quality, robustness and speed vary
integration of approaches is difficult
Ingo L¨ utkebohle Hopalong Casualty 25
Introduction Analysis Methods Tracking Tools Conclusion Systems
OpenCV
Open Source Computer Vision Library Intel Corporation and contributors Comprehensive algorithm supports Pretty fast, can use Intel Performance Primitives (x86) Written in ’C’, bindings for Python Supported on Win32 and Linux Main drawback: Just a library http://www.intel.com/technology/computing/opencv/
Ingo L¨ utkebohle Hopalong Casualty 26
Introduction Analysis Methods Tracking Tools Conclusion Systems
iceWing
Open source integration environment for algorithms Basic algorithms included Extension via plugins, operating in a processing chain FireWire, V4L, AVIs, PNGs, . . . Plugins in ’C’, C++, Python or Matlab Various unices and Mac OS X http://icewing.sf.net/
Ingo L¨ utkebohle Hopalong Casualty 27
Introduction Analysis Methods Tracking Tools Conclusion
Conclusion
Indoor presence detection works The rest is a world full of edge cases Current methods are not robust enough for public areas Human-like results require a lot of human help The Roadrunner problem will be with us for a while “I wouldn’t stake my life on this technology and I wouldn’t pay for it either.”
Ingo L¨ utkebohle Hopalong Casualty 28
Introduction Analysis Methods Tracking Tools Conclusion
Outlook: Where is it going?
Research Integration 30 pixel man, i.e. coping with bad resolution Interaction analysis Congress Maybe a hands-on workshop? Talk to me afterwards!
Ingo L¨ utkebohle Hopalong Casualty 29
Introduction Analysis Methods Tracking Tools Conclusion
Acknowledgements
Credits to Frank L¨
capture), Britta Wrede (experimental data) and Julia L¨ uning (22C3).
Ingo L¨ utkebohle Hopalong Casualty 30