SLIDE 1
Developing Objects Segregation capabilities and the notion of Object Containment from unlabeled natural videos
Joint work with Nimrod Dorfman and Shimon Ullman
Object Segregation
Object 1 Object 2 Background
Developing Objects Segregation capabilities and the notion of - - PowerPoint PPT Presentation
Developing Objects Segregation capabilities and the notion of - - PowerPoint PPT Presentation
Developing Objects Segregation capabilities and the notion of Object Containment from unlabeled natural videos Daniel Harari Joint work with Nimrod Dorfman and Shimon Ullman Object Segregation Object 2 Object 1 Background Object
SLIDE 2
SLIDE 3
Object segregation is learned
[Kellman & Spelke 1983; Spelke 1990; Kestenbaum et al., 1987]
3
5 months
Even basic Gestalt cues are initially missing
[Schmidt et al. 1986]
SLIDE 4
Object segregation is learned
4
Adults
SLIDE 5
It all begins with motion
5
SLIDE 6
It all begins with motion
Grouping by common motion precedes figural goodness
[Spelke 1990 - review]
Motion discontinuities provide an early cue for occlusion boundaries
[Granrud et al. 1984]
6
SLIDE 7
Our model
7
Static segregation Local occlusion boundaries Object form Motion discontinuities Common motion
Boundary General Accurate Noisy Incomplete Global Object-specific Complete Inaccurate
Motion-based segregation
CogSci 2013
SLIDE 8
Intensity edges?
8
Boundary
SLIDE 9
Occlusion cues
9
Extremal edges Convexity T-junctions
[Ghose & Palmer 2010]
Boundary
SLIDE 10
Familiar object
10
Global
SLIDE 11
11
How does it actually work?
SLIDE 12
Moving object
12
Motion
SLIDE 13
13
Motion Boundary Global
Figure Ground Unknown
SLIDE 14
14
Need many examples for good results (1000+) Boundary
Informative boundary features
SLIDE 15
Prediction
15
Figure
- r
Ground? Figure
- r
Ground? Novel object, novel background
78% success Using 100,000 training examples
Boundary
SLIDE 16
Entire image
16
Boundary
Figure Background
SLIDE 17
Learning an object
17
Standard object recognition algorithm
Learns local features and their relative locations
Global
SLIDE 18
Detection
18
Global
SLIDE 19
Combining information sources
19
Combined Boundary
Accurate Noisy & Incomplete
Global
Complete Inaccurate
Figure Background
SLIDE 20
More complex algorithms
Default GrabCut With segregation cue
[Rother et al. 2004]
20
SLIDE 21
Summary
- Static segregation is learned from motion
- Two simple mechanisms:
Boundary
Motion discontinuities Occlusion boundaries
(Need a rich library, including extremal edges)
Global
Common motion Object form
- These mechanisms work in synergy
- This is enough to get started,
adult segregation is much more complex
21
SLIDE 22
Object Containment
Object 1 Object 2 Object 2 Object 2
SLIDE 23
Occlusion vs. Containment
C A A occludes C C occludes A A C A occludes C & C occludes A A C
Occlusion Containment = a paradoxical occlusion
SLIDE 24
Developmental path
2.5 months 6 months 18 months
Dynamic occlusion Dynamic containment Static containment High angle containment Tight and loose fit
SLIDE 25
Reasoning about spatial relations
[Hespos & Baillargeon 2001]
Occlusion and Containment at 2.5 months
SLIDE 26
Dynamic containment
SLIDE 27
Familiarization events
External boundary Internal boundary External object region Internal object region
SLIDE 28
Motion boundaries
Optical flow Flow boundary t-Δt t t+Δt
SLIDE 29
Motion boundaries ownership
Between (t-Δt) and t
SLIDE 30
Motion boundaries ownership
Between t and (t+Δt) The boundary is owned by the GREEN object (basket)
SLIDE 31
Temporal dynamics of containment
C A A occludes C A C
Occlusion Containment
A occludes C A occludes C & C occludes A
SLIDE 32
Detection of dynamic containment
Simple Occlusion Event Containment Event
Frame number Frame number Occluder/total Container/total
SLIDE 33
Static containment
33
SLIDE 34
Familiarization events
External boundary Internal boundary External object region Internal object region
SLIDE 35
Main idea
35
Use motion to learn about
- bject regions and boundaries
SLIDE 36
Advanced notions of containment: Tight vs. Loose Fit and High angle
[Casasola & Cohen 2002] High Angle Tight/loose fit
At 6 months
SLIDE 37
Tight vs. Loose Fit
SLIDE 38
The computational challenge
- Learning to identify containers and containment
events in dynamic and static visual inputs without labels.
- Learning more conceptual aspects of what
containment means, for example that a contained
- bject moves together with the container to a new
location, loose- and tight-fit, etc.
- In the future, learn other aspects of conceptual
knowledge about container (containing liquids, pouring from a container, etc.)
SLIDE 39
Summary
Object Segregation
- Dynamic grouping -
global object model
- Dynamic discontinuities-
figure-ground
- Acquiring the capability
to segregate objects in static images Object Containment
- Dynamic grouping –
- bject occlusion
- Dynamic discontinuities-
internal and external
- bject boundaries
- Acquiring the notion of
paradoxical occlusion,
- bject containment, in
both dynamic and static images
SLIDE 40