SLIDE 1 Visual Object Recognition Computational Models and Neurophysiological Mechanisms Neurobiology 130/230. Harvard College/GSAS 78454
Web site: http://tinyurl.com/visionclass (Class notes, readings, etc) Location: Biolabs 2062 Time: Mondays 03:30 – 05:30 Lectures: Faculty: Gabriel Kreiman and invited guests Contact information:
Gabriel Kreiman Joseph Olson gabriel.kreiman@tch.harvard.edu josepholson@fas.harvard.edu 617-919-2530 Office Hours: After Class. Mon 05:30-06:30
SLIDE 2 Visual Object Recognition Computational Models and Neurophysiological Mechanisms Neurobiology 230. Harvard College/GSAS 78454
Class 1. Sep-12 Introduction to pattern recognition. Why is vision difficult? Visual input. Natural image
Class 2. Sep-19 Lesion studies in animal models. Neurological studies of cortical visual deficits in humans. Class 3. Sep-26 Psychophysics of visual object recognition [Joseph Olson] Class 4. Oct-03 Introduction to the thalamus and primary visual cortex [Camille Gomez-Laberge] Oct-10 Columbus Day. No class. Class 5. Oct-17 Adventures into terra incognita. Neurophysiology beyond V1 [Hanlin Tang] Class 6. Oct-24 First steps into inferior temporal cortex [Carlos Ponce] Class 7. Oct-31 From the highest echelons of visual processing to cognition [Leyla Isik] Class 8. Nov-07 Correlation and causality. Electrical stimulation in visual cortex. Class 9. Nov-14 Theoretical neuroscience. Computational models of neurons and neural networks. [Bill Lotter] Class 10. Nov-21 Computer vision. Towards artificial intelligence systems for cognition [David Cox] Class 11. Nov-28 Computational models of visual object recognition. [Kreiman] Class 12. Dec-05 [Extra class] Towards understanding subjective visual perception. Visual consciousness.
SLIDE 3 Psychophysics: The study of the dependencies of
psychological experiences upon the physical stimuli that generate them
Basic measures:
- Reaction time — The time taken by subjects to perform a task or make a judgment can
give an indication (or at least an upper bound) of how long the necessary psychological (and hence neural) processing takes.
- Performance — Often inversely related to reaction time. There are techniques for
mitigating response biases.
- Threshold — Stimuli can be varied to determine the threshold for detection, discrimination,
- r some more complex psychological phenomenon.
SLIDE 4
- What are the theories / evidence / questions driving the
motivation behind some psychophysics experiments of visual recognition? – Atoms of recognition – Gestalt (whole vs sum of the parts) – Context – Tolerance and Invariance to image transformations – Fundamental properties of visual system (e.g. speed)
SLIDE 5 Gestalt laws of grouping
Basic phenomenological constraints
- Law of Closure — The mind may experience elements it does not
perceive through sensation, in order to complete a regular figure (that is, to increase regularity).
- Law of Similarity — The mind groups similar elements into collective
entities or totalities. This similarity might depend on relationships of form, color, size, or brightness.
- Law of Proximity — Spatial or temporal proximity of elements may
induce the mind to perceive a collective or totality.
- Law of Symmetry (Figure ground relationships)— Symmetrical
images are perceived collectively, even in spite of distance.
- Law of Continuity — The mind continues visual, auditory, and kinetic
patterns.
- Law of Common Fate — Elements with the same moving direction
are perceived as a collective or unit.
SLIDE 6 Law of closure: perceiving objects as whole even if they are not complete
The mind may experience elements it does not perceive through sensation, in order to complete a regular figure (that is, to increase regularity)
SLIDE 7 Law of closure: perceiving objects as whole even if they are not complete
The mind may experience elements it does not perceive through sensation, in order to complete a regular figure (that is, to increase regularity)
SLIDE 8 Law of similarity
The mind groups similar elements into collective entities or totalities. This similarity might depend on relationships of form, color, size, or brightness
SLIDE 9 Law of proximity
- Spatial or temporal proximity of elements may induce
the mind to perceive a collective or totality.
SLIDE 10 Law of symmetry
[ ] { } [ ] { } [ ] { } [ { } { } { } { } { } { } { | | | | | | | | | | | | |
http://isle.hanover.edu/Ch05O bject/Ch05SymmetryLaw.html
- The Law of Symmetry is the gestalt grouping law that states that elements
that are symmetrical to each other tend to be perceived as a unified group
SLIDE 11 Law of continuity
The mind continues visual, auditory, and kinetic patterns
SLIDE 12 Law of continuity
The mind continues visual, auditory, and kinetic patterns
SLIDE 13
Law of common fate
SLIDE 14
MIRCs Minimal Recognizable Configurations
SLIDE 15
Holistic representation of faces
McKone et al, Frontiers in Psychology, 2013
SLIDE 16
Holistic representation of faces
McKone et al, Frontiers in Psychology, 2013
SLIDE 17
Holistic representation of faces
McKone et al, Frontiers in Psychology, 2013 Composite illusion
SLIDE 18
Beyond pixels – Context matters
SLIDE 19
Tolerance to image transformations
Scale Position Rotation (2D) Rotation (3D) – viewpoint Color Illumination Cues Clutter Occlusion Other non-rigid transformations (aging, expressions, etc)
SLIDE 20 Scale tolerance
x
A A A A A
SLIDE 21
One-shot learning for scale tolerance
Which one is it?
SLIDE 22 Position tolerance
x
bd db bd bd bd db db db db bd bd
SLIDE 23
Tolerance to viewpoint and illumination changes
SLIDE 24
Recognition from minimal features
SLIDE 25 Recognition of caricatures
Images: Hanoch Piven
SLIDE 26
Visual recognition depends on experience
SLIDE 27
Recognition of images flashed for ~100 ms (demo)
NEED MOVIE
SLIDE 28 Visual recognition can be extremely fast
Kirchner, H., & Thorpe, S. J. (2006). Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vision Res, 46(11), 1762-1776.
SLIDE 29 Is information integrated over time?
Singer and Kreiman, 2014
SLIDE 30 Rapid decay in recognition of asynchronously presented object parts
Brief asynchronies disrupt object recognition
Singer and Kreiman, 2014
SLIDE 31
The visual system has a very large capacity
SLIDE 32
Occlusion
SLIDE 33
Pattern completion: Objects can be recognized from partial information
SLIDE 34
Amodal completion
SLIDE 35
Object recognition from partial information
SLIDE 36
Object completion task
SLIDE 37
Object completion (unmasked condition)
NO MASK MASK Whole Partial
SLIDE 38
Partial Information induces latencies
SLIDE 39 Backward masking
10 ms 20 ms 30 ms 40 ms 50 ms 100 ms 200 ms
SLIDE 40 Doubles?
Francois Brunelle
http://www.francoisbrunelle.com/
SLIDE 41
Object completion task (masking)
SLIDE 42
Object completion (unmasked condition)
NO MASK MASK Whole Partial Whole Partial Masked Unmasked
SLIDE 43 Further reading
- Regan, D. Human Perception of Objects (2000). Sinauer Associates. Sunderland,
Massachusets.
- Frisby, JP and Stone JV. Seeing (2010). MIT Press. Cambridge, Massachusetts.
Original articles cited in class (see lecture notes for complete list)
Recognition memory for a rapid sequence of pictures. Journal of Experimental Psychology 81:10-15.
- Kirchner, H., & Thorpe, S. J. (2006). Ultra-rapid object detection with saccadic eye movements: visual processing speed
- revisited. Vision Res, 46(11), 1762-1776.
- Brady, T. F., Konkle, T., Alvarez, G. A., & Oliva, A. (2008). Visual long-term memory has a massive storage capacity for object
- details. Proc Natl Acad Sci U S A, 105(38), 14325-14329
- Mooney CM. (1957). Age in the development of closure ability in children. Canadian Journal of Psychology 11: 219-226
- McKone et al, Frontiers in Psychology, 2013
- Singer and Kreiman (2014). Short temporal asynchrony disrupts visual object recognition. Journal of Vision 12:14.
- Tang, H., et al. (2014). "Spatiotemporal dynamics underlying object completion in human ventral visual cortex." Neuron 83: 736-
748.
- Tang, H., et al. (2014). "A role for recurrent processing in object completion: neurophysiological, psychophysical and
computational evidence." CBMM Memo(9).
