Visual Cognition Computer Vision 3D Vision Mobile multimedia - - PDF document

1

Visual Cognition

An Imaginary experiment in Neuroimaging

UPC March 2009

Mehdi Mirza-Mohammadi

Computer Vision



3D Vision



3D TV



Biometrics



Color and texture



Document analysis



Graph-based Methods



Image and video indexing and database retrieval



Image and video processing



Image-based modeling



Kernel methods



Medical imaging



Mobile multimedia



Model-based vision approaches



Motion Analysis



Non-photorealistic animation and modeling



Object recognition



Performance evaluation



Segmentation and grouping



Shape representation and analysis



Structural pattern recognition



Tracking

Pattern Recognition

Main steps in current methods

 Feature detection  Machine learning

These approaches works in completely different way that brain visual cognition works

Objective

 By study of brain visual cognition process we may

found a new approach for computer vision or even pattern recognition in general

2

Hypothesis

 Record neuro activity of visual cognition  Analyze and train machine with these data  Simulate visual signals form eye from a digital

image

 Develop a commuter vision by these data

Brain-computer interface

 There are some research in which, machine can

learn brain signal activities

Neuroimaging Techniques

 Magnetoencephalography (MEG)  functional magnetic resonance imaging (fMRI)  Electroencephalography (EEG)

Training

 We record brain activity during vision process  Then try to find a pattern on these data and learn it

via machine learning methods

3

Eye Signals

 Next step is to find a way to produce same signals as

eye produce, in computer by a digital image

Result

 If we input these artificial eye's vision signals form

an image to our trained machine we can have visual cognition as our brain do

Problems

 There are huge amount of data, we need to know

where exactly to look and record

 Machine training with these amount of data needs

lots of computing resources and time

 Eye signal simulation  Even if it is successful it may not be practical due to

huge computation cost

Good News

There have been some very interesting experiments which have been able to detect image object category from fMRI and MEG images and even some to reconstruct images and display them on a computer!



"Decoding the Mind's Eye - Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders", Neuron (Elsevier, Cell Press) 60 (5): 915-929, 10 December 2008, doi:10.1016/j.neuron.2008.11.004



Cox, D.D., and Savoy, R.L. (2003). Functional magnetic resonance imaging (fMRI) ‘‘brain reading’’: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270.