binocular cells Tues. Jan. 30, 2018 1 Recall last lecture: - - PowerPoint PPT Presentation
COMP 546 Lecture 6 orientation 2: complex cells binocular cells Tues. Jan. 30, 2018 1 Recall last lecture: simple Cell Linear response half wave rectification + - - + - - + - - + 2 Recall last lecture: simple Cell Linear
1
2
+ + + +
half wave rectification
3
+ + + +
half wave rectification + + +
+ +
Responds to preferred orientation of line anywhere in receptive field.
4
5
+ + + +
+ + +
Use several simple cells with common orientation and neighboring receptive field
structure of that orientation anywhere in the overlapping receptive fields. + + + +
+ +
+ + Complex cell
6
+ + + +
+ + +
Now suppose these even cell and odd cells have the same receptive field locations (perfect overlap). Again sum up their rectified responses and the result is a response anywhere in the receptive field. even odd + + +
+ + +
+ + Complex cell
7
+ + + +
+ + +
This is the same as the last model but now we square the positive values. This model is more commonly used than model (2) and so we’ll use this one. even odd + + +
+ + +
+ + Complex cell rectify and square the response
8
(cos 𝜒, sin 𝜒)
9
( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 >, < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > )
The response to an image 𝐽 𝑦 , 𝑧 is modelled as the Euclidean length of the vector, i.e. L2 norm
( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 >, < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > )
10
Complex cell response modelled as the length of 2D vector:
( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 >, < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 , 𝑧 , 𝐽 𝑦, 𝑧 > )
11
Cosine Gabor Sine Gabor
12
Example: image cross correlated with four complex cells
13
𝜚 = 90 𝜚 = −90 𝜚 = 90 𝜚 = −90
Right halves of retina map to V1 in right hemisphere Left halves of retina map to V1 in left hemisphere eccentricity eccentricity
−90
𝜚 = 𝜚 =
−90
90
15
𝑦 𝑧
Negative disparity Zero disparity (eyes verged at this depth) Positive disparity
Computer vision-ish approach: For each 𝑦0 , 𝑧0 , find disparity value 𝑒 that minimizes: where sum is over a neighborhood of 𝑦0 , 𝑧0 .
i.e. Shift the left image to undo the disparity and register the left and right images.
𝑦,𝑧
( 𝐽𝑚𝑓𝑔𝑢 𝑦 + 𝑒, 𝑧 − 𝐽𝑠𝑗ℎ𝑢 𝑦 , 𝑧 )2
16
17
We use vertically
18
Right eye 𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0 , 𝑧 − 𝑧0 Left eye 𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0 − 𝑒 , 𝑧 − 𝑧0
𝑦0 , 𝑧0 𝑦0 , 𝑧0 𝑦0 + 𝑒 , 𝑧0 𝑦0 + 𝑒 , 𝑧0 𝑒
Idea 1: (analogous to computer vision) To compute disparity at 𝑦0, 𝑧0 , find the 𝑒 that minimizes: ( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0 − 𝑒, 𝑧 − 𝑧0 , 𝐽𝑚𝑓𝑔𝑢 (𝑦, 𝑧) > − < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠(𝑦 − 𝑦0, 𝑧 − 𝑧0), 𝐽𝑠𝑗ℎ𝑢 𝑦, 𝑧 > )2 + ( < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0 − 𝑒, 𝑧 − 𝑧0 , 𝐽𝑚𝑓𝑔𝑢(𝑦, 𝑧) > − < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0, 𝑧 − 𝑧0 , 𝐽𝑠𝑗ℎ𝑢(𝑦, 𝑧) > )2
19
( < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0 − 𝑒, 𝑧 − 𝑧0 , 𝐽𝑚𝑓𝑔𝑢 (𝑦, 𝑧) > − < 𝑑𝑝𝑡𝐻𝑏𝑐𝑝𝑠(𝑦 − 𝑦0, 𝑧 − 𝑧0), 𝐽𝑠𝑗ℎ𝑢 𝑦, 𝑧 > )2 + ( < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0 − 𝑒, 𝑧 − 𝑧0 , 𝐽𝑚𝑓𝑔𝑢(𝑦, 𝑧) > − < 𝑡𝑗𝑜𝐻𝑏𝑐𝑝𝑠 𝑦 − 𝑦0, 𝑧 − 𝑧0 , 𝐽𝑠𝑗ℎ𝑢(𝑦, 𝑧) > )2
20
If 𝐽𝑚𝑓𝑔𝑢 𝑦 + 𝑒, 𝑧 = 𝐽𝑠𝑗ℎ𝑢 (𝑦, 𝑧) for all (𝑦, 𝑧) in receptive fields, then the minimum should be 0. Idea 1: (analogous to computer vision) To compute disparity at 𝑦0, 𝑧0 , find the 𝑒 that minimizes:
21
+ + + +
+ + +
+ + + +
+ + +
𝑚𝑓𝑔𝑢 𝑠𝑗ℎ𝑢 𝑠𝑗ℎ𝑢
Let y position of these cells be the same and x positions
is tuned to disparity 𝑒 .
𝑒
22
(𝑑𝑚 − 𝑑𝑠)2 + (𝑡𝑚 − 𝑡𝑠)2 = 𝑑𝑚2 + 𝑑𝑠2 + 𝑡𝑚2 + 𝑡𝑠2 − 2 ( 𝑑𝑚𝑑𝑠 + 𝑡𝑚𝑡𝑠) where 𝑑𝑚 and 𝑡𝑚 depend on 𝑒. Idea 1 (computer vision): find the disparity 𝑒 that minimizes the squared differences:
23
(𝑑𝑚 + 𝑑𝑠)2 + (𝑡𝑚 + 𝑡𝑠)2 = 𝑑𝑚2 + 𝑑𝑠2 + 𝑡𝑚2 + 𝑡𝑠2 + 2 ( 𝑑𝑚𝑑𝑠 + 𝑡𝑚𝑡𝑠) where 𝑑𝑚 and 𝑡𝑚 depend on 𝑒. Idea 2 (biological vision): find the shift 𝑒 that maximizes the squared sums :
24
+
+ + + +
+ + +
+ + +
+ + +
𝑚𝑓𝑔𝑢 𝑠𝑗ℎ𝑢 𝑠𝑗ℎ𝑢 𝑒
25
26
Response
cell
Recall (monocular) complex cell response to white line
27
Response of binocular complex cell tuned to 𝑒 = 0 when disparity of white line is 2 pixels.
shift the line
28
shift the line
lower
Response of binocular complex cell tuned to 𝑒 = 0 when disparity of white line is 10 pixels.
29
Response of binocular complex cell tuned to 𝑒 = 0 when disparity of white line is 18 pixels.
shift the line
lower
30
31
Each binocular cell has receptive field location centered at 𝑦𝑚, 𝑧𝑚 and 𝑦𝑠, 𝑧𝑠 in the two eyes. Q: What disparity is each cell tuned for ? A: We just discussed this. Q: How to visualize the set (“population”) of cells ?
32
𝑒 = 0 𝑒 > 0 𝑒 < 0
33
𝑒 = 0 𝑒 > 0 𝑒 < 0
Previous examples of vertical lines
𝑦0
34
𝑒 = 0 𝑒 > 0 𝑒 < 0
35
𝑒 = 0 𝑒 > 0 𝑒 < 0
To estimating the disparity at an image point, the visual system must select which of these binocular complex cells gives the largest response?