Modeling Orientation LISSOM Orientation Maps Starting point: LISSOM - PowerPoint PPT Presentation

Modeling Orientation LISSOM Orientation Maps • Starting point: LISSOM retinotopy model • Exactly the same architecture, different input pattern Dr. James A. Bednar • Three dimensions of variance: x, y, orientation jbednar@inf.ed.ac.uk • How will that fit into a 2D map? http://homepages.inf.ed.ac.uk/jbednar CNV Spring 2015: LISSOM Orientation Maps 1 CNV Spring 2015: LISSOM Orientation Maps 2 Retinotopy input and response Orientation input and response CMVC figure 4.4 CMVC figure 5.6 Retinal LGN Iteration 0: Iteration 0: 10,000: 10,000: Retinal LGN Iteration 0: Iteration 0: 10,000: 10,000: activation response Initial V1 Settled V1 Initial V1 Settled V1 activation response Initial V1 Settled V1 Initial V1 Settled V1 response response response response response response response response • Response before training similar to retinotopy case (Reminder from previous slides) • Response after training has multiple activity blobs per input pattern • Final blobs are orientation-specific CNV Spring 2015: LISSOM Orientation Maps 3 CNV Spring 2015: LISSOM Orientation Maps 4

Self-organized V1 weights Self-organized weights across V1 CMVC figure 5.7 CMVC figure 5.8 Afferent (ON − OFF) Lateral excitatory Lateral inhibitory Typical: • Gabor-like afferent CF • Nearly uniform short-range lateral excitatory Afferent (ON − OFF) Lateral inhibitory • Patchy, orientation-specific long-range lateral inhibitory CNV Spring 2015: LISSOM Orientation Maps 5 CNV Spring 2015: LISSOM Orientation Maps 6 OR map self-organization Macaque ORmap: Fourier,gradient Iteration 0 CMVC figure 5.1 CMVC figure 5.9 Iteration 10,000 Fourier spectrum Gradient In monkeys: • Ring-shaped spectrum: repeats regularly in all directions • High gradient at fractures, pinwheels. OR preference OR selectivity OR preference & OR H selectivity CNV Spring 2015: LISSOM Orientation Maps 7 CNV Spring 2015: LISSOM Orientation Maps 8

OR Map: Fourier, gradient OR Map: Retinotopic organization CMVC figure 5.11 CMVC figure 5.10 • Retinotopy is distorted locally by orientation prefs Fourier spectrum Gradient LISSOM model has similar spectrum, gradient • Matches distortions found in animal maps? CNV Spring 2015: LISSOM Orientation Maps 9 CNV Spring 2015: LISSOM Orientation Maps 10 OR Map: Lateral connections Effect of initial weights OR weights Weights 2 CMVC figure 5.12 CMVC figure 8.5 OR CH OR connections Weights 1 Connections Connections Connections Connections in iso-OR in OR in OR in OR ( a ) Iteration 0 ( b ) Iteration 50 ( c ) Iteration 10,000 patches pinwheels saddles fractures Changing weights doesn’t change map folding pattern. CNV Spring 2015: LISSOM Orientation Maps 11 CNV Spring 2015: LISSOM Orientation Maps 12

Effect of input streams Scaling retinal and cortical area Inputs 1 CMVC figure 8.5 CMVC figure 15.1a,b Inputs 2 ( a ) Iteration 0 ( b ) Iteration 50 ( c ) Iteration 10,000 ( a ) Original retina: R = 24 ( b ) Retinal area scaled by 4.0: Changing inputs changes entire pattern. R = 96 CNV Spring 2015: LISSOM Orientation Maps 13 CNV Spring 2015: LISSOM Orientation Maps 14 Scaling retinal and cortical area Scaling retinal density Retina CMVC figure 15.1c,d CMVC figure 15.2 V1 ( c ) Original V1: ( d ) V1 area scaled by 4.0: Original retina Retina scaled by 2 Retina scaled by 3 N = 54 , 0.4 hours, 8 MB N = 216 , 9 hours, 148 MB CNV Spring 2015: LISSOM Orientation Maps 15 CNV Spring 2015: LISSOM Orientation Maps 16

Scaling cortical density Full-size V1 Map • Map scaled to cover most of CMVC figure 15.3 visual field • Allows testing ( a ) ( b ) ( c ) ( d ) ( e ) 36 × 36 : 48 × 48 : 72 × 72 : 96 × 96 : 144 × 144 : with full-size 0.17 hours, 0.32 hours, 0.77 hours, 1.73 hours, 5.13 hours, images 2.0 MB 5.2 MB 22 MB 65 MB 317 MB • 30 million Above minimum density (due to lateral radii), connections density not crucial for organization CNV Spring 2015: LISSOM Orientation Maps 17 CNV Spring 2015: LISSOM Orientation Maps 18 Sample Image RGC/LGN Response CNV Spring 2015: LISSOM Orientation Maps 19 CNV Spring 2015: LISSOM Orientation Maps 20

V1 Response with γ n V1 Orientation Map CNV Spring 2015: LISSOM Orientation Maps 21 CNV Spring 2015: LISSOM Orientation Maps 22 Afferent normalization RGC/LGN response to large image LISSOM mechanism for contrast invariant tuning: 0 1 @ X γ A ξ ρab A ρab,ij A ρab CMVC figure 8.2a,b s ij = , (1) 0 1 @ X 1 + γ n ξ ρab A ρab ξ ρab : activation of unit ( a, b ) in afferent CF ρ of neuron ( i, j ) A ab,ij is the corresponding afferent weight γ A , γ n are constant scaling factors Retinal activation LGN response RGC/LGN responds to most of the visible contours GCAL achieves similar results with lateral inhibition in RGC/LGN CNV Spring 2015: LISSOM Orientation Maps 23 CNV Spring 2015: LISSOM Orientation Maps 24

V1 without afferent normalization V1 with afferent normalization CMVC figure 8.2c-e CMVC figure 8.2c-e V1 response: V1 response: V1 response: V1 response: γ n = 0 , γ A = 3 . 25 γ n = 0 , γ A = 7 . 5 γ n = 0 , γ A = 3 . 25 γ n = 80 , γ A = 30 Cannot get selective response to all contours Responds based on contour, not contrast CNV Spring 2015: LISSOM Orientation Maps 25 CNV Spring 2015: LISSOM Orientation Maps 26 Tuning with afferent normalization OR Map: Gaussian 1.0 1.0 100% 90% 80% Peak settled response Peak settled response 0.8 0.8 70% 60% 50% CMVC figure 8.3 0.6 0.6 40% 30% CMVC figure 5.13 20% 0.4 0.4 10% 0.2 0.2 White line CFs only Retina LGN RFs LIs 0.0 0.0 o o o o o o o o o o o o 0 30 60 90 120 150 0 30 60 90 120 150 Orientation Orientation γ n = 0 , γ A = 3 . 25 γ n = 80 , γ A = 30 Sine grating tuning curve: • Without γ n : selectivity lost as contrast increases ORpref.&sel. OR H OR FFT • With γ n : always orientation-specific CNV Spring 2015: LISSOM Orientation Maps 27 CNV Spring 2015: LISSOM Orientation Maps 28

OR Map: +/- Gaussian OR Map: Retinal wave model White or black CMVC figure 5.13 CMVC figure 5.13 line CFs Some line, mostly Retina LGN RFs LIs Retina LGN RFs LIs OR map disrupted edge CFs due to phase columns ORpref.&sel. OR H OR FFT ORpref.&sel. OR H OR FFT CNV Spring 2015: LISSOM Orientation Maps 29 CNV Spring 2015: LISSOM Orientation Maps 30 OR Map: Smooth disks OR Map: Natural images All types of CFs CMVC figure 5.13 CMVC figure 5.13 Longer range lateral weights All edge CFs Retina LGN RFs LIs Retina LGN RFs LIs Histogram: horizontal, vertical bias ORpref.&sel. OR H OR FFT ORpref.&sel. OR H OR FFT CNV Spring 2015: LISSOM Orientation Maps 31 CNV Spring 2015: LISSOM Orientation Maps 32

OR Map: Uniform noise Modeling pre/post-natal phases Input patterns CMVC figure 5.13 Relatively Retina LGN RFs LIs 0 1000 5000 10000 unselective CFs • Prenatal: internal activity • Postnatal: natural images (Shouval et al. 1996) ORpref.&sel. OR H OR FFT CNV Spring 2015: LISSOM Orientation Maps 33 CNV Spring 2015: LISSOM Orientation Maps 34 Pre/post-natal V1 development Statistics drive development Input patterns Input patterns Orientation maps Orientation maps 0 1000 5000 10000 0 1000 5000 10000 • Biased image dataset: mostly landscapes • Neonatal map smoothly becomes more selective • Smoothly changes into horizontal-dominated map CNV Spring 2015: LISSOM Orientation Maps 35 CNV Spring 2015: LISSOM Orientation Maps 36

OR Histograms Stable development Ferret (Stevens et al. 2013) GCAL 0 ◦ 90 ◦ 180 ◦ 0 ◦ 90 ◦ 180 ◦ HLISSOM model Adult ferret V1 (Coppola et al. 1998) L • After postnatal training on Shouval natural images, orientation histogram matches results from ferrets GCAL map development is stable like ferret V1; LISSOM is • Model adapts to statistical structure of images unstable even w/o threshold changes, radius shrinking (L) CNV Spring 2015: LISSOM Orientation Maps 37 CNV Spring 2015: LISSOM Orientation Maps 38 Pinwheel density Summary (Stevens et al. 2013) • Development depends on features of input pattern • Orientation maps develop with many different patterns • Develops Gabor-type CFs with most inputs • Breaks up image into oriented patches • Animal orientation maps have an average of π • Scale response by local contrast to work for large images pinwheels per hypercolumn (Kaschube et al. 2010) • Matching biology requires prenatal, postnatal phases • GCAL is so far the only mechanistic model shown to have this property • Can get more elaborate: complex cells, multiple laminae/cell types, short-range inhibition, feedback, ... • LISSOM probably would as well, but requires unrealistic mechanisms to do so, since L does not CNV Spring 2015: LISSOM Orientation Maps 39 CNV Spring 2015: LISSOM Orientation Maps 40