Deep Neural Networks Reveal a Gradient in the Complexity of Neural - - PowerPoint PPT Presentation

▶
deep neural networks reveal a gradient in the complexity
SMART_READER_LITE
LIVE PREVIEW

Deep Neural Networks Reveal a Gradient in the Complexity of Neural - - PowerPoint PPT Presentation

Jul 29, 2023 216 likes •857 views

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Umut Gcl and Marcel A. J. van Gerven Article overview by Ilya Kuzovkin Computational Neuroscience Seminar University of Tartu

slide-1
SLIDE 1

Article overview by Ilya Kuzovkin

Umut Güclü and Marcel A. J. van Gerven

Computational Neuroscience Seminar University of Tartu 2015

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-2
SLIDE 2

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-3
SLIDE 3

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels classes

Linear

“spider” “cat”

slide-4
SLIDE 4

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels classes

…

hidden layer

Non-linear

“cat” “spider”

slide-5
SLIDE 5

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels classes

…

hidden layer

…

hidden layer

Deep

“cat” “spider”

slide-6
SLIDE 6

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

“spider” “cat” important feature

slide-7
SLIDE 7

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

“spider” important feature

RUN!

“cat”

slide-8
SLIDE 8

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

“spider” important feature

RUN!

Convolutional filter

“cat”

slide-9
SLIDE 9

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Convolutional (and pooling) layer

slide-10
SLIDE 10

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

…

pixels classes

…

hidden layer

…

hidden layer

…

convolutional layer

Deep Convolutional Neural Network

“cat” “spider”

slide-11
SLIDE 11

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-12
SLIDE 12

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-13
SLIDE 13

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-14
SLIDE 14

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013
slide-15
SLIDE 15 Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-16
SLIDE 16 Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-17
SLIDE 17 Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-18
SLIDE 18 Matthew D. Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks 2013

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-19
SLIDE 19

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-20
SLIDE 20

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

Two-stream hypothesis

slide-21
SLIDE 21

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-22
SLIDE 22

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-23
SLIDE 23

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-24
SLIDE 24

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-25
SLIDE 25

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-26
SLIDE 26

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-27
SLIDE 27

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

?

slide-28
SLIDE 28

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-29
SLIDE 29

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-30
SLIDE 30
slide-31
SLIDE 31
slide-32
SLIDE 32

96 x 37x 37 = 131,424

slide-33
SLIDE 33

96 x 37x 37 = 131,424

slide-34
SLIDE 34

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

slide-35
SLIDE 35

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

slide-36
SLIDE 36

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

slide-37
SLIDE 37

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

slide-38
SLIDE 38

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

Train linear regression model

slide-39
SLIDE 39

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

Train linear regression model Test it

slide-40
SLIDE 40

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

Train linear regression model Test it

r = 0.22

slide-41
SLIDE 41

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

Train linear regression model Test it

r = 0.22

Train linear regression model Test it

slide-42
SLIDE 42

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

Train linear regression model Test it

r = 0.22

Train linear regression model Test it

r = 0.67

slide-43
SLIDE 43

96 x 37x 37 = 131,424 256 x 17x 17 = 73,984

Train linear regression model Test it

r = 0.22

Train linear regression model Test it

r = 0.67

slide-44
SLIDE 44

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream

slide-45
SLIDE 45

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

slide-46
SLIDE 46

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... .

1888

slide-47
SLIDE 47

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... .

1888

slide-48
SLIDE 48

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

slide-49
SLIDE 49

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

slide-50
SLIDE 50

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

Low Mid High

  • blob
  • contrast
  • edge
  • contour
  • shape
  • texture
  • pattern
  • object
  • object part

human-assigned to 9 categories

slide-51
SLIDE 51

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

Low Mid High

  • blob
  • contrast
  • edge
  • contour
  • shape
  • texture
  • pattern
  • object
  • object part

human-assigned to 9 categories

  • 1. Divide 1888 neurons into 9

categories

slide-52
SLIDE 52

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

Low Mid High

  • blob
  • contrast
  • edge
  • contour
  • shape
  • texture
  • pattern
  • object
  • object part

human-assigned to 9 categories

  • 1. Divide 1888 neurons into 9

categories

  • 2. Predict activity of each voxel

from group-by-group

slide-53
SLIDE 53

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

Low Mid High

  • blob
  • contrast
  • edge
  • contour
  • shape
  • texture
  • pattern
  • object
  • object part

human-assigned to 9 categories

  • 1. Divide 1888 neurons into 9

categories

  • 2. Predict activity of each voxel

from group-by-group

  • 3. For each voxel find the

group, which best predicts voxel’s activity

slide-54
SLIDE 54

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

Low Mid High

  • blob
  • contrast
  • edge
  • contour
  • shape
  • texture
  • pattern
  • object
  • object part

human-assigned to 9 categories

  • 1. Divide 1888 neurons into 9

categories

  • 2. Predict activity of each voxel

from group-by-group

  • 3. For each voxel find the

group, which best predicts voxel’s activity

  • 4. Assign each of 1888 DNN

neurons to a visual layer: V1, V2, V4, LO

slide-55
SLIDE 55

NEXT COOL THING: CATEGORIES OF FEATURES

…

ImageNet validation set

... . .

1888

deconvolution

.

Low Mid High

  • blob
  • contrast
  • edge
  • contour
  • shape
  • texture
  • pattern
  • object
  • object part

human-assigned to 9 categories

  • 1. Divide 1888 neurons into 9

categories

  • 2. Predict activity of each voxel

from group-by-group

  • 3. For each voxel find the

group, which best predicts voxel’s activity

  • 4. Assign each of 1888 DNN

neurons to a visual layer: V1, V2, V4, LO

  • 5. Map visual layers to

categories

slide-56
SLIDE 56

NEXT COOL THING: CATEGORIES OF FEATURES

slide-57
SLIDE 57

OTHER RESULTS

Correlation between predicted responses between pairs of voxel groups

slide-58
SLIDE 58

OTHER RESULTS

Selectivity of visual areas to feature maps of varying complexity

slide-59
SLIDE 59

OTHER RESULTS

Distribution of the receptive field centers

slide-60
SLIDE 60

OTHER RESULTS

Biclustering of voxels and feature maps

slide-61
SLIDE 61

SUMMARY

slide-62
SLIDE 62

An intracranial dataset we have. How to repeat the result?

slide-63
SLIDE 63

An intracranial dataset we have. How to repeat the result?

vs.