RESTRICTED BOLTZMANN MACHINES DANIEL KOHLSDORF LAST LECTURE: DEEP - - PowerPoint PPT Presentation

▶
restricted boltzmann machines
SMART_READER_LITE
LIVE PREVIEW

RESTRICTED BOLTZMANN MACHINES DANIEL KOHLSDORF LAST LECTURE: DEEP - - PowerPoint PPT Presentation

Aug 17, 2023 125 likes •381 views

RESTRICTED BOLTZMANN MACHINES DANIEL KOHLSDORF LAST LECTURE: DEEP AUTO ENCODERS Directed Model Reconstructs the input Back propagation Today: Probabilistic Interpretation Undirected Model DIRECTED VS UNDIRECTED MODELS VS PROBABILISTIC

slide-1
SLIDE 1

RESTRICTED BOLTZMANN MACHINES

DANIEL KOHLSDORF

slide-2
SLIDE 2

LAST LECTURE: DEEP AUTO ENCODERS

Directed Model Reconstructs the input Back propagation Today: Probabilistic Interpretation Undirected Model

slide-3
SLIDE 3

DIRECTED VS UNDIRECTED MODELS VS

slide-4
SLIDE 4

PROBABILISTIC UNDIRECTED MODELS

slide-5
SLIDE 5

PRELIMINARIES: MARKOV RANDOM FIELD

Cliques Probability Distribution

slide-6
SLIDE 6

RESTRICTED BOLTZMANN MACHINE

Hinton: A Practical Guide to Training Restricted Boltzmann Machines

slide-7
SLIDE 7

GIBBS SAMPLING

slide-8
SLIDE 8

GIBBS SAMPLING FOR RBM

h0 ~ p(h0 | v0,v1,v2,v3, h1,h2) h1 ~ p(h1 | v0,v1,v2,v3, h0,h2) h2 ~ p(h2 | v0,v1,v2,v3, h1,h0) h0, h1, h2 are independent

slide-9
SLIDE 9

GIBBS SAMPLING FOR RBM

h0 ~ p(h0 | v0,v1,v2,v3) h1 ~ p(h1 | v0,v1,v2,v3) h2 ~ p(h2 | v0,v1,v2,v3)

slide-10
SLIDE 10

RBM HIDDEN CONDITIONAL

p(h0 | v0,v1,v2,v3)

slide-11
SLIDE 11

RBM HIDDEN CONDITIONAL

h0 = p(h0 = 1 | v0,v1,v2,v3) > Uniform(0, 1) h1 = p(h1 = 1 | v0,v1,v2,v3) > Uniform(0, 1) h2 = p(h2 = 1 | v0,v1,v2,v3) > Uniform(0, 1)

slide-12
SLIDE 12

RBM VISIBLE CONDITIONAL

v0 = p(v0 = 1 | h0,h1,h2) > Uniform(0, 1) v1 = p(v1 = 1 | h0,h1,h2) > Uniform(0, 1) v2 = p(v2 = 1 | h0,h1,h2) > Uniform(0, 1) v3 = p(v3 = 1 | h0,h1,h2) > Uniform(0, 1)

slide-13
SLIDE 13

ALTERNATE GIBBS SAMPLING

h0, h1, h2 v0, v1, v2, v3 h0, h1, h2 v0, v1, v2, v3

…

slide-14
SLIDE 14

LEARNING

h0, h1, h2 v0, v1, v2, v3 h0, h1, h2 v0, v1, v2, v3

… Input Reconstruction Weight Update

slide-15
SLIDE 15

LEARNING: CONTRASTIVE DIVERGENCE

h0, h1, h2 v0, v1, v2, v3 v0, v1, v2, v3

Input Reconstruction

Just do it once!

slide-16
SLIDE 16

DEEP!

Hinton, G. E. and Salakhutdinov, R. R. (2006) Reducing the dimensionality of data with neural networks. Science

slide-17
SLIDE 17

CONVOLUTIONAL RESTRICTED BOLTZMANN MACHINES

Convolutional deep belief networks for scalable unsupervised learning of hierarchical

  • representations. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng.
slide-18
SLIDE 18

Edge Detector Gaussian From Aaron

slide-19
SLIDE 19

From Aaron

slide-20
SLIDE 20

Convolutional deep belief networks for scalable unsupervised learning of hierarchical

  • representations. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng.
slide-21
SLIDE 21

Convolutional deep belief networks for scalable unsupervised learning of hierarchical

  • representations. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng.
slide-22
SLIDE 22

Convolutional deep belief networks for scalable unsupervised learning of hierarchical

  • representations. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng.
slide-23
SLIDE 23

Convolutional deep belief networks for scalable unsupervised learning of hierarchical

  • representations. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng.
slide-24
SLIDE 24

Convolutional deep belief networks for scalable unsupervised learning of hierarchical

  • representations. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng.