Introduction to Deep Learning Outline Deep Learning RNN - - PowerPoint PPT Presentation

▶
introduction to deep learning outline
SMART_READER_LITE
LIVE PREVIEW

Introduction to Deep Learning Outline Deep Learning RNN - - PowerPoint PPT Presentation

Aug 14, 2023 531 likes •1.11k views

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples RNNs Some slides borrowed from Fei-Fei Li & Justin Johnson &

slide-1
SLIDE 1

Introduction to Deep Learning

slide-2
SLIDE 2

Outline

  • Deep Learning

○ RNN ○ CNN ○ Attention ○ Transformer

  • Pytorch

○ Introduction ○ Basics ○ Examples

slide-3
SLIDE 3

RNNs

Some slides borrowed from Fei-Fei Li & Justin Johnson & Serena Yeung at Stanford.

slide-4
SLIDE 4

Vanilla Neural Networks

Input Output Hidden Layers Input Output Hidden Layers

House Price Prediction

slide-5
SLIDE 5

How to model sequences?

  • Text Classification: Input Sequence -> Output label
  • Translation: Input Sequence -> Output Sequence
  • Image Captioning: Input image -> Output Sequence
slide-6
SLIDE 6

RNN- Recurrent Neural Networks

Vanilla Neural Networks e.g.- Image Captioning e.g.- Text Classification e.g.- Translation e.g.- POS tagging

slide-7
SLIDE 7

RNN- Representation

Input Vector Output Vector Hidden state fed back into the RNN cell

slide-8
SLIDE 8

RNN- Recurrence Relation

Input Vector Output Vector Hidden state fed back into the RNN cell The RNN cell consists of a hidden state that is updated whenever a new input is received. At every time step, this hidden state is fed back into the RNN cell.

slide-9
SLIDE 9

RNN- Rolled out representation

slide-10
SLIDE 10

RNN- Rolled out representation

Same Weight matrix- W Individual Losses Li

slide-11
SLIDE 11

RNN- Backpropagation Through Time

Forward pass through entire sequence to produce intermediate hidden states, output sequence and finally the loss. Backward pass through the entire sequence to compute gradient.

slide-12
SLIDE 12

RNN- Backpropagation Through Time

Running Backpropagation through time for the entire text would be very slow. Switch to an approximation- Truncated Backpropagation Through Time

slide-13
SLIDE 13

RNN- Truncated Backpropagation Through Time

Run forward and backward through chunks of the sequence instead of whole sequence

slide-14
SLIDE 14

RNN- Truncated Backpropagation Through Time

Carry hidden states forward in time forever, but only backpropagate for some smaller number of steps

slide-15
SLIDE 15

RNN- Types

The 3 most common types of Recurrent Neural Networks are- 1. Vanilla RNN 2. LSTM (Long Short-Term Memory) 3. GRU (Gated Recurrent Units) Some good resources- Understanding LSTM Networks An Empirical Exploration of Recurrent Network Architectures Recurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano Stanford CS231n: Lecture 10 | Recurrent Neural Networks

slide-16
SLIDE 16

CNNs

Some slides borrowed from Fei-Fei Li & Justin Johnson & Serena Yeung at Stanford.

slide-17
SLIDE 17

Fully Connected Layer

Input 32x32x3 image Flattened image 32*32*3 = 3072 Weight Matrix Output

slide-18
SLIDE 18

Convolutional Layer

Input 32x32x3 image Filter 5x5x3

Convolve the filter with the image i.e. “slide over the image spatially, computing dot products” Filters always extend the full depth of the input volume.

slide-19
SLIDE 19

Convolutional Layer

At each step during the convolution, the filter acts on a region in the input image and results in a single number as

  • utput.

This number is the result of the dot product between the values in the filter and the values in the 5x5x3 chunk in the image that the filter acts on. Combining these together for the entire image results in the activation map.

slide-20
SLIDE 20

Convolutional Layer

Filters can be stacked together. Example- If we had 6 filters of shape 5x5, each would produce an activation map of 28x28x1 and

  • ur output would be a “new

image” of shape 28x28x6.

slide-21
SLIDE 21

Convolutional Layer

Visualizations borrowed from Irhum Shafkat’s blog.

slide-22
SLIDE 22

Convolutional Layer

Visualizations borrowed from vdumoulin’s github repo.

Standard Convolution Convolution with Padding Convolution with strides

slide-23
SLIDE 23

Convolutional Layer

Output Size: (N - F)/stride + 1 e.g. N = 7, F = 3, stride 1 => (7 - 3)/1 + 1 = 5 e.g. N = 7, F = 3, stride 2 => (7 - 3)/2 + 1 = 3

slide-24
SLIDE 24

Pooling Layer

  • makes the

representations smaller and more manageable

  • perates over each

activation map independently

slide-25
SLIDE 25

Max Pooling

slide-26
SLIDE 26

ConvNet Layer

Image credits- Saha’s blog.

slide-27
SLIDE 27

ConvNet Layer

Image credits- Saha’s blog.

slide-28
SLIDE 28

Attention

Some slides borrowed from Sarah Wiegreffe at Georgia Tech.

slide-29
SLIDE 29

RNN

slide-30
SLIDE 30

RNN - Attention

slide-31
SLIDE 31

RNN - Attention

slide-32
SLIDE 32

RNN - Attention

slide-33
SLIDE 33

RNN - Attention

slide-34
SLIDE 34

RNN - Attention

slide-35
SLIDE 35

RNN - Attention

slide-36
SLIDE 36

RNN - Attention

slide-37
SLIDE 37

RNN - Attention

slide-38
SLIDE 38

RNN - Attention

slide-39
SLIDE 39

Attention

slide-40
SLIDE 40
slide-41
SLIDE 41

Drawbacks of RNN

slide-42
SLIDE 42

Transformer

Some slides borrowed from Sarah Wiegreffe at Georgia Tech.

slide-43
SLIDE 43

Transformer

slide-44
SLIDE 44

Self-Attention

slide-45
SLIDE 45

Self-Attention

slide-46
SLIDE 46

Self-Attention

slide-47
SLIDE 47

Self-Attention

slide-48
SLIDE 48

Multi-Head Self-Attention

slide-49
SLIDE 49

Retaining Hidden State Size

slide-50
SLIDE 50

Details of Each Attention Sub-Layer of Transformer Encoder

slide-51
SLIDE 51

Each Layer of Transformer Encoder

slide-52
SLIDE 52

Positional Encoding

slide-53
SLIDE 53

Each Layer of Transformer Decoder

slide-54
SLIDE 54

Transformer Decoder - Masked Multi-Head Attention

Problem of Encoder self-attention: we can’t see the future !

slide-55
SLIDE 55

Transformer