Beginning Neural Networks [Assignment 6] Paolo Gabriel ECE 228, - - PowerPoint PPT Presentation

▶
beginning neural networks assignment 6 paolo gabriel
SMART_READER_LITE
LIVE PREVIEW

Beginning Neural Networks [Assignment 6] Paolo Gabriel ECE 228, - - PowerPoint PPT Presentation

Jan 25, 2023 280 likes •395 views

Beginning Neural Networks [Assignment 6] Paolo Gabriel ECE 228, Spring 2018 Objectives Data description Introduce a simple neural network model Implement getGradient() for sum-of-squares error function Optimize weights of NN

slide-1
SLIDE 1

Beginning Neural Networks [Assignment 6] Paolo Gabriel

ECE 228, Spring 2018

slide-2
SLIDE 2

Objectives

  • Data description
  • Introduce a simple neural network model
  • Implement getGradient() for sum-of-squares error function
  • Optimize weights of NN using gradient descent
slide-3
SLIDE 3

The data

  • Sunspot dataset (unit-scaled)
  • Time series of relative sunspot

activity

Input Target x_1 x_2 x_3 x_4 x_5 x_6 … … … … … … x_(n-5) x_(n-4) x_(n-3) x_(n-2) x_(n-1) x_n

slide-4
SLIDE 4

The network

x1 x2 x3 x4 x5 Input Hidden Output z1 z2 z3 z4 z5 z6 z7 z8 w(1) y1 w(2) 1 1 bias bias Given

  • Input, Output (x, y)
  • NN construction parameters (5x8x1)
  • Error function parameters (sum-of-squares, α)
  • Gradient descent parameters (stop condition, η)

Goal

  • find network parameters that predict value

based on previous 5 ordered observations

slide-5
SLIDE 5

Finding network weights

  • Specifically, gradient of network error function E(w)
  • Find parameter changes that cause most change in error (make
  • pposite changes)

{Prediction error} Noise control

slide-6
SLIDE 6

x1 x2 x3 x4 x5 1 z1 z2 z3 z4 z5 z6 z7 z8 1 y1 bias bias Input Hidden Output w(1) w(2) Goal: calculate estimated output using current network weights

Forward pass

slide-7
SLIDE 7

x1 x2 x3 x4 x5 1 z1 z2 z3 z4 z5 z6 z7 z8 1 y1 bias bias Input Hidden Output w(1) w(2) Goal: calculate gradient of error function based on estimates from current network weights

Backward pass

Input -> hidden Hidden ->

  • utput

“ “

Error function x6 True value

slide-8
SLIDE 8

[6.1] getGradient()

Input -> hidden Hidden ->

  • utput
slide-9
SLIDE 9

[6.2] Gradient Descent

Single step Typical stopping criteria can be based on number of iterations or change in error Multiple steps

Step size, η

slide-10
SLIDE 10

Final comments

  • Bias added to end of arrays in this

assignment… generally start/end ok

  • tanh used to keep hidden node

values near [-1,1]

  • Things that affect neural network:
  • Parameters: step size, stop criteria,

iteration range, initialization, network size and shape, error/optimization algorithms, hidden unit functions