SLIDE 1
61A Extra Lecture 13 Announcements Prediction Regression 4 - - PowerPoint PPT Presentation
61A Extra Lecture 13 Announcements Prediction Regression 4 - - PowerPoint PPT Presentation
61A Extra Lecture 13 Announcements Prediction Regression 4 Regression Given a set of (x, y) pairs, find a function f(x) that returns good y values 4 Regression Given a set of (x, y) pairs, find a function f(x) that returns good y values
SLIDE 2
SLIDE 3
Prediction
SLIDE 4
Regression
4
SLIDE 5
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
SLIDE 6
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...]
SLIDE 7
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Data from home sales records in Ames, Iowa
SLIDE 8
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Data from home sales records in Ames, Iowa
SLIDE 9
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Price (thousands) Data from home sales records in Ames, Iowa
SLIDE 10
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Price (thousands) Data from home sales records in Ames, Iowa Measuring error: |y-f(x)| or (y-f(x))2 are both typical
SLIDE 11
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Price (thousands) Data from home sales records in Ames, Iowa Measuring error: |y-f(x)| or (y-f(x))2 are both typical Over the whole set of (x, y) pairs, we can compute the mean of the squared error
SLIDE 12
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Price (thousands) Data from home sales records in Ames, Iowa Measuring error: |y-f(x)| or (y-f(x))2 are both typical Over the whole set of (x, y) pairs, we can compute the mean of the squared error Squared error has the wrong units, so it's common to take the square root
SLIDE 13
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Price (thousands) Data from home sales records in Ames, Iowa Measuring error: |y-f(x)| or (y-f(x))2 are both typical Over the whole set of (x, y) pairs, we can compute the mean of the squared error Squared error has the wrong units, so it's common to take the square root The result is the "root mean squared error" of a predictor f on a set of (x, y) pairs
SLIDE 14
Regression
Given a set of (x, y) pairs, find a function f(x) that returns good y values
4
pairs = [(1656, 215.0), (896, 105.0), (1329, 172.0), ...] Square feet Price (thousands) Data from home sales records in Ames, Iowa Measuring error: |y-f(x)| or (y-f(x))2 are both typical Over the whole set of (x, y) pairs, we can compute the mean of the squared error Squared error has the wrong units, so it's common to take the square root The result is the "root mean squared error" of a predictor f on a set of (x, y) pairs (Demo)
SLIDE 15
Purpose of Newton's Method
Quickly finds accurate approximations to zeroes of differentiable functions!
5
SLIDE 16
Purpose of Newton's Method
Quickly finds accurate approximations to zeroes of differentiable functions! f(x) = x2 - 2
5
SLIDE 17
Purpose of Newton's Method
Quickly finds accurate approximations to zeroes of differentiable functions!
- 5
- 2.5
2.5 5
- 2.5
2.5
f(x) = x2 - 2
5
SLIDE 18
Purpose of Newton's Method
Quickly finds accurate approximations to zeroes of differentiable functions!
- 5
- 2.5
2.5 5
- 2.5
2.5
f(x) = x2 - 2 A "zero" of a function f is an input x such that f(x)=0
5
SLIDE 19
Purpose of Newton's Method
Quickly finds accurate approximations to zeroes of differentiable functions!
- 5
- 2.5
2.5 5
- 2.5
2.5
f(x) = x2 - 2 A "zero" of a function f is an input x such that f(x)=0 x=1.414213562373095
5
SLIDE 20
Purpose of Newton's Method
Quickly finds accurate approximations to zeroes of differentiable functions!
- 5
- 2.5
2.5 5
- 2.5
2.5
f(x) = x2 - 2 A "zero" of a function f is an input x such that f(x)=0 Application: Find the minimum of a function by finding the zero of its derivative x=1.414213562373095
5
SLIDE 21
Approximate Differentiation
6
SLIDE 22
Approximate Differentiation
6
SLIDE 23
Approximate Differentiation
Differentiation can be performed symbolically or numerically
6
SLIDE 24
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16
6
SLIDE 25
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x
6
SLIDE 26
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
SLIDE 27
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
SLIDE 28
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
f 0(x) = lim
a!0
f(x + a) − f(x) a
SLIDE 29
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
f 0(x) = lim
a!0
f(x + a) − f(x) a f 0(x) ≈ f(x + a) − f(x) a
SLIDE 30
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
f 0(x) = lim
a!0
f(x + a) − f(x) a f 0(x) ≈ f(x + a) − f(x) a
(if 𝑏 is small)
SLIDE 31
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
f 0(x) = lim
a!0
f(x + a) − f(x) a f 0(x) ≈ f(x + a) − f(x) a
(if 𝑏 is small)
SLIDE 32
Approximate Differentiation
Differentiation can be performed symbolically or numerically f(x) = x2 - 16 f'(x) = 2x f'(2) = 4
6
f 0(x) = lim
a!0
f(x + a) − f(x) a f 0(x) ≈ f(x + a) − f(x) a
(if 𝑏 is small)
SLIDE 33
Critical Points
7
SLIDE 34
Critical Points
Maxima, minima, and inflection points of a differentiable function occur when the derivative is 0
7
SLIDE 35
Critical Points
Maxima, minima, and inflection points of a differentiable function occur when the derivative is 0
7
http://upload.wikimedia.org/wikipedia/commons/f/fd/Stationary_vs_inflection_pts.svg
SLIDE 36
Critical Points
Maxima, minima, and inflection points of a differentiable function occur when the derivative is 0
7
http://upload.wikimedia.org/wikipedia/commons/f/fd/Stationary_vs_inflection_pts.svg
The global minimum of convex functions that are (mostly) twice-differentiable can be computed numerically using techniques that are similar to Newton's method
SLIDE 37
Critical Points
Maxima, minima, and inflection points of a differentiable function occur when the derivative is 0
7
http://upload.wikimedia.org/wikipedia/commons/f/fd/Stationary_vs_inflection_pts.svg
The global minimum of convex functions that are (mostly) twice-differentiable can be computed numerically using techniques that are similar to Newton's method (Demo)
SLIDE 38
Multiple Linear Regression
Given a set of (xs, y) pairs, find a linear function f(xs) that returns good y values
8
SLIDE 39
Multiple Linear Regression
Given a set of (xs, y) pairs, find a linear function f(xs) that returns good y values
8
A linear function has the form w • xs + b for vectors w and xs and scalar b
SLIDE 40
Multiple Linear Regression
Given a set of (xs, y) pairs, find a linear function f(xs) that returns good y values
8
A linear function has the form w • xs + b for vectors w and xs and scalar b (Demo)
SLIDE 41
Multiple Linear Regression
Given a set of (xs, y) pairs, find a linear function f(xs) that returns good y values
8
A linear function has the form w • xs + b for vectors w and xs and scalar b (Demo) Note: Root mean squared error can be optimized through linear algebra alone, but numerical
- ptimization works for a much larger class of related error measures