What makes a model linear IN TR OD U C TION TO L IN E AR MOD E L - - PowerPoint PPT Presentation

▶
what makes a model linear
SMART_READER_LITE
LIVE PREVIEW

What makes a model linear IN TR OD U C TION TO L IN E AR MOD E L - - PowerPoint PPT Presentation

Dec 27, 2023 222 likes •561 views

What makes a model linear IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON Jason Vest u to Data Scientist Ta y lor Series Things to kno w: 1. appro x imate an y c u r v e 2. pol y nomial form : y = a0 + a1*x + a2*x**2 + a3*x**3 + ....

slide-1
SLIDE 1

What makes a model linear

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

Jason Vestuto

Data Scientist

slide-2
SLIDE 2

INTRODUCTION TO LINEAR MODELING IN PYTHON

Taylor Series

Things to know:

  • 1. approximate any curve
  • 2. polynomial form: y = a0 + a1*x + a2*x**2 + a3*x**3 + .... + an*x**n
  • 3. oen, rst order is enough: y = a0 + a1*x
slide-3
SLIDE 3

INTRODUCTION TO LINEAR MODELING IN PYTHON

Series Terms: a0=1

slide-4
SLIDE 4

INTRODUCTION TO LINEAR MODELING IN PYTHON

Series Terms: a1=1

slide-5
SLIDE 5

INTRODUCTION TO LINEAR MODELING IN PYTHON

Series Terms: a2=1

slide-6
SLIDE 6

INTRODUCTION TO LINEAR MODELING IN PYTHON

Combining all Terms

slide-7
SLIDE 7

INTRODUCTION TO LINEAR MODELING IN PYTHON

Real Data

slide-8
SLIDE 8

INTRODUCTION TO LINEAR MODELING IN PYTHON

Zeroth Order

slide-9
SLIDE 9

INTRODUCTION TO LINEAR MODELING IN PYTHON

First Order

slide-10
SLIDE 10

INTRODUCTION TO LINEAR MODELING IN PYTHON

Higher Order

slide-11
SLIDE 11

INTRODUCTION TO LINEAR MODELING IN PYTHON

Over-fitting

slide-12
SLIDE 12

Let's practice!

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

slide-13
SLIDE 13

Interpreting Slope and Intercept

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

Jason Vestuto

Data Scientist

slide-14
SLIDE 14

INTRODUCTION TO LINEAR MODELING IN PYTHON

Reminder: Terminology

Review:

y = a0 + a1*x x = independent variable, e.g. time y = dependent variable, e.g. distance traveled xp = 10; yp = a0 + a1*xp , "model prediction"

slide-15
SLIDE 15

INTRODUCTION TO LINEAR MODELING IN PYTHON

Intercept

x0 = 0 print(y(x0)) 100

slide-16
SLIDE 16

INTRODUCTION TO LINEAR MODELING IN PYTHON

Slope

slope = (225 - 100) / (5 - 0) print(slope) 25

slide-17
SLIDE 17

INTRODUCTION TO LINEAR MODELING IN PYTHON

Average Slope

slope = (350 - 100) / (10 - 0) print(slope) 25

slide-18
SLIDE 18

INTRODUCTION TO LINEAR MODELING IN PYTHON

Rescaling versus Dependency

slope = (212-32)/(100-0) # 180/100 = 9/5 intercept = 32

slide-19
SLIDE 19

Let's practice!

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

slide-20
SLIDE 20

Model Optimization

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

Jason Vestuto

Data Scientist

slide-21
SLIDE 21

INTRODUCTION TO LINEAR MODELING IN PYTHON

Residuals

residuals = y_model - y_data len(residuals) == len(y_data) True

slide-22
SLIDE 22

INTRODUCTION TO LINEAR MODELING IN PYTHON

Residuals Summed

residuals = y_model - y_data print(np.sum(residuals)) 0.0

slide-23
SLIDE 23

INTRODUCTION TO LINEAR MODELING IN PYTHON

Residuals Squared

residuals_squared = np.square(y_model - y_ print(np.sum(residuals_squared)) 65.1

slide-24
SLIDE 24

INTRODUCTION TO LINEAR MODELING IN PYTHON

RSS

resid_squared = np.square(y_model - y_data RSS = np.sum(resid_squared)

slide-25
SLIDE 25

INTRODUCTION TO LINEAR MODELING IN PYTHON

RSS

RSS = np.sum(np.square(y_model - y_data)) print(RSS) 5.9

slide-26
SLIDE 26

INTRODUCTION TO LINEAR MODELING IN PYTHON

Variation of RSS

Minimum value of RSS gives minimum residuals Minimum residuals give the best model

slide-27
SLIDE 27

Let's practice!

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

slide-28
SLIDE 28

Least-Squares Optimization

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON

Jason Vestuto

Data Scientist

slide-29
SLIDE 29

INTRODUCTION TO LINEAR MODELING IN PYTHON

Minima of RSS

Seing RSS slope = zero, and some calculus, yields:

a = covariance(x,y)/variance(x) a = mean(y) − a × mean(x)

1 1

slide-30
SLIDE 30

INTRODUCTION TO LINEAR MODELING IN PYTHON

Optimized by Numpy

Numpy expressions of optimal slope and intercept

x_mean = np.mean(x) y_mean = np.mean(y) x_dev = x - x_mean y_dev = y - y_mean a1 = np.sum( x_dev * y_dev ) / np.sum( x_dev**2 ) a0 = y_mean - (a1*x_mean)

slide-31
SLIDE 31

INTRODUCTION TO LINEAR MODELING IN PYTHON

Optimized by Scipy

from scipy import optimize x_data, y_data = load_data() def model_func(x, a0, a1): return a0 + (a1*x) param_opt, param_cov = optimize.curve_fit(model_func, x_data, y_data) a0 = param_opt[0] # a0 is the intercept in y = a0 + a1*x a1 = param_opt[1] # a1 is the slope in y = a0 + a1*x

slide-32
SLIDE 32

INTRODUCTION TO LINEAR MODELING IN PYTHON

Optimized by Statsmodels

from statsmodels.formula.api import ols x_data, y_data = load_data() df = pd.DataFrame(dict(x_name=x_data, y_name=y_data)) model_fit = ols(formula="y_name ~ x_name", data=df).fit() y_model = model_fit.predict(df) x_model = x_data a0 = model_fit.params['Intercept'] a1 = model_fit.params['x_name']

slide-33
SLIDE 33

Let's practice!

IN TR OD U C TION TO L IN E AR MOD E L IN G IN P YTH ON