Outline The Simple Linear Regression Model (12.1) Fitting the - - PDF document
2/22/2007 219323 Probability and Statistics for Software Statistics for Software and Knowledge Engineers Lecture 13: Simple Linear Regression Simple Linear Regression and Correlation Monchai Sopitkamon, Ph.D. Outline The Simple Linear
Monchai Sopitkamon, Ph.D.
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
Purpose of regression analysis: predict the
Purpose of correlation analysis: measure
yi = β0 + β1xi Yi ∼ N(β0 + β1xi, σ2) Yi N(β0 β1xi, σ )
Intercept parameter Slope parameter
Sim ple linear regression m odel
I nterpretation of the error variance σ2 error variance σ
β1 > 0 positive relationship β1 = 0 No relationship
30 35
β1 < 0 negative relationship SLR model is not appropriate for nonlinear relationship
5 10 15 20 25 2 4 6 8 10 12 14 16
for nonlinear relationship
Ex.67 pg.536: Car Plant Electricity Usage
2 2.2 2.4 2.6 2.8 3 3.2 3.4 3.6 3.8 3 3.5 4 4.5 5 5.5 6 6.5 Electricity usage Productiom
Excel sheet
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
(errors)
error The least squares fit
i i
1
i
1
2
2 1 1 1 2 1 2
) ( ) ˆ (
= = =
+ − = − = =
n i i i n i i i n i i
x y y y e SSE β β
1
=
= n i i
e
Method of Least Squares
n i i n i i i 1 2 2 1 1
= =
1
2 ˆ 2 − = n SSE σ
n-2 since two regression parameters need to be computed first
Ex.67 pg.545: Car Plant Electricity Usage x 4 885 =
n i i n i i i 1 1 2 2 1 1
= =
y x y x
i i i i
846 . 2 885 . 4 12 253 . 169 169.253 291.231 x 2.846 4.885
12 1 12 1 2 i
× × − = = = = =
= =
β x y 499 . 409 . 409 . 885 . 4 4998 . 846 . 2 4988 . 885 . 4 12 231 . 291
2 1
+ = ∴ = × − = = × − = β β
Excel sheet
Ex.67 pg.545: Car Plant Electricity Usage
y = 0.498x + 0.409 R² = 0.802
2.6 2.8 3 3.2 3.4 3.6 3.8 Electricity usage 2 2.2 2.4 3 3.5 4 4.5 5 5.5 6 6.5 Productiom
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
Apply the similar ANOVA approach as the
Consider the variability in the dependent
Hypothesis test:
2
n i −
1
i i
=
( )
SSE SST y y SSR
n i i
− = − =∑
=1 2
ˆ
=
− =
n i i i
y y SSE
1 2
ˆ
The sum of squares for a sim ple linear regression
The analysis of variance table for a sim ple linear regression analysis Hypothesis test:
The two-sided p-value is
Coefficient of determination (R2): fraction
2 The closer R2 is to one, the better is the regression model.
The coefficient of determination R2 is larger in scenario I I than in scenario I
Ex.67 pg.572: Car Plant Electricity Usage
2124 1 53 . 40 0299 . 2124 . 1 = = = SSR MSE MSR F
Excel sheet
802 . 5115 . 1 2124 . 1
2
= = = SST SSR R
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
Residuals: differences between the
Residual analysis can be used to
i i i
– Identify outliers – Check if the fitted model is good – Check if the variance of error is constant – Check if the error terms are normally distributed
Excel sheet
Plot the residuals ei against the values of the
Random scatter plot indicates no problem
If (standardized residual) is > 3, data
If there are outliers, they should be removed
σ ˆ /
i
e
Excel sheet
Residual plot indicating points that m ay be outliers
If residual plots show positive and negative
A grouping of positive d i id l and negative residuals indicates that the linear m odel is inappropriate
If the residual plot shows a “funnel
A funnel shape in the residual plot indicates residual plot indicates a non-constant error variance
Normal probability plot (normal scores plot)
A norm al scores plot of a sim ulated sam ple from a cores sam ple from a norm al distribution, w hich show s the points lying approxim ately
Normal sc
Exhibits non-normal
distribution of
res
residuals
Linear modeling
approach may not be used
Norm al scores plots
Normal scor
Norm al scores plots
from non-norm al distributions, w hich show nonlinear patterns
Normal scores
The Simple Linear Regression Model
Fitting the Regression Line (12.2) The Analysis of Variance Table (12.6) Residual Analysis (12.7) Correlation Analysis (12.9)
From the correlation eq. in Section 2.5.4,
) , ( Cov ) , ( Corr Y X Y X = = ρ
which measures the strength of linear association between two jointly distributed RVs X and Y
The sample correlation coefficient r for a set of
paired data observations (xi, yi) is
) ( Var ) ( Var ) , ( Corr Y X Y X ρ
− −
n i i
y y x x ) )( (
= = = = = =
− − − = − − =
n i i n i i n i i i n i i n i i i i i
y n y x n x y x n y x y y x x y y r
1 2 2 1 2 2 1 1 2 1 2 1
) ( ) ( ) )( (
(-1 ≤ r ≤ 1)
r = 0 no linear association r < 0 negative linear association r > 0 positive linear association
( l l ti ft )2 ( ft f d t i ti )
r is unchanged if x and y are swapped, which is in
contrast to regression analysis, which requires that
explanatory
r is also not affected by any linear combination of
the variables, e.g., (sample correlation coeft.)2 = (coeft. of determination)
d cy y b ax x
i i i i
+ = ′ + = ′ and
the variables, e.g.,
d cy y b ax x
i i i i
a d
Hypothesis test:
2
1 2 r n r t − − =
Ex.69 pg.588: Cranial Circumferences 255 10745 . 3
XY
S r
255 . 457 . 99 489 . 1 = = =
YY XX XY
S S r
2 2
255 . 1 18 255 . 1 2 − × = − − = r n r t
Excel sheet