1 KULKUNYA PRAYARACH, PH.D. Multiple Regression Analysis I. Basic - - PowerPoint PPT Presentation
Multiple Regression Analysis I. Basic Concepts II. Multicollinearity III. Autocorrelation IV. Heteroscedasticity V. Research & Group Work 1 KULKUNYA PRAYARACH, PH.D. Multiple Regression Analysis I. Basic Concepts II.
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Basic Concept: Multiple Regression MULTICOLLINEARITY AUTOCORRELATION HETEROSCEDASTICITY REASEARCH IN FINANCE
2
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
𝑍
𝑗 = 𝛾1 + 𝛾2𝑌1𝑗 + 𝛾3𝑌2𝑗 + 𝛾4𝑌3𝑗 + 𝑣𝑗
BASIC CONCEPTS: Multiple Regression
3
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
BASIC CONCEPTS: Normality Assumption for
𝑍
𝑗 = 𝛾1 + 𝛾2𝑌1𝑗 + 𝛾3𝑌2𝑗 + 𝛾4𝑌3𝑗 + 𝑣𝑗
4
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
BASIC CONCEPTS: Why we need Normality Assumptions of
5
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1. Influence of the omitted or neglected variables is small and at best random Central Limit Theorem (CLT)
are not strictly independent, their sum may still be normally distributed 3. Must be normally distributed in order to make assumption of OLS estimators , are normally distributed 4. Normal distribution is a comparatively simple distribution involving
5. Let’s say sample < 100 , normality assumption assumes a critical
6. Large samples, t and F statistics have appropriately.
TEST ‘BLUE’ Condition
BASIC CONCEPTS: Why we need Normality Assumptions of
6
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
20 40 60 80 100 120 140 ม.ค.-09 เม.ย.-09 ก.ค.-09 ต.ค.-09 ม.ค.-10 เม.ย.-10 ก.ค.-10 ต.ค.-10 ม.ค.-11 เม.ย.-11 ก.ค.-11 ต.ค.-11 ม.ค.-12 เม.ย.-12 OIL OIL_SA
component of a time series that is used when analyzing non-seasonal trends
Seasonally Adjusted : Census X12 Method 20 40 60 80 100 120 140 Jan Feb Mar Apr MayJune Jul Aug Sep Oct Nov Dec
Dubai Crude Oil Price
2009 2010 2011 2012
DATA PREPARATION: Seasonally Adjusted
7
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
DATA PREPARATION: Seasonally Adjusted
8
KULKUNYA PRAYARACH, PH.D.
: William H. Greene, Dr. Kulkunya Prayarach
VIF (βi) = 1 / (1-R2)
If Autocorrelation D.W. not 2, then AR(1) If Multicollinearity VIF > 10, then drop variable If Heteroscedasticity (p ≤ 0.05) Transform Regression Yi /xi = b0\Xi, +b1 Yi/Xi2 = b0\ Xi2, +b1/Xi Yi/ 2
i = b0, +b1Xi /2 iECONOMETRIC PROBLEMS
Multicollinearity Run: Xi = f(X1, X2,..,Xk)
Rule of Thumb: VIF ≤ 10 No Multi
VIF (i) = 1 / 1 –R2)
Stationary
(Unit Root Test: ADF) H0: Non Station (unit root)
Stationary : I(0) (Reject H0), p ≤ 0.05 Non Stationary : I(1)
(Fail to Reject H0) p> 0.05
Stationary Data at I(0) or I(1)
First Diff D(data)
Autocorrelation Test: Durbin Watson (D.W.) 2 No Autocorrelation Heteroscedasticity Test: White Test H0 : Homoscedasticity, p > 0.05 Clean Econometrix Problems
GO AHEAD!!! RUN OLS
ALTERNATIVE MODELS
VAR/VECM
Granger Causality Test
ARCH/GARCH
9
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
not change when shifted in time or space
>>> Parameters (mean, variance) will not change overtime or position
I(0)
Stationary at level
DATA PREPARATION: Stationary
10
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Random Walk without Drift
DATA PREPARATION: Random Walk (Unit Root Process)
Random Walk with Drift
11
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
… a test of stationary (or nonstationary) Where ut is a white noise error term. Test Augmented Dickey-Fuller (ADF) Test for Unit Root Test Test H0 : then UNIT ROOT (nonstationary) ~ Random walk without drift >>> CANNOT simply regress Yt on its lagged value Yt-1
where
DATA PREPARATION: Unit Root Test
12
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
STEP 1: First Differentiate STEP 2 : Test Unit Root again
Test H0: ~ >>> Unit root (ACCEPT)
STEP 3 : Second Differentiate Test H0: if reject then NO Unit root DATA PREPARATION: How to Solve Unit Root Problem
13
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Exchange Rate
27 29 31 33 35 37
1/1/2009 1/4/2009 1/7/2009 1/10/2009 1/1/2010 1/4/2010 1/7/2010 1/10/2010 1/1/2011 1/4/2011 1/7/2011 1/10/2011 1/1/2012 1/4/2012
20 40 60 80 100 120 140 160
1/3/2006 3/22/2006 6/8/2006 8/24/2006 11/9/2006 1/30/2007 4/18/2007 7/5/2007 9/20/2007 5 Dec 07 19 Feb 08 5 May 08 18 Jul 08 2 Oct 08 17 Dec 08 3 Mar 09 18 May 09 31 Jul 09 15 Oct 09 30 Dec 09 16 Mar 10 31 May 10 13 Aug 10 28 Oct 10 12 Jan 11 29 Mar 11 13 Jun 11 26 Aug 11 10 Nov 11 25 Jan 12 10 Apr 12Oil Price (WTI)
14
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
DATA PREPARATION: Gaussian, Standard or Classical Linear Regression Model (CLRM)
15
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
# of stock Abnormal profit % Assumption 1:
16
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Taylor Series Expansion Gauss-Newton iterative
Newton-Raphson iterative Method
Nonlinear Regression Assumption 2:
17
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Assumption 3:
18
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Assumption 4:
19
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Assumption 5:
20
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Internal Factor, External Factor, Shock
There must be sufficient variability in the values taken by the regressors. Assumption 6:
21
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Should be vary
Assumption 7:
22
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
problem of multicollinearity?
Assumption 8: MULTICOLLINEARITY: Is Multicollinearity seriously Problem?
23
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
MULTICOLLINEARITY: Is Multicollinearity seriously Problem?
linear relationship among some or all explanatory variables of a regression model
24
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Best Linear Unbiased Estimator
Collinearity does not destroy the property of BLUE
MULTICOLLINEARITY: Consequences of Multicollinearity
25
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Example: R2 = 0.8 but individual t tests wil show that none or few of the partial slope coefficients are statisticallly different from zero.
MULTICOLLINEARITY: Detecting of Multicollinearity
26
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
if 100 < k <1000 moderate multicollinearity k > 1000 severe multicollinearity
TOL >>> 0 or VIF > 10
MULTICOLLINEARITY: Detecting of Multicollinearity
27
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1. Do nothing “Multicollinearity is God’s will, not a problem with OLS or statistical techique in general” (Blanchard)
(1) A priori information (2) Combining cross-sectional and time series data (3) Dropping variable(s) and specification bias (4) Transformation of variables (5) (Additional or new data) Increase a size of sample (6) Polynomial Regression (7) Factor analysis
MULTICOLLINEARITY: Remedial Measures
28
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1. What is the nature of autocorrelation? 2. What are the theoretical and practical consequences of autocorrelation? 3. How does one remedy the problem of autocorrelation?
Assumption 9:
Autocorrelation: Nature of Autocorrelation
29
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Positive serial correlation Negative serial correlation Zero correlation
Autocorrelation: Nature of Autocorrelation
30
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Autocorrelation: Types of Autocorrelation
31
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Best Linear Unbiased Estimator
Autocorrelation destroy property of BLUE
variance
applied, are likely to give seriously misleading conclusions about the statiscal signifcance of the estimated regression coefficients
Autocorrelation: Consequences of using OLS in the Presence of Autocorrelation
32
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1. Graph Residual Plot 2. Run Test 3. Durbin-Watson Test 4. Breusch-Godfrey (BG) test ~ LM test
nonstochastic regressors, higher-order autoregressive : AR(1) , AR(2)) Autocorrelation: Detecting Autocorrelation
33
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1. Transform the original model >>>
2. First-Difference Method 3. When is not known then estimate from the residuals AR(1) 4. Change Model to ARCH and GARCH Models 5. Change Model to ARMA or ARIMA
Autocorrelation: Remedial Measure
34
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Assumption 10:
Heteroscedasticity: Nature of Heteroscedasticity
35
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
What is the nature of heteroscedasticity? What are its consequences? How does one detect it? What are the remedial measures?
Heteroscedasticity: Nature of Heteroscedasticity
36
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Why the variances of ui may be variable? 1. Following the error-learning models, as people learn their errors of behavior become smaller over time. 2. Growth oriented companies 3. As data collecting techniques improves, is likely to decrease. 4. The presence of outliers 5. Skewness
Heteroscedasticity: Nature of Heteroscedasticity
37
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Best Linear Unbiased Estimator “If we persist in using the usual testing procedure despite heteroscedasticity, whatever conclusions we draw or inferences we make may be very misleading”
Heteroscedasticity destroy property of BLUE
Heteroscedasticity: Consequences of using OLS in the Presence of Heteroscedasticity
38
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
1. Graph Residual Plot against Y and X
Heteroscedasticity: Detecting of Heteroscedasticity
39
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
, 1/X, Different variables
Heteroscedasticity: Remedial Measures
40
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Omitting Variables
Assumption 11:
41
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
42
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
43
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Heteroscedasticity
44
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
Variable Definitions
45
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
46
KULKUNYA PRAYARACH, PH.D.
Multiple Regression Analysis
WORK ORDERS : Multiple Regression
(1) Run Multiple Regression
Take care of seasonal effect and smooth data (by taking log)
(2) Test Multicollinearity and remedy if happens (3) Test Autocorrelation and remedy if happens (4) Test Heteroscedasticity and remedy if happens (5) Analyze your results 47