Business Statistics CONTENTS Multiple regression Dummy regressors - - PowerPoint PPT Presentation

MULTIPLE REGRESSION ANALYSIS AND OTHER ISSUES

Business Statistics

Multiple regression Dummy regressors Assumptions of regression analysis Predicting with regression analysis Old exam question Further study CONTENTS

The regression model so far is for one dependent variable (𝑍) and one independent (explanatory) variable (𝑌) ▪ There are many cases where several explanatory variables might play a role

▪ ... might “explain” the dependent variable 𝑍

▪ Example: house prices depend on

▪ floor area ▪ ground area (first floor + garden) ▪ number of rooms ▪ age of the house ▪ etc.

MULTIPLE REGRESSION

Generalize simple regression model ▪ from 𝑍 = 𝛾0 + 𝛾1𝑌1 + 𝜁 ▪ to 𝑍 = 𝛾0 + 𝛾1𝑌1 + 𝛾2𝑌2 + 𝜁 ▪ or even to 𝑍 = 𝛾0 + 𝛾1𝑌1 + 𝛾2𝑌2 + ⋯ + 𝛾𝑙𝑌𝑙 + 𝜁 Multiple regression ▪ a quite obvious extension ▪ we can reuse much of the theory of simple regression ▪ still based on OLS, 𝑆2, 𝐺-test, and 𝑢-test MULTIPLE REGRESSION

Now, you’ll understand why we used a subscript 0 for the constant in 𝛾0 ...

SPSS output Estimated model: ෠ 𝑍 = −217603 + 5347𝑌1 + 225𝑌2 MULTIPLE REGRESSION

“Step 0” (statistical model): 𝑍 = 𝛾0 + 𝛾1𝑌1 + 𝛾2𝑌2 + 𝜁, with 𝜁~𝑂 0, 𝜏2 Step 1: ▪ 𝐼0: 𝛾1 = 𝛾2 = 0; 𝐼1: at least one of these not 0 Step 2: ▪ Sample statistic: 𝐺 =

𝑁𝑇𝑆 𝑁𝑇𝐹; reject for “too large” values

Step 3: ▪ Under 𝐼0: 𝐺~𝐺2,𝑜−3; assumption: see model (step 0) Step 4: ▪ 𝐺calc = ⋯; 𝐺crit = 𝐺2,𝑜−3;𝛽 Step 5: ▪ reject/not reject 𝐼0 MULTIPLE REGRESSION

with 𝑙 regressors: df1 = 𝑙 df2 = 𝑜 − 𝑙 − 1 mind that the null hypothesis does not include the constant (intercept) 𝛾0

Rejecting the 𝐺-test in multiple regressions means: ▪ at least one of the slope coefficients differs from 0

▪ “not 𝛾1 = 𝛾2 = 0”

▪ which one differs (or differ) from 0 must be investigated by separate 𝑢-tests So, ▪ while in simple regression the overall 𝐺-test and the 𝑢-test for 𝛾1 do exactly the same thing ... ▪ ... the two tests have a complimentary role in multiple regression

▪ first look at overall 𝐺, then go to the individual 𝑢s

MULTIPLE REGRESSION

First, overall model test, using 𝐺-test Next, test each slope coefficient, using 𝑙 times a 𝑢-test MULTIPLE REGRESSION

not interesting

What does it mean when in multiple regression

• a. the overall 𝐺-test yields a significant result?
• b. a 𝑢-test of an individual coefficient 𝛾3 yields a significant

result? EXERCISE 1

Example: ▪ overall 𝐺-test: highly significant ▪ both regression slopes: highly significant ▪ coefficient of determination (𝑆2): very high (90%) ▪ a very useful model ▪ in fact: better than the simple regression model with 𝑆2 = 82% MULTIPLE REGRESSION

Observe: ▪ including more explanatory variables will in general improve the model ▪ 𝑆2 will increase, even if we include “non-sense” variables (e.g., street number of the house) ▪ 𝑆adj

2

(“R-square-adjusted”) penalizes for including “too many” regressors ▪ 𝑆adj

2

= 1 −

𝑇𝑇𝐹/𝑜−𝑙−1 𝑇𝑇𝑈/𝑜−1 while 𝑆2 = 1 − 𝑇𝑇𝐹 𝑇𝑇𝑈

MULTIPLE REGRESSION

House prices (numerical) depend on: ▪ numerical variables (floor area, ground area, etc.) ▪ binary categorical variables (with/without garage, etc.) ▪ other categorical variables (no/free/paid parking, etc.) However: ▪ regression for numerical 𝑌 and numerical 𝑍 ▪ ANOVA for categorical 𝑌 and numerical 𝑍 So, how to combine numerical 𝑌1 and categorical 𝑌2? Solution: dummy variables for categorical variable ▪ dummy regressors/dummy regression DUMMY REGRESSORS

We can include dummy variables in multiple regression ▪ Splitting binary in several binary

▪ original variable: garage = no/yes ▪ garage: 0=no; 1=yes

▪ Splitting non-binary in several binary

▪ original variable: parking = no/free/paid ▪ free_parking: 0=no; 1=yes ▪ paid_parking: 0=no; 1=yes

▪ Dummy variables only for independent (𝑌) variables

▪ never for dependent (𝑍) variable ▪ 𝑍 must be numerical (think about 𝜁~𝑂)

DUMMY REGRESSORS

Omitted variable: no_parking (redundant): free=0, paid=0 Omitted variable: no_garage (redundant): garage=0

Example ▪ House price (𝑍) as a function of

▪ floor area (𝑌1) ▪ dummy for garden (𝑌2; 0=No, 1=Yes) ▪ 𝑄𝑠𝑗𝑑𝑓 = −261741 + 6040𝐺𝑚𝑝𝑝𝑠𝐵𝑠𝑓𝑏 + 21825𝐻𝑏𝑠𝑒𝑓𝑜

DUMMY REGRESSORS

meaning 21825 € extra when there is a garden (whatever the size)

▪ Use dummy variables only for the independent (explanatory) variable

▪ not for the dependent variable.(logistic regression, not in this course!)

▪ It is quite common to indicate dummy explanatory variables with a 𝐸 instead of an 𝑌

▪ for instance: 𝑍 = 𝛾0 + 𝛾1𝑌1 + 𝛾2𝐸2 + 𝛾3𝐸3 + 𝜁

DUMMY REGRESSORS

We want to explain car prices in terms of 1) engine power 2) number of seats 3) gas/diesel/electric. What is the theoretical model? EXERCISE 2

The OLS equations always find coefficients 𝑐0, 𝑐1, … that minimize the residual sum of squares (𝑇𝑇𝐹) ▪ so no assumptions required for that part But when testing the model (and when testing the coefficients 𝛾1, 𝛾2, …) ▪ we need to assume a statistical model with 𝜁~𝑂 0, 𝜏2 :

▪ the residual terms should be normally distributed ▪ the residual terms should come from a distribution with constant variance ▪ the residual terms should be independent of each other ▪ there should be a linear relationship between the 𝑌-variable(s) and 𝑍

ASSUMPTIONS OF REGRESSION ANALYSIS

A final word on the residual 𝜁~𝑂 0, 𝜏2 ▪ Theoretical regression model

▪ 𝑍 = 𝛾0 + 𝛾1𝑌1 + 𝛾2𝑌2 + ⋯ + 𝛾𝑙𝑌𝑙 + 𝜁

▪ Estimated regression model

▪ ෠ 𝑍 = 𝑐0 + 𝑐1𝑌1 + 𝑐2𝑌2 + ⋯ + 𝛾𝑙𝑌𝑙

▪ Observations

▪ 𝑍

𝑗 = 𝑐0 + 𝑐1𝑌1,𝑗 + 𝑐2𝑌2,𝑗 + ⋯ + 𝛾𝑙𝑌𝑙,𝑗 + 𝑓𝑗

▪ And the standard deviation of the residual term 𝜏 = 𝜏2

▪ is estimated by 𝑡 =

𝑇𝑇𝐹 𝑜−𝑙−1 =

𝑁𝑇𝐹 ▪ is known as the standard error of the regression or standard error

• f the estimate

ASSUMPTIONS OF REGRESSION ANALYSIS

Given a sample of data 𝑦1𝑗, 𝑦2𝑗, … , 𝑧𝑗 with 𝑗 = 1, … , 𝑜 ▪ we can use OLS to estimate the regression model ෠ 𝑍 = 𝑐0 + 𝑐1𝑌1 + 𝑐2𝑌2 + ⋯ ▪ subsequently, given the floor area, we can estimate the price of the house Now, a new “ incomplete” observations arrives ▪ for instance, a new house with known floor area (𝑦𝑜+1), but with unknown price (no 𝑧𝑜+1) We can use the regression model to estimate the house price ▪ so to predict ෟ 𝑧𝑜+1 PREDICTION WITH REGRESSION ANALYSIS

Example: ▪ ෠ 𝑍 = −264749 + 6152𝑌 ▪ a house with floor area 𝑦 = 85 m2 has an estimated price ො 𝑧 = −264748 + 6152 × 85 = 258142 (€) PREDICTION WITH REGRESSION ANALYSIS

So, we can predict a value of ො 𝑧 ▪ for a given 𝑦 (or 𝑦1, 𝑦2, …) ▪ and given estimated regression coefficients (𝑐0, 𝑐1, …) The quality of this estimate depends obviously on the quality

• f the regression model

▪ try to find a confidence interval for the estimated ො 𝑧-value ▪ two types:

▪ the confidence interval for the average price of a house of 85 m2 ▪ the confidence interval for a particular house of 85 m2

PREDICTION WITH REGRESSION ANALYSIS

Point prediction: 258142 Case 1: confidence interval (95%) for prediction of mean price ▪ 212866, 303419 Case 2: confidence interval (95%) for individual prediction ▪ −96372, 612658 PREDICTION WITH REGRESSION ANALYSIS

Individual predictions are always less accurate  wider confidence interval (this one even includes 0) Price (𝑍) unknown, area (𝑌) known

26 March 2015, Q3a OLD EXAM QUESTION

Doane & Seward 5/E 12.7, 13.1-13.5 Tutorial exercises week 4 multiple regression dummy regression prediction interval FURTHER STUDY