Generalized Degrees of Freedom (GDF) 9 June 2015 Dr. Shu-Ping Hu - - PowerPoint PPT Presentation

PRT-191 30 Mar 2015 1 Approved for Public Release PRT-191 30 Mar 2015 Approved for Public Release

 Los Angeles  Washington, D.C.  Boston  Chantilly  Huntsville  Dayton  Santa Barbara  Albuquerque  Colorado Springs  Goddard Space Flight Center  Johnson Space Center  Ogden  Patuxent River  Washington Navy Yard  Ft. Meade  Ft. Monmouth  Dahlgren  Quantico  Cleveland  Montgomery  Silver Spring  San Diego  Tampa  Tacoma  Aberdeen  Oklahoma City  Eglin AFB  San Antonio  New Orleans  Denver  Vandenberg AFB

Generalized Degrees of Freedom (GDF)

9 June 2015

• Dr. Shu-Ping Hu

PRT-191 30 Mar 2015 2 Approved for Public Release

 Constrained Process (Background Info)  Objectives  Error Terms (Additive vs. Multiplicative)  Multiplicative-Error Models



ZMPE CER Unbiased?



SPE Comparison: ZMPE vs. MUPE

 Definitions of DF and GDF  Calculate Fit Statistics Using GDF  Examples  Conclusions

Outline

Note: SPE is standard percent error and MUPE stands for minimum-unbiased-percent error. Other acronyms will be explained on next page

PRT-191 30 Mar 2015 3 Approved for Public Release

 Solver (an Excel add-in program) is a popular tool used to generate

nonlinear cost estimating relationships (CER), especially when constraints are specified. A few examples are given below:

 Minimizing the sum of squared percentage errors under the Zero-

Percentage Bias constraint (i.e., the ZMPE CER)

 Minimizing the sum of squared residuals under the Zero-Percentage

Bias constraint, (i.e., the mean of the % errors is zero)

 Minimizing the sum of squared percentage errors or residuals in log

space under the Zero-Bias constraint (i.e., the mean of the residuals is zero) using the Balance-Adjustment Factor (BAF)1

 In the above examples, we may not have the degrees of freedom (DF)

as given by the traditional definition when no constraints are specified

Constrained Process (1/2) Introduction

1. Book, S., “Significant Reasons to Eschew Log-Log OLS Regression when Deriving Estimating Relationships,” 2012 ISPA/SCEA Joint Annual Conference, Orlando, FL, 26-29 June.

PRT-191 30 Mar 2015 4 Approved for Public Release

 Do not abuse Solver

 Do not specify constraints excessively just because it is easy to do so

in Solver

 Explore different starting points to see if the solution stabilizes when

using Solver

 Solver can be sensitive to starting points—different starting points may

lead to different solutions

 Solver can be trapped in local minima, especially when fitting

complicated or ZMPE equations

 Specify “meaningful” constraints

 Make sure the constraints are necessary, logical, and statistically

sound as DF can be reduced by additional constraints

 Calculate the DF properly when constraints are specified

Constrained Process (2/2) Suggestions

PRT-191 30 Mar 2015 5 Approved for Public Release

 Explain why degrees of freedom (DF) should be adjusted if

constraints are specified in the curve-fitting process

 Recommend a Generalized Degrees of Freedom (GDF) measure to

compute fit statistics properly for constraint-driven equations

 Explain why ZMPE’s standard error underestimates the spread of the

CER error distribution

 We will illustrate how to calculate standard error properly for ZMPE

CERs

Objectives

PRT-191 30 Mar 2015 6 Approved for Public Release

Cost variation is independent of the scale of the project

Additive Error Term : y = aX^b + 

X Note: This requires non-linear regression. Y

Additive Error Term : y = f(x) + 

X Note: Error distribution is independent of the scale of the project. (OLS) Y

Additive Error Term: Y = f(X) + 

PRT-191 30 Mar 2015 7 Approved for Public Release

Multiplicative Error Term : y = ax^b * 

X Note: This equation is linear in log space. Y UpperBound f(x) LowerBound

Cost variance is proportional to the scale of the project

Multiplicative Error Term : y = (a + bx) * 

X Note: This requires non-linear regression. Y UpperBound f(x) LowerBound

Multiplicative Error Term: Y = f(X)*

Multiplicative error assumption is appropriate when

• Errors in the dependent variable are

believed to be proportional to the level of the function (the value of the variable)

• Dependent variable ranges over

more than one order of magnitude

PRT-191 30 Mar 2015 8 Approved for Public Release

 Log-Error:  ~ LN(0, s2)  Least squares in log space

 Error = Log (Y) - Log f(X)  Minimize the sum of squared errors; process is done in log space

 MUPE: E() = 1, V() = s2

 Least squares in weighted space

 Error = (Y-f(X))/f(X)  Minimize the sum of squared (%) errors

iteratively (i.e., minimize Si {(yi-f(xi))/fk-1(xi)}2, k is the iteration number)

 MUPE (an iterative, weighted least squares) has zero sample bias

 ZMPE: E() = 1, V() = s2

 Least squares in weighted space

 Error = (Y-f(X))/f(X)  Minimize the sum of squared (percentage) errors with a constraint:  ZMPE is a constrained minimization process  Average sample bias is eliminated by the constraint

Note: E((Y-f(X))/f(X)) = 0 V((Y-f(X))/f(X)) = s2

Multiplicative Error Model: Y = f(X)*

If f(x) is linear in log space, it is termed log-linear

• r LOLS CER

Si(yi - f(xi))/f(xi) = 0

variance of error term

We will focus on MUPE/ZMPE equations in this paper

PRT-191 30 Mar 2015 9 Approved for Public Release



Both MUPE and ZMPE methods have zero percentage bias (ZPB) for the sample data points:



For MUPE, this condition is achieved through the iterative minimization process; for ZMPE, ZPB is obtained by using a constraint



If a CER is unbiased, then E(Ŷ) = E(Y) = f(X,b)



Does the “ZPB” property imply that the CER is unbiased?



The ZPB constraint can be applied to any proposed methodologies (i.e., objective functions), but there is no guarantee that the CER result will be unbiased; namely, this condition “E(Ŷ) = f(X,b)” may not be satisfied



MUPE is the best linear unbiased estimator (BLUE) for linear models



For linear CERs, e.g., Y = (a + bX1 + cX2)*, the MUPE method produces unbiased estimates of the parameters and the function mean; it also provides smaller variances for the parameters and for any linear function of the parameters



MUPE’s parameter estimators are the quasi maximum likelihood estimators (QMLE) of the parameters; MUPE also provides consistent estimates of the parameters. ZMPE CERs, however, do not have statistical properties readily available.

ZMPE CER Unbiased?

ˆ ˆ 1

1

 



 n i i i i

y y y n

Not necessarily Don’t Know

y = actual value ŷ = predicted value

PRT-191 30 Mar 2015 10 Approved for Public Release

 The standard percent error (SPE) for Y = f(X)* is given by

 n = sample size, p = total # of estimated parameters, y = actual value, and

ŷ = predicted value

 SPE2 (i.e., MSE) is used to estimate s2, the variance of   SPE(ZMPE) ≤ SPE(MUPE)

 ZMPE always produces a smaller SPE when compared to MUPE except

for simple factor CERs (Book, 2006)

 Is a smaller SPE better?

 No, not necessarily. If it is true, we should develop MPE CERs, which are

proven to be over-estimating (see Hu and Sjovoid, 1994)

 Beware of using SPE alone for selecting CERs; we should also evaluate

• ther useful stats (see Hu, 2010)

SPE Comparison: ZMPE vs. MUPE (1/5)

Equal sign holds only for simple factor equations

SPE is CER’s standard error of estimate, which is used to measure the model’s

• verall error of estimation. It is the one-

sigma spread of the MUPE or ZMPE CER.

PRT-191 30 Mar 2015 11 Approved for Public Release

Q: Is ZMPE’s SPE2 (i.e., MSE) an unbiased estimator of s2? (I) When the CER is linear:

 MUPE’s SSE = Si wi(yi – ŷi)2 = Z’(I – H)Z

 MUPE can be converted to OLS in weighted space; wi (= 1/(ŷi)2) is the

weighting factor of the ith observation

 Z is the new vector variable in the weighted space ( ). V(Z) = Is2

and H is Z’s hat matrix. See Morrison (1983) & Draper and Smith (1981).

 For MUPE CERs: E(SSE) = s2(n – p) and E(SPE2(MUPE)) = s2

 E(SSE/(n-p)) = E(MSE) = E(SPE2(MUPE)) = s2  This equation is true regardless of the distribution type  This is an approximation if the CER is nonlinear

 ZMPE’s SPE underestimates the true s, except for simple factor CERs

 Since SPE2(ZMPE) ≤ SPE2(MUPE), E(SPE2(ZMPE)) ≤ E(SPE2(MUPE)) = s2

SPE Comparison: ZMPE vs. MUPE (2/5)

E(SPE2(ZMPE)) ≤ E(SPE2(MUPE)) = s2

Equal sign holds

• nly for simple

factor equations

Caution: ZMPE’s SPE underestimates the spread of the CER error distribution

i i i

y w z 

No

PRT-191 30 Mar 2015 12 Approved for Public Release

 If E(y) = m, V(y) = S  E(y’Ay) = m’Am + trace(SA)

 y is a vector of random variables; apostrophe is used to denote transpose  m is y’s expected value; S is y’s variance/covariance matrix (S = E(y-m)(y-m)’)  A is a symmetric matrix  trace is defined to be the sum of the diagonal elements of a square matrix  This equation holds regardless of the distribution assumption

 For WLS: If y = Xb + , where E() = 0 and V() = Vs2  there exists a

nonsingular symmetric matrix P such that PP = V

 Z = P-1y = P-1Xb+P-1 = Qb + P-1 &V(Z) = V(P-1) = P-1V()(P-1)’s2 = Is2  SSE(MUPE)

 MUPE is a WLS. P-1P-1 = V-1 = W (weighting matrix), H = Q(Q’Q)-1Q’ (hat

matrix), and is the LS solution for the unknown parameters.

 E(SSE) = E(Z’(I – H)Z) = (E(Z))’(I – H)E(Z) + trace[(I – H)V(Z)] = b’Q’(I – H)Qb

+ s2(n – trace[Q(Q’Q)-1Q’)]) = 0 + s2(n – trace[(Q’Q)-1Q’Q)]) = s2(n – p)

SPE Comparison: ZMPE vs. MUPE (3/5)

Formula to Prove E(SSE(MUPE)) = s2(n-p)

) ˆ ( )' ˆ ( ) ( ' ) ( )' ( ) ' ) ' ( )' ' ) ' ( ( ) ˆ ( )' ˆ ( ) ˆ ( )' ˆ ( β X Y P P β X Y Z H I Z HZ Z HZ Z Z Q Q Q Q (Z Z Q Q Q Q Z β Q Z β Q Z Z Z Z Z

1 1 1 1

                

   

β

ˆ

See Morrison (1983) for details

PRT-191 30 Mar 2015 13 Approved for Public Release

Q: Is ZMPE’s SPE2 (i.e., MSE) an unbiased estimator of s2? (II) When the CER error term is also normally distributed:

 MUPE’s SSE = S wi(yi – ŷi)2 ~ s2c2

(n-p)

 See Morrison (1983) and Draper and Smith (1981) for details  The proof is also given in the back-up section (slide #31)

 For MUPE CERs: E(SSE) = E(s2c2

(n-p)) = s2(n – p)

 E(SSE/(n-p)) = E(MSE) = E(SPE2(MUPE)) = s2

 ZMPE’s SPE underestimates the true s, except for simple factor CERs

 Since SPE2(ZMPE) ≤ SPE2(MUPE), E(SPE2(ZMPE) ) ≤ E(SPE2(MUPE)) = s2  The equal sign holds only for simple factor equations

SPE Comparison: ZMPE vs. MUPE (4/5)

E(SPE2(ZMPE)) ≤ E(SPE2(MUPE)) = s2

Caution: ZMPE’s SPE underestimates the spread of the CER error distribution

No

PRT-191 30 Mar 2015 14 Approved for Public Release

 The fitted regression coefficients may falsely become significant due

to the biased low SPE

 Given E(SPE2(ZMPE)) < s2 (except for simple factor CERs), using ZMPE

CERs in cost uncertainty analysis may unduly tighten the S-curve

 Prediction Intervals (PI) are specified for cost uncertainty analysis  The smaller the SPE, the tighter the PI becomes

• A PI is a function of the standard error of the regression (e.g., SPE), the

sample size, the “distance” of the estimating point from the center of the database used to generate the CER, etc.

 The impact on the risk session can be substantial when using

underestimated SPEs in numerous work breakdown structure (WBS) elements

SPE Comparison: ZMPE vs. MUPE (5/

(5/5) 5)

Concerns About Underestimating CER Errors

PRT-191 30 Mar 2015 15 Approved for Public Release 15

Q: Why is ZMPE's SPE biased low? A: Its degrees of freedom did not get adjusted properly

PRT-191 30 Mar 2015 16 Approved for Public Release

Definition – Degrees of Freedom (1/2)

 DF = number of data points (n) - number of parameters estimated in

the regression equation (p)

 DF is used to characterize the number of “independent” pieces of

information contained in a statistic

 Any sum of squares (SS) has a DF associated with it

 DF indicates how many pieces of independent information from the n

independent observations are needed to compile the sum of squares

 Given a simple OLS model: Y = a + b X + 

 SST = Si(yi -

)2 has (n – 1) DF because Si(yi - ) = 0

 SSR = b2Si(xi -

)2 = (SSxy)2/SSxx has one DF

 SSE = Si(yi - ŷi)2 has (n – 2) DF as intercept and slope are estimated

y

 

     

2

) ( ) )( ( ˆ x x y y x x SS SS b

i i i xx xy

b

x y b a ˆ ˆ  

x

The DF for SSE is the excess data points that can be used to judge the quality of the fit

The term “independent” means “free to vary”

y

PRT-191 30 Mar 2015 17 Approved for Public Release

Definition – Degrees of Freedom (2/2) (a ZMPE Model)

 The objective function, F, for “Y = (a + bX)*” is  The solutions for a and b are derived by solving the two normal

equations:









 The total number of constraints becomes three when including an

additional constraint in the system:

 Adjust DF accordingly for additional constraints: DF = n – 2 – 1

 Adjust DF when solutions are derived in a constrained domain. When

additional constraints are present, we cannot search as freely as we can in an unconstrained domain to find a solution, which results in a loss of DF.

   a F    b F

two constraints

 

 

      

n i n i i i i i i

x y x x y

1 1

1 * * * b a b a b a

Each normal equation in the system can be viewed as a constraint that restricts one DF ZMPE CER applies the Zero- Percentage Bias constraint to the curve-fitting process.





          

n i i i

x y F

1 2

1 * b a





   

n i i i i i

x x y y

1 3

) * ( ) * ( b a b a





   

n i i i i i i

x x y x y

1 3

) * ( ) * ( b a b a

PRT-191 30 Mar 2015 18 Approved for Public Release

 DF should be adjusted to reflect (1) additional constraints and (2)

redundancies in CER/PER development

 One restriction is equivalent to a loss of one DF  We should take redundancy into account when counting the DF

• For example, if two additional constraints are specified for a regression model

but one constraint can be derived by the other, then we should only count a loss of one DF rather than two

• Another example, if a constraint is directly related to the normal equations, then

it does not count towards a loss of DF  Generalized DF (GDF) = n – p – (# constraints) + (# redundancies)

 n = the sample size; p = total number of estimated parameters

 GDF =

Definition – Generalized DF:

Adjusting DF for Constraint driven equations DF should be adjusted for ZMPE CERs

      ) ( 1 CERs factor simple for except ZMPE for p n MUPE for p n

See Slide #32 in the back-up section

PRT-191 30 Mar 2015 19 Approved for Public Release 19

Q: Why is GDF important? A: DF is the basis for fit measures. All fit statistics (e.g., SEE and SPE) should be updated by GDF if constraints are included. Note: SEE and SPE are also used in cost uncertainty analysis.

PRT-191 30 Mar 2015 20 Approved for Public Release

 Standard error of estimate (SEE) for an additive-error model is given

by:

 Standard percent error (SPE) for a multiplicative-error model is given

by:

 GDF = n – p when no constraints are specified  GDF = n – p for MUPE; GDF = n – p – 1 for ZMPE (except for factor CERs)  SPE(ZMPE) =

 We can now compare MUPE’s SPE with ZMPE’s SPE

Calculate SEE and SPE Using GDF

% 100 * ˆ ˆ 1

1 2





         

n i i i i

y y y GDF SPE





 

n i i i

y y GDF SEE

1 2

) ˆ ( 1

SPE, as well as SEE, is CER’s standard error of estimate, which is used to measure the model’s overall error of estimation

        equations factor simple for SPE p n p n SPE

c c

1

SPEc stands for the current calculation, not using GDF

y = actual value ŷ = predicted value

PRT-191 30 Mar 2015 21 Approved for Public Release

 Adjusted R2 for Additive-Error CERs, i.e., Y = f(X)+:



Adjusted R2 for Multiplicative-Error CERs, i.e., Y = f(X)*:

 This adjusted R2 compares MUPE and ZMPE’s SPE2 to its baseline (i.e., SPE2

• f an average CER, ); it is more pertinent to the fitting method

 This measure “Adjusted R2

(MUPE/ZMPE)” puts SPE2 in perspective

 GDF = n – p when no constraints are specified  GDF = n – p for MUPE; GDF = n – p – 1 for ZMPE (except for factor CERs)

Calculate Adjusted R2 Using GDF

2 2 2 2 2 2

) 1 /( ) / ) (( / ) ˆ / ) ˆ (( 1 ) / ( .

Y f Y i i i i

SPE SPE SPE n y y y GDF y y y ZMPE MUPE R Adj       

 

Y

2 2 2 2 2 2

) 1 /( ) ( / ) ˆ ( 1 ) ( .

Y f Y i i i

SEE SEE SEE n y y GDF y y Additive R Adj       

 

 We can compare MUPE’s Adjusted R2 with ZMPE’s Adjusted R2

• Adj. R2 measures the % difference

between the CER’s estimated variance and the sample variance

• f an average CER. For example, if

a CER’s estimated variance is 0.35 while the sample variance of an average CER is 1.4, then the CER’s variance is only 25% of the variance of an average CER. This reduction of variance, 75%, is the Adjusted R2. The reduction of variance is considered to be an improvement when applying the CER (see Hu, 2010).

PRT-191 30 Mar 2015 22 Approved for Public Release

 Consider sample size when applying correlation to a cost risk model

 A correlation of 0.8 derived from 30 data points is much more

significant than the same correlation computed from just 5 data points

 Neither the sample size nor the degrees of freedom (DF) adjustment is

accounted for in the Pearson’s correlation formula (see below)

 In OLS: Adjusted R2 = 1 – (1–R2)*(n–1)/(n–p) = R2 – (1–R2)*(p–1)/(n–p)  Compute Pearson’s Adjusted r (Adj. r) for two sets of numbers, e.g.,

“cost vs. cost” or %error vs. %error correlation between two CERs: r2 = r2 – (1 – r2)/(n – 2)

 Instead of sample correlations, suggest using Pearson’s Adjusted r

(Adj. r) for correlation analysis

Modify Pearson’s r by Sample Size

(Between CERs)

      * ) ( sign .

2 2

r if r if r r r Adj

Assume p = 2 in this situation

Constraints and DF adjustment are not relevant to derive Adj. r if not comparing y versus ŷ for a CER

Not CER related

  

  

    

n i i n i i n i i i

y y x x y y x x y x r

1 2 1 2 1

) ( ) ( ) )( ( ) , ( : Definition

PRT-191 30 Mar 2015 23 Approved for Public Release

 Generalized R2 (GRSQ): GRSQ is Pearson’s r2 between a CER’s actual

{y} and its predicted {ŷ} values in unit space, i.e., GRSQ = r2(y, ŷ)

 GRSQ takes neither the sample size nor degrees of freedom into account

 We should modify GRSQ (denoted by r2) for DF using GDF:

 GDF = n – p if no constraints are specified  GDF = n – p for MUPE; GDF = n – p – 1 for ZMPE (except for factor CERs)

 This modified GRSQ (GRSQ(GDF)) takes the DF, numbers of estimated

parameters, as well as the constraints, into consideration

Modify GRSQ (r2) Using GDF

(Within a CER)

   

  

  

           

n i i n i i n i i i

y y y y y y y y y y r GRSQ

1 2 1 2 2 1 2

ˆ ˆ ) ( ˆ ˆ ) ( ) ˆ , (

Note: GRSQ, as well as Pearson’s correlation,

• nly measures the linear association between

two sets of numbers rather than the actual deviation between them.

  

  

    

n i i n i i n i i i

y y x x y y x x y x r

1 2 1 2 1

) ( ) ( ) )( ( ) , ( : Definition

PRT-191 30 Mar 2015 24 Approved for Public Release

 A hypothetical data set to predict the cost of a black box using weight

Example – Weight-Based CERs (1/3)

Data Point Cost $K Weight (lbs) Obs 1 135.0 4.18 Obs 2 6.5 0.32 Obs 3 8.0 0.57 Obs 4 64.6 2.34 Obs 5 32.9 0.50 Obs 6 95.4 2.70 Obs 7 67.0 4.54 Obs 8 112.2 4.42 Obs 9 29.2 0.55 Obs 10 22.7 0.20 Obs 11 16.9 0.80 Obs 12 35.0 1.75

PRT-191 30 Mar 2015 25 Approved for Public Release



Four different CERs are generated using both MUPE and ZMPE methods



The unadjusted SPE measures for these ZMPE CERs are all smaller than their respective SPE measures generated by the MUPE method



ZMPE’s updated SPE measures using GDF become larger than their MUPE counter-part (numbers in dark red)



ZMPE’s updated adjusted R2 and GRSQ are all smaller than those generated by the MUPE method when GDF is applied (numbers in purple and green)



MUPE outperforms ZMPE based upon all three statistics (SPE, Adj. R2, & GRSQ)

Example – Weight-Based CER (2/3)

Although the semi-log equation is tighter than the log-linear and triad equations, its GRSQ is much worse than those of the log-linear and triad CERs. Caution: Beware of using GRSQ alone for selecting CERs as it only measures the linear association between y and ŷ, not the actual difference between them.

ZMPE CERs SPE SPE(GDF)

• Adj. R2
• Adj. R2

(GDF)

GRSQ GRSQ(GDF) Linear 12.794 + 19.16*Weight 45.7% 48.2% 68.9% 65.4% 77.5% 75.0% LogLinear 36.889*Weight^(0.5882) 48.9% 51.5% 66.9% 60.4% 77.4% 74.9% Semi-Log 17.881*(1.5314)^Weight 47.1% 49.7% 64.4% 63.2% 68.9% 65.4% Triad 16.785+12.034*Weight^(1.349) 47.5% 50.4% 66.4% 62.2% 75.5% 69.4% MUPE CERs SPE

• Adj. R2

GRSQ Linear 10.528 + 21.2975*Weight 46.5% 67.8% 77.5% LogLinear 36.2953*Weight^(0.6635) 50.2% 65.5% 77.7% Semi-Log 16.756*(1.5835)^Weight 47.4% 66.5% 67.0% Triad 15.026+12.9863*Weight^(1.386) 48.4% 65.1% 75.3%

PRT-191 30 Mar 2015 26 Approved for Public Release



The linear CER seems to be the best choice among the four CERs



Use the semi-log equation with caution because it goes up exponentially

Example – Weight-Based CER (3/3)

Scatter Plot for MUPE CERs

PRT-191 30 Mar 2015 27 Approved for Public Release

 Make sure the constraints (if any) are meaningful, logical, and

statistically sound when adding them to the curve-fitting process

 Explore different starting points in Solver to ensure the solution stabilizes

 Adjust DF for constraint-driven equations  Define GDF = n – p – (# constraints) + (# redundancies)

 Subtract one from DF for ZMPE equations except for simple factor CERs

 Calculate SEE, SPE, Adjusted R2, and GRSQ (r2) using GDF  No need to adjust DF and goodness-of-fit measures for MUPE CERs  ZMPE’s SPE underestimates the spread of the CER error distribution

 Using ZMPE CERs without adjustment in cost uncertainty analysis may

unduly tighten the S-curve

Conclusions

PRT-191 30 Mar 2015 28 Approved for Public Release

 Adding excessive constraints into the process may cause the

unknown parameters in the CER to be determined completely by the

• constraints. (See Slide #32 for an example.) When it happens, there is

no need to run regression analysis. How do we define DF properly for this case?

 If # of constraints = # of estimated parameters, we do not use any

curve-fitting methods to derive a solution. Consequently, we may have no degrees of freedom left to judge the quality of the fit due to lack of regression.

 An inequality constraint may not be treated the same as an equality

• constraint. How to treat inequality constraints in the curve-fitting or

distribution-finding process is another topic.

Future Study Items for Constraint Driven Regression

PRT-191 30 Mar 2015 29 Approved for Public Release



Book, S., “IRLS/MUPE CERs are not MPE-ZPB CERs,” 28th Annual ISPA International Conference, Seattle, WA, 23-26 May 2006.



Book, S., “Significant Reasons to Eschew Log-Log OLS Regression when Deriving Estimating Relationships,” 2012 ISPA/SCEA Joint Annual Conference, Orlando, FL, 26-29 June.



Book, S. A. and N. Y. Lao, “Minimum-Percentage-Error Regression under Zero-Bias Constraints,” Proceedings of the 4th Annual U.S. Army Conference on Applied Statistics, 21-23 Oct 1998, U.S. Army Research Laboratory, Report No. ARL-SR- 84, November 1999, pages 47-56.



Cochran, W. G., “The distribution of quadratic forms in a normal system, with applications to the analysis of covariance,” Proceedings of the Cambridge Philosophical Society, vol. 30, 1934, pages 178-191.



Draper, N. R. and H. Smith, “Applied Regression Analysis (2nd edition),” New York: John Wiley & Sons, Inc., 1981.



Hu, S., “The Minimum-Unbiased-Percentage-Error (MUPE) Method in CER Development,” 3rd Joint Annual ISPA/SCEA International Conference, Vienna, VA, 12-15 June 2001.



Hu, S., “R2 versus r2,” 2008 SCEA/ISPA Joint Annual Conference, Industry Hills, CA, 24-27 June 2008.



Hu, S., “Simple Mean, Weighted Mean, or Geometric Mean?” 2010 ISPA/SCEA Joint Annual Conference, San Diego, CA, 8- 11 June 2010



Hu, S. and A. R. Sjovold, "Multiplicative Error Regression Techniques," 62nd MORS Symposium, Colorado Springs, Colorado, 7-9 June 1994.



Hu, S. and A. Smith, “Why ZMPE When You Can MUPE,” 6th Joint Annual ISPA/SCEA International Conference, New Orleans, LA, 12-15 June 2007.



Morrison, D. F., “Applied Linear Statistical Methods”, New Jersey: Prentice-Hall, Inc. 1983.



Nguyen, P., B. Kwok, et al., “Unmanned Spacecraft Cost Model, Ninth Edition,” U. S. Air Force Space and Missile Systems Center (SMC/FMC), Los Angeles AFB, CA, August 2010.



Searle, S. R. “Linear Models,” New York: John Wiley & Sons, Inc., 1981.



Seber, G. A. F. and C. J. Wild, “Nonlinear Regression,” New York: John Wiley & Sons (1989), pages 37, 46, 86-88.



Weisberg, S., “Applied Linear Regression, 2nd Edition,” New York: John Wiley & Sons, 1985, pages 87-88.



Wedderburn, R. W. M., “Quasi-Likelihood Functions, Generalized Linear Models, and the Gauss-Newton Method,” Biometrika,

• Vol. 61, Issue 3 (December 1974), pages 439-447.

References

PRT-191 30 Mar 2015 30 Approved for Public Release 30

Backup Slides

PRT-191 30 Mar 2015 31 Approved for Public Release

 Given a multivariate normal distribution y ~ N(m,S), the quadratic form

Q = y’Ay has the noncentral c2 distribution with r DF & noncentrality parameter m’Am if and only if AS is an idempotent matrix

 m is y’s expected value  S is y’s variance/covariance matrix (i.e., S = E(y – m)(y – m)’)  A is a symmetric matrix of rank r  See Searle (1973) and Cochran (1934) for the proof of the theorem

 If y = Xb + , where  ~ N(0,Vs2)  P-1y=P-1Xb + P-1 and P-1 ~ N(0, Is2)

 PP = V; V-1 (also denoted by W) is a weighting matrix

 The SSE for MUPE CER can be expressed below  Since (V-1 – V-1RV-1)*V is an idempotent matrix and the noncentrality

parameter is zero, SSE follows a c2 distribution with (n – p) DF

Theorem to Prove SSE(MUPE) ~ s2c2

(n-p)

WRW)Y (W Y' )Y RV V (V Y' )Y V X' X) V X(X' V (V Y' Y V X' ' β Y V Y' β X V X' ' β β X V Y' Y V X' ' β Y V Y' ) β X (Y V )' β X (Y ) β X (Y P P )' β X (Y ) β X P Y (P )' β X P Y (P

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

                    

                    

ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ ˆ SSE

PRT-191 30 Mar 2015 32 Approved for Public Release

 The MPE solution for b is derived by minimizing the objective

function:

 The MPE solution for b is given by  The ZMPE solution for b is given by  Note: The ZMPE solution is derived by the ZMPE constraint, not the

minimization process:

Example of a Factor CER (y = bx*)

DF should be subtracted by one for ZMPE CERs except for simple factor CERs





         

n i i i

x y F

1 2

1 b

   

 

 



n i i i n i i i

x y x y

1 1 2

/ / ˆ b

 







n i i i

x y n

1

/ 1 ˆ b

1 1

                   

 

 

n x y x x y

n i i i n i i i i

b b b

See Hu (June 2010)