linear regression estimating parameters
play

Linear Regression - Estimating Parameters Bernd Schr oder logo1 - PowerPoint PPT Presentation

Aug 18, 2022 •133 likes •1.16k views

Coefficients Examples Error Sum of Squares Coefficient of Determination Linear Regression - Estimating Parameters Bernd Schr oder logo1 Bernd Schr oder Louisiana Tech University, College of Engineering and Science Linear Regression -

  1. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∂ f = ∂ [ y i − ( b 0 + b 1 x i )] ! [ y i − ( b 0 + b 1 x i )] 2 ∑ ∑ = − 2 = 0 ∂ b 0 ∂ b 0 i = 1 i = 1 n n ∂ f = ∂ x i [ y i − ( b 0 + b 1 x i )] ! [ y i − ( b 0 + b 1 x i )] 2 ∑ ∑ = = 0 − 2 ∂ b 1 ∂ b 1 i = 1 i = 1 n n ∑ ∑ y i − nb 0 − b 1 x i = 0 i = 1 i = 1 n n n x 2 ∑ ∑ ∑ = x i y i − b 0 x i − b 1 0 i i = 1 i = 1 i = 1 n n ∑ ∑ nb 0 + b 1 x i = y i i = 1 i = 1 n n n x 2 ∑ ∑ ∑ x i + b 1 = b 0 x i y i i i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  2. Coefficients Examples Error Sum of Squares Coefficient of Determination logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  3. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  4. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 � � n n 1 ∑ ∑ = b 0 y i − b 1 x i n i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  5. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 � � n n 1 ∑ ∑ = b 0 y i − b 1 x i n i = 1 i = 1 � � n n n n n 1 x 2 ∑ ∑ ∑ ∑ ∑ x i + b 1 = y i − b 1 x i x i y i i n i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  6. Coefficients Examples Error Sum of Squares Coefficient of Determination n n ∑ ∑ nb 0 + b 1 = x i y i i = 1 i = 1 n n n ∑ ∑ x 2 ∑ b 0 x i + b 1 = x i y i i i = 1 i = 1 i = 1 � � n n 1 ∑ ∑ = b 0 y i − b 1 x i n i = 1 i = 1 � � n n n n n 1 x 2 ∑ ∑ ∑ ∑ ∑ x i + b 1 = y i − b 1 x i x i y i i n i = 1 i = 1 i = 1 i = 1 i = 1  � 2  � n n n n n i − 1 x i y i − 1 x 2 ∑ ∑ ∑ ∑ ∑ b 1 x i = y i x i   n n i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  7. Coefficients Examples Error Sum of Squares Coefficient of Determination logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  8. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  9. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  10. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x ∑ n i = 1 y i ( x i − x ) − ∑ n i = 1 y ( x i − x ) = ∑ n i = 1 x i ( x i − x ) − ∑ n i = 1 x ( x i − x ) logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  11. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x ∑ n i = 1 y i ( x i − x ) − ∑ n i = 1 y ( x i − x ) = ∑ n i = 1 x i ( x i − x ) − ∑ n i = 1 x ( x i − x ) ∑ n i = 1 ( x i − x )( y i − y ) = i = 1 ( x i − x ) 2 ∑ n logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  12. Coefficients Examples Error Sum of Squares Coefficient of Determination ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i = b 1 � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 ∑ n i = 1 x i y i − ∑ n i = 1 y i x = � ∑ n i = 1 x 2 i − ∑ n � i = 1 x i x ∑ n i = 1 y i ( x i − x ) − ∑ n i = 1 y ( x i − x ) = ∑ n i = 1 x i ( x i − x ) − ∑ n i = 1 x ( x i − x ) ∑ n i = 1 ( x i − x )( y i − y ) = i = 1 ( x i − x ) 2 ∑ n = b 0 y − b 1 x logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  13. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  14. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  15. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n i = 1 x i � i = 1 x i ) 2 � ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  16. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  17. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  18. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 2. b 0 = y − b 1 x logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  19. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 2. b 0 = y − b 1 x y i : = ˆ β 0 + ˆ β 1 x i . 3. Fitted (or predicted) values: ˆ logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  20. Coefficients Examples Error Sum of Squares Coefficient of Determination Summarizing the Formulas 1. b 1 = ∑ n i = 1 x i y i − 1 n ∑ n i = 1 y i ∑ n = ∑ n i = 1 x i i = 1 ( x i − x )( y i − y ) = S xy � i = 1 x i ) 2 � i = 1 ( x i − x ) 2 ∑ n S xx ∑ n i − 1 n ( ∑ n i = 1 x 2 2. b 0 = y − b 1 x y i : = ˆ β 0 + ˆ β 1 x i . 3. Fitted (or predicted) values: ˆ 4. Residuals: y i − ˆ y i . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  21. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  22. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  23. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  24. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  25. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  26. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  27. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  28. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  29. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  30. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  31. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  32. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  33. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  34. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  35. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  36. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  37. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  38. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 Estimate of σ 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  39. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  40. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 2. We divide by n − 2 because we have only n − 2 degrees of freedom after determining two parameters ˆ β 0 and ˆ β 1 . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  41. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 2. We divide by n − 2 because we have only n − 2 degrees of freedom after determining two parameters ˆ β 0 and ˆ β 1 . (Previously, using an estimate of µ resulted in the loss of one degree of freedom.) logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  42. Coefficients Examples Error Sum of Squares Coefficient of Determination How Well Does the Line Fit the Data? 1. Error sum of squares (or residual sum of squares). n n �� 2 y i ) 2 = � � β 0 + ˆ ˆ ∑ ∑ SSE = ( y i − ˆ β 1 x i y i − i = 1 i = 1 y i ) 2 n − 2 = ∑ n i = 1 ( y i − ˆ σ 2 = s 2 = SSE Estimate of σ 2 : ˆ . n − 2 2. We divide by n − 2 because we have only n − 2 degrees of freedom after determining two parameters ˆ β 0 and ˆ β 1 . (Previously, using an estimate of µ resulted in the loss of one degree of freedom.) 3. It can be shown (we cannot present the proof) that S 2 is an unbiased estimator for σ 2 . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  43. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  44. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE SSE logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  45. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  46. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 n �� 2 � � β 0 + ˆ ˆ ∑ = y i − β 1 x i i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  47. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 n �� 2 � � β 0 + ˆ ˆ ∑ = y i − β 1 x i i = 1 n i − 2 ˆ β 0 y i − 2 ˆ β 1 x i y i + ˆ 0 + 2 ˆ β 0 ˆ β 1 x i + ˆ y 2 β 2 β 2 1 x 2 ∑ = i i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  48. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n y i ) 2 ∑ = ( y i − ˆ SSE i = 1 n �� 2 � � β 0 + ˆ ˆ ∑ = y i − β 1 x i i = 1 n i − 2 ˆ β 0 y i − 2 ˆ β 1 x i y i + ˆ 0 + 2 ˆ β 0 ˆ β 1 x i + ˆ y 2 β 2 β 2 1 x 2 ∑ = i i = 1 n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ ∑ y 2 ∑ ∑ β 2 ∑ β 2 ∑ x 2 = β 0 β 1 β 1 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  49. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  50. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 � � n n n n n y 2 i − 2 ˆ y i − 2 ˆ x i y i + ˆ y i − ˆ ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 0 β 1 x i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  51. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 � � n n n n n y 2 i − 2 ˆ y i − 2 ˆ x i y i + ˆ y i − ˆ ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 0 β 1 x i i = 1 i = 1 i = 1 i = 1 i = 1 n n + 2 ˆ β 0 ˆ x i + ˆ β 2 x 2 ∑ ∑ β 1 1 i i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  52. Coefficients Examples Error Sum of Squares Coefficient of Determination Formula For SSE n n n n n i − 2 ˆ y i − 2 ˆ x i y i + n ˆ 0 + 2 ˆ β 0 ˆ x i + ˆ y 2 β 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 � � n n n n n y 2 i − 2 ˆ y i − 2 ˆ x i y i + ˆ y i − ˆ ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 0 β 1 x i i = 1 i = 1 i = 1 i = 1 i = 1 n n + 2 ˆ β 0 ˆ x i + ˆ β 2 x 2 ∑ ∑ β 1 1 i i = 1 i = 1 n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  53. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  54. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ y 2 ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1  � 2  � n n i ± 1 + ˆ β 2 x 2 ∑ ∑ x i 1   n i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  55. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ x i + ˆ y 2 β 2 x 2 ∑ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE 1 i i = 1 i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ y 2 ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1  � 2  � n n i ± 1 + ˆ β 2 x 2 ∑ ∑ x i 1   n i = 1 i = 1 n n n n y 2 i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1  � 2  � 2 � � n n n i − 1 1 + ˆ β 2 x 2  + ˆ β 2 ∑ ∑ ∑ x i x i 1  1 n n i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  56. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1  � 2  � 2 � � n n n i − 1 1 + ˆ β 2 x 2  + ˆ β 2 ∑ ∑ ∑ x i x i 1  1 n n i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  57. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1  � 2  � 2 � � n n n i − 1 1 + ˆ β 2 x 2  + ˆ β 2 ∑ ∑ ∑ x i x i 1  1 n n i = 1 i = 1 i = 1 n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ ∑ y 2 ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  58. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  59. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1 � � n n n 1 1 − ˆ y i − ˆ ∑ ∑ ∑ β 1 β 1 x i x i n n i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  60. Coefficients Examples Error Sum of Squares Coefficient of Determination n n n n i − ˆ y i − 2 ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 SSE x i i = 1 i = 1 i = 1 i = 1 � 2 � � � n n n n x i y i − 1 1 + ˆ + ˆ β 2 ∑ ∑ ∑ ∑ β 1 y i x i x i 1 n n i = 1 i = 1 i = 1 i = 1 n n n n i − ˆ y i − ˆ x i y i + ˆ β 0 ˆ y 2 ∑ ∑ ∑ ∑ = β 0 β 1 β 1 x i i = 1 i = 1 i = 1 i = 1 � � n n n 1 1 − ˆ y i − ˆ ∑ ∑ ∑ β 1 β 1 x i x i n n i = 1 i = 1 i = 1 n n n i − ˆ y i − ˆ ∑ y 2 ∑ ∑ = β 0 β 1 x i y i i = 1 i = 1 i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  61. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  62. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  63. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. SSE ≈ 1 . 1194 and s 2 ≈ 0 . 0622 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  64. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  65. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  66. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. SSE ≈ 79 . 7958 and s 2 ≈ 4 . 4331 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  67. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  68. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. ∑ 1. Let SST : = i = 1 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  69. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  70. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  71. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. 3. SST measures the total variation in y . logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  72. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. 3. SST measures the total variation in y . 4. The quotient gives the proportion of the variation not attributed to the linear relationship. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  73. Coefficients Examples Error Sum of Squares Coefficient of Determination Coefficient of Determination n ( y i − y ) 2 be the total sum of squares. The ∑ 1. Let SST : = i = 1 coefficient of determination is given by r 2 = 1 − SSE SST . 2. SSE measures how much variation is not attributed to the linear relationship. 3. SST measures the total variation in y . 4. The quotient gives the proportion of the variation not attributed to the linear relationship. 5. r 2 gives the proportion of the variation that is attributed to the linear relationship. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  74. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  75. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  76. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 1. r 2 ≈ . 9516 logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

  77. Coefficients Examples Error Sum of Squares Coefficient of Determination Example 2. logo1 Bernd Schr¨ oder Louisiana Tech University, College of Engineering and Science Linear Regression - Estimating Parameters

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Estimating the Error at Given Test Estimating the Error at Given Test Input Points for Linear

Estimating the Error at Given Test Estimating the Error at Given Test Input Points for Linear

IASTED-NCI2004 Feb. 23-25, 2004 Estimating the Error at Given Test Estimating the Error at Given Test Input Points for Linear Regression Input Points for Linear Regression Masashi Sugiyama Fraunhofer FIRST-IDA, Berlin, Germany Tokyo

336 views • 21 slides
LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR

LINEAR REGRESSION LINEAR REGRESSION - FROM A MACHINE LEARNING POINT OF VIEW 25 SIMPLE LINEAR REGRESSION 20 Sales 15 Starting point 10 Simplest parametric function 5 Easy to interpret the parameters: 0 50 100 150 200 250

1.12k views • 13 slides
Week 3: Linear Regression Instructor: Sergey Levine 1 The regression problem We saw how we can

Week 3: Linear Regression Instructor: Sergey Levine 1 The regression problem We saw how we can

Week 3: Linear Regression Instructor: Sergey Levine 1 The regression problem We saw how we can estimate the parameters of probability distributions over a random variable x . However, in a supervised learning setting, we might be interested in

64 views • 5 slides
Estimating the parameters of some probability distributions: Exemplifications 1. Estimating the

Estimating the parameters of some probability distributions: Exemplifications 1. Estimating the

0. Estimating the parameters of some probability distributions: Exemplifications 1. Estimating the parameter of the Bernoulli distribution: the MLE and MAP approaches CMU, 2015 spring, Tom Mitchell, Nina Balcan, HW2, pr. 2 2. Suppose we

889 views • 54 slides
EC3062 ECONOMETRICS ELEMENTARY REGRESSION ANALYSIS We shall consider three methods for estimating

EC3062 ECONOMETRICS ELEMENTARY REGRESSION ANALYSIS We shall consider three methods for estimating

EC3062 ECONOMETRICS ELEMENTARY REGRESSION ANALYSIS We shall consider three methods for estimating statistical parameters. These are the method of moments, the method of least squares and the principle of maximum likelihood. In the case of the

624 views • 23 slides
Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common

0. Regression Methods 1. Linear Regression and Logistic Regression: definitions, and a common property CMU, 2004 fall, Andrew Moore, HW2, pr. 4 2. Linear Regression and Logistic Regression: Definitions Given an input vector X , linear

1.07k views • 22 slides
Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear

Notes on the Non-linear Regression The model Non-linear regression models, like ordinary linear models, assume that the response, Y , has a normal distribution with constant variance. Unlike linear models, however, the mean function may depend

159 views • 4 slides
Estimating Parameters of Pareto Distribution Under Interval and Fuzzy Uncertainty Nitaya Buntao

Estimating Parameters of Pareto Distribution Under Interval and Fuzzy Uncertainty Nitaya Buntao

Formulation of the Problem First Result: Estimating x 0 Under Interval Uncertainty Estimating Under Interval Uncertainty Algorithm for Computing the Smallest Value of Algorithm for Computing the Largest Value of Estimating Parameters of

977 views • 48 slides
Parametric Methods Steven J Zeil Old Dominion Univ. Fall 2010 1 Distributions Estimating

Parametric Methods Steven J Zeil Old Dominion Univ. Fall 2010 1 Distributions Estimating

Distributions Estimating Distribution Parameters Parametric Classification Regression Parametric Methods Steven J Zeil Old Dominion Univ. Fall 2010 1 Distributions Estimating Distribution Parameters Parametric Classification Regression

889 views • 59 slides
The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study

The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study

The General Linear Model. April 22, 2008 Multiple regression Data: The Faroese Mercury Study Simple linear regression Confounding The multiple linear regression model Interpretation of parameters Model control

564 views • 39 slides
Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied

Logistic regression CS 446 1. Linear classifiers Linear regression Last two lectures, we studied linear regression ; the output/label space Y was R . 90 80 delay 70 60 50 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 duration 1 / 68 Linear

607 views • 38 slides
Analysis of variance and regression November 22, 2007 Parametrisations : Choice of parameters

Analysis of variance and regression November 22, 2007 Parametrisations : Choice of parameters

Analysis of variance and regression November 22, 2007 Parametrisations : Choice of parameters Comparison of models Test for linearity Linear splines Lene Theil Skovgaard, Dept. of Biostatistics, Institute of Public Health,

966 views • 71 slides
Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3)

Which models can be fit with linear regression? Simple linear regression in Matlab X = rand(3,3) b = pinv(X) * y X = b = 0.0467 0.5188 0.5317 14.3966 0.6587 0.3323 0.5070 50.9428 0.7573 0.0428 0.2532 -50.8102 y = rand(3,1) b = X \

643 views • 13 slides
Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model

Trendlines Simple Linear Regression Multiple Linear Regression Systematic Model Building Practical Issues Overfitting Categorical Variables Interaction Terms Non-linear Terms Linear y = a + bx Logarithmic

945 views • 65 slides
Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions Linear in Parameters : 1)

Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions Linear in Parameters : 1)

Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions Linear in Parameters : 1) Y X X X 0 1 1 2 2 i k k Random Sampling : we have a random sample from 2) the

776 views • 39 slides
Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall

Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall

Announcements Unit 7: Multiple linear regression 1. Introduction to multiple linear regression Sta 101 - Fall 2018 Project questions? Duke University, Department of Statistical Science Dr. Abrahamsen Slides posted at

537 views • 10 slides
Error Handling Marco Chiarandini Department of Mathematics & Computer Science University of

Error Handling Marco Chiarandini Department of Mathematics & Computer Science University of

DM560 Introduction to Programming in C++ Error Handling Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark [ Based on slides by Bjarne Stroustrup ] Error Handling Outline 1. Error Handling 2

573 views • 36 slides
CS7015 (Deep Learning) : Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks,

CS7015 (Deep Learning) : Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks,

CS7015 (Deep Learning) : Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute of

869 views • 70 slides
Training Neural Networks with Local Error Signals Arild Nkland Lars H. Eidnes Local learning

Training Neural Networks with Local Error Signals Arild Nkland Lars H. Eidnes Local learning

Training Neural Networks with Local Error Signals Arild Nkland Lars H. Eidnes Local learning Typically we train neural networks by backpropagating errors from the loss function and back through the layers. Hard to explain how the

647 views • 13 slides
Bounds for the capacity error function for unidirectional channels with noiseless feedback

Bounds for the capacity error function for unidirectional channels with noiseless feedback

Bounds for the capacity error function for unidirectional channels with noiseless feedback Christian Deppe 1 , Vladimir Lebedev 2 and Georg Maringer 1 1 Technical University of Munich Institute for Communications Engineering 2 Kharkevich Institute

351 views • 19 slides
BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe

BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe

Illustration: detail from The Alchemist Discovering Phosphorus by Joseph Wright (1771) BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe University // Fall 2019 About class projects This semester

1.42k views • 46 slides
Clustering / Unsupervised Learning The target features are not given in the training examples The

Clustering / Unsupervised Learning The target features are not given in the training examples The

Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. D. Poole and A. Mackworth 2010 c Artificial

741 views • 19 slides
Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron ,

Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron ,

Certifying Non-negativity with Lasserres Hierarchy and Semidefinite Programming Victor Magron , LAASCNRS 5 March 2019 Faculty of Mechanical Engineering University of Ljubljana Victor Magron Certifying Non-negativity with Lasserres

1.73k views • 85 slides
Lecture 10. Simple linear regression 2020 (1) Using one r.v. to predict another X and Y are

Lecture 10. Simple linear regression 2020 (1) Using one r.v. to predict another X and Y are

Lecture 10. Simple linear regression 2020 (1) Using one r.v. to predict another X and Y are random variables. What is the best linear predictor b 0 + b 1 X of Y ? Prediction error is e = Y ` b 0 ` b 1 X . For the best predictor, there is

523 views • 40 slides

More recommend