Chapter 8: Hypothesis Testing STK4011/9011: Statistical Inference - PowerPoint PPT Presentation

Chapter 8: Hypothesis Testing STK4011/9011: Statistical Inference Theory Johan Pensar STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 1 / 36

Overview Methods of Finding Tests 1 Likelihood Ratio Tests Union-Intersection and Intersection-Union Tests Methods of Evaluating Tests 2 Error Probabilities and The Power Function Most Powerful Tests Covers Sec 8.1, 8.2.1, 8.2.3, 8.3.1, and 8.3.2 in CB. STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 2 / 36

Hypothesis Testing A hypothesis is here a statement about a population parameter θ . The goal of a hypothesis test is to decide, based on a sample from the population, which of two complementary hypotheses is true: H 0 : θ 2 Θ 0 ( null hypothesis ) , H 1 : θ 2 Θ c 0 ( alternative hypothesis ) . Example: Let θ denote the average change in a patient’s blood pressure after taking a drug. Then, an experimenter might want to test H 0 : θ = 0 , H 1 : θ 6 = 0 . STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 3 / 36

Hypothesis Testing A hypothesis test is thus basically a rule that specifies: For which sample values x , H 0 is accepted as true (or not rejected), For which sample values x , H 0 is rejected and H 1 is instead accepted as true. The subset of the sample space for which H 0 is rejected is called the rejection region . Typically, a hypothesis test is specified in terms of a test statistic W ( X ). Example: A test could be that H 0 is to be rejected if W ( X ) = ¯ X is larger than some constant c , that is, the rejection region is { x : ¯ x > c } . STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 4 / 36

Overview Methods of Finding Tests 1 Likelihood Ratio Tests Union-Intersection and Intersection-Union Tests Methods of Evaluating Tests 2 Error Probabilities and The Power Function Most Powerful Tests STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 5 / 36

The Likelihood Ratio Test (LRT) Recall, for a random sample X 1 , . . . , X n from a population with pdf/pmf f ( x | θ ). the likelihood function is n Y L ( θ | x ) = f ( x | θ ) = f ( x i | θ ) . i =1 Def: Let Θ denote the entire parameter space. The likelihood ratio test statistic for testing H 0 : θ 2 Θ 0 versus H 1 : θ 2 Θ c 0 is λ ( x ) = sup Θ 0 L ( θ | x ) sup Θ L ( θ | x ) , and a likelihood ratio test (LRT) is any test that has a rejection region of the form { x : λ ( x )  c } , where 0  c  1. STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 6 / 36

The Rationale Behind the LRT Let ˆ θ be the MLE of θ obtained by doing unrestricted maximization of L ( θ | x ), that is, w.r.t the entire parameter space Θ = Θ 0 [ Θ c 0 . Let ˆ θ 0 be the MLE of θ obtained by doing restricted maximization of L ( θ | x ) w.r.t the “null parameter space” Θ 0 . Then, the LRT statistic is λ ( x ) = L (ˆ θ 0 | x ) , L (ˆ θ | x ) and it has a small value if the observed sample is much more likely for a parameter point in Θ c 0 than for any parameter point in Θ 0 (in which case H 0 is rejected). STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 7 / 36

Example: Normal LRT STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 8 / 18

Example: Exponential LRT STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 9 / 18

LRT with Su ffi cient Statistics Let T ( X ) be a su ffi cient statistic for θ , that is, it contains all the information about θ available in the sample. Then, we can construct an LRT statistic λ ⇤ ( t ), based on T and with likelihood function L ⇤ ( θ | t ) = g ( t | θ ), which is equivalent to the LRT statistic λ ( x ) based on the complete sample. Thm 8.2.4: If T ( X ) is a su ffi cient statistic for θ , and λ ⇤ ( t ) and λ ( x ) are the LRT statistics based on T and X , respectively, then λ ⇤ � � T ( x ) = λ ( x ) for every x in the sample space. STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 10 / 36

Proof of Thm 8.2.4 STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 11 / 18

Examples: LRT and Su ffi ciency Let X 1 , . . . , X n be iid N ( θ , 1). Then, ¯ X is a su ffi cient statistic for θ and the likelihood function associated with ¯ X ⇠ N ( θ , 1 / n ) could be used to conclude that an LRT of H 0 : θ = θ 0 versus H 1 : θ 6 = θ 0 rejects H 0 for large values of | ¯ X � θ 0 | . Let X 1 , . . . , X n be iid with an exponential pdf f ( x | θ ) = e � ( x � θ ) , x � θ . Then, X (1) is a su ffi cient statistic for θ and the likelihood function of X (1) could be used to conclude that an LRT of H 0 : θ  θ 0 versus H 1 : θ > θ 0 rejects H 0 for large values of X (1) . STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 12 / 36

Nuisance Parameters Likelihood ratio tests are also useful in situations where there are nuisance parameters present in the model. A nuisance parameter is not of direct inferential interest but it is present in the model. The presence of nuisance parameters does not a ff ect the LRT construction method, but it might lead to a di ff erent test. STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 13 / 36

Example: Normal LRT with Unknown Variance STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 14 / 18

Overview Methods of Finding Tests 1 Likelihood Ratio Tests Union-Intersection and Intersection-Union Tests Methods of Evaluating Tests 2 Error Probabilities and The Power Function Most Powerful Tests STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 15 / 36

The Union-Intersection Method The union-intersection method can be used when the null hypothesis can be expressed as an intersection: \ H 0 : θ 2 Θ γ . γ 2 Γ Assume that there are tests available for each of the problems of testing H 1 γ : θ 2 Θ c H 0 γ : θ 2 Θ γ versus γ , with the rejection region of H 0 γ being { x : T γ ( x ) 2 R γ } . Then, the rejection region for H 0 is [ { x : T γ ( x ) 2 R γ } . γ 2 Γ STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 16 / 36

Example: Normal Union-Intersection Test STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 17 / 18

The Intersection-Union Method The intersection-union method can be used when the null hypothesis can be expressed as a union: [ H 0 : θ 2 Θ γ . γ 2 Γ Assume that there are tests available for each of the problems of testing H 1 γ : θ 2 Θ c H 0 γ : θ 2 Θ γ versus γ , with the rejection region of H 0 γ being { x : T γ ( x ) 2 R γ } . Then, the rejection region for H 0 is \ { x : T γ ( x ) 2 R γ } . γ 2 Γ STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 18 / 36

Overview Methods of Finding Tests 1 Likelihood Ratio Tests Union-Intersection and Intersection-Union Tests Methods of Evaluating Tests 2 Error Probabilities and The Power Function Most Powerful Tests STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 19 / 36

Evaluating Tests When deciding whether to reject or accept (or not reject) the null hypothesis H 0 , the experimenter might make a mistake. Hypothesis tests are evaluated by how likely they are to make di ff erent type of mistakes or errors. While the error probabilities can be controlled (to some extent), there is typically a tradeo ff between di ff erent types of errors. STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 20 / 36

The Two Types of Errors in Hypothesis Testing A hypothesis test of H 0 : θ 2 Θ 0 versus H 1 : θ 2 Θ c 0 can make two types of mistakes: Type I error: H 0 is rejected when θ 2 Θ 0 , Type II error: H 0 is accepted when θ 2 Θ c 0 . Decision Accept H 0 Reject H 0 H 0 Correct decision Type I error Truth H 1 Type II error Correct decision STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 21 / 36

The Power Function Let R denote a test’s rejection region (samples for which H 0 is rejected): If θ 2 Θ 0 , the probability of a Type I error is P θ ( X 2 R ), If θ 2 Θ c 0 , the probability of a Type II error is P θ ( X 2 R c ) = 1 � P θ ( X 2 R ). Thus, we have that ( P (Type I Error) , if θ 2 Θ 0 , P θ ( X 2 R ) = if θ 2 Θ c 1 � P (Type II Error) , 0 . Def: The power function of a hypothesis test with rejection region R is the function of θ defined by β ( θ ) = P θ ( X 2 R ). STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 22 / 36

Example: Binomial Power Function STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 23 / 36

Example: Normal Power Function STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 24 / 36

Controlling the Probability of Type I Errors For a fixed sample size, it is typically impossible make the probabilities of both errors arbitrarily small. The common approach is to only consider tests that control the Type 1 error probability at a specified level. In the above class of tests, one then wants to find the test with smallest Type II error probability. Def: For 0  α  1, a test with power function β ( θ ) is A size α test if sup θ ∈ Θ 0 β ( θ ) = α , A level α test if sup θ ∈ Θ 0 β ( θ )  α . STK4011/9011: Statistical Inference Theory Chapter 8: Hypothesis Testing 25 / 36