STAT 113 Hypothesis Testing I Colin Reimer Dawson Oberlin College - - PowerPoint PPT Presentation

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

STAT 113 Hypothesis Testing I

Colin Reimer Dawson

Oberlin College

October 5, 2017 1 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Outline

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 2 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Self-Check (Pairs): Find a 90% CI for a Correlation

• What percentiles of the bootstrap distribution are needed to

get a 90% CI? Work these out before moving on.

• Go to http://lock5stat.com/statkey/ and select CI for

slope/correlation, under “Bootstrap CI”.

• Select “Restaurant Tips (Pct Tip as a Function of Bill)” from

the first dropdown menu. (Vars are % Tip and $s of Bill)

• Generate 10,000 bootstrap samples, and click the “Two-tail”

check box.

• Click on the box that says 0.950, and change the value to 0.90

(this is the confidence level).

• What is the resulting confidence interval for the population

correlation? Interpret it in real world terms. Is it wider or narrower than the 95% one? Explain why that makes sense in the context of estimation. 3 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Outline

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 4 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Hypothesis Testing

Sometimes we are not interested so much in a specific quantitative estimate as we are in evaluating a qualitative claim:

• 1. Do more people disapprove than approve of Donald Trump’s

job performance?

• 2. Do people in the population tip more (as a %) for more

expensive restaurant meals?

• 3. Does a new treatment work better than the old one on

average in the population? 5 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

The Lady Tasting Tea

In a party in the 1920s in Cambridge, a lady (Dr. Muriel Bristol, a psychologist) claimed she could tell a cup of tea had been prepared by adding milk before or after the tea was poured. The statistician Ronald Fisher, who was also in attendance, proposed to put it to a blind taste test w/ 10 cups of tea prepared in random order.

• Is her claim plausible if she gets 5 of 10 correct? 10 of 10? 9
• f 10?
• How much success is enough to believe her? Why?

6 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Outline

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 7 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Falsification

Karl Popper: scientific theories can’t be fully verified (there is always another possible explanation), only falsified. 8 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Falsification With Randomness

• When sampling, we will occasionally get strange results just by
• chance. So we can’t falsify absolutely.
• But we can say a hypothesis is implausible if the data would

be very unlikely if the hypothesis were true. 9 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

The Null Hypothesis

• R.A. Fisher: Formulate the negation of your research

hypothesis, and establish conditions under which it can be rejected.

• Fisher called this “antihypothesis” the null hypothesis, and

developed null hypothesis significance testing (NHST). 10 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

The Alternative Hypothesis

• Jerzy Neyman and Egon Pearson added the idea of the

alternative hypothesis to Fisher’s null hypothesis formulation.

• Idea: don’t reject H0 in a vacuum — reject in favor of another

hypothesis, the alternative hypothesis (or H1).

• This is usually the one you set out to investigate: the drug is

better; the correlation is positive. 11 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Outline

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 12 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

The Null and Alternative Hypothesis

What are the null and alternative hypotheses (abbreviated H0 and H1) corresponding to each of the following research claims?

• Dr. Bristol can tell the difference between cups of tea more
• ften than random guessing.
• There is a positive linear association between pH and mercury

in Florida lakes.

• Lab mice eat more on average when the room is light.

13 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Statistics vs. Parameters

• Summary values (like mean, median, standard deviation) can

be computed for populations or for samples.

• In a population, such a summary value is called a parameter
• In a sample, these values are called statistics, and are used to

estimate the corresponding parameter Value Population Parameter Sample Statistic Mean µ ¯ X Proportion p ˆ p Correlation ρ r Slope of a Line β1 ˆ β1 Difference in Means µ1 − µ2 ¯ X1 − ¯ X2 . . . . . . . . . 14 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Quantifying H0 and H1

• Pairs/Threes (5 min.): All of these hypotheses are statements

about the population (or about “long run behavior”). Can we “quantify” them by translating them into true/false statements about population parameters? Identify the relevant population parameter for each of the following claims and state the null and alternative hypotheses (abbreviated H0 and H1), as statements about that parameter.

• Dr. Bristol can tell the difference between cups of tea more
• ften than random guessing. H0: pcorrect = 0.5, H1:

pcorrect > 0.5, where pcorrect is her “long run” success rate

• There is a positive linear association between pH and mercury

in Florida lakes. H0: ρ = 0, H1: ρ > 0, where ρ is the correlation coefficient between pH and Hg in all Florida lakes

• Lab mice eat more on average when the room is light. H0:

µlight − µdark = 0, H1: µlight − µdark > 0, where µ are “long run”/population means for an appropriate measure of amount

• f food consumed

15 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Outline

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 16 / 17

Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence

Quantifying the Evidence

• Idea: if our sample would be very unlikely assuming the null

hypothesis (H0), but not so unlikely assuming H1, then we will reject H0 and accept H1.

• We say the result is statistically significant.
• If the sample statistic is not that surprising, our test is

inconclusive (the result is not statistically significant)

• Key questions:
• How do we quantify how unlikely the sample is?
• What are we even measuring the likelihood of?

17 / 17