Inference Statistical inference Definition: Definition: The act - - PDF document

10/20/2016 1

S L I D E 0

Statistics in medicine

Lecture 2: Hypothesis testing and inference

Fatma Shebl, MD, MS, MPH, PhD Assistant Professor Chronic Disease Epidemiology Department Yale School of Public Health Fatma.shebl@yale.edu

S L I D E 1

Outline

• Inference
• Hypothesis testing
• Type I error
• Type II error
• Confidence interval

S L I D E 2

Inference

• Definition:

–The act or process of reaching a conclusion about something from known facts or evidence

http://www.merriam-webster.com/dictionary/inference

S L I D E 3

Statistical inference

• Definition:

–The process of drawing conclusions about a population from quantitative or qualitative information obtained from a sample of observations using methods

• f statistics to describe the data and to

test suitable hypothesis. Or simply, –Drawing conclusions about population from a sample.

S L I D E 4

Hypothesis testing

• Hypothesis:

– Definition: Is a prediction about what the examination of appropriately collected data will show – Usually investigator starts with a research question such as “Are individuals infected with hepatitis C virus (HCV) are at excess risk of hepatocellular carcinoma (HCC)?” therefore the researcher states a hypothesis such as, “HCV infection is associated with excess risk of HCC”

S L I D E 5

Hypothesis testing

• Hypothesis testing:

– An approach to statistical inference resulting in a decision to reject or not to reject the null hypothesis

10/20/2016 2

S L I D E 6

Hypothesis testing

• Steps of hypothesis testing
• 1- Postulate hypotheses
• 2- Design a study, and collect data
• 3- Perform tests of statistical significance
• 4- Assess the evidence
• 5- Draw a conclusion

S L I D E 7

1- Postulate hypotheses

• Two hypotheses are postulated

– Null – Alternative

S L I D E 8

1- Postulate hypotheses

• Null:

– Written as H0 – The hypothesis that there is no real difference between groups (means, proportion, …etc.) being compared

• eg., there is no real difference in HCC risk

between HCV+ and HCV- subjects.

– If the data is consistent with the H0 hypothesis fail to reject H0 – If the data is not consistent with the H0 hypothesis reject H0

S L I D E 9

1- Postulate hypotheses

• Alternative

– Written as HA – The hypothesis that a real difference exists between groups (means, proportion,…etc.) being compared

• eg., there is real difference in HCC risk

between HCV+ and HCV- subjects.

– If the data is consistent with the HA hypothesis accept HA (reject H0)

S L I D E 10

2- Design a study and collect data

• Choose the best study design
• Determine the acceptable risk of error (type I

and Type II errors)

• Calculate the required sample size based on the

above

• Collect the data needed to test the study’s

research question

S L I D E 11

Type I error

• Definition:

– Rejecting the null hypothesis when it is true – Accepting the alternative hypothesis when it is false

• aka.

– Alpha error – False-positive error

• Generally we would like very small chance of rejecting the null

when it is true (to minimize harm)

Truth Statistical test results H0 False (there is a difference) H0 True (there is no difference) Decision

• f

statistical test Reject H0 Correct Power Type I error False-positive α=P(reject H0| H0 true) Statistically significant Fail to reject H0 Type II error False-negative β=P(accept H0| H0 false) Correct Statistically insignificant

10/20/2016 3

S L I D E 12

Type II error

• Definition:

– Accepting the null hypothesis when it is false – Rejecting the alternative hypothesis when it is true

• Aka.

– Beta error – False-negative error

• Generally we would like small chance of rejecting the alternative when it

is true – 20% is acceptable

Truth Statistical test results H0 False (there is a difference) H0 True (there is no difference) Decision

• f

statistical test Reject H0 Correct Power Type I error False-positive α=P(reject H0| H0 true) Statistically significant Fail to reject H0 Type II error False-negative β=P(accept H0| H0 false) Correct Statistically insignificant

S L I D E 13

Power

• Definition:
• Is the probability of detecting a difference if it actually exists
• Is the complement of type II error
• Is the true positive rate (sensitivity)
• Power= P(reject H0| H0 false) = 1- β

Truth Statistical test results H0 False (there is a difference) H0 True (there is no difference) Decision

• f

statistical test Reject H0 Correct Power Type I error False-positive α=P(reject H0| H0 true) Statistically significant Fail to reject H0 Type II error False-negative β=P(accept H0| H0 false) Correct Statistically insignificant

S L I D E 14

Power

• Definition:
• Is the probability of detecting a difference if it

actually exists

• Is the complement of type II error
• Is the true positive rate (sensitivity)
• Power= P(reject H0| H0 false) = 1- β
• 80% is acceptable

– Interpretation: There is 80% chance that the statistical test will detect the proposed difference, given that difference exists

S L I D E 15

Power

• Depends on the true magnitude of the association

– Strong association is easier to detect than a weak association

• Depends on the variance of the measure of effect

– The lesser the variance in the measure of effect, the less difficult to detect it

• Increase with

– Increasing effect size (RR) – Cohort study/RCT: increasing frequency of the outcome in source population (up to about 0.5) – Case-control study: increasing frequency of exposure in source population (up to about 0.5) – Decreasing variance in the measure of effect – Increasing sample size – Increasing significance level

S L I D E 16

3- Perform test of statistical significance

• Calculate the estimates
• Calculate a test statistic

– A measure of the difference between the actual sample estimate and the population parameter proposed by the H0

• Obtain the p value for the data
• Calculate the confidence interval of the estimate

S L I D E 17

4- Assess the evidence

• Compare the p value with the preselected

alpha level

• Inspect whether the confidence interval

include the fixed value

10/20/2016 4

S L I D E 18

4- Assess the evidence

• Alpha level:

– The highest acceptable risk of committing type I error

• 5% is acceptable

–Arbitrary level –<5%  reject the null –>5%  fail to reject the null

• Decided upon prior to data collection

S L I D E 19

4- Assess the evidence

• Alpha level:

–Interpretation of .05 alpha level

• The investigator is willing to take not

more than 5% risk of erroneously rejecting the null hypothesis when it is in fact true

S L I D E 20

4- Assess the evidence

P value

• The probability of observing statistic as extreme or

more extreme than the observed statistic given that the null hypothesis is true

• The p value is obtained from the test of significance

S L I D E 21

Figure modified from a graph generated by http://www.imathas.com/stattools/norm.html

4- Assess the evidence

Two-tailed test One-tailed test

1.645 SE 1.96 SE

One-tailed test (one-sided) When the hypothesis is “difference in

• ne direction”

A>B or B>A Two-tailed test (two-sided) When the hypothesis is “difference in both directions” A>B and A<B

Shaded area=.05 of total area under the curve Shaded area=.025 of total area under the curve Shaded area=.025 of total area under the curve

S L I D E 22

4- Assess the evidence

P value

• Magnitude of p value depends on

– Sample size (n): as n increases, p decreases – Standard deviation (SD): as SD increases, p increases

S L I D E 23

4- Assess the evidence

Meaning of statistical significance

• Small p value

– The results are unlikely to occur by chance – It is not equivalent to clinical or biological relevance – It is not equivalent to true association

• It could be true association
• It could be artifactitious due to confounding
• Large p value

– Null is true – The power of the study is low to detect a difference

10/20/2016 5

S L I D E 24

4- Assess the evidence

P value misconceptions (i.e. p value IS NOT)

• If P = .05, the null hypothesis has only a 5%

chance of being true

• With a P = .05 threshold for significance, the

chance of a type I error will be 5%.

• P = .05 means that if you reject the null

hypothesis, the probability of a type I error is only 5%

• P = .05 means that we have observed data that

would occur only 5% of the time under the null hypothesis

Goodman S (2008). A Dirty Dozen: twelve p-value misconceptions. Semin Hematol 45(3):135–140.

S L I D E 25

4- Assess the evidence

Confidence interval (CI)

• Definition:

– Range of values that describe uncertainty about an estimate – A set of parameter values most compatible with the data

• Another method of estimating population values and

indicating significance

S L I D E 26

4- Assess the evidence

Confidence interval

• General formula

100(1-α) CI= estimate+(confidence coefficient x standard error of the estimate) – If the population SD is known:

• 100(1-α) confidence interval of the mean= mean + (SE)
• eg. 95%CI of mean= mean + (SE) = mean + 1.96(SE)

– If the population SD is unknown:

• 100(1-α) confidence interval of the mean= mean + (SE)

– Single population proportion:

• 100(1-α) confidence interval of the proportion= + (SE)

S L I D E 27

Standard error

• Is the standard deviation of the distribution of the means
• Is used in calculating CI
• Is used in tests of statistical significance
• SE is dependent on the size of the sample

– Increasing the size of the sample decreasing the SE

• Standard error= SE

– SE of proportion= – SE of the mean =

S L I D E 28

4- Assess the evidence

Confidence interval

• Interpretation
• For a series of samples, all of the same sample size n obtained from

a population, and 100(1-α)% CI estimating population parameter are constructed for each sample, then the relative frequency with which these intervals contain the true population parameter is 100(1-α)% . – Ex. 95% CI: is a set of parameter values formed by a procedure, which if used repeatedly, will contain the true parameter 95%

• f the time (Statistical analysis of epidemiologic data, text for Prev 720, LS Magder)
• For a single interval obtained from a single sample, a 100(1-α)% CI

signifies that the investigator can be 100(1-α)% confident that this interval contains the unknown population parameter – Ex. 90% CI means that the investigator can be 90% confident that this interval contains the unknown population parameter

S L I D E 29

4- Assess the evidence

Confidence interval

• The width of the CI depends on the sample size

– Increasing sample size  decreasing the width of the CI

• The width of the CI depends on the standard error

– Increasing SE increasing the width of the CI

• The width of the CI depends on the selected percentage of confidence

– Increasing the confidence percentage  increasing the width of the CI e.g.. 95% CI is wider than a 90% CI

10/20/2016 6

S L I D E 30

4- Assess the evidence

Confidence interval

• Misconceptions (i.e. CI IS NOT)

– For a 95% CI, there is a 95% probability that the true value of the population mean is between lower and upper limits of the interval – For a 95% CI, there is 95% chance the CI includes the sample mean – For a 95% CI, 95% of the sample means drawn from the population should fall between lower and upper limits of the interval – For a 95% CI, there is 95% chance the population mean will be included between lower and upper limits of the interval – For a 95% CI, 95% of the data are included in the interval – CI are range of plausible values for the sample mean – CI are range of individual scores

S L I D E 31

4- Assess the evidence

Confidence interval

• Usage to test significance

– Could be used to test whether an estimate (mean, proportion,

• dds ratio, hazard ratio…etc.) differs significantly from a

fixed value

• Examples:

– For ratio: differs from 1 (i.e. interval crosses 1) – For mean difference: differs from 0 (i.e. interval crosses 0)

S L I D E 32

5- Draw a conclusion

• Reject or fail to reject the null hypothesis

– Fail to reject the null

• If the p value > alpha level
• If the confidence interval crosses the fixed

value – Reject the null

• If the p value < alpha level
• If the confidence interval does not include

the fixed value

S L I D E 33

5- Draw a conclusion

• Statistical versus clinical significance

– Statistically significant difference might not be clinically significant

• With large sample sizes, small differences

would be statistically significant, although clinically is not important –e.g. a 100 gm change in an adult’s body weight in response to an intervention might not be clinically important, even though it would be statistically significant

S L I D E 34

Readings and resources

• Chapter 10, p119-123: Jekel's

epidemiology, biostatistics, preventive medicine, and public health by David L. Katz et al (4th edition).