Sample Size Vorasith Sornsrivichai, MD., FETP Epidemiology Unit, - - PowerPoint PPT Presentation
Sample Size Vorasith Sornsrivichai, MD., FETP Epidemiology Unit, Faculty of Medicine Prince of Songkla University All nature is but art, unknown to thee; All chance, direction, which thou canst not see; All discord, harmony not understood;
Vorasith Sornsrivichai, MD., FETP Epidemiology Unit, Faculty of Medicine Prince of Songkla University
“All nature is but art, unknown to thee; All chance, direction, which thou canst not see; All discord, harmony not understood; All partial evil, universal good; And spite of pride, in erring reason's spite, One truth is clear, Whatever is, is right” ~ Alexander Pope ~
3
How Much Is Enough?
“Is sample size of 30 subjects enough?” “If I sampling 10% of population will it be
OK?”
“Can I just use all 24 patients I have?”
4
Objectives
To learn how to calculate the sample size needed to
parameter
To learn how to calculate the sample size needed to
provide a specified power for a comparative study
5
Outline of Presentation
Review of basic principle Determination of sample size Sample size calculation
6
Source: http://trochim.human.cornell.edu/kb/random.htm
7
Two Types Of Study Objective
Estimation: Approximation of some
parameters (magnitude or difference or ratio)
Critical feature is the precision of the estimation. e.g. “A public health officer seeks to estimate the
proportion of children in the district receiving vaccinations.”
Hypothesis testing: Examination of
proposed assumption
Critical feature is the power of the study e.g. “Is drug B more effective than drug A?”
8
Determinants of The Sample Size
Effect size Level of significance Power of the test Variation of the outcome
9
Other Determinants of The Sample Size
Research questions and objective of the study Defining the population and the population size Type of outcome e.g. dichotomous, continuous Outcome measurement e.g. single, repeated
measurements
Sampling technique e.g. cluster sampling Type of statistical methods Type of analysis e.g. subgroup analysis Non-responses or lost to follow-up
10
Effect Size
RR, OR, RD, etc. The higher the effect size, the lower the
sample size needed
(, to forgive divine) ~ Alexander Pope~
12
Errors
Truth Study Results
Ho is not true H0 is true Reject Ho
1 –
Power Type I error Fail to reject H0 Type II error 1 – Confidence
β
α
β
α
13
Significance
False detection of difference/association by
“chance”
Statistical significance VS Epidemiological &
Clinical significance
α
14
Power (1-
) is the probability of rejecting Ho when Ho is not true
Power of the test
Ha H0 Study number
Power = 9/10 *100 = 90%
β
“Knowledge is an unending adventure at the edge of uncertainty.”
~ Jacob Bronowski ~
16
Uncertainty…
Variability in the population: not all samples would
give exactly the same finding, i.e., there is uncertainty in making an inference
However, the uncertainty can usually be quantified Uncertainty can be reduced by using a sufficiently
large sample
Population Sample n = 2 n = 5 n = 20
Central Limit Theorem
If samples are drawn from a non-normally distributed
parent population, the frequency distribution of the population of sample means approaches the normal distribution as the sample size increases.
Population Sample n = 2 n = 5 n = 20
19
Sampling Distributions
As the sample size increases:
the sample means tend to be distributed normally the width of the distribution decreases
As the number of samples increases:
the mean of the distribution of sample means tends to the
mean of the population
The above is also true for sample estimates of population proportion
as long as the proportion is not too close to 0 or 1
20
Standard Normal Distribution
0 1 1.96 2.56
0.6826 0.9545 0.9973
X-3SE X-2SE X-1SE X X+1SE X+2SE X+3SE
21
Estimation
Distribution of estimate of the means from many samples Big n Small n
Narrow SE Wide SE
22
Range of population values
Estimation
(large sample)
Sampling distributions from populations with various values of X bar
Study value
d d
d = precision
23
Estimation
(small sample)
Sampling distributions from populations with various values of X bar
d d
Study value
Range of population values
d = precision
Population1
μ1~150 cm.
σ1~ 5 cm. Population2
μ2~150 cm. σ2~ 10 cm. d
X
d
Estimate of mean height α = 0.05 d=3 cm.
x1 x2 n1=12 n2=45
Distribution of means of hypothetical samples
25
SDA ~ σ SDB~ σ
25
SEB=SDB/ N= ∞
μ
N= ∞
μ
σ σ σ σ Population Population Uncertainty in measure sample A
X
Estimation Uncertainty in measure sample B
X
Estimation Sample A n=100 Data
A
X
Sample B n=25 Data
B
X 100
SEA=SDA/
27
Sample Size Calculation
Available tables Nomogram Manual calculation Software: EpiInfo, STATA, R, OpenEpi
28
Available Table
e.g. sample size to estimate P within d absolute
percentage points with 99% confidence
29
http://ccforum.com/content/6/4/335
Open Source Epidemiologic Statistics for Public Health http://www.openepi.com
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
Considerations
The appropriate sample size may not be the
same for all objectives in a study.
Therefore calculate for all objectives then
decide
All sample size calculations considered here
and in most computer programs assume simple random sampling
Other sampling method e.g. cluster
sampling may require adjustments
50
Considerations
Calculated sample size is the minimum sample
needed
Add more (~10–30%) for non-response and
lost to follow up
E.g. suppose 10% of subjects in the study are
expected to refuse to participate or to drop out before the study ends.
The total number of n/(1-0.1) eligible
subjects would have to be approached in the first instance
51
Inappropriate Sample Size
Too SMALL wide CI unable to detect a
real effect
may miss important
association
Too BIG waste of reource
(effort, time, money)
even very small
effects become statistical significant
may be unethical
“Although our intellect always longs for clarity and certainty,
~ Karl von Clausewitz ~
53
Sample Size Calculation
One sample
Estimating: proportion, mean Hypothesis testing: proportion, mean
Two sample
Estimating: difference between two proportions,
two means
Hypothesis testing: difference between two
proportions, two means
54
2 / 1
−α
d
Sample size calculations for estimation are based on : In each case, we just put in the appropriate expression for standard error e.g.
n SD/
55
Estimating A Population Mean
n SE σ =
2 2 2
d Z n
1
σ
α/2 −
= ∴
n Z d
1
σ
α/2 −
= ∴
SE Z d
1
× =
− 2 / α
56
Example 1
(Estimating a population mean)
An estimate is desired of the average retail price of 20
tablets of a tranquilizer. It is required to be within 10 %
was estimated as 85 %. How many pharmacies should be randomly selected?
n = (1.96)2(0.85)2/(0.1)2 ~278
2 2 2 / 2
d Z n σ
α
=
57
Estimating A Population Proportion
n p 1 p SE ) ( − =
2 2
) ( d p 1 p Z n
1
− = ∴
/2 −α
SE Z d
1
× =
− 2 / α
58
Example 2
(Estimating a population proportion)
A district public health officer seeks to estimate the
proportion of children in the district receiving appropriate childhood vaccinations. How many children must be studied if the resulting estimate is to fall within 10 % of the true proportion with 95% CI.
n = (1.96)2(0.25)/(0.1)2 = 96.04
2 1
)/d p
p Z n (
2 / 2 α −
=
59
The sample selected will
be largest when P = 0.5
When one has no idea
what the level of P is in the population, choosing 0.5 for P will always provide enough
P P(1-P) 0.5 0.25 0.4 0.24 0.3 0.21 0.2 0.16 0.1 0.09
60
mean of B - mean of A
β
α) ( ) ( ) ( ) ( *
1 2 / 1 2 /
SE Z SE Z SE Z SE Z
a a
× + × = Δ × = × − Δ =
β β
Δ
*
1
Δ is minimum effect worth detecting
61
Basic Equations Underlying Sample Size
) (
/ β β α
Z Z SE SE Z SE Z
α/2 1 1 2 1
+ = Δ × + × = Δ
− −
2a
If SE0= SE1 = SE then
2b
SE Z d
1
Most sample size calculations for estimation and hypothesis testing are based on these equations.
× =
− 2 / α
1
62
How to Choose Δ
Δ should be the “minimum difference of
clinical significance”, or the “minimum difference worth detecting”.
Previously reported differences may
not be suitable for your study.
It may be useful to consider the
standardized effect size (ε = Δ/σ) when the outcome is a continuous variable.
63
Estimating The Difference Between Two Means
2 1
n 1 n 1 SE + = σ
2 2 2
) ( d Z r 1 1 n
1 1
σ
α/2 −
+ = ∴
SE Z d
1
× =
− 2 / α
r n 1 n 1 SE
1 1
+ = σ
then r n n If
1 2
=
64
Example 3
(Estimating the difference between two means)
Nutritionists wish to estimate the difference in caloric intake
at lunch between children in a school offering hot lunches and children in a school which does not. From other studies, they estimate that the SD of caloric intake among schoolchildren is 75 calories, and they wish to make their estimate to within 20 calories of the true difference with 95% confidence. (Equal numbers in each group)
= 108.05 ~ 109 (Note that r = n2/n1)
2 2 2 / 2
] / [ d Z r 1 1 n
1 1
σ
α −
+ =
65
Estimating the difference between two proportions
r n p 1 p n p 1 p SE
1 2 2 1 1 1
) ( ) ( − + − =
2 2
) ( ) ( d r p 1 p p 1 p Z n
2 2 1 1 1 1
⎟ ⎠ ⎞ ⎜ ⎝ ⎛ − + − = ∴
/2 −α
SE Z d
1
× =
− 2 / α
⎟ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎜ ⎝ ⎛ − + − = ∴
− 1 2 1 1 1
n p 1 r p p 1 p Z d ) ( ) (
2 2 α
66
Example 4 (Estimating the difference between two proportions)
It is desired to estimate a risk difference in two
industrial groups. How large a sample should be selected in each group for the estimate to be within 5 percentage points of the true difference with 95% confidence. It was observed that P1 = 0.4, P2 = 0.32. (Equal numbers in each group)
n1 = 1.96 [(0.40)(0.60) +
(0.32)(0.68)]/(0.05)2 ~ 704
2 2 / 2 1
] / ) ( ) ( [ d r p 1 p p 1 p Z n
2 2 1 1 1
− + − =
−α
67
Testing the hypothesis of a difference between two means
r n 1 n 1 SE
1 1
+ = σ
β α
Z Z SE
1
+ = Δ
− 2 /
Z
r n 1 n 1
2 1 1 1
σ
β α
+ + = Δ ∴
( )
Z Z r n 1 r
2 2
1 2 2
σ
β α +
⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ + = Δ ∴
( )
2 2 2
) ( Δ + + = ∴
/2 −
σ
β α
Z Z r 1 1 n
1 1
68
Example 5
(Testing the hypothesis of a difference between two means)
A study is being designed to measure the effect, on systolic
blood pressure, of lowering sodium in the diet. From a pilot study it is observed that the SD of SBP in a community with high sodium diet is 12 mm Hg, while that in a group with low sodium diet is 10.3 mm Hg. If alpha is 0.05 and beta is 0.10, how large a sample from each community should be selected in order to detect a 2 mm Hg difference in blood pressure between the communities? (Equal group size and use pooled variance)
2 2 2 2 /
) )( / ( Δ + + =
−
σ
β α
Z Z r 1 1 n
1 1
69
Testing the hypothesis of a difference between two proportions) Ho true: Ho not true:
1 2 1
SE Z SE Z × + × = Δ
− β α /
2 2 2 2 /
/ ) 1 ( ) 1 ( ) / )( ( Δ − + − + + − = ∴
−
r p p p p Z r 1 1 p 1 p Z n
1 1 2 1 1 β α
1
n r 1 1 p 1 p SE + × − = ) (
( ) ( )
r 1 rp p p
2 1
+ + =
1 2 2 1 1 1 1
rn p 1 p n p 1 p SE ) ( ) ( − + − =
1 2 2 1 1 1 a 1
n r p 1 p p 1 p Z n r 1 1 p 1 p Z ) ( ) ( ) )( (
2
− + − + + − = Δ ∴
− β
70
Example 6
(Testing the hypothesis of a difference between two proportions)
) /( ) ( / ) ( ) ( ) / )( (
2 2 2 /
] [
r 1 rp p p r p 1 p p 1 p Z r 1 1 p 1 p Z n
2 1 2 2 1 1 1 1
+ + = Δ − + − + + − =
− β α
A case-control study is to be conducted with a case:control ratio of 1:2. Exposure to the potential risk factor of interest among controls is expected to be 20%. How many cases and controls will be needed to detect an odds ratio of at least 2.0, at a significance level of 0.05 with a power of 80 percent? (Let n1=number of cases, and n2 = number of controls)
( )
2 2
20 . 33 . 2 / ) 20 . 1 ( 20 . ) 33 . 1 ( 33 . 84 . ) 2 1 1 )( 243 . 1 ( 243 . 96 . 1
] [
− − + − + + − = / n1
n1 = 132 and n2 = 2 * 132 = 264
71
Power Determination
Power = 1- =1-function(A)=1-P(Z )
Continuous data
(nt=nc) (nt nc)
c t c t t c c t c
n n n n Z A n Z A / ) ( / 2
2 2 / 2 2 /
σ μ μ σ μ μ
α α
+ − − = − − =
β
β
≠
72
To compare a new antihypertensive drug with the standard
treatment (n=150 in each group). The difference in BP treated with these two drugs was 4 mmHg. The variance was 140
difference in these two drugs. Do you agree with this conclusion?
A= Power = 1-function(A)
=1-f(-0.9677) =1-0.1685 =0.8315
Power = 83.15%
Exercise
9677 . 150 / ) 140 ( 2 4 96 . 1 − = −
73
Power Determination
Discrete data and proportion
nt=nc nt nc
c t t c c c t c t c c t t c c t c c
n Q P n Q P P P n Q P n Q P Z A n Q P Q P P P n Q P Z A / ) / / / / ) / 2 + − − + = + − − =
α α
P Q P P P
c t
− = + = 1 ) ( 2 / 1
≠
74
Exercise
To compare between 2 kinds of anti UV cream, A and B.
Seventy five of 100 patients treated with cream A whereas 65 out of 100 patients treated with cream B improved. The researcher concluded that these two kinds of cream were not different at 5% of level of significance. Do you agree with this conclusion?
Power = 1- f(0.1026) = 1-0.46 = 0.54 ~54%
1026 . 100 / )] 25 . )( 75 . ( ) 35 . )( 65 . [( 65 . 75 . 100 / ) 30 )(. 70 (. 2 645 . 1 = + − − = A
75
Interpretations Of “Negative Findings” - Power Calculations
For a hypothesis-testing study which fails to reject
the null hypothesis, it is useful to conduct a post- hoc power calculation.
We can use a rearrangement of the relevant
sample-size equation.
This should be done using the clinically relevant
difference for the Δ of the equation (not the difference found in the study).
76
β − 1
Power depends on:
the size of difference the
treatment makes
the rates of events
among control patients
the alpha level in use the number of patients in
the trial
β − 1
Nonrejection region Rejection region