SLIDE 1
Balkin, R. S. (2008). Balkin, R. S. (2008). 1 1
Balkin, R. S. (2008). Balkin, R. S. (2008). 2 2
Statistical Power in Statistical Power in ANOVA ANOVA Rick Balkin - - PowerPoint PPT Presentation
Statistical Power in Statistical Power in ANOVA ANOVA Rick Balkin - - PowerPoint PPT Presentation
Statistical Power in Statistical Power in ANOVA ANOVA Rick Balkin Balkin, Ph.D., LPC , Ph.D., LPC Rick Department of Counseling Department of Counseling Texas A&M University-Commerce Texas A&M University-Commerce
SLIDE 2
SLIDE 3
Balkin, R. S. (2008). Balkin, R. S. (2008). 3 3
Power Power
Power, therefore is directly related to type II
Power, therefore is directly related to type II
- error. The more power in a study, the less
- error. The more power in a study, the less
chance there is to identify a non-significant chance there is to identify a non-significant difference when there actually is a difference when there actually is a significant difference. significant difference.
Statistically, power is expressed by 1-
Statistically, power is expressed by 1-β β , , and therefore, type II error is expressed as and therefore, type II error is expressed as β β. .
SLIDE 4
Balkin, R. S. (2008). Balkin, R. S. (2008). 4 4
Power Power
The power of a study is dependent
The power of a study is dependent upon several factors: upon several factors:
– – Sample size Sample size – – Effect size Effect size – – Alpha level Alpha level
SLIDE 5
Balkin, R. S. (2008). Balkin, R. S. (2008). 5 5
Power and sample size Power and sample size
As discussed in the lecture on effect size, a
As discussed in the lecture on effect size, a large sample size increases the likelihood of large sample size increases the likelihood of finding statistically significant differences. finding statistically significant differences.
Thus larger sample sizes increase statistical
Thus larger sample sizes increase statistical power power
Often, statistical tests show significance, not
Often, statistical tests show significance, not because the results are meaningful, but because the results are meaningful, but simply because the sample size is so large simply because the sample size is so large that the test picks up on very minor that the test picks up on very minor deviations/differences. deviations/differences.
SLIDE 6
Balkin, R. S. (2008). Balkin, R. S. (2008). 6 6
Power and alpha level Power and alpha level
The alpha level also has an impact. The alpha level also has an impact.
When the alpha is at the .10 level of significance, as opposed When the alpha is at the .10 level of significance, as opposed to .05, the critical value is lowered and the likelihood of to .05, the critical value is lowered and the likelihood of finding a statistically significant difference increases ( finding a statistically significant difference increases (Fobs Fobs is is more likely to be larger than more likely to be larger than Fcrit Fcrit). ).
As the likelihood of masking a type I error is increased, the As the likelihood of masking a type I error is increased, the likelihood of making a type II error is decreased. Therefore, likelihood of making a type II error is decreased. Therefore, there is an inverse relationship between type I and type II there is an inverse relationship between type I and type II error. error.
While procedures exist to decrease the chance of making a While procedures exist to decrease the chance of making a type I error, researchers run this risk of increasing the chance type I error, researchers run this risk of increasing the chance
- f making a type II error, especially when smaller sample
- f making a type II error, especially when smaller sample
sizes are involved. sizes are involved.
SLIDE 7
Balkin, R. S. (2008). Balkin, R. S. (2008). 7 7
Power and effect size Power and effect size
Additionally, effect size is pertinent.
Additionally, effect size is pertinent.
The greater the magnitudes of the
The greater the magnitudes of the differences between groups, the fewer differences between groups, the fewer participants are needed to identify participants are needed to identify statistical significance. statistical significance.
SLIDE 8
Balkin, R. S. (2008). Balkin, R. S. (2008). 8 8
Power and error Power and error
Finally, power is influenced by error; the
Finally, power is influenced by error; the less error measured in a study, the more less error measured in a study, the more power. power.
While issues like the magnitude of the
While issues like the magnitude of the treatment effect or the error variance are treatment effect or the error variance are minimally influenced by the researcher, the minimally influenced by the researcher, the establishment of an alpha level and the establishment of an alpha level and the sample size are easily controlled. sample size are easily controlled.
The easiest method of increasing power in a
The easiest method of increasing power in a study is to increase sample size. study is to increase sample size.
SLIDE 9
Balkin, R. S. (2008). Balkin, R. S. (2008). 9 9
Power and research Power and research
Research methods may be wrought with emphasis
Research methods may be wrought with emphasis
- n statistical significance.
- n statistical significance.
An unfortunate trend is to discount meaningful
An unfortunate trend is to discount meaningful findings because no statistical difference or findings because no statistical difference or relationship exists. Perhaps not enough emphasis is relationship exists. Perhaps not enough emphasis is placed on practical significance. placed on practical significance.
Thompson (1999) identified the over-emphasis on
Thompson (1999) identified the over-emphasis on tests for statistical significance and emphasized the tests for statistical significance and emphasized the need to report practical significance along with need to report practical significance along with statistical significance. statistical significance.
SLIDE 10
Balkin, R. S. (2008). Balkin, R. S. (2008). 10 10
Power and research Power and research
Knowing where not to look for answers can be just
Knowing where not to look for answers can be just as important as knowing where to look for answers. as important as knowing where to look for answers.
However, moderate and large effect sizes may be
However, moderate and large effect sizes may be found when statistically significant differences do found when statistically significant differences do not exist, and this is usually due to a lack of not exist, and this is usually due to a lack of statistical power. statistical power.
When sample size is increased, statistical
When sample size is increased, statistical significance will be evident. Thus, having sufficient significance will be evident. Thus, having sufficient power in a design can be very important to the power in a design can be very important to the manner in which results are reported and ultimately manner in which results are reported and ultimately published. published.
SLIDE 11
Balkin, R. S. (2008). Balkin, R. S. (2008). 11 11
Power and research Power and research
For social sciences, power is usually
For social sciences, power is usually deemed sufficient at .80 deemed sufficient at .80— —80% chance 80% chance
- f finding statistically significant
- f finding statistically significant
differences when they actually do exist differences when they actually do exist and a 20% of type II error. Statistical and a 20% of type II error. Statistical packages do not compute power. packages do not compute power.
SLIDE 12
Balkin, R. S. (2008). Balkin, R. S. (2008). 12 12
Evaluating Power Evaluating Power
Three types of power analyses Three types of power analyses
A priori
A priori
Post hoc
Post hoc
Sensitivity
Sensitivity
SLIDE 13
Balkin, R. S. (2008). Balkin, R. S. (2008). 13 13
A priori A priori
The purpose is to identify the appropriate The purpose is to identify the appropriate sample size to conduct the analysis before sample size to conduct the analysis before data is even collected data is even collected
The researcher must be able to The researcher must be able to
1.
- 1. Estimate the effect size that would define
Estimate the effect size that would define statistical significance statistical significance 2.
- 2. Identify the number of groups in the study
Identify the number of groups in the study 3.
- 3. Set a minimum level of power (usually .80)
Set a minimum level of power (usually .80) 4.
- 4. Identify an alpha level for the study
Identify an alpha level for the study
SLIDE 14
Balkin, R. S. (2008). Balkin, R. S. (2008). 14 14
Post hoc Post hoc
The purpose of a post hoc power analysis The purpose of a post hoc power analysis is to identify whether power was adequate is to identify whether power was adequate for the study. for the study.
The researcher must be able to The researcher must be able to
1. 1. Identify the effect size in your study Identify the effect size in your study 2. 2. Identify the number of groups in the study Identify the number of groups in the study 3. 3. Identify the total sample size Identify the total sample size 4. 4. Identify an alpha level for the study Identify an alpha level for the study
SLIDE 15
Balkin, R. S. (2008). Balkin, R. S. (2008). 15 15
Sensitivity Sensitivity
The purpose of a sensitivity power analysis The purpose of a sensitivity power analysis is to identify the necessary effect size to is to identify the necessary effect size to detect statistical significance. detect statistical significance.
The researcher must be able to The researcher must be able to
1.
- 1. Set a minimum level of power (usually .80)
Set a minimum level of power (usually .80) 2.
- 2. Identify the number of groups in the study
Identify the number of groups in the study 3.
- 3. Identify the total sample size
Identify the total sample size 4.
- 4. Identify an alpha level for the study
Identify an alpha level for the study
SLIDE 16
Balkin, R. S. (2008). Balkin, R. S. (2008). 16 16
Power summary Power summary
Power is an under-reported but important
Power is an under-reported but important aspect in social science research. aspect in social science research.
Computing power is a little complicated but
Computing power is a little complicated but the interested reader may wish to refer to the interested reader may wish to refer to the text or pp. 10-12 in my the text or pp. 10-12 in my notepack notepack. .
A free program called G*Power will conduct