TY - JOUR
T1 - Sample-Size Planning for More Accurate Statistical Power
T2 - A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty
AU - Anderson, Samantha F.
AU - Kelley, Ken
AU - Maxwell, Scott E.
N1 - Publisher Copyright:
© 2017, © The Author(s) 2017.
PY - 2017/11/1
Y1 - 2017/11/1
N2 - The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty. We show that the use of this approach often results in underpowered studies, sometimes to an alarming degree. We present an alternative approach that adjusts sample effect sizes for bias and uncertainty, and we demonstrate its effectiveness for several experimental designs. Furthermore, we discuss an open-source R package, BUCSS, and user-friendly Web applications that we have made available to researchers so that they can easily implement our suggested methods.
AB - The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty. We show that the use of this approach often results in underpowered studies, sometimes to an alarming degree. We present an alternative approach that adjusts sample effect sizes for bias and uncertainty, and we demonstrate its effectiveness for several experimental designs. Furthermore, we discuss an open-source R package, BUCSS, and user-friendly Web applications that we have made available to researchers so that they can easily implement our suggested methods.
KW - effect size
KW - methodology
KW - publication bias
KW - sample size
KW - statistical power
UR - http://www.scopus.com/inward/record.url?scp=85033228473&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85033228473&partnerID=8YFLogxK
U2 - 10.1177/0956797617723724
DO - 10.1177/0956797617723724
M3 - Article
C2 - 28902575
AN - SCOPUS:85033228473
SN - 0956-7976
VL - 28
SP - 1547
EP - 1562
JO - Psychological Science
JF - Psychological Science
IS - 11
ER -