TY - JOUR
T1 - Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials
AU - Brincks, Ahnalee
AU - Montag, Samantha
AU - Howe, George W.
AU - Huang, Shi
AU - Siddique, Juned
AU - Ahn, Soyeon
AU - Sandler, Irwin
AU - Pantin, Hilda
AU - Brown, C. Hendricks
N1 - Funding Information:
We gratefully acknowledge the National Institute of Mental Health Collaborative Data Synthesis for Adolescent Depression Trials Study Team, composed of our many colleagues who generously provided their data to be used in this study, obtained access to key datasets, reviewed coding decisions, provided substantive or methodologic recommendations, and reviewed this manuscript. We also thank NIMH for their support through grant number R01-MH040859 (Collaborative Data Synthesis for Adolescent Depression Trials, Brown PI) and NCATS for their support through grant number UL1-TR000460 (Miami Clinical and Translational Science Institute, Szapocznik PI), as well as the following grants that financially supported the trials: NIMH R01-MH048696 (Beardslee), NIMH R01-MH064717 (Beardslee & Gladstone), NIMH R01-MH064503 (Brent), NIMH R03-MH048118 (Clarke), NIDA R01-DA07031 (Dishion), NIDA R01-DA13773 (Dishion), NIMH R01-MH064735 (Garber), WT Grant Foundation 961730 (Garber), NIMH R01-MH064707 (Gonzales), NIDA R01-DA017462 (Pantin), NIDA R01-DA025894 (Prado), CDC U01-PS000671 (Prado), NIMH P30-MH068685 (Sandler), NIMH R01-MH049155 (Sandler & Ayers), NIMH R01-MH052270 (Seligman), NIDA R01-DA007029 (Spoth), NIDA R01-DA018374 (Stormshak), NIMH MH063042 (Szapocznik), NIMH K08-MH072918 (Van Voorhees), NIMH K23-MH071320 (Young), and NARSAD Young Investigator Grant (Young). The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies nor that of our collaborators who provided access to their data. CHB was supported as a consultant on one of these projects (New Beginnings) and received funding support on the Familias Unidas trials as did Brincks, Huang, and Pantin who developed and directed much of research on this intervention. Sandler developed and directed much of the work on two trials in this synthesis.
Funding Information:
Acknowledgments We gratefully acknowledge the National Institute of Mental Health Collaborative Data Synthesis for Adolescent Depression Trials Study Team, composed of our many colleagues who generously provided their data to be used in this study, obtained access to key datasets, reviewed coding decisions, provided substantive or methodologic recommendations, and reviewed this manuscript. We also thank NIMH for their support through grant number R01-MH040859 (Collaborative Data Synthesis for Adolescent Depression Trials, Brown PI) and NCATS for their support through grant number UL1-TR000460 (Miami Clinical and Translational Science Institute, Szapocznik PI), as well as the following grants that financially supported the trials: NIMH R01-MH048696 (Beardslee), NIMH R01-MH064717 (Beardslee & Gladstone), NIMH R01-MH064503 (Brent), NIMH R03-MH048118 (Clarke), NIDA R01-DA07031 (Dishion), NIDA R01-DA13773 (Dishion), NIMH R01-MH064735 (Garber), WT Grant Foundation 961730 (Garber), NIMH R01-MH064707 (Gonzales), NIDA R01-DA017462 (Pantin), NIDA R01-DA025894 (Prado), CDC U01-PS000671 (Prado), NIMH P30-MH068685 (Sandler), NIMH R01-MH049155 (Sandler & Ayers), NIMH R01-MH052270 (Seligman), NIDA R01-DA007029 (Spoth), NIDA R01-DA018374 (Stormshak), NIMH MH063042 (Szapocznik), NIMH K08-MH072918 (Van Voorhees), NIMH K23-MH071320 (Young), and NARSAD Young Investigator Grant (Young). The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies nor that of our collaborators who provided access to their data.
Publisher Copyright:
© 2017, Society for Prevention Research.
PY - 2018/2/1
Y1 - 2018/2/1
N2 - Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.
AB - Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.
KW - Harmonization
KW - Integrative data analysis
KW - Participant-level meta-analysis
KW - Synthesis methodology
UR - http://www.scopus.com/inward/record.url?scp=85018848570&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85018848570&partnerID=8YFLogxK
U2 - 10.1007/s11121-017-0769-1
DO - 10.1007/s11121-017-0769-1
M3 - Article
C2 - 28434055
AN - SCOPUS:85018848570
SN - 1389-4986
VL - 19
SP - 60
EP - 73
JO - Prevention Science
JF - Prevention Science
ER -