Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Ahnalee Brincks, Samantha Montag, George W. Howe, Shi Huang, Juned Siddique, Soyeon Ahn, Irwin N. Sandler, Hilda Pantin, C. Hendricks Brown

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.

Original languageEnglish (US)
Pages (from-to)1-14
Number of pages14
JournalPrevention Science
DOIs
StateAccepted/In press - Apr 22 2017

Fingerprint

Randomized Controlled Trials
Time and Motion Studies
Sample Size
Observational Studies
Longitudinal Studies
Meta-Analysis
Research Personnel
Outcome Assessment (Health Care)
Depression
Growth
Datasets

Keywords

  • Harmonization
  • Integrative data analysis
  • Participant-level meta-analysis
  • Synthesis methodology

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health

Cite this

Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials. / Brincks, Ahnalee; Montag, Samantha; Howe, George W.; Huang, Shi; Siddique, Juned; Ahn, Soyeon; Sandler, Irwin N.; Pantin, Hilda; Brown, C. Hendricks.

In: Prevention Science, 22.04.2017, p. 1-14.

Research output: Contribution to journalArticle

Brincks, Ahnalee ; Montag, Samantha ; Howe, George W. ; Huang, Shi ; Siddique, Juned ; Ahn, Soyeon ; Sandler, Irwin N. ; Pantin, Hilda ; Brown, C. Hendricks. / Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials. In: Prevention Science. 2017 ; pp. 1-14.
@article{e4218e02b92a4a36a2e54aa4f8dee566,
title = "Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials",
abstract = "Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.",
keywords = "Harmonization, Integrative data analysis, Participant-level meta-analysis, Synthesis methodology",
author = "Ahnalee Brincks and Samantha Montag and Howe, {George W.} and Shi Huang and Juned Siddique and Soyeon Ahn and Sandler, {Irwin N.} and Hilda Pantin and Brown, {C. Hendricks}",
year = "2017",
month = "4",
day = "22",
doi = "10.1007/s11121-017-0769-1",
language = "English (US)",
pages = "1--14",
journal = "Prevention Science",
issn = "1389-4986",
publisher = "Springer New York",

}

TY - JOUR

T1 - Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

AU - Brincks, Ahnalee

AU - Montag, Samantha

AU - Howe, George W.

AU - Huang, Shi

AU - Siddique, Juned

AU - Ahn, Soyeon

AU - Sandler, Irwin N.

AU - Pantin, Hilda

AU - Brown, C. Hendricks

PY - 2017/4/22

Y1 - 2017/4/22

N2 - Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.

AB - Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.

KW - Harmonization

KW - Integrative data analysis

KW - Participant-level meta-analysis

KW - Synthesis methodology

UR - http://www.scopus.com/inward/record.url?scp=85018848570&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85018848570&partnerID=8YFLogxK

U2 - 10.1007/s11121-017-0769-1

DO - 10.1007/s11121-017-0769-1

M3 - Article

C2 - 28434055

AN - SCOPUS:85018848570

SP - 1

EP - 14

JO - Prevention Science

JF - Prevention Science

SN - 1389-4986

ER -