Performance of evidence-based youth psychotherapies compared with usual clinical care a multilevelmeta-analysis

John R. Weisz, Sofie Kuppens, Dikla Eckshtain, Ana M. Ugueto, Kristin M. Hawley, Amanda Doss

Research output: Contribution to journalArticle

151 Citations (Scopus)

Abstract

IMPORTANCE Research across more than 4 decades has produced numerous empirically tested evidence-based psychotherapies (EBPs) for psychopathology in children and adolescents. The EBPs were developed to improve on usual clinical interventions. Advocates argue that the EBPs should replace usual care, but this assumes that EBPs produce better outcomes than usual care. OBJECTIVE To determine whether EBPs do in fact produce better outcomes than usual care in youth psychotherapy.We performed ameta-analysis of 52 randomized trials directly comparing EBPs with usual care. Analyses assessed the overall effect of EBPs vs usual care and candidate moderators; we used multilevel analysis to address the dependency among effect sizes (ES) that is common but typically unaddressed in psychotherapy syntheses. DATA SOURCES We searched the PubMed, PsychINFO, and Dissertation Abstracts International databases for studies from January 1, 1960, through December 31, 2010. STUDY SELECTION We identified 507 randomized youth psychotherapy trials. Of these, the 52 studies that compared EBPs with usual care were included in the meta-analysis. DATA EXTRACTION AND SYNTHESIS Sixteen variables (participant, treatment, outcome, and study characteristics) were extracted from studies, and ESs were calculated for all comparisons of EBP vs usual care.We used an extension of the commonly used random-effects meta-analytic model to obtain an overall estimate of the difference between EBP and usual care while accounting for the dependency among ESs.We then fitted a 3-level mixed-effects model to identify moderators that might explain variation in ESs within and between studies by adding study or ES characteristics as fixed predictors. MAIN OUTCOMES AND MEASURES Primary outcomes of our meta-analysis were mean ES estimates across all studies and for levels of candidate moderators. These ES values were based on measures of symptoms, functioning, and other outcomes assessed within the 52 randomized trials. RESULTS Evidence-based psychotherapies outperformed usual care. Mean ES was 0.29; the probability was 58%that a randomly selected youth would have a better outcome after EBP than a randomly selected youth after receiving usual care. The following 3 variables moderated treatment benefit: ESs decreased for studies conducted outside North America, for studies in which all participants were impaired enough to qualify for diagnoses, and for outcomes reported by informants other than the youths and parents in therapy. For certain key groups (eg, studies of clinically referred samples and youths with diagnoses), significant EBP effects were not demonstrated. CONCLUSIONS AND RELEVANCE Evidence-based psychotherapies outperform usual care, but the EBP advantage is modest and moderated by youth, location, and assessment characteristics. The EBPs have room for improvement in the magnitude and range of their benefit relative to usual clinical care.

Original languageEnglish
Pages (from-to)750-761
Number of pages12
JournalJAMA Psychiatry
Volume70
Issue number7
DOIs
StatePublished - Jul 1 2013

Fingerprint

Psychotherapy
Meta-Analysis
Multilevel Analysis
North America
Psychopathology
PubMed

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Psychiatry and Mental health

Cite this

Performance of evidence-based youth psychotherapies compared with usual clinical care a multilevelmeta-analysis. / Weisz, John R.; Kuppens, Sofie; Eckshtain, Dikla; Ugueto, Ana M.; Hawley, Kristin M.; Doss, Amanda.

In: JAMA Psychiatry, Vol. 70, No. 7, 01.07.2013, p. 750-761.

Research output: Contribution to journalArticle

Weisz, John R. ; Kuppens, Sofie ; Eckshtain, Dikla ; Ugueto, Ana M. ; Hawley, Kristin M. ; Doss, Amanda. / Performance of evidence-based youth psychotherapies compared with usual clinical care a multilevelmeta-analysis. In: JAMA Psychiatry. 2013 ; Vol. 70, No. 7. pp. 750-761.
@article{337ce9647a074d818d6f26d9a19c619d,
title = "Performance of evidence-based youth psychotherapies compared with usual clinical care a multilevelmeta-analysis",
abstract = "IMPORTANCE Research across more than 4 decades has produced numerous empirically tested evidence-based psychotherapies (EBPs) for psychopathology in children and adolescents. The EBPs were developed to improve on usual clinical interventions. Advocates argue that the EBPs should replace usual care, but this assumes that EBPs produce better outcomes than usual care. OBJECTIVE To determine whether EBPs do in fact produce better outcomes than usual care in youth psychotherapy.We performed ameta-analysis of 52 randomized trials directly comparing EBPs with usual care. Analyses assessed the overall effect of EBPs vs usual care and candidate moderators; we used multilevel analysis to address the dependency among effect sizes (ES) that is common but typically unaddressed in psychotherapy syntheses. DATA SOURCES We searched the PubMed, PsychINFO, and Dissertation Abstracts International databases for studies from January 1, 1960, through December 31, 2010. STUDY SELECTION We identified 507 randomized youth psychotherapy trials. Of these, the 52 studies that compared EBPs with usual care were included in the meta-analysis. DATA EXTRACTION AND SYNTHESIS Sixteen variables (participant, treatment, outcome, and study characteristics) were extracted from studies, and ESs were calculated for all comparisons of EBP vs usual care.We used an extension of the commonly used random-effects meta-analytic model to obtain an overall estimate of the difference between EBP and usual care while accounting for the dependency among ESs.We then fitted a 3-level mixed-effects model to identify moderators that might explain variation in ESs within and between studies by adding study or ES characteristics as fixed predictors. MAIN OUTCOMES AND MEASURES Primary outcomes of our meta-analysis were mean ES estimates across all studies and for levels of candidate moderators. These ES values were based on measures of symptoms, functioning, and other outcomes assessed within the 52 randomized trials. RESULTS Evidence-based psychotherapies outperformed usual care. Mean ES was 0.29; the probability was 58{\%}that a randomly selected youth would have a better outcome after EBP than a randomly selected youth after receiving usual care. The following 3 variables moderated treatment benefit: ESs decreased for studies conducted outside North America, for studies in which all participants were impaired enough to qualify for diagnoses, and for outcomes reported by informants other than the youths and parents in therapy. For certain key groups (eg, studies of clinically referred samples and youths with diagnoses), significant EBP effects were not demonstrated. CONCLUSIONS AND RELEVANCE Evidence-based psychotherapies outperform usual care, but the EBP advantage is modest and moderated by youth, location, and assessment characteristics. The EBPs have room for improvement in the magnitude and range of their benefit relative to usual clinical care.",
author = "Weisz, {John R.} and Sofie Kuppens and Dikla Eckshtain and Ugueto, {Ana M.} and Hawley, {Kristin M.} and Amanda Doss",
year = "2013",
month = "7",
day = "1",
doi = "10.1001/jamapsychiatry.2013.1176",
language = "English",
volume = "70",
pages = "750--761",
journal = "JAMA Psychiatry",
issn = "2168-622X",
publisher = "American Medical Association",
number = "7",

}

TY - JOUR

T1 - Performance of evidence-based youth psychotherapies compared with usual clinical care a multilevelmeta-analysis

AU - Weisz, John R.

AU - Kuppens, Sofie

AU - Eckshtain, Dikla

AU - Ugueto, Ana M.

AU - Hawley, Kristin M.

AU - Doss, Amanda

PY - 2013/7/1

Y1 - 2013/7/1

N2 - IMPORTANCE Research across more than 4 decades has produced numerous empirically tested evidence-based psychotherapies (EBPs) for psychopathology in children and adolescents. The EBPs were developed to improve on usual clinical interventions. Advocates argue that the EBPs should replace usual care, but this assumes that EBPs produce better outcomes than usual care. OBJECTIVE To determine whether EBPs do in fact produce better outcomes than usual care in youth psychotherapy.We performed ameta-analysis of 52 randomized trials directly comparing EBPs with usual care. Analyses assessed the overall effect of EBPs vs usual care and candidate moderators; we used multilevel analysis to address the dependency among effect sizes (ES) that is common but typically unaddressed in psychotherapy syntheses. DATA SOURCES We searched the PubMed, PsychINFO, and Dissertation Abstracts International databases for studies from January 1, 1960, through December 31, 2010. STUDY SELECTION We identified 507 randomized youth psychotherapy trials. Of these, the 52 studies that compared EBPs with usual care were included in the meta-analysis. DATA EXTRACTION AND SYNTHESIS Sixteen variables (participant, treatment, outcome, and study characteristics) were extracted from studies, and ESs were calculated for all comparisons of EBP vs usual care.We used an extension of the commonly used random-effects meta-analytic model to obtain an overall estimate of the difference between EBP and usual care while accounting for the dependency among ESs.We then fitted a 3-level mixed-effects model to identify moderators that might explain variation in ESs within and between studies by adding study or ES characteristics as fixed predictors. MAIN OUTCOMES AND MEASURES Primary outcomes of our meta-analysis were mean ES estimates across all studies and for levels of candidate moderators. These ES values were based on measures of symptoms, functioning, and other outcomes assessed within the 52 randomized trials. RESULTS Evidence-based psychotherapies outperformed usual care. Mean ES was 0.29; the probability was 58%that a randomly selected youth would have a better outcome after EBP than a randomly selected youth after receiving usual care. The following 3 variables moderated treatment benefit: ESs decreased for studies conducted outside North America, for studies in which all participants were impaired enough to qualify for diagnoses, and for outcomes reported by informants other than the youths and parents in therapy. For certain key groups (eg, studies of clinically referred samples and youths with diagnoses), significant EBP effects were not demonstrated. CONCLUSIONS AND RELEVANCE Evidence-based psychotherapies outperform usual care, but the EBP advantage is modest and moderated by youth, location, and assessment characteristics. The EBPs have room for improvement in the magnitude and range of their benefit relative to usual clinical care.

AB - IMPORTANCE Research across more than 4 decades has produced numerous empirically tested evidence-based psychotherapies (EBPs) for psychopathology in children and adolescents. The EBPs were developed to improve on usual clinical interventions. Advocates argue that the EBPs should replace usual care, but this assumes that EBPs produce better outcomes than usual care. OBJECTIVE To determine whether EBPs do in fact produce better outcomes than usual care in youth psychotherapy.We performed ameta-analysis of 52 randomized trials directly comparing EBPs with usual care. Analyses assessed the overall effect of EBPs vs usual care and candidate moderators; we used multilevel analysis to address the dependency among effect sizes (ES) that is common but typically unaddressed in psychotherapy syntheses. DATA SOURCES We searched the PubMed, PsychINFO, and Dissertation Abstracts International databases for studies from January 1, 1960, through December 31, 2010. STUDY SELECTION We identified 507 randomized youth psychotherapy trials. Of these, the 52 studies that compared EBPs with usual care were included in the meta-analysis. DATA EXTRACTION AND SYNTHESIS Sixteen variables (participant, treatment, outcome, and study characteristics) were extracted from studies, and ESs were calculated for all comparisons of EBP vs usual care.We used an extension of the commonly used random-effects meta-analytic model to obtain an overall estimate of the difference between EBP and usual care while accounting for the dependency among ESs.We then fitted a 3-level mixed-effects model to identify moderators that might explain variation in ESs within and between studies by adding study or ES characteristics as fixed predictors. MAIN OUTCOMES AND MEASURES Primary outcomes of our meta-analysis were mean ES estimates across all studies and for levels of candidate moderators. These ES values were based on measures of symptoms, functioning, and other outcomes assessed within the 52 randomized trials. RESULTS Evidence-based psychotherapies outperformed usual care. Mean ES was 0.29; the probability was 58%that a randomly selected youth would have a better outcome after EBP than a randomly selected youth after receiving usual care. The following 3 variables moderated treatment benefit: ESs decreased for studies conducted outside North America, for studies in which all participants were impaired enough to qualify for diagnoses, and for outcomes reported by informants other than the youths and parents in therapy. For certain key groups (eg, studies of clinically referred samples and youths with diagnoses), significant EBP effects were not demonstrated. CONCLUSIONS AND RELEVANCE Evidence-based psychotherapies outperform usual care, but the EBP advantage is modest and moderated by youth, location, and assessment characteristics. The EBPs have room for improvement in the magnitude and range of their benefit relative to usual clinical care.

UR - http://www.scopus.com/inward/record.url?scp=84880563714&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880563714&partnerID=8YFLogxK

U2 - 10.1001/jamapsychiatry.2013.1176

DO - 10.1001/jamapsychiatry.2013.1176

M3 - Article

VL - 70

SP - 750

EP - 761

JO - JAMA Psychiatry

JF - JAMA Psychiatry

SN - 2168-622X

IS - 7

ER -