Intraclass correlation associated with therapists: Estimates and applications in planning psychotherapy research

Scott A. Baldwin, David M. Murray, William R. Shadish, Sherri L. Pals, Jason M. Holland, Jonathan S. Abramowitz, Gerhard Andersson, David C. Atkins, Per Carlbring, Kathleen M. Carroll, Andrew Christensen, Kari M. Eddington, Anke Ehlers, Daniel J Feaster, Ger P J Keijsers, Ellen Koch, Willem Kuyken, Alfred Lange, Tania Lincoln, Robert S. StephensSteven Taylor, Chris Trepka, Jeanne Watson

Research output: Contribution to journalArticle

41 Citations (Scopus)

Abstract

It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.

Original languageEnglish
Pages (from-to)15-33
Number of pages19
JournalCognitive Behaviour Therapy
Volume40
Issue number1
DOIs
StatePublished - Mar 1 2011

Fingerprint

Psychotherapy
Outcome Assessment (Health Care)
Research
Research Personnel
Databases
Power (Psychology)
Dependency (Psychology)

Keywords

  • Intraclass correlation
  • Power
  • Psychotherapy research
  • Statistical dependence
  • Therapist effects

ASJC Scopus subject areas

  • Clinical Psychology

Cite this

Baldwin, S. A., Murray, D. M., Shadish, W. R., Pals, S. L., Holland, J. M., Abramowitz, J. S., ... Watson, J. (2011). Intraclass correlation associated with therapists: Estimates and applications in planning psychotherapy research. Cognitive Behaviour Therapy, 40(1), 15-33. https://doi.org/10.1080/16506073.2010.520731

Intraclass correlation associated with therapists : Estimates and applications in planning psychotherapy research. / Baldwin, Scott A.; Murray, David M.; Shadish, William R.; Pals, Sherri L.; Holland, Jason M.; Abramowitz, Jonathan S.; Andersson, Gerhard; Atkins, David C.; Carlbring, Per; Carroll, Kathleen M.; Christensen, Andrew; Eddington, Kari M.; Ehlers, Anke; Feaster, Daniel J; Keijsers, Ger P J; Koch, Ellen; Kuyken, Willem; Lange, Alfred; Lincoln, Tania; Stephens, Robert S.; Taylor, Steven; Trepka, Chris; Watson, Jeanne.

In: Cognitive Behaviour Therapy, Vol. 40, No. 1, 01.03.2011, p. 15-33.

Research output: Contribution to journalArticle

Baldwin, SA, Murray, DM, Shadish, WR, Pals, SL, Holland, JM, Abramowitz, JS, Andersson, G, Atkins, DC, Carlbring, P, Carroll, KM, Christensen, A, Eddington, KM, Ehlers, A, Feaster, DJ, Keijsers, GPJ, Koch, E, Kuyken, W, Lange, A, Lincoln, T, Stephens, RS, Taylor, S, Trepka, C & Watson, J 2011, 'Intraclass correlation associated with therapists: Estimates and applications in planning psychotherapy research', Cognitive Behaviour Therapy, vol. 40, no. 1, pp. 15-33. https://doi.org/10.1080/16506073.2010.520731
Baldwin, Scott A. ; Murray, David M. ; Shadish, William R. ; Pals, Sherri L. ; Holland, Jason M. ; Abramowitz, Jonathan S. ; Andersson, Gerhard ; Atkins, David C. ; Carlbring, Per ; Carroll, Kathleen M. ; Christensen, Andrew ; Eddington, Kari M. ; Ehlers, Anke ; Feaster, Daniel J ; Keijsers, Ger P J ; Koch, Ellen ; Kuyken, Willem ; Lange, Alfred ; Lincoln, Tania ; Stephens, Robert S. ; Taylor, Steven ; Trepka, Chris ; Watson, Jeanne. / Intraclass correlation associated with therapists : Estimates and applications in planning psychotherapy research. In: Cognitive Behaviour Therapy. 2011 ; Vol. 40, No. 1. pp. 15-33.
@article{20aeb83f305b4a53906aa7cc418392ed,
title = "Intraclass correlation associated with therapists: Estimates and applications in planning psychotherapy research",
abstract = "It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.",
keywords = "Intraclass correlation, Power, Psychotherapy research, Statistical dependence, Therapist effects",
author = "Baldwin, {Scott A.} and Murray, {David M.} and Shadish, {William R.} and Pals, {Sherri L.} and Holland, {Jason M.} and Abramowitz, {Jonathan S.} and Gerhard Andersson and Atkins, {David C.} and Per Carlbring and Carroll, {Kathleen M.} and Andrew Christensen and Eddington, {Kari M.} and Anke Ehlers and Feaster, {Daniel J} and Keijsers, {Ger P J} and Ellen Koch and Willem Kuyken and Alfred Lange and Tania Lincoln and Stephens, {Robert S.} and Steven Taylor and Chris Trepka and Jeanne Watson",
year = "2011",
month = "3",
day = "1",
doi = "10.1080/16506073.2010.520731",
language = "English",
volume = "40",
pages = "15--33",
journal = "Cognitive Behaviour Therapy",
issn = "1650-6073",
publisher = "Taylor and Francis AS",
number = "1",

}

TY - JOUR

T1 - Intraclass correlation associated with therapists

T2 - Estimates and applications in planning psychotherapy research

AU - Baldwin, Scott A.

AU - Murray, David M.

AU - Shadish, William R.

AU - Pals, Sherri L.

AU - Holland, Jason M.

AU - Abramowitz, Jonathan S.

AU - Andersson, Gerhard

AU - Atkins, David C.

AU - Carlbring, Per

AU - Carroll, Kathleen M.

AU - Christensen, Andrew

AU - Eddington, Kari M.

AU - Ehlers, Anke

AU - Feaster, Daniel J

AU - Keijsers, Ger P J

AU - Koch, Ellen

AU - Kuyken, Willem

AU - Lange, Alfred

AU - Lincoln, Tania

AU - Stephens, Robert S.

AU - Taylor, Steven

AU - Trepka, Chris

AU - Watson, Jeanne

PY - 2011/3/1

Y1 - 2011/3/1

N2 - It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.

AB - It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.

KW - Intraclass correlation

KW - Power

KW - Psychotherapy research

KW - Statistical dependence

KW - Therapist effects

UR - http://www.scopus.com/inward/record.url?scp=79951780040&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79951780040&partnerID=8YFLogxK

U2 - 10.1080/16506073.2010.520731

DO - 10.1080/16506073.2010.520731

M3 - Article

C2 - 21337212

AN - SCOPUS:79951780040

VL - 40

SP - 15

EP - 33

JO - Cognitive Behaviour Therapy

JF - Cognitive Behaviour Therapy

SN - 1650-6073

IS - 1

ER -