Online Training of Community Therapists in Observational Coding of Family Therapy Techniques: Reliability and Accuracy

Aaron Hogue, Nicole Porter, Molly Bobek, Alexandra MacLean, Lila Bruynesteyn, Amanda Jensen-Doss, Sarah Dauber, Craig E. Henderson

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


A foundational strategy to promote implementation of evidence-based interventions (EBIs) is providing EBI training to therapists. This study tested an online training system in which therapists practiced observational coding of mock video vignettes demonstrating family therapy techniques for adolescent behavior problems. The study compared therapists ratings to gold-standard scores to measure therapist reliability (consistency across vignettes) and accuracy (approximation to gold scores); tested whether reliability and accuracy improved during training; and tested therapist-level predictors of overall accuracy and change in accuracy over time. Participants were 48 therapists working in nine community behavioral health clinics. The 32-exercise training course provided online instruction (about 15 min/week) in 13 core family therapy techniques representing three modules: Family Engagement, Relational Orientation, Interactional Change. Therapist reliability in rating technique presence (i.e., technique recognition) remained moderate across training; reliability in rating extensiveness of technique delivery (i.e., technique judgment) improved sharply over time, from poor to good. Whereas therapists on average overestimated extensiveness for almost every technique, their tendency to give low-accuracy scores decreased. Therapist accuracy improved significantly over time only for Interactional Change techniques. Baseline digital literacy and submission of self-report checklists on use of the techniques in their own sessions predicted coding accuracy. Training therapists to be more reliable and accurate coders of EBI techniques can potentially yield benefits in increased EBI self-report acumen and EBI use in daily practice. However, training effects may need to improve from those reported here to avail meaningful impact on EBI implementation. Trial Registration: The parent clinical trial is registered at, ID: NCT03342872 (registration date: 11.10.17).

Original languageEnglish (US)
Pages (from-to)139-151
Number of pages13
JournalAdministration and Policy in Mental Health and Mental Health Services Research
Issue number1
StatePublished - Jan 2022


  • Evidence-based intervention
  • Family therapy
  • Observational coding
  • Online training
  • Reliability
  • Usual care

ASJC Scopus subject areas

  • Phychiatric Mental Health
  • Health Policy
  • Public Health, Environmental and Occupational Health
  • Psychiatry and Mental health


Dive into the research topics of 'Online Training of Community Therapists in Observational Coding of Family Therapy Techniques: Reliability and Accuracy'. Together they form a unique fingerprint.

Cite this