Identification of predicted individual treatment effects in randomized clinical trials

Andrea Lamont, Michael D. Lyons, Thomas Jaki, Elizabeth Stuart, Daniel J Feaster, Kukatharmini Tharmaratnam, Daniel Oberski, Hemant Ishwaran, Dawn K. Wilson, M. Lee Van Horn

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

In most medical research, treatment effectiveness is assessed using the average treatment effect or some version of subgroup analysis. The practice of individualized or precision medicine, however, requires new approaches that predict how an individual will respond to treatment, rather than relying on aggregate measures of effect. In this study, we present a conceptual framework for estimating individual treatment effects, referred to as predicted individual treatment effects. We first apply the predicted individual treatment effect approach to a randomized controlled trial designed to improve behavioral and physical symptoms. Despite trivial average effects of the intervention, we show substantial heterogeneity in predicted individual treatment response using the predicted individual treatment effect approach. The predicted individual treatment effects can be used to predict individuals for whom the intervention may be most effective (or harmful). Next, we conduct a Monte Carlo simulation study to evaluate the accuracy of predicted individual treatment effects. We compare the performance of two methods used to obtain predictions: multiple imputation and nonparametric random decision trees. Results showed that, on average, both predictive methods produced accurate estimates at the individual level; however, the random decision trees tended to underestimate the predicted individual treatment effect for people at the extreme and showed more variability in predictions across repetitions compared to the imputation approach. Limitations and future directions are discussed.

Original languageEnglish (US)
Pages (from-to)142-157
Number of pages16
JournalStatistical Methods in Medical Research
Volume27
Issue number1
DOIs
StatePublished - Jan 1 2018

Fingerprint

Randomized Clinical Trial
Treatment Effects
Randomized Controlled Trials
Decision tree
Therapeutics
Decision Trees
Precision Medicine
Average Treatment Effect
Randomized Controlled Trial
Predict
Multiple Imputation
Prediction
Imputation
Medicine
Behavioral Symptoms
Trivial
Extremes
Monte Carlo Simulation
Simulation Study
Subgroup

Keywords

  • Heterogeneity in treatment effects
  • Individual predictions
  • Individualized medicine
  • Multiple imputation
  • Predicted individual treatment effects
  • Random decision trees
  • Random forests

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability
  • Health Information Management

Cite this

Identification of predicted individual treatment effects in randomized clinical trials. / Lamont, Andrea; Lyons, Michael D.; Jaki, Thomas; Stuart, Elizabeth; Feaster, Daniel J; Tharmaratnam, Kukatharmini; Oberski, Daniel; Ishwaran, Hemant; Wilson, Dawn K.; Van Horn, M. Lee.

In: Statistical Methods in Medical Research, Vol. 27, No. 1, 01.01.2018, p. 142-157.

Research output: Contribution to journalArticle

Lamont, A, Lyons, MD, Jaki, T, Stuart, E, Feaster, DJ, Tharmaratnam, K, Oberski, D, Ishwaran, H, Wilson, DK & Van Horn, ML 2018, 'Identification of predicted individual treatment effects in randomized clinical trials', Statistical Methods in Medical Research, vol. 27, no. 1, pp. 142-157. https://doi.org/10.1177/0962280215623981
Lamont, Andrea ; Lyons, Michael D. ; Jaki, Thomas ; Stuart, Elizabeth ; Feaster, Daniel J ; Tharmaratnam, Kukatharmini ; Oberski, Daniel ; Ishwaran, Hemant ; Wilson, Dawn K. ; Van Horn, M. Lee. / Identification of predicted individual treatment effects in randomized clinical trials. In: Statistical Methods in Medical Research. 2018 ; Vol. 27, No. 1. pp. 142-157.
@article{cca4346fba574dfda1688950c634e018,
title = "Identification of predicted individual treatment effects in randomized clinical trials",
abstract = "In most medical research, treatment effectiveness is assessed using the average treatment effect or some version of subgroup analysis. The practice of individualized or precision medicine, however, requires new approaches that predict how an individual will respond to treatment, rather than relying on aggregate measures of effect. In this study, we present a conceptual framework for estimating individual treatment effects, referred to as predicted individual treatment effects. We first apply the predicted individual treatment effect approach to a randomized controlled trial designed to improve behavioral and physical symptoms. Despite trivial average effects of the intervention, we show substantial heterogeneity in predicted individual treatment response using the predicted individual treatment effect approach. The predicted individual treatment effects can be used to predict individuals for whom the intervention may be most effective (or harmful). Next, we conduct a Monte Carlo simulation study to evaluate the accuracy of predicted individual treatment effects. We compare the performance of two methods used to obtain predictions: multiple imputation and nonparametric random decision trees. Results showed that, on average, both predictive methods produced accurate estimates at the individual level; however, the random decision trees tended to underestimate the predicted individual treatment effect for people at the extreme and showed more variability in predictions across repetitions compared to the imputation approach. Limitations and future directions are discussed.",
keywords = "Heterogeneity in treatment effects, Individual predictions, Individualized medicine, Multiple imputation, Predicted individual treatment effects, Random decision trees, Random forests",
author = "Andrea Lamont and Lyons, {Michael D.} and Thomas Jaki and Elizabeth Stuart and Feaster, {Daniel J} and Kukatharmini Tharmaratnam and Daniel Oberski and Hemant Ishwaran and Wilson, {Dawn K.} and {Van Horn}, {M. Lee}",
year = "2018",
month = "1",
day = "1",
doi = "10.1177/0962280215623981",
language = "English (US)",
volume = "27",
pages = "142--157",
journal = "Statistical Methods in Medical Research",
issn = "0962-2802",
publisher = "SAGE Publications Ltd",
number = "1",

}

TY - JOUR

T1 - Identification of predicted individual treatment effects in randomized clinical trials

AU - Lamont, Andrea

AU - Lyons, Michael D.

AU - Jaki, Thomas

AU - Stuart, Elizabeth

AU - Feaster, Daniel J

AU - Tharmaratnam, Kukatharmini

AU - Oberski, Daniel

AU - Ishwaran, Hemant

AU - Wilson, Dawn K.

AU - Van Horn, M. Lee

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In most medical research, treatment effectiveness is assessed using the average treatment effect or some version of subgroup analysis. The practice of individualized or precision medicine, however, requires new approaches that predict how an individual will respond to treatment, rather than relying on aggregate measures of effect. In this study, we present a conceptual framework for estimating individual treatment effects, referred to as predicted individual treatment effects. We first apply the predicted individual treatment effect approach to a randomized controlled trial designed to improve behavioral and physical symptoms. Despite trivial average effects of the intervention, we show substantial heterogeneity in predicted individual treatment response using the predicted individual treatment effect approach. The predicted individual treatment effects can be used to predict individuals for whom the intervention may be most effective (or harmful). Next, we conduct a Monte Carlo simulation study to evaluate the accuracy of predicted individual treatment effects. We compare the performance of two methods used to obtain predictions: multiple imputation and nonparametric random decision trees. Results showed that, on average, both predictive methods produced accurate estimates at the individual level; however, the random decision trees tended to underestimate the predicted individual treatment effect for people at the extreme and showed more variability in predictions across repetitions compared to the imputation approach. Limitations and future directions are discussed.

AB - In most medical research, treatment effectiveness is assessed using the average treatment effect or some version of subgroup analysis. The practice of individualized or precision medicine, however, requires new approaches that predict how an individual will respond to treatment, rather than relying on aggregate measures of effect. In this study, we present a conceptual framework for estimating individual treatment effects, referred to as predicted individual treatment effects. We first apply the predicted individual treatment effect approach to a randomized controlled trial designed to improve behavioral and physical symptoms. Despite trivial average effects of the intervention, we show substantial heterogeneity in predicted individual treatment response using the predicted individual treatment effect approach. The predicted individual treatment effects can be used to predict individuals for whom the intervention may be most effective (or harmful). Next, we conduct a Monte Carlo simulation study to evaluate the accuracy of predicted individual treatment effects. We compare the performance of two methods used to obtain predictions: multiple imputation and nonparametric random decision trees. Results showed that, on average, both predictive methods produced accurate estimates at the individual level; however, the random decision trees tended to underestimate the predicted individual treatment effect for people at the extreme and showed more variability in predictions across repetitions compared to the imputation approach. Limitations and future directions are discussed.

KW - Heterogeneity in treatment effects

KW - Individual predictions

KW - Individualized medicine

KW - Multiple imputation

KW - Predicted individual treatment effects

KW - Random decision trees

KW - Random forests

UR - http://www.scopus.com/inward/record.url?scp=85041407704&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041407704&partnerID=8YFLogxK

U2 - 10.1177/0962280215623981

DO - 10.1177/0962280215623981

M3 - Article

C2 - 26988928

AN - SCOPUS:85041407704

VL - 27

SP - 142

EP - 157

JO - Statistical Methods in Medical Research

JF - Statistical Methods in Medical Research

SN - 0962-2802

IS - 1

ER -