Dynamic wait-listed designs for randomized trials: New designs for prevention of youth suicide

C. Hendricks Brown, Peter A. Wyman, Jing Guo, Juan Peña

Research output: Contribution to journalArticle

62 Citations (Scopus)

Abstract

Background: The traditional wait-listed design, where half are randomly assigned to receive the intervention early and half are randomly assigned to receive it later, is often acceptable to communities who would not be comfortable with a no-treatment group. As such this traditional wait-listed design provides an excellent opportunity to evaluate short-term impact of an intervention. We introduce a new class of wait-listed designs for conducting randomized experiments where all subjects receive the intervention, and the timing of the intervention is randomly assigned. We use the term "dynamic wait-listed designs" to describe this new class. Purpose: This paper examines a new class of statistical designs where random assignment to intervention condition occurs at multiple times in a trial. As an extension of a traditional wait-listed design, this dynamic design allows all subjects to receive the intervention at a random time. Motivated by our search for increased statistical power in an ongoing school-based trial that is testing a program of gatekeeper training to identify suicidal youth and refer them to treatment, this new design class is especially useful when the primary outcome is a count or rate of occurrence, such as suicidal behavior, whose rate can fluctuate over time due to uncontrolled factors. Methods: Statistical power is computed for various dynamic wait-listed designs under conditions where the underlying rate of occurrence is allowed to vary non-systematically. We also present as an example a large ongoing trial to evaluate a gatekeeper training suicide prevention program in 32 schools which we initially began as a classic randomized wait-listed design. The primary outcome of interest in this study is the count of the number of children who are identified by the school system as having suicidal thoughts or behaviors who are then validated as being suicidal by mental health professionals in the community. Results: A general result shows that dynamic wait-listed designs always have higher statistical power over a traditional wait-listed design. This power increase can be substantial. Efficiency gains of 33% are easy to obtain for situations where the intervention has a small effect and the variation in rate across time is quite high. When the rate variation for an outcome is very low or the intervention effect is large, efficiency gains approach 100%. A small increase in the number of times where random assignment occurs from 2 - for the standard wait-listed design, to say 4 - can provide a large reduction in variance. Efficiency gains can also be high when converting standard wait-listed design to a dynamic one half-way into the study. Limitations As with all wait-listed designs, dynamic wait-listed designs can only be used to evaluate short-term impact. Since all subjects eventually receive the intervention, no comparison can be made after the end of the random assignment period. The statistical power benefits are primarily limited to outcomes that can be treated as count or time to event data. Conclusions: A dynamic design randomly assigns units - either individuals or groups - to start the intervention at varying times during the course of the study. This design is useful in testing interventions that screen for new or existing cases, as well as testing the scalability of interventions as they are disseminated or expanded system wide. They can improve on the traditional wait-listed design both in terms of statistical power and robustness in the presence of exogenous factors. This paper demonstrates that such designs yield smaller standard errors and can achieve higher statistical power than that of a standard wait-listed design. Just as important, dynamic designs can also help reduce the logistical challenges of implementing an intervention on a wide scale. When the intervention requires that significant training resources be allocated throughout the study, the dynamic wait-listed design is likely to increase the rate of training and lead to a higher level of program implementation.

Original languageEnglish
Pages (from-to)259-271
Number of pages13
JournalClinical Trials
Volume3
Issue number3
DOIs
StatePublished - Jul 27 2006

Fingerprint

Suicide
Mental Health
Education
Therapeutics

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Dynamic wait-listed designs for randomized trials : New designs for prevention of youth suicide. / Brown, C. Hendricks; Wyman, Peter A.; Guo, Jing; Peña, Juan.

In: Clinical Trials, Vol. 3, No. 3, 27.07.2006, p. 259-271.

Research output: Contribution to journalArticle

Brown, C. Hendricks ; Wyman, Peter A. ; Guo, Jing ; Peña, Juan. / Dynamic wait-listed designs for randomized trials : New designs for prevention of youth suicide. In: Clinical Trials. 2006 ; Vol. 3, No. 3. pp. 259-271.
@article{c5734a8433c14b2ebf9cf8b5d9bb2d6d,
title = "Dynamic wait-listed designs for randomized trials: New designs for prevention of youth suicide",
abstract = "Background: The traditional wait-listed design, where half are randomly assigned to receive the intervention early and half are randomly assigned to receive it later, is often acceptable to communities who would not be comfortable with a no-treatment group. As such this traditional wait-listed design provides an excellent opportunity to evaluate short-term impact of an intervention. We introduce a new class of wait-listed designs for conducting randomized experiments where all subjects receive the intervention, and the timing of the intervention is randomly assigned. We use the term {"}dynamic wait-listed designs{"} to describe this new class. Purpose: This paper examines a new class of statistical designs where random assignment to intervention condition occurs at multiple times in a trial. As an extension of a traditional wait-listed design, this dynamic design allows all subjects to receive the intervention at a random time. Motivated by our search for increased statistical power in an ongoing school-based trial that is testing a program of gatekeeper training to identify suicidal youth and refer them to treatment, this new design class is especially useful when the primary outcome is a count or rate of occurrence, such as suicidal behavior, whose rate can fluctuate over time due to uncontrolled factors. Methods: Statistical power is computed for various dynamic wait-listed designs under conditions where the underlying rate of occurrence is allowed to vary non-systematically. We also present as an example a large ongoing trial to evaluate a gatekeeper training suicide prevention program in 32 schools which we initially began as a classic randomized wait-listed design. The primary outcome of interest in this study is the count of the number of children who are identified by the school system as having suicidal thoughts or behaviors who are then validated as being suicidal by mental health professionals in the community. Results: A general result shows that dynamic wait-listed designs always have higher statistical power over a traditional wait-listed design. This power increase can be substantial. Efficiency gains of 33{\%} are easy to obtain for situations where the intervention has a small effect and the variation in rate across time is quite high. When the rate variation for an outcome is very low or the intervention effect is large, efficiency gains approach 100{\%}. A small increase in the number of times where random assignment occurs from 2 - for the standard wait-listed design, to say 4 - can provide a large reduction in variance. Efficiency gains can also be high when converting standard wait-listed design to a dynamic one half-way into the study. Limitations As with all wait-listed designs, dynamic wait-listed designs can only be used to evaluate short-term impact. Since all subjects eventually receive the intervention, no comparison can be made after the end of the random assignment period. The statistical power benefits are primarily limited to outcomes that can be treated as count or time to event data. Conclusions: A dynamic design randomly assigns units - either individuals or groups - to start the intervention at varying times during the course of the study. This design is useful in testing interventions that screen for new or existing cases, as well as testing the scalability of interventions as they are disseminated or expanded system wide. They can improve on the traditional wait-listed design both in terms of statistical power and robustness in the presence of exogenous factors. This paper demonstrates that such designs yield smaller standard errors and can achieve higher statistical power than that of a standard wait-listed design. Just as important, dynamic designs can also help reduce the logistical challenges of implementing an intervention on a wide scale. When the intervention requires that significant training resources be allocated throughout the study, the dynamic wait-listed design is likely to increase the rate of training and lead to a higher level of program implementation.",
author = "Brown, {C. Hendricks} and Wyman, {Peter A.} and Jing Guo and Juan Pe{\~n}a",
year = "2006",
month = "7",
day = "27",
doi = "10.1191/1740774506cn152oa",
language = "English",
volume = "3",
pages = "259--271",
journal = "Clinical Trials",
issn = "1740-7745",
publisher = "SAGE Publications Ltd",
number = "3",

}

TY - JOUR

T1 - Dynamic wait-listed designs for randomized trials

T2 - New designs for prevention of youth suicide

AU - Brown, C. Hendricks

AU - Wyman, Peter A.

AU - Guo, Jing

AU - Peña, Juan

PY - 2006/7/27

Y1 - 2006/7/27

N2 - Background: The traditional wait-listed design, where half are randomly assigned to receive the intervention early and half are randomly assigned to receive it later, is often acceptable to communities who would not be comfortable with a no-treatment group. As such this traditional wait-listed design provides an excellent opportunity to evaluate short-term impact of an intervention. We introduce a new class of wait-listed designs for conducting randomized experiments where all subjects receive the intervention, and the timing of the intervention is randomly assigned. We use the term "dynamic wait-listed designs" to describe this new class. Purpose: This paper examines a new class of statistical designs where random assignment to intervention condition occurs at multiple times in a trial. As an extension of a traditional wait-listed design, this dynamic design allows all subjects to receive the intervention at a random time. Motivated by our search for increased statistical power in an ongoing school-based trial that is testing a program of gatekeeper training to identify suicidal youth and refer them to treatment, this new design class is especially useful when the primary outcome is a count or rate of occurrence, such as suicidal behavior, whose rate can fluctuate over time due to uncontrolled factors. Methods: Statistical power is computed for various dynamic wait-listed designs under conditions where the underlying rate of occurrence is allowed to vary non-systematically. We also present as an example a large ongoing trial to evaluate a gatekeeper training suicide prevention program in 32 schools which we initially began as a classic randomized wait-listed design. The primary outcome of interest in this study is the count of the number of children who are identified by the school system as having suicidal thoughts or behaviors who are then validated as being suicidal by mental health professionals in the community. Results: A general result shows that dynamic wait-listed designs always have higher statistical power over a traditional wait-listed design. This power increase can be substantial. Efficiency gains of 33% are easy to obtain for situations where the intervention has a small effect and the variation in rate across time is quite high. When the rate variation for an outcome is very low or the intervention effect is large, efficiency gains approach 100%. A small increase in the number of times where random assignment occurs from 2 - for the standard wait-listed design, to say 4 - can provide a large reduction in variance. Efficiency gains can also be high when converting standard wait-listed design to a dynamic one half-way into the study. Limitations As with all wait-listed designs, dynamic wait-listed designs can only be used to evaluate short-term impact. Since all subjects eventually receive the intervention, no comparison can be made after the end of the random assignment period. The statistical power benefits are primarily limited to outcomes that can be treated as count or time to event data. Conclusions: A dynamic design randomly assigns units - either individuals or groups - to start the intervention at varying times during the course of the study. This design is useful in testing interventions that screen for new or existing cases, as well as testing the scalability of interventions as they are disseminated or expanded system wide. They can improve on the traditional wait-listed design both in terms of statistical power and robustness in the presence of exogenous factors. This paper demonstrates that such designs yield smaller standard errors and can achieve higher statistical power than that of a standard wait-listed design. Just as important, dynamic designs can also help reduce the logistical challenges of implementing an intervention on a wide scale. When the intervention requires that significant training resources be allocated throughout the study, the dynamic wait-listed design is likely to increase the rate of training and lead to a higher level of program implementation.

AB - Background: The traditional wait-listed design, where half are randomly assigned to receive the intervention early and half are randomly assigned to receive it later, is often acceptable to communities who would not be comfortable with a no-treatment group. As such this traditional wait-listed design provides an excellent opportunity to evaluate short-term impact of an intervention. We introduce a new class of wait-listed designs for conducting randomized experiments where all subjects receive the intervention, and the timing of the intervention is randomly assigned. We use the term "dynamic wait-listed designs" to describe this new class. Purpose: This paper examines a new class of statistical designs where random assignment to intervention condition occurs at multiple times in a trial. As an extension of a traditional wait-listed design, this dynamic design allows all subjects to receive the intervention at a random time. Motivated by our search for increased statistical power in an ongoing school-based trial that is testing a program of gatekeeper training to identify suicidal youth and refer them to treatment, this new design class is especially useful when the primary outcome is a count or rate of occurrence, such as suicidal behavior, whose rate can fluctuate over time due to uncontrolled factors. Methods: Statistical power is computed for various dynamic wait-listed designs under conditions where the underlying rate of occurrence is allowed to vary non-systematically. We also present as an example a large ongoing trial to evaluate a gatekeeper training suicide prevention program in 32 schools which we initially began as a classic randomized wait-listed design. The primary outcome of interest in this study is the count of the number of children who are identified by the school system as having suicidal thoughts or behaviors who are then validated as being suicidal by mental health professionals in the community. Results: A general result shows that dynamic wait-listed designs always have higher statistical power over a traditional wait-listed design. This power increase can be substantial. Efficiency gains of 33% are easy to obtain for situations where the intervention has a small effect and the variation in rate across time is quite high. When the rate variation for an outcome is very low or the intervention effect is large, efficiency gains approach 100%. A small increase in the number of times where random assignment occurs from 2 - for the standard wait-listed design, to say 4 - can provide a large reduction in variance. Efficiency gains can also be high when converting standard wait-listed design to a dynamic one half-way into the study. Limitations As with all wait-listed designs, dynamic wait-listed designs can only be used to evaluate short-term impact. Since all subjects eventually receive the intervention, no comparison can be made after the end of the random assignment period. The statistical power benefits are primarily limited to outcomes that can be treated as count or time to event data. Conclusions: A dynamic design randomly assigns units - either individuals or groups - to start the intervention at varying times during the course of the study. This design is useful in testing interventions that screen for new or existing cases, as well as testing the scalability of interventions as they are disseminated or expanded system wide. They can improve on the traditional wait-listed design both in terms of statistical power and robustness in the presence of exogenous factors. This paper demonstrates that such designs yield smaller standard errors and can achieve higher statistical power than that of a standard wait-listed design. Just as important, dynamic designs can also help reduce the logistical challenges of implementing an intervention on a wide scale. When the intervention requires that significant training resources be allocated throughout the study, the dynamic wait-listed design is likely to increase the rate of training and lead to a higher level of program implementation.

UR - http://www.scopus.com/inward/record.url?scp=33746227594&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33746227594&partnerID=8YFLogxK

U2 - 10.1191/1740774506cn152oa

DO - 10.1191/1740774506cn152oa

M3 - Article

C2 - 16895043

AN - SCOPUS:33746227594

VL - 3

SP - 259

EP - 271

JO - Clinical Trials

JF - Clinical Trials

SN - 1740-7745

IS - 3

ER -