Psychometric comparison of computerized and standard administration of the neurocognitive assessment instruments selected by the CATIE and MATRICS consortia among patients with schizophrenia

James P. O'Halloran, Aaron S. Kemp, Kirstan N. Gooch, Philip D Harvey, Barton W. Palmer, Christopher Reist, Lon S. Schneider

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

Neurocognitive assessment is an essential component for clinical trials of candidate "cognitive-enhancing" treatments for schizophrenia. However, manual administration of large, paper-based, neurocognitive batteries is often inefficient, error-prone, and inconsistent across multiple sites. Existing computerized testing systems are also limited both in the assessment instruments available and in the range of impairments that can be accommodated with the subject sitting alone at a single display. Therefore, a dual-display computerized testing system was developed, with funding from the National Institute of Mental Health (NIMH), that integrates (rather than replaces) the examiner for computerized administration of standard neurocognitive assessment batteries. Purpose: To compare standard administration versions (SAVs) of tests selected by the NIMH-sponsored CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness) and MATRICS™ (Measurement and Treatment Research to Improve Cognition in Schizophrenia) consortia, with structurally- and functionally-equivalent, computerized administration versions (CAVs). Method: 116 outpatients with schizophrenia received both the SAVs and CAVs within one week and again, approximately 30 days later. Results: Intraclass Correlation Coefficient (ICC) comparisons between SAVs and CAVs yielded highly significant measures of absolute agreement for all tests, ranging 0.61-0.95. ICCs for test-retest reliability, ranging 0.56-0.94 for SAVs and 0.59-0.98 for CAVs, were also highly significant for both batteries, though significantly higher for CAVs overall. Conclusions: The CAVs of the neurocognitive assessment instruments selected by the CATIE and MATRICS™ consortia are substantially equivalent to antecedent SAVs. Importantly, the increased reliability afforded by computerization highlight the potential for increasing power, thereby decreasing sample size requirements, for clinical evaluations of putative "cognitive-enhancing" treatments.

Original languageEnglish
Pages (from-to)33-41
Number of pages9
JournalSchizophrenia Research
Volume106
Issue number1
DOIs
StatePublished - Nov 1 2008
Externally publishedYes

Fingerprint

Psychometrics
Cognition
Antipsychotic Agents
Schizophrenia
Clinical Trials
Research
National Institute of Mental Health (U.S.)
Reproducibility of Results
Sample Size
Outpatients
Therapeutics

Keywords

  • CMINDS
  • Computerized Assessment
  • Concurrent Validity
  • Neurocognition
  • Reliability
  • Schizophrenia

ASJC Scopus subject areas

  • Psychiatry and Mental health
  • Biological Psychiatry

Cite this

Psychometric comparison of computerized and standard administration of the neurocognitive assessment instruments selected by the CATIE and MATRICS consortia among patients with schizophrenia. / O'Halloran, James P.; Kemp, Aaron S.; Gooch, Kirstan N.; Harvey, Philip D; Palmer, Barton W.; Reist, Christopher; Schneider, Lon S.

In: Schizophrenia Research, Vol. 106, No. 1, 01.11.2008, p. 33-41.

Research output: Contribution to journalArticle

O'Halloran, James P. ; Kemp, Aaron S. ; Gooch, Kirstan N. ; Harvey, Philip D ; Palmer, Barton W. ; Reist, Christopher ; Schneider, Lon S. / Psychometric comparison of computerized and standard administration of the neurocognitive assessment instruments selected by the CATIE and MATRICS consortia among patients with schizophrenia. In: Schizophrenia Research. 2008 ; Vol. 106, No. 1. pp. 33-41.
@article{8bc645df43d84bbab6edd42578b0b3ad,
title = "Psychometric comparison of computerized and standard administration of the neurocognitive assessment instruments selected by the CATIE and MATRICS consortia among patients with schizophrenia",
abstract = "Neurocognitive assessment is an essential component for clinical trials of candidate {"}cognitive-enhancing{"} treatments for schizophrenia. However, manual administration of large, paper-based, neurocognitive batteries is often inefficient, error-prone, and inconsistent across multiple sites. Existing computerized testing systems are also limited both in the assessment instruments available and in the range of impairments that can be accommodated with the subject sitting alone at a single display. Therefore, a dual-display computerized testing system was developed, with funding from the National Institute of Mental Health (NIMH), that integrates (rather than replaces) the examiner for computerized administration of standard neurocognitive assessment batteries. Purpose: To compare standard administration versions (SAVs) of tests selected by the NIMH-sponsored CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness) and MATRICS™ (Measurement and Treatment Research to Improve Cognition in Schizophrenia) consortia, with structurally- and functionally-equivalent, computerized administration versions (CAVs). Method: 116 outpatients with schizophrenia received both the SAVs and CAVs within one week and again, approximately 30 days later. Results: Intraclass Correlation Coefficient (ICC) comparisons between SAVs and CAVs yielded highly significant measures of absolute agreement for all tests, ranging 0.61-0.95. ICCs for test-retest reliability, ranging 0.56-0.94 for SAVs and 0.59-0.98 for CAVs, were also highly significant for both batteries, though significantly higher for CAVs overall. Conclusions: The CAVs of the neurocognitive assessment instruments selected by the CATIE and MATRICS™ consortia are substantially equivalent to antecedent SAVs. Importantly, the increased reliability afforded by computerization highlight the potential for increasing power, thereby decreasing sample size requirements, for clinical evaluations of putative {"}cognitive-enhancing{"} treatments.",
keywords = "CMINDS, Computerized Assessment, Concurrent Validity, Neurocognition, Reliability, Schizophrenia",
author = "O'Halloran, {James P.} and Kemp, {Aaron S.} and Gooch, {Kirstan N.} and Harvey, {Philip D} and Palmer, {Barton W.} and Christopher Reist and Schneider, {Lon S.}",
year = "2008",
month = "11",
day = "1",
doi = "10.1016/j.schres.2007.11.015",
language = "English",
volume = "106",
pages = "33--41",
journal = "Schizophrenia Research",
issn = "0920-9964",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Psychometric comparison of computerized and standard administration of the neurocognitive assessment instruments selected by the CATIE and MATRICS consortia among patients with schizophrenia

AU - O'Halloran, James P.

AU - Kemp, Aaron S.

AU - Gooch, Kirstan N.

AU - Harvey, Philip D

AU - Palmer, Barton W.

AU - Reist, Christopher

AU - Schneider, Lon S.

PY - 2008/11/1

Y1 - 2008/11/1

N2 - Neurocognitive assessment is an essential component for clinical trials of candidate "cognitive-enhancing" treatments for schizophrenia. However, manual administration of large, paper-based, neurocognitive batteries is often inefficient, error-prone, and inconsistent across multiple sites. Existing computerized testing systems are also limited both in the assessment instruments available and in the range of impairments that can be accommodated with the subject sitting alone at a single display. Therefore, a dual-display computerized testing system was developed, with funding from the National Institute of Mental Health (NIMH), that integrates (rather than replaces) the examiner for computerized administration of standard neurocognitive assessment batteries. Purpose: To compare standard administration versions (SAVs) of tests selected by the NIMH-sponsored CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness) and MATRICS™ (Measurement and Treatment Research to Improve Cognition in Schizophrenia) consortia, with structurally- and functionally-equivalent, computerized administration versions (CAVs). Method: 116 outpatients with schizophrenia received both the SAVs and CAVs within one week and again, approximately 30 days later. Results: Intraclass Correlation Coefficient (ICC) comparisons between SAVs and CAVs yielded highly significant measures of absolute agreement for all tests, ranging 0.61-0.95. ICCs for test-retest reliability, ranging 0.56-0.94 for SAVs and 0.59-0.98 for CAVs, were also highly significant for both batteries, though significantly higher for CAVs overall. Conclusions: The CAVs of the neurocognitive assessment instruments selected by the CATIE and MATRICS™ consortia are substantially equivalent to antecedent SAVs. Importantly, the increased reliability afforded by computerization highlight the potential for increasing power, thereby decreasing sample size requirements, for clinical evaluations of putative "cognitive-enhancing" treatments.

AB - Neurocognitive assessment is an essential component for clinical trials of candidate "cognitive-enhancing" treatments for schizophrenia. However, manual administration of large, paper-based, neurocognitive batteries is often inefficient, error-prone, and inconsistent across multiple sites. Existing computerized testing systems are also limited both in the assessment instruments available and in the range of impairments that can be accommodated with the subject sitting alone at a single display. Therefore, a dual-display computerized testing system was developed, with funding from the National Institute of Mental Health (NIMH), that integrates (rather than replaces) the examiner for computerized administration of standard neurocognitive assessment batteries. Purpose: To compare standard administration versions (SAVs) of tests selected by the NIMH-sponsored CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness) and MATRICS™ (Measurement and Treatment Research to Improve Cognition in Schizophrenia) consortia, with structurally- and functionally-equivalent, computerized administration versions (CAVs). Method: 116 outpatients with schizophrenia received both the SAVs and CAVs within one week and again, approximately 30 days later. Results: Intraclass Correlation Coefficient (ICC) comparisons between SAVs and CAVs yielded highly significant measures of absolute agreement for all tests, ranging 0.61-0.95. ICCs for test-retest reliability, ranging 0.56-0.94 for SAVs and 0.59-0.98 for CAVs, were also highly significant for both batteries, though significantly higher for CAVs overall. Conclusions: The CAVs of the neurocognitive assessment instruments selected by the CATIE and MATRICS™ consortia are substantially equivalent to antecedent SAVs. Importantly, the increased reliability afforded by computerization highlight the potential for increasing power, thereby decreasing sample size requirements, for clinical evaluations of putative "cognitive-enhancing" treatments.

KW - CMINDS

KW - Computerized Assessment

KW - Concurrent Validity

KW - Neurocognition

KW - Reliability

KW - Schizophrenia

UR - http://www.scopus.com/inward/record.url?scp=54149086168&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=54149086168&partnerID=8YFLogxK

U2 - 10.1016/j.schres.2007.11.015

DO - 10.1016/j.schres.2007.11.015

M3 - Article

VL - 106

SP - 33

EP - 41

JO - Schizophrenia Research

JF - Schizophrenia Research

SN - 0920-9964

IS - 1

ER -