Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity

Gregg C. Fonarow, Wenqin Pan, Jeffrey L. Saver, Eric E. Smith, Mathew J. Reeves, Joseph P. Broderick, Dawn O. Kleindorfer, Ralph L Sacco, DaiWai M. Olson, Adrian F. Hernandez, Eric D. Peterson, Lee H. Schwamm

Research output: Contribution to journalArticle

118 Citations (Scopus)

Abstract

Context: There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied. Objective: To evaluate the degree to which hospital outcome ratings and potential eligibility for financial incentives are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke. Design, Setting, and Patients: Data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127 950 fee-for-service Medicare beneficiaries with ischemic stroke who had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared. Main Outcomes Measures: Model discrimination, hospital 30-day mortality outcome rankings, and value-based purchasing financial incentive categories. Results: Across the study population, the mean (SD) NIHSS score was 8.23 (8.11) (median, 5; interquartile range, 2-12). There were 18 186 deaths (14.5%) within the first 30 days, including 7430 deaths (5.8%) during the index hospitalization. The hospital mortality model with NIHSS scores had significantly better discrimination than the model without (C statistic, 0.864; 95% CI, 0.861-0.867, vs 0.772; 95% CI, 0.769-0.776; P<.001). Among hospitals ranked in the top 20% or bottom 20% of performers by the claims model without NIHSS scores, 26.3% were ranked differently by the model with NIHSS scores. Of hospitals initially classified as having "worse than expected" mortality, 57.7% were reclassified to "as expected" by the model with NIHSS scores. The net reclassification improvement (93.1%; 95% CI, 91.6%-94.6%; P<.001) and integrated discrimination improvement (15.0%; 95% CI, 14.6%-15.3%; P<.001) indexes both demonstrated significant enhancement of model performance after the addition of NIHSS. Explained variance and model calibration was also improved with the addition of NIHSS scores. Conclusion: Adding stroke severity as measured by the NIHSS to a hospital 30-day risk model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with considerably improved model discrimination and change in mortality performance rankings for a substantial portion of hospitals.

Original languageEnglish
Pages (from-to)257-264
Number of pages8
JournalJAMA - Journal of the American Medical Association
Volume308
Issue number3
DOIs
StatePublished - Jul 18 2012

Fingerprint

Stroke
Mortality
Medicare
Hospital Mortality
Motivation
Value-Based Purchasing
Fee-for-Service Plans
Neurologic Examination
National Institutes of Health (U.S.)
Calibration
Hospitalization
Outcome Assessment (Health Care)
Guidelines
Population

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity. / Fonarow, Gregg C.; Pan, Wenqin; Saver, Jeffrey L.; Smith, Eric E.; Reeves, Mathew J.; Broderick, Joseph P.; Kleindorfer, Dawn O.; Sacco, Ralph L; Olson, DaiWai M.; Hernandez, Adrian F.; Peterson, Eric D.; Schwamm, Lee H.

In: JAMA - Journal of the American Medical Association, Vol. 308, No. 3, 18.07.2012, p. 257-264.

Research output: Contribution to journalArticle

Fonarow, GC, Pan, W, Saver, JL, Smith, EE, Reeves, MJ, Broderick, JP, Kleindorfer, DO, Sacco, RL, Olson, DM, Hernandez, AF, Peterson, ED & Schwamm, LH 2012, 'Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity', JAMA - Journal of the American Medical Association, vol. 308, no. 3, pp. 257-264. https://doi.org/10.1001/jama.2012.7870
Fonarow, Gregg C. ; Pan, Wenqin ; Saver, Jeffrey L. ; Smith, Eric E. ; Reeves, Mathew J. ; Broderick, Joseph P. ; Kleindorfer, Dawn O. ; Sacco, Ralph L ; Olson, DaiWai M. ; Hernandez, Adrian F. ; Peterson, Eric D. ; Schwamm, Lee H. / Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity. In: JAMA - Journal of the American Medical Association. 2012 ; Vol. 308, No. 3. pp. 257-264.
@article{c063feb4a4834f488e9849e4c73e4f9a,
title = "Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity",
abstract = "Context: There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied. Objective: To evaluate the degree to which hospital outcome ratings and potential eligibility for financial incentives are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke. Design, Setting, and Patients: Data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127 950 fee-for-service Medicare beneficiaries with ischemic stroke who had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared. Main Outcomes Measures: Model discrimination, hospital 30-day mortality outcome rankings, and value-based purchasing financial incentive categories. Results: Across the study population, the mean (SD) NIHSS score was 8.23 (8.11) (median, 5; interquartile range, 2-12). There were 18 186 deaths (14.5{\%}) within the first 30 days, including 7430 deaths (5.8{\%}) during the index hospitalization. The hospital mortality model with NIHSS scores had significantly better discrimination than the model without (C statistic, 0.864; 95{\%} CI, 0.861-0.867, vs 0.772; 95{\%} CI, 0.769-0.776; P<.001). Among hospitals ranked in the top 20{\%} or bottom 20{\%} of performers by the claims model without NIHSS scores, 26.3{\%} were ranked differently by the model with NIHSS scores. Of hospitals initially classified as having {"}worse than expected{"} mortality, 57.7{\%} were reclassified to {"}as expected{"} by the model with NIHSS scores. The net reclassification improvement (93.1{\%}; 95{\%} CI, 91.6{\%}-94.6{\%}; P<.001) and integrated discrimination improvement (15.0{\%}; 95{\%} CI, 14.6{\%}-15.3{\%}; P<.001) indexes both demonstrated significant enhancement of model performance after the addition of NIHSS. Explained variance and model calibration was also improved with the addition of NIHSS scores. Conclusion: Adding stroke severity as measured by the NIHSS to a hospital 30-day risk model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with considerably improved model discrimination and change in mortality performance rankings for a substantial portion of hospitals.",
author = "Fonarow, {Gregg C.} and Wenqin Pan and Saver, {Jeffrey L.} and Smith, {Eric E.} and Reeves, {Mathew J.} and Broderick, {Joseph P.} and Kleindorfer, {Dawn O.} and Sacco, {Ralph L} and Olson, {DaiWai M.} and Hernandez, {Adrian F.} and Peterson, {Eric D.} and Schwamm, {Lee H.}",
year = "2012",
month = "7",
day = "18",
doi = "10.1001/jama.2012.7870",
language = "English",
volume = "308",
pages = "257--264",
journal = "JAMA - Journal of the American Medical Association",
issn = "0002-9955",
publisher = "American Medical Association",
number = "3",

}

TY - JOUR

T1 - Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity

AU - Fonarow, Gregg C.

AU - Pan, Wenqin

AU - Saver, Jeffrey L.

AU - Smith, Eric E.

AU - Reeves, Mathew J.

AU - Broderick, Joseph P.

AU - Kleindorfer, Dawn O.

AU - Sacco, Ralph L

AU - Olson, DaiWai M.

AU - Hernandez, Adrian F.

AU - Peterson, Eric D.

AU - Schwamm, Lee H.

PY - 2012/7/18

Y1 - 2012/7/18

N2 - Context: There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied. Objective: To evaluate the degree to which hospital outcome ratings and potential eligibility for financial incentives are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke. Design, Setting, and Patients: Data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127 950 fee-for-service Medicare beneficiaries with ischemic stroke who had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared. Main Outcomes Measures: Model discrimination, hospital 30-day mortality outcome rankings, and value-based purchasing financial incentive categories. Results: Across the study population, the mean (SD) NIHSS score was 8.23 (8.11) (median, 5; interquartile range, 2-12). There were 18 186 deaths (14.5%) within the first 30 days, including 7430 deaths (5.8%) during the index hospitalization. The hospital mortality model with NIHSS scores had significantly better discrimination than the model without (C statistic, 0.864; 95% CI, 0.861-0.867, vs 0.772; 95% CI, 0.769-0.776; P<.001). Among hospitals ranked in the top 20% or bottom 20% of performers by the claims model without NIHSS scores, 26.3% were ranked differently by the model with NIHSS scores. Of hospitals initially classified as having "worse than expected" mortality, 57.7% were reclassified to "as expected" by the model with NIHSS scores. The net reclassification improvement (93.1%; 95% CI, 91.6%-94.6%; P<.001) and integrated discrimination improvement (15.0%; 95% CI, 14.6%-15.3%; P<.001) indexes both demonstrated significant enhancement of model performance after the addition of NIHSS. Explained variance and model calibration was also improved with the addition of NIHSS scores. Conclusion: Adding stroke severity as measured by the NIHSS to a hospital 30-day risk model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with considerably improved model discrimination and change in mortality performance rankings for a substantial portion of hospitals.

AB - Context: There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied. Objective: To evaluate the degree to which hospital outcome ratings and potential eligibility for financial incentives are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke. Design, Setting, and Patients: Data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127 950 fee-for-service Medicare beneficiaries with ischemic stroke who had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared. Main Outcomes Measures: Model discrimination, hospital 30-day mortality outcome rankings, and value-based purchasing financial incentive categories. Results: Across the study population, the mean (SD) NIHSS score was 8.23 (8.11) (median, 5; interquartile range, 2-12). There were 18 186 deaths (14.5%) within the first 30 days, including 7430 deaths (5.8%) during the index hospitalization. The hospital mortality model with NIHSS scores had significantly better discrimination than the model without (C statistic, 0.864; 95% CI, 0.861-0.867, vs 0.772; 95% CI, 0.769-0.776; P<.001). Among hospitals ranked in the top 20% or bottom 20% of performers by the claims model without NIHSS scores, 26.3% were ranked differently by the model with NIHSS scores. Of hospitals initially classified as having "worse than expected" mortality, 57.7% were reclassified to "as expected" by the model with NIHSS scores. The net reclassification improvement (93.1%; 95% CI, 91.6%-94.6%; P<.001) and integrated discrimination improvement (15.0%; 95% CI, 14.6%-15.3%; P<.001) indexes both demonstrated significant enhancement of model performance after the addition of NIHSS. Explained variance and model calibration was also improved with the addition of NIHSS scores. Conclusion: Adding stroke severity as measured by the NIHSS to a hospital 30-day risk model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with considerably improved model discrimination and change in mortality performance rankings for a substantial portion of hospitals.

UR - http://www.scopus.com/inward/record.url?scp=84863928399&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863928399&partnerID=8YFLogxK

U2 - 10.1001/jama.2012.7870

DO - 10.1001/jama.2012.7870

M3 - Article

C2 - 22797643

AN - SCOPUS:84863928399

VL - 308

SP - 257

EP - 264

JO - JAMA - Journal of the American Medical Association

JF - JAMA - Journal of the American Medical Association

SN - 0002-9955

IS - 3

ER -