Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics

Natasha Chida, Christopher Brown, Jyoti Mathad, Kelly Carpenter, George Nelson, Marcos C. Schechter, Paulina A. Rebolledo, Valeria Fabre, Diana Silva Cantillo, Sarah Longworth, Valerianna Amorosa, Christian Petrauskis, Catherine V Boulanger, Natalie Cain, Amita Gupta, Jane McKenzie-White, Robert Bollinger, Michael Melia

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background : Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.

Objective : We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.

Methods : We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.

Results : Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61% (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57% correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43% (113 of 261), except for chest x-rays (96%, 328 of 343).

Conclusions : Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.

Original languageEnglish (US)
Pages (from-to)331-335
Number of pages5
JournalJournal of graduate medical education
Volume10
Issue number3
DOIs
StatePublished - Jun 1 2018

Fingerprint

Internal Medicine
Communicable Diseases
Volunteers
Tuberculosis
Thorax
Observation
X-Rays

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Chida, N., Brown, C., Mathad, J., Carpenter, K., Nelson, G., Schechter, M. C., ... Melia, M. (2018). Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics. Journal of graduate medical education, 10(3), 331-335. https://doi.org/10.4300/JGME-D-17-00377.1

Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics. / Chida, Natasha; Brown, Christopher; Mathad, Jyoti; Carpenter, Kelly; Nelson, George; Schechter, Marcos C.; Rebolledo, Paulina A.; Fabre, Valeria; Cantillo, Diana Silva; Longworth, Sarah; Amorosa, Valerianna; Petrauskis, Christian; Boulanger, Catherine V; Cain, Natalie; Gupta, Amita; McKenzie-White, Jane; Bollinger, Robert; Melia, Michael.

In: Journal of graduate medical education, Vol. 10, No. 3, 01.06.2018, p. 331-335.

Research output: Contribution to journalArticle

Chida, N, Brown, C, Mathad, J, Carpenter, K, Nelson, G, Schechter, MC, Rebolledo, PA, Fabre, V, Cantillo, DS, Longworth, S, Amorosa, V, Petrauskis, C, Boulanger, CV, Cain, N, Gupta, A, McKenzie-White, J, Bollinger, R & Melia, M 2018, 'Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics', Journal of graduate medical education, vol. 10, no. 3, pp. 331-335. https://doi.org/10.4300/JGME-D-17-00377.1
Chida, Natasha ; Brown, Christopher ; Mathad, Jyoti ; Carpenter, Kelly ; Nelson, George ; Schechter, Marcos C. ; Rebolledo, Paulina A. ; Fabre, Valeria ; Cantillo, Diana Silva ; Longworth, Sarah ; Amorosa, Valerianna ; Petrauskis, Christian ; Boulanger, Catherine V ; Cain, Natalie ; Gupta, Amita ; McKenzie-White, Jane ; Bollinger, Robert ; Melia, Michael. / Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics. In: Journal of graduate medical education. 2018 ; Vol. 10, No. 3. pp. 331-335.
@article{ca1d3ddb10354e8cb8adf9256ad102d8,
title = "Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics",
abstract = "Background : Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.Objective : We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.Methods : We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.Results : Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61{\%} (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57{\%} correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43{\%} (113 of 261), except for chest x-rays (96{\%}, 328 of 343).Conclusions : Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.",
author = "Natasha Chida and Christopher Brown and Jyoti Mathad and Kelly Carpenter and George Nelson and Schechter, {Marcos C.} and Rebolledo, {Paulina A.} and Valeria Fabre and Cantillo, {Diana Silva} and Sarah Longworth and Valerianna Amorosa and Christian Petrauskis and Boulanger, {Catherine V} and Natalie Cain and Amita Gupta and Jane McKenzie-White and Robert Bollinger and Michael Melia",
year = "2018",
month = "6",
day = "1",
doi = "10.4300/JGME-D-17-00377.1",
language = "English (US)",
volume = "10",
pages = "331--335",
journal = "Journal of graduate medical education",
issn = "1949-8349",
publisher = "University of Finance and Management",
number = "3",

}

TY - JOUR

T1 - Developing an Assessment Framework for Essential Internal Medicine Subspecialty Topics

AU - Chida, Natasha

AU - Brown, Christopher

AU - Mathad, Jyoti

AU - Carpenter, Kelly

AU - Nelson, George

AU - Schechter, Marcos C.

AU - Rebolledo, Paulina A.

AU - Fabre, Valeria

AU - Cantillo, Diana Silva

AU - Longworth, Sarah

AU - Amorosa, Valerianna

AU - Petrauskis, Christian

AU - Boulanger, Catherine V

AU - Cain, Natalie

AU - Gupta, Amita

AU - McKenzie-White, Jane

AU - Bollinger, Robert

AU - Melia, Michael

PY - 2018/6/1

Y1 - 2018/6/1

N2 - Background : Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.Objective : We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.Methods : We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.Results : Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61% (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57% correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43% (113 of 261), except for chest x-rays (96%, 328 of 343).Conclusions : Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.

AB - Background : Assessing residents by direct observation is the preferred assessment method for infrequently encountered subspecialty topics, but this is logistically challenging.Objective : We developed an assessment framework for internal medicine (IM) residents in subspecialty topics, using tuberculosis diagnosis for proof of concept.Methods : We used a 4-step process at 8 academic medical centers that entailed (1) creating a 10-item knowledge assessment tool; (2) pilot testing on a sample of 129 IM residents and infectious disease fellow volunteers to evaluate validity evidence; (3) implementing the final tool among 886 resident volunteers; and (4) assessing outcomes via retrospective chart review. Outcomes included tool score, item performance, and rates of obtaining recommended diagnostics.Results : Following tool development, 10 infectious disease experts provided content validity. Pilot testing showed higher mean scores for fellows compared with residents (7 [SD = 1.8] versus 3.8 [SD = 1.7], respectively, P < .001) and a satisfactory Kuder-Richardson Formula 20 (0.72). Implementation of the tool revealed a 14-minute (SD = 2.0) mean completion time, 61% (541 of 886) response rate, 4.4 (SD = 1.6) mean score, and ≤ 57% correct response rate for 9 of 10 items. On chart review (n = 343), the rate of obtaining each recommended test was ≤ 43% (113 of 261), except for chest x-rays (96%, 328 of 343).Conclusions : Our assessment framework revealed knowledge and practice gaps in tuberculosis diagnosis in IM residents. Adopting this approach may help ensure assessment is not limited to frequently encountered topics.

UR - http://www.scopus.com/inward/record.url?scp=85050851712&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050851712&partnerID=8YFLogxK

U2 - 10.4300/JGME-D-17-00377.1

DO - 10.4300/JGME-D-17-00377.1

M3 - Article

C2 - 29946392

AN - SCOPUS:85050851712

VL - 10

SP - 331

EP - 335

JO - Journal of graduate medical education

JF - Journal of graduate medical education

SN - 1949-8349

IS - 3

ER -