Assessing cardiac physical examination skills using simulation technology and real patients

A comparison study

Rose Hatala, Barry Issenberg, Barry Kassen, Gary Cole, C. Maria Bacchus, Ross Scalese

Research output: Contribution to journalArticle

28 Citations (Scopus)

Abstract

Objective: High-stakes assessments of doctors' physical examination skills often employ standardised patients (SPs) who lack physical abnormalities. Simulation technology provides additional opportunities to assess these skills by mimicking physical abnormalities. The current study examined the relationship between internists' cardiac physical examination competence as assessed with simulation technology compared with that assessed with real patients (RPs). Methods: The cardiac physical examination skills and bedside diagnostic accuracy of 28 internists were assessed during an objective structured clinical examination (OSCE). The OSCE included 3 modalities of cardiac patients: RPs with cardiac abnormalities; SPs combined with computer-based, audio-video simulations of auscultatory abnormalities, and a cardiac patient simulator (CPS) manikin. Four cardiac diagnoses and their associated cardiac findings were matched across modalities. At each station, 2 examiners independently rated a participant's physical examination technique and global clinical competence. Two investigators separately scored diagnostic accuracy. Results: Inter-rater reliability between examiners for global ratings (GRs) ranged from 0.75-0.78 for the different modalities. Although there was no significant difference between participants' mean GRs for each modality, the correlations between participants' performances on each modality were low to modest: RP versus SP, r = 0.19; RP versus CPS, r = 0.22; SP versus CPS, r = 0.57 (P < 0.01). Conclusions: Methodological limitations included variability between modalities in the components contributing to examiners' GRs, a paucity of objective outcome measures and restricted case sampling. No modality provided a clear 'gold standard' for the assessment of cardiac physical examination competence. These limitations need to be addressed before determining the optimal patient modality for high-stakes assessment purposes.

Original languageEnglish
Pages (from-to)628-636
Number of pages9
JournalMedical Education
Volume42
Issue number6
DOIs
StatePublished - Jun 1 2008

Fingerprint

Physical Examination
Technology
examination
simulation
examiner
rating
diagnostic
gold standard
Mental Competency
video
Manikins
Clinical Competence
lack
performance
Research Personnel
Outcome Assessment (Health Care)

Keywords

  • *internship and residency
  • Cardiology/*education
  • Clinical competence/*standards
  • Comparative study [publication type]
  • Multicentre study [publication type]
  • Observer variation
  • Patient simulation
  • Physical examination/*standards

ASJC Scopus subject areas

  • Nursing(all)
  • Education
  • Public Health, Environmental and Occupational Health

Cite this

Assessing cardiac physical examination skills using simulation technology and real patients : A comparison study. / Hatala, Rose; Issenberg, Barry; Kassen, Barry; Cole, Gary; Bacchus, C. Maria; Scalese, Ross.

In: Medical Education, Vol. 42, No. 6, 01.06.2008, p. 628-636.

Research output: Contribution to journalArticle

@article{0a06a82793f74a2a87f5dd04714d763c,
title = "Assessing cardiac physical examination skills using simulation technology and real patients: A comparison study",
abstract = "Objective: High-stakes assessments of doctors' physical examination skills often employ standardised patients (SPs) who lack physical abnormalities. Simulation technology provides additional opportunities to assess these skills by mimicking physical abnormalities. The current study examined the relationship between internists' cardiac physical examination competence as assessed with simulation technology compared with that assessed with real patients (RPs). Methods: The cardiac physical examination skills and bedside diagnostic accuracy of 28 internists were assessed during an objective structured clinical examination (OSCE). The OSCE included 3 modalities of cardiac patients: RPs with cardiac abnormalities; SPs combined with computer-based, audio-video simulations of auscultatory abnormalities, and a cardiac patient simulator (CPS) manikin. Four cardiac diagnoses and their associated cardiac findings were matched across modalities. At each station, 2 examiners independently rated a participant's physical examination technique and global clinical competence. Two investigators separately scored diagnostic accuracy. Results: Inter-rater reliability between examiners for global ratings (GRs) ranged from 0.75-0.78 for the different modalities. Although there was no significant difference between participants' mean GRs for each modality, the correlations between participants' performances on each modality were low to modest: RP versus SP, r = 0.19; RP versus CPS, r = 0.22; SP versus CPS, r = 0.57 (P < 0.01). Conclusions: Methodological limitations included variability between modalities in the components contributing to examiners' GRs, a paucity of objective outcome measures and restricted case sampling. No modality provided a clear 'gold standard' for the assessment of cardiac physical examination competence. These limitations need to be addressed before determining the optimal patient modality for high-stakes assessment purposes.",
keywords = "*internship and residency, Cardiology/*education, Clinical competence/*standards, Comparative study [publication type], Multicentre study [publication type], Observer variation, Patient simulation, Physical examination/*standards",
author = "Rose Hatala and Barry Issenberg and Barry Kassen and Gary Cole and Bacchus, {C. Maria} and Ross Scalese",
year = "2008",
month = "6",
day = "1",
doi = "10.1111/j.1365-2923.2007.02953.x",
language = "English",
volume = "42",
pages = "628--636",
journal = "Medical Education",
issn = "0308-0110",
publisher = "Wiley-Blackwell",
number = "6",

}

TY - JOUR

T1 - Assessing cardiac physical examination skills using simulation technology and real patients

T2 - A comparison study

AU - Hatala, Rose

AU - Issenberg, Barry

AU - Kassen, Barry

AU - Cole, Gary

AU - Bacchus, C. Maria

AU - Scalese, Ross

PY - 2008/6/1

Y1 - 2008/6/1

N2 - Objective: High-stakes assessments of doctors' physical examination skills often employ standardised patients (SPs) who lack physical abnormalities. Simulation technology provides additional opportunities to assess these skills by mimicking physical abnormalities. The current study examined the relationship between internists' cardiac physical examination competence as assessed with simulation technology compared with that assessed with real patients (RPs). Methods: The cardiac physical examination skills and bedside diagnostic accuracy of 28 internists were assessed during an objective structured clinical examination (OSCE). The OSCE included 3 modalities of cardiac patients: RPs with cardiac abnormalities; SPs combined with computer-based, audio-video simulations of auscultatory abnormalities, and a cardiac patient simulator (CPS) manikin. Four cardiac diagnoses and their associated cardiac findings were matched across modalities. At each station, 2 examiners independently rated a participant's physical examination technique and global clinical competence. Two investigators separately scored diagnostic accuracy. Results: Inter-rater reliability between examiners for global ratings (GRs) ranged from 0.75-0.78 for the different modalities. Although there was no significant difference between participants' mean GRs for each modality, the correlations between participants' performances on each modality were low to modest: RP versus SP, r = 0.19; RP versus CPS, r = 0.22; SP versus CPS, r = 0.57 (P < 0.01). Conclusions: Methodological limitations included variability between modalities in the components contributing to examiners' GRs, a paucity of objective outcome measures and restricted case sampling. No modality provided a clear 'gold standard' for the assessment of cardiac physical examination competence. These limitations need to be addressed before determining the optimal patient modality for high-stakes assessment purposes.

AB - Objective: High-stakes assessments of doctors' physical examination skills often employ standardised patients (SPs) who lack physical abnormalities. Simulation technology provides additional opportunities to assess these skills by mimicking physical abnormalities. The current study examined the relationship between internists' cardiac physical examination competence as assessed with simulation technology compared with that assessed with real patients (RPs). Methods: The cardiac physical examination skills and bedside diagnostic accuracy of 28 internists were assessed during an objective structured clinical examination (OSCE). The OSCE included 3 modalities of cardiac patients: RPs with cardiac abnormalities; SPs combined with computer-based, audio-video simulations of auscultatory abnormalities, and a cardiac patient simulator (CPS) manikin. Four cardiac diagnoses and their associated cardiac findings were matched across modalities. At each station, 2 examiners independently rated a participant's physical examination technique and global clinical competence. Two investigators separately scored diagnostic accuracy. Results: Inter-rater reliability between examiners for global ratings (GRs) ranged from 0.75-0.78 for the different modalities. Although there was no significant difference between participants' mean GRs for each modality, the correlations between participants' performances on each modality were low to modest: RP versus SP, r = 0.19; RP versus CPS, r = 0.22; SP versus CPS, r = 0.57 (P < 0.01). Conclusions: Methodological limitations included variability between modalities in the components contributing to examiners' GRs, a paucity of objective outcome measures and restricted case sampling. No modality provided a clear 'gold standard' for the assessment of cardiac physical examination competence. These limitations need to be addressed before determining the optimal patient modality for high-stakes assessment purposes.

KW - internship and residency

KW - Cardiology/education

KW - Clinical competence/standards

KW - Comparative study [publication type]

KW - Multicentre study [publication type]

KW - Observer variation

KW - Patient simulation

KW - Physical examination/standards

UR - http://www.scopus.com/inward/record.url?scp=43549099503&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=43549099503&partnerID=8YFLogxK

U2 - 10.1111/j.1365-2923.2007.02953.x

DO - 10.1111/j.1365-2923.2007.02953.x

M3 - Article

VL - 42

SP - 628

EP - 636

JO - Medical Education

JF - Medical Education

SN - 0308-0110

IS - 6

ER -