Communication Skills Assessment Using Human Avatars: Piloting a Virtual World Objective Structured Clinical Examination

Bruce Kava, Allen D. Andrade, Robert Marcovich, Thaer Idress, Jorge G. Ruiz

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Introduction Proficiency in communication skills is a core competency of residency training. We evaluated the feasibility, acceptability and applicability of a virtual world objective structured clinical examination that enables practice based learning and assessment of resident communication skills. Methods A virtual clinical encounter situated in 2 practice settings was developed that uses a human avatar physician and a standardized patient. Following an online tutorial house staff participated in 4 communication tasks, including shared decision making, delivering bad news, obtaining informed consent and disclosing a medical error. Validated instruments and semistructured interviews were used to assess house staff acceptability and applicability of the platform. Three faculty members used ACS (Affective Competency Scale) and communication specific assessment instruments to evaluate house staff performance. Results A total of 12 urology house staff completed the simulation. Direct costs were approximately $1,000. The virtual world was easy to use and immersive. Applicability directly correlated with presence (Pearson r = 0.67, p = 0.01) and co-presence (Pearson r = 0.8, p = 0.002). House staff identified problems with 1) limited nonverbal cues, 2) too much information presented and 3) a lack of immediate feedback. The ICC (intraclass correlation) of faculty assessments was high for ACS at 0.53 (95% CI 0.36–0.69) for single measures, 0.77 (95% CI 0.63–0.86) for average measures and less for other assessment instruments. Conclusions A virtual world objective structured clinical examination is a feasible, acceptable and applicable method of communication skills assessment. Improving nonverbal cues, focusing on individual skill sets and providing immediate feedback are measures to be adopted in future iterations of this platform.

Original languageEnglish (US)
Pages (from-to)76-84
Number of pages9
JournalUrology Practice
Volume4
Issue number1
DOIs
StatePublished - Jan 1 2017

Fingerprint

Internship and Residency
Communication
Cues
Medical Errors
Urology
Informed Consent
Decision Making
Learning
Interviews
Physicians
Costs and Cost Analysis

Keywords

  • clinical competence
  • computer simulation
  • physicians
  • social skills
  • urology

ASJC Scopus subject areas

  • Urology

Cite this

Communication Skills Assessment Using Human Avatars : Piloting a Virtual World Objective Structured Clinical Examination. / Kava, Bruce; Andrade, Allen D.; Marcovich, Robert; Idress, Thaer; Ruiz, Jorge G.

In: Urology Practice, Vol. 4, No. 1, 01.01.2017, p. 76-84.

Research output: Contribution to journalArticle

@article{5c241799458c47aab4e6ace7f720d77f,
title = "Communication Skills Assessment Using Human Avatars: Piloting a Virtual World Objective Structured Clinical Examination",
abstract = "Introduction Proficiency in communication skills is a core competency of residency training. We evaluated the feasibility, acceptability and applicability of a virtual world objective structured clinical examination that enables practice based learning and assessment of resident communication skills. Methods A virtual clinical encounter situated in 2 practice settings was developed that uses a human avatar physician and a standardized patient. Following an online tutorial house staff participated in 4 communication tasks, including shared decision making, delivering bad news, obtaining informed consent and disclosing a medical error. Validated instruments and semistructured interviews were used to assess house staff acceptability and applicability of the platform. Three faculty members used ACS (Affective Competency Scale) and communication specific assessment instruments to evaluate house staff performance. Results A total of 12 urology house staff completed the simulation. Direct costs were approximately $1,000. The virtual world was easy to use and immersive. Applicability directly correlated with presence (Pearson r = 0.67, p = 0.01) and co-presence (Pearson r = 0.8, p = 0.002). House staff identified problems with 1) limited nonverbal cues, 2) too much information presented and 3) a lack of immediate feedback. The ICC (intraclass correlation) of faculty assessments was high for ACS at 0.53 (95{\%} CI 0.36–0.69) for single measures, 0.77 (95{\%} CI 0.63–0.86) for average measures and less for other assessment instruments. Conclusions A virtual world objective structured clinical examination is a feasible, acceptable and applicable method of communication skills assessment. Improving nonverbal cues, focusing on individual skill sets and providing immediate feedback are measures to be adopted in future iterations of this platform.",
keywords = "clinical competence, computer simulation, physicians, social skills, urology",
author = "Bruce Kava and Andrade, {Allen D.} and Robert Marcovich and Thaer Idress and Ruiz, {Jorge G.}",
year = "2017",
month = "1",
day = "1",
doi = "10.1016/j.urpr.2016.01.006",
language = "English (US)",
volume = "4",
pages = "76--84",
journal = "Urology Practice",
issn = "2352-0779",
publisher = "Elsevier Inc.",
number = "1",

}

TY - JOUR

T1 - Communication Skills Assessment Using Human Avatars

T2 - Piloting a Virtual World Objective Structured Clinical Examination

AU - Kava, Bruce

AU - Andrade, Allen D.

AU - Marcovich, Robert

AU - Idress, Thaer

AU - Ruiz, Jorge G.

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Introduction Proficiency in communication skills is a core competency of residency training. We evaluated the feasibility, acceptability and applicability of a virtual world objective structured clinical examination that enables practice based learning and assessment of resident communication skills. Methods A virtual clinical encounter situated in 2 practice settings was developed that uses a human avatar physician and a standardized patient. Following an online tutorial house staff participated in 4 communication tasks, including shared decision making, delivering bad news, obtaining informed consent and disclosing a medical error. Validated instruments and semistructured interviews were used to assess house staff acceptability and applicability of the platform. Three faculty members used ACS (Affective Competency Scale) and communication specific assessment instruments to evaluate house staff performance. Results A total of 12 urology house staff completed the simulation. Direct costs were approximately $1,000. The virtual world was easy to use and immersive. Applicability directly correlated with presence (Pearson r = 0.67, p = 0.01) and co-presence (Pearson r = 0.8, p = 0.002). House staff identified problems with 1) limited nonverbal cues, 2) too much information presented and 3) a lack of immediate feedback. The ICC (intraclass correlation) of faculty assessments was high for ACS at 0.53 (95% CI 0.36–0.69) for single measures, 0.77 (95% CI 0.63–0.86) for average measures and less for other assessment instruments. Conclusions A virtual world objective structured clinical examination is a feasible, acceptable and applicable method of communication skills assessment. Improving nonverbal cues, focusing on individual skill sets and providing immediate feedback are measures to be adopted in future iterations of this platform.

AB - Introduction Proficiency in communication skills is a core competency of residency training. We evaluated the feasibility, acceptability and applicability of a virtual world objective structured clinical examination that enables practice based learning and assessment of resident communication skills. Methods A virtual clinical encounter situated in 2 practice settings was developed that uses a human avatar physician and a standardized patient. Following an online tutorial house staff participated in 4 communication tasks, including shared decision making, delivering bad news, obtaining informed consent and disclosing a medical error. Validated instruments and semistructured interviews were used to assess house staff acceptability and applicability of the platform. Three faculty members used ACS (Affective Competency Scale) and communication specific assessment instruments to evaluate house staff performance. Results A total of 12 urology house staff completed the simulation. Direct costs were approximately $1,000. The virtual world was easy to use and immersive. Applicability directly correlated with presence (Pearson r = 0.67, p = 0.01) and co-presence (Pearson r = 0.8, p = 0.002). House staff identified problems with 1) limited nonverbal cues, 2) too much information presented and 3) a lack of immediate feedback. The ICC (intraclass correlation) of faculty assessments was high for ACS at 0.53 (95% CI 0.36–0.69) for single measures, 0.77 (95% CI 0.63–0.86) for average measures and less for other assessment instruments. Conclusions A virtual world objective structured clinical examination is a feasible, acceptable and applicable method of communication skills assessment. Improving nonverbal cues, focusing on individual skill sets and providing immediate feedback are measures to be adopted in future iterations of this platform.

KW - clinical competence

KW - computer simulation

KW - physicians

KW - social skills

KW - urology

UR - http://www.scopus.com/inward/record.url?scp=85007309769&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85007309769&partnerID=8YFLogxK

U2 - 10.1016/j.urpr.2016.01.006

DO - 10.1016/j.urpr.2016.01.006

M3 - Article

AN - SCOPUS:85007309769

VL - 4

SP - 76

EP - 84

JO - Urology Practice

JF - Urology Practice

SN - 2352-0779

IS - 1

ER -