Behavioral Coding for Evaluation of Medical Student Communication: Clarification or Obfuscation?

Philip D. Sloane, Rainer Beck, Vicki Kowlowitz, Angela M. Blotzer, Lily Wang, Luzmila Akins, Foy White-Chu, C. Madeline Mitchell

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Purpose. To understand which specific student behaviors predict performance ratings from standardized patients and behavioral scientist preceptors. Method. In 1996-98, objective, real-time ratings of student verbal and nonverbal behaviors were conducted on 75 videotaped interviews between second-year medical students and standardized patients at University of North Carolina at Chapel Hill School of Medicine. The coding system used in these analyses was developed based on evidence-based literature reviews and used software that provides for real-time recording; 30 nonverbal and 33 behaviors were coded. The coded behaviors were then compared with four ratings: a global rating of the encounter by the standardized patient, a summary score derived from a checklist completed by the standardized patient, a global rating of the encounter by a behavioral science preceptor who observed the encounter, and a summary score derived from a checklist completed by the preceptor. Results. Analyses identified strong correlations between all four of the preceptor and patient ratings, a strong independent effect of case scenario, and significant between-rater variation. When multivariable analysis was used to predict these global ratings based on coding of specific behaviors, a relatively high proportion of observer variation was explained by a small group of coded behaviors. Conclusions. This study suggests that the coding of specific behaviors may hold promise as a student evaluation technique thereby improving medical training techniques and ultimately enhancing the communication skills in physician-patient encounters. It also illustrates the need to better clarify which specific behaviors are most critical in influencing patient satisfaction.

Original languageEnglish (US)
Pages (from-to)162-170
Number of pages9
JournalAcademic Medicine
Volume79
Issue number2
DOIs
StatePublished - Feb 2004
Externally publishedYes

Fingerprint

medical student
coding
rating
communication
evaluation
performance behavior
behavioral science
student
communication skills
small group
recording
physician
medicine
scenario
interview
school
evidence

ASJC Scopus subject areas

  • Education

Cite this

Sloane, P. D., Beck, R., Kowlowitz, V., Blotzer, A. M., Wang, L., Akins, L., ... Mitchell, C. M. (2004). Behavioral Coding for Evaluation of Medical Student Communication: Clarification or Obfuscation? Academic Medicine, 79(2), 162-170. https://doi.org/10.1097/00001888-200402000-00013

Behavioral Coding for Evaluation of Medical Student Communication : Clarification or Obfuscation? / Sloane, Philip D.; Beck, Rainer; Kowlowitz, Vicki; Blotzer, Angela M.; Wang, Lily; Akins, Luzmila; White-Chu, Foy; Mitchell, C. Madeline.

In: Academic Medicine, Vol. 79, No. 2, 02.2004, p. 162-170.

Research output: Contribution to journalArticle

Sloane, PD, Beck, R, Kowlowitz, V, Blotzer, AM, Wang, L, Akins, L, White-Chu, F & Mitchell, CM 2004, 'Behavioral Coding for Evaluation of Medical Student Communication: Clarification or Obfuscation?', Academic Medicine, vol. 79, no. 2, pp. 162-170. https://doi.org/10.1097/00001888-200402000-00013
Sloane, Philip D. ; Beck, Rainer ; Kowlowitz, Vicki ; Blotzer, Angela M. ; Wang, Lily ; Akins, Luzmila ; White-Chu, Foy ; Mitchell, C. Madeline. / Behavioral Coding for Evaluation of Medical Student Communication : Clarification or Obfuscation?. In: Academic Medicine. 2004 ; Vol. 79, No. 2. pp. 162-170.
@article{56a567a54dfd468aaba67a73c634c406,
title = "Behavioral Coding for Evaluation of Medical Student Communication: Clarification or Obfuscation?",
abstract = "Purpose. To understand which specific student behaviors predict performance ratings from standardized patients and behavioral scientist preceptors. Method. In 1996-98, objective, real-time ratings of student verbal and nonverbal behaviors were conducted on 75 videotaped interviews between second-year medical students and standardized patients at University of North Carolina at Chapel Hill School of Medicine. The coding system used in these analyses was developed based on evidence-based literature reviews and used software that provides for real-time recording; 30 nonverbal and 33 behaviors were coded. The coded behaviors were then compared with four ratings: a global rating of the encounter by the standardized patient, a summary score derived from a checklist completed by the standardized patient, a global rating of the encounter by a behavioral science preceptor who observed the encounter, and a summary score derived from a checklist completed by the preceptor. Results. Analyses identified strong correlations between all four of the preceptor and patient ratings, a strong independent effect of case scenario, and significant between-rater variation. When multivariable analysis was used to predict these global ratings based on coding of specific behaviors, a relatively high proportion of observer variation was explained by a small group of coded behaviors. Conclusions. This study suggests that the coding of specific behaviors may hold promise as a student evaluation technique thereby improving medical training techniques and ultimately enhancing the communication skills in physician-patient encounters. It also illustrates the need to better clarify which specific behaviors are most critical in influencing patient satisfaction.",
author = "Sloane, {Philip D.} and Rainer Beck and Vicki Kowlowitz and Blotzer, {Angela M.} and Lily Wang and Luzmila Akins and Foy White-Chu and Mitchell, {C. Madeline}",
year = "2004",
month = "2",
doi = "10.1097/00001888-200402000-00013",
language = "English (US)",
volume = "79",
pages = "162--170",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "2",

}

TY - JOUR

T1 - Behavioral Coding for Evaluation of Medical Student Communication

T2 - Clarification or Obfuscation?

AU - Sloane, Philip D.

AU - Beck, Rainer

AU - Kowlowitz, Vicki

AU - Blotzer, Angela M.

AU - Wang, Lily

AU - Akins, Luzmila

AU - White-Chu, Foy

AU - Mitchell, C. Madeline

PY - 2004/2

Y1 - 2004/2

N2 - Purpose. To understand which specific student behaviors predict performance ratings from standardized patients and behavioral scientist preceptors. Method. In 1996-98, objective, real-time ratings of student verbal and nonverbal behaviors were conducted on 75 videotaped interviews between second-year medical students and standardized patients at University of North Carolina at Chapel Hill School of Medicine. The coding system used in these analyses was developed based on evidence-based literature reviews and used software that provides for real-time recording; 30 nonverbal and 33 behaviors were coded. The coded behaviors were then compared with four ratings: a global rating of the encounter by the standardized patient, a summary score derived from a checklist completed by the standardized patient, a global rating of the encounter by a behavioral science preceptor who observed the encounter, and a summary score derived from a checklist completed by the preceptor. Results. Analyses identified strong correlations between all four of the preceptor and patient ratings, a strong independent effect of case scenario, and significant between-rater variation. When multivariable analysis was used to predict these global ratings based on coding of specific behaviors, a relatively high proportion of observer variation was explained by a small group of coded behaviors. Conclusions. This study suggests that the coding of specific behaviors may hold promise as a student evaluation technique thereby improving medical training techniques and ultimately enhancing the communication skills in physician-patient encounters. It also illustrates the need to better clarify which specific behaviors are most critical in influencing patient satisfaction.

AB - Purpose. To understand which specific student behaviors predict performance ratings from standardized patients and behavioral scientist preceptors. Method. In 1996-98, objective, real-time ratings of student verbal and nonverbal behaviors were conducted on 75 videotaped interviews between second-year medical students and standardized patients at University of North Carolina at Chapel Hill School of Medicine. The coding system used in these analyses was developed based on evidence-based literature reviews and used software that provides for real-time recording; 30 nonverbal and 33 behaviors were coded. The coded behaviors were then compared with four ratings: a global rating of the encounter by the standardized patient, a summary score derived from a checklist completed by the standardized patient, a global rating of the encounter by a behavioral science preceptor who observed the encounter, and a summary score derived from a checklist completed by the preceptor. Results. Analyses identified strong correlations between all four of the preceptor and patient ratings, a strong independent effect of case scenario, and significant between-rater variation. When multivariable analysis was used to predict these global ratings based on coding of specific behaviors, a relatively high proportion of observer variation was explained by a small group of coded behaviors. Conclusions. This study suggests that the coding of specific behaviors may hold promise as a student evaluation technique thereby improving medical training techniques and ultimately enhancing the communication skills in physician-patient encounters. It also illustrates the need to better clarify which specific behaviors are most critical in influencing patient satisfaction.

UR - http://www.scopus.com/inward/record.url?scp=1542318884&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=1542318884&partnerID=8YFLogxK

U2 - 10.1097/00001888-200402000-00013

DO - 10.1097/00001888-200402000-00013

M3 - Article

C2 - 14744718

AN - SCOPUS:1542318884

VL - 79

SP - 162

EP - 170

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 2

ER -