Telemedical diagnosis of retinopathy of prematurity: Accuracy of expert versus non-expert graders

Steven L. Williams, Lu Wang, Steven A. Kane, Thomas C. Lee, David J. Weissgold, Audina Berrocal, Daniel Rabinowitz, Justin Starren, John T. Flynn, Michael F. Chiang

Research output: Contribution to journalArticle

33 Citations (Scopus)

Abstract

Background/aims: To assess accuracy of telemedical retinopathy of prematurity (ROP) diagnosis by trained non-expert graders compared with expert graders. Methods: Eye examinations (n=248) from 67 consecutive infants were captured using wide-angle retinal photography (RetCam-II, Clarity Medical Systems, Pleasanton, California, USA). Non-expert graders attended two 1-h training sessions on image-based ROP diagnosis. Using a web-based telemedicine system, 14 non-expert and three expert graders provided a diagnosis for each eye: no ROP, mild ROP, type 2 pre-threshold ROP or treatment-requiring ROP. All diagnoses were compared with a reference standard of dilated indirect ophthalmoscopy by an experienced paediatric ophthalmologist. Results: For detection of type 2 or worse ROP, the mean (range) sensitivities and specificities were 0.95 (0.94-0.97) and 0.93 (0.91-0.96) for experts, 0.87 (0.71-0.97) and 0.73 (0.39-0.95) for resident non-experts, and 0.73 (0.41-0.88) and 0.91 (0.84-0.96) for student non-experts, respectively. For detection of treatment-requiring ROP, the mean (range) sensitivities and specificities were 1.00 (1.00-1.00) and 0.93 (0.88-0.96) for experts, 0.88 (0.50-1.00) and 0.84 (0.71-0.98) for resident non-experts, and 0.82 (0.42-1.00) and 0.92 (0.83-0.97) for student non-experts, respectively. Conclusions: Mean sensitivity and specificity of trained non-experts were lower than that of experts, although several non-experts had high accuracy. Development of methods for training non-expert graders may help support telemedical ROP evaluation.

Original languageEnglish
Pages (from-to)351-356
Number of pages6
JournalBritish Journal of Ophthalmology
Volume94
Issue number3
DOIs
StatePublished - Mar 1 2010

Fingerprint

Retinopathy of Prematurity
Sensitivity and Specificity
Students
Ophthalmoscopy
Telemedicine
Photography
Pediatrics

ASJC Scopus subject areas

  • Ophthalmology
  • Sensory Systems
  • Cellular and Molecular Neuroscience

Cite this

Telemedical diagnosis of retinopathy of prematurity : Accuracy of expert versus non-expert graders. / Williams, Steven L.; Wang, Lu; Kane, Steven A.; Lee, Thomas C.; Weissgold, David J.; Berrocal, Audina; Rabinowitz, Daniel; Starren, Justin; Flynn, John T.; Chiang, Michael F.

In: British Journal of Ophthalmology, Vol. 94, No. 3, 01.03.2010, p. 351-356.

Research output: Contribution to journalArticle

Williams, SL, Wang, L, Kane, SA, Lee, TC, Weissgold, DJ, Berrocal, A, Rabinowitz, D, Starren, J, Flynn, JT & Chiang, MF 2010, 'Telemedical diagnosis of retinopathy of prematurity: Accuracy of expert versus non-expert graders', British Journal of Ophthalmology, vol. 94, no. 3, pp. 351-356. https://doi.org/10.1136/bjo.2009.166348
Williams, Steven L. ; Wang, Lu ; Kane, Steven A. ; Lee, Thomas C. ; Weissgold, David J. ; Berrocal, Audina ; Rabinowitz, Daniel ; Starren, Justin ; Flynn, John T. ; Chiang, Michael F. / Telemedical diagnosis of retinopathy of prematurity : Accuracy of expert versus non-expert graders. In: British Journal of Ophthalmology. 2010 ; Vol. 94, No. 3. pp. 351-356.
@article{f46f9485a69447c185cc85e102910ed8,
title = "Telemedical diagnosis of retinopathy of prematurity: Accuracy of expert versus non-expert graders",
abstract = "Background/aims: To assess accuracy of telemedical retinopathy of prematurity (ROP) diagnosis by trained non-expert graders compared with expert graders. Methods: Eye examinations (n=248) from 67 consecutive infants were captured using wide-angle retinal photography (RetCam-II, Clarity Medical Systems, Pleasanton, California, USA). Non-expert graders attended two 1-h training sessions on image-based ROP diagnosis. Using a web-based telemedicine system, 14 non-expert and three expert graders provided a diagnosis for each eye: no ROP, mild ROP, type 2 pre-threshold ROP or treatment-requiring ROP. All diagnoses were compared with a reference standard of dilated indirect ophthalmoscopy by an experienced paediatric ophthalmologist. Results: For detection of type 2 or worse ROP, the mean (range) sensitivities and specificities were 0.95 (0.94-0.97) and 0.93 (0.91-0.96) for experts, 0.87 (0.71-0.97) and 0.73 (0.39-0.95) for resident non-experts, and 0.73 (0.41-0.88) and 0.91 (0.84-0.96) for student non-experts, respectively. For detection of treatment-requiring ROP, the mean (range) sensitivities and specificities were 1.00 (1.00-1.00) and 0.93 (0.88-0.96) for experts, 0.88 (0.50-1.00) and 0.84 (0.71-0.98) for resident non-experts, and 0.82 (0.42-1.00) and 0.92 (0.83-0.97) for student non-experts, respectively. Conclusions: Mean sensitivity and specificity of trained non-experts were lower than that of experts, although several non-experts had high accuracy. Development of methods for training non-expert graders may help support telemedical ROP evaluation.",
author = "Williams, {Steven L.} and Lu Wang and Kane, {Steven A.} and Lee, {Thomas C.} and Weissgold, {David J.} and Audina Berrocal and Daniel Rabinowitz and Justin Starren and Flynn, {John T.} and Chiang, {Michael F.}",
year = "2010",
month = "3",
day = "1",
doi = "10.1136/bjo.2009.166348",
language = "English",
volume = "94",
pages = "351--356",
journal = "British Journal of Ophthalmology",
issn = "0007-1161",
publisher = "BMJ Publishing Group",
number = "3",

}

TY - JOUR

T1 - Telemedical diagnosis of retinopathy of prematurity

T2 - Accuracy of expert versus non-expert graders

AU - Williams, Steven L.

AU - Wang, Lu

AU - Kane, Steven A.

AU - Lee, Thomas C.

AU - Weissgold, David J.

AU - Berrocal, Audina

AU - Rabinowitz, Daniel

AU - Starren, Justin

AU - Flynn, John T.

AU - Chiang, Michael F.

PY - 2010/3/1

Y1 - 2010/3/1

N2 - Background/aims: To assess accuracy of telemedical retinopathy of prematurity (ROP) diagnosis by trained non-expert graders compared with expert graders. Methods: Eye examinations (n=248) from 67 consecutive infants were captured using wide-angle retinal photography (RetCam-II, Clarity Medical Systems, Pleasanton, California, USA). Non-expert graders attended two 1-h training sessions on image-based ROP diagnosis. Using a web-based telemedicine system, 14 non-expert and three expert graders provided a diagnosis for each eye: no ROP, mild ROP, type 2 pre-threshold ROP or treatment-requiring ROP. All diagnoses were compared with a reference standard of dilated indirect ophthalmoscopy by an experienced paediatric ophthalmologist. Results: For detection of type 2 or worse ROP, the mean (range) sensitivities and specificities were 0.95 (0.94-0.97) and 0.93 (0.91-0.96) for experts, 0.87 (0.71-0.97) and 0.73 (0.39-0.95) for resident non-experts, and 0.73 (0.41-0.88) and 0.91 (0.84-0.96) for student non-experts, respectively. For detection of treatment-requiring ROP, the mean (range) sensitivities and specificities were 1.00 (1.00-1.00) and 0.93 (0.88-0.96) for experts, 0.88 (0.50-1.00) and 0.84 (0.71-0.98) for resident non-experts, and 0.82 (0.42-1.00) and 0.92 (0.83-0.97) for student non-experts, respectively. Conclusions: Mean sensitivity and specificity of trained non-experts were lower than that of experts, although several non-experts had high accuracy. Development of methods for training non-expert graders may help support telemedical ROP evaluation.

AB - Background/aims: To assess accuracy of telemedical retinopathy of prematurity (ROP) diagnosis by trained non-expert graders compared with expert graders. Methods: Eye examinations (n=248) from 67 consecutive infants were captured using wide-angle retinal photography (RetCam-II, Clarity Medical Systems, Pleasanton, California, USA). Non-expert graders attended two 1-h training sessions on image-based ROP diagnosis. Using a web-based telemedicine system, 14 non-expert and three expert graders provided a diagnosis for each eye: no ROP, mild ROP, type 2 pre-threshold ROP or treatment-requiring ROP. All diagnoses were compared with a reference standard of dilated indirect ophthalmoscopy by an experienced paediatric ophthalmologist. Results: For detection of type 2 or worse ROP, the mean (range) sensitivities and specificities were 0.95 (0.94-0.97) and 0.93 (0.91-0.96) for experts, 0.87 (0.71-0.97) and 0.73 (0.39-0.95) for resident non-experts, and 0.73 (0.41-0.88) and 0.91 (0.84-0.96) for student non-experts, respectively. For detection of treatment-requiring ROP, the mean (range) sensitivities and specificities were 1.00 (1.00-1.00) and 0.93 (0.88-0.96) for experts, 0.88 (0.50-1.00) and 0.84 (0.71-0.98) for resident non-experts, and 0.82 (0.42-1.00) and 0.92 (0.83-0.97) for student non-experts, respectively. Conclusions: Mean sensitivity and specificity of trained non-experts were lower than that of experts, although several non-experts had high accuracy. Development of methods for training non-expert graders may help support telemedical ROP evaluation.

UR - http://www.scopus.com/inward/record.url?scp=77949505912&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77949505912&partnerID=8YFLogxK

U2 - 10.1136/bjo.2009.166348

DO - 10.1136/bjo.2009.166348

M3 - Article

C2 - 19955195

AN - SCOPUS:77949505912

VL - 94

SP - 351

EP - 356

JO - British Journal of Ophthalmology

JF - British Journal of Ophthalmology

SN - 0007-1161

IS - 3

ER -