TY - JOUR
T1 - Variability in Plus Disease Identified Using a Deep Learning-Based Retinopathy of Prematurity Severity Scale
AU - Imaging and Informatics in Retinopathy of Prematurity Consortium
AU - Choi, Rene Y.
AU - Brown, James M.
AU - Kalpathy-Cramer, Jayashree
AU - Chan, R. V.Paul
AU - Ostmo, Susan
AU - Chiang, Michael F.
AU - Campbell, J. Peter
AU - Kim, Sang Jin
AU - Sonmez, Kemal
AU - Jonas, Karyn
AU - Horowitz, Jason
AU - Coki, Osode
AU - Eccles, Cheryl Ann
AU - Sarna, Leora
AU - Orlin, Anton
AU - Berrocal, Audina
AU - Negron, Catherin
AU - Denser, Kimberly
AU - Cumming, Kristi
AU - Osentoski, Tammy
AU - Check, Tammy
AU - Zajechowski, Mary
AU - Lee, Thomas
AU - Kruger, Evan
AU - McGovern, Kathryn
AU - Simmons, Charles
AU - Murthy, Raghu
AU - Galvis, Sharon
AU - Rotter, Jerome
AU - Chen, Ida
AU - Li, Xiaohui
AU - Taylor, Kent
AU - Roll, Kaye
AU - Erdogmus, Deniz
AU - Ioannidis, Stratis
AU - Martinez-Castellanos, Maria Ana
AU - Salinas-Longoria, Samantha
AU - Romero, Rafael
AU - Arriola, Andrea
AU - Olguin-Manriquez, Francisco
AU - Meraz-Gutierrez, Miroslava
AU - Dulanto-Reinoso, Carlos M.
AU - Montero-Mendoza, Cristina
N1 - Publisher Copyright:
© 2020 American Academy of Ophthalmology
PY - 2020/10
Y1 - 2020/10
N2 - Purpose: Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score. Design: This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained. Participants: Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results. Methods: An ROP severity score (1–9) was generated for each image using a deep learning (DL) algorithm. Main Outcome Measures: The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made. Results: A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8–8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P < 0.01). Conclusions: We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.
AB - Purpose: Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score. Design: This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained. Participants: Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results. Methods: An ROP severity score (1–9) was generated for each image using a deep learning (DL) algorithm. Main Outcome Measures: The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made. Results: A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8–8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P < 0.01). Conclusions: We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.
UR - http://www.scopus.com/inward/record.url?scp=85087684676&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087684676&partnerID=8YFLogxK
U2 - 10.1016/j.oret.2020.04.022
DO - 10.1016/j.oret.2020.04.022
M3 - Article
C2 - 32380115
AN - SCOPUS:85087684676
SN - 2468-7219
VL - 4
SP - 1016
EP - 1021
JO - Ophthalmology Retina
JF - Ophthalmology Retina
IS - 10
ER -