TY - JOUR
T1 - Multinational External Validation of Autonomous Retinopathy of Prematurity Screening
AU - Coyner, Aaron S.
AU - Murickan, Tom
AU - Oh, Minn A.
AU - Young, Benjamin
AU - Ostmo, Susan R.
AU - Singh, Praveer
AU - Paul Chan, R. V.
AU - Moshfeghi, Darius M.
AU - Shah, Parag K.
AU - Venkatapathy, Narendran
AU - Chiang, Michael
AU - Kalpathy-Cramer, Jayashree
AU - Peter Campbell, J.
N1 - Publisher Copyright:
© 2024 American Medical Association. All rights reserved.
PY - 2024
Y1 - 2024
N2 - IMPORTANCE Retinopathy of prematurity (ROP) is a leading cause of blindness in children, with significant disparities in outcomes between high-income and low-income countries, due in part to insufficient access to ROP screening. OBJECTIVE To evaluate how well autonomous artificial intelligence (AI)–based ROP screening can detect more-than-mild ROP (mtmROP) and type 1 ROP. DESIGN, SETTING, AND PARTICIPANTS This diagnostic study evaluated the performance of an AI algorithm, trained and calibrated using 2530 examinations from 843 infants in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) study, on 2 external datasets (6245 examinations from 1545 infants in the Stanford University Network for Diagnosis of ROP [SUNDROP] and 5635 examinations from 2699 infants in the Aravind Eye Care Systems [AECS] telemedicine programs). Data were taken from 11 and 48 neonatal care units in the US and India, respectively. Data were collected from January 2012 to July 2021, and data were analyzed from July to December 2023. EXPOSURES An imaging processing pipeline was created using deep learning to autonomously identify mtmROP and type 1 ROP in eye examinations performed via telemedicine. MAIN OUTCOMES AND MEASURES The area under the receiver operating characteristics curve (AUROC) as well as sensitivity and specificity for detection of mtmROP and type 1 ROP at the eye examination and patient levels. RESULTS The prevalence of mtmROP and type 1 ROP were 5.9% (91 of 1545) and 1.2% (18 of 1545), respectively, in the SUNDROP dataset and 6.2% (168 of 2699) and 2.5% (68 of 2699) in the AECS dataset. Examination-level AUROCs for mtmROP and type 1 ROP were 0.896 and 0.985, respectively, in the SUNDROP dataset and 0.920 and 0.982 in the AECS dataset. At the cross-sectional examination level, mtmROP detection had high sensitivity (SUNDROP: mtmROP, 83.5%; 95% CI, 76.6-87.7; type 1 ROP, 82.2%; 95% CI, 81.2-83.1; AECS: mtmROP, 80.8%; 95% CI, 76.2-84.9; type 1 ROP, 87.8%; 95% CI, 86.8-88.7). At the patient level, all infants who developed type 1 ROP screened positive (SUNDROP: 100%; 95% CI, 81.4-100; AECS: 100%; 95% CI, 94.7-100) prior to diagnosis. CONCLUSIONS AND RELEVANCE Where and when ROP telemedicine programs can be implemented, autonomous ROP screening may be an effective force multiplier for secondary prevention of ROP.
AB - IMPORTANCE Retinopathy of prematurity (ROP) is a leading cause of blindness in children, with significant disparities in outcomes between high-income and low-income countries, due in part to insufficient access to ROP screening. OBJECTIVE To evaluate how well autonomous artificial intelligence (AI)–based ROP screening can detect more-than-mild ROP (mtmROP) and type 1 ROP. DESIGN, SETTING, AND PARTICIPANTS This diagnostic study evaluated the performance of an AI algorithm, trained and calibrated using 2530 examinations from 843 infants in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) study, on 2 external datasets (6245 examinations from 1545 infants in the Stanford University Network for Diagnosis of ROP [SUNDROP] and 5635 examinations from 2699 infants in the Aravind Eye Care Systems [AECS] telemedicine programs). Data were taken from 11 and 48 neonatal care units in the US and India, respectively. Data were collected from January 2012 to July 2021, and data were analyzed from July to December 2023. EXPOSURES An imaging processing pipeline was created using deep learning to autonomously identify mtmROP and type 1 ROP in eye examinations performed via telemedicine. MAIN OUTCOMES AND MEASURES The area under the receiver operating characteristics curve (AUROC) as well as sensitivity and specificity for detection of mtmROP and type 1 ROP at the eye examination and patient levels. RESULTS The prevalence of mtmROP and type 1 ROP were 5.9% (91 of 1545) and 1.2% (18 of 1545), respectively, in the SUNDROP dataset and 6.2% (168 of 2699) and 2.5% (68 of 2699) in the AECS dataset. Examination-level AUROCs for mtmROP and type 1 ROP were 0.896 and 0.985, respectively, in the SUNDROP dataset and 0.920 and 0.982 in the AECS dataset. At the cross-sectional examination level, mtmROP detection had high sensitivity (SUNDROP: mtmROP, 83.5%; 95% CI, 76.6-87.7; type 1 ROP, 82.2%; 95% CI, 81.2-83.1; AECS: mtmROP, 80.8%; 95% CI, 76.2-84.9; type 1 ROP, 87.8%; 95% CI, 86.8-88.7). At the patient level, all infants who developed type 1 ROP screened positive (SUNDROP: 100%; 95% CI, 81.4-100; AECS: 100%; 95% CI, 94.7-100) prior to diagnosis. CONCLUSIONS AND RELEVANCE Where and when ROP telemedicine programs can be implemented, autonomous ROP screening may be an effective force multiplier for secondary prevention of ROP.
UR - http://www.scopus.com/inward/record.url?scp=85187537781&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85187537781&partnerID=8YFLogxK
U2 - 10.1001/jamaophthalmol.2024.0045
DO - 10.1001/jamaophthalmol.2024.0045
M3 - Article
C2 - 38451496
AN - SCOPUS:85187537781
SN - 2168-6165
JO - JAMA ophthalmology
JF - JAMA ophthalmology
ER -