TY - JOUR
T1 - Automated Fundus Image Quality Assessment in Retinopathy of Prematurity Using Deep Convolutional Neural Networks
AU - Imaging and Informatics in Retinopathy of Prematurity Research Consortium
AU - Coyner, Aaron S.
AU - Swan, Ryan
AU - Campbell, J. Peter
AU - Ostmo, Susan
AU - Brown, James M.
AU - Kalpathy-Cramer, Jayashree
AU - Kim, Sang Jin
AU - Jonas, Karyn E.
AU - Chan, R. V.Paul
AU - Chiang, Michael F.
AU - Sonmez, Kemal
AU - Chan, R. V.Paul
AU - Jonas, Karyn
AU - Horowitz, Jason
AU - Coki, Osode
AU - Eccles, Cheryl Ann
AU - Sarna, Leora
AU - Orlin, Anton
AU - Berrocal, Audina
AU - Negron, Catherin
AU - Denser, Kimberly
AU - Cumming, Kristi
AU - Osentoski, Tammy
AU - Check, Tammy
AU - Zajechowski, Mary
AU - Lee, Thomas
AU - Kruger, Evan
AU - McGovern, Kathryn
AU - Simmons, Charles
AU - Murthy, Raghu
AU - Galvis, Sharon
AU - Rotter, Jerome
AU - Chen, Ida
AU - Li, Xiaohui
AU - Taylor, Kent
AU - Roll, Kaye
AU - Chang, Ken
AU - Beers, Andrew
AU - Erdogmus, Deniz
AU - Ioannidis, Stratis
AU - Martinez-Castellanos, Maria Ana
AU - Salinas-Longoria, Samantha
AU - Romero, Rafael
AU - Arriola, Andrea
AU - Olguin-Manriquez, Francisco
AU - Meraz-Gutierrez, Miroslava
AU - Dulanto-Reinoso, Carlos M.
AU - Montero-Mendoza, Cristina
N1 - Publisher Copyright:
© 2019 American Academy of Ophthalmology
PY - 2019/5
Y1 - 2019/5
N2 - Purpose: Accurate image-based ophthalmic diagnosis relies on fundus image clarity. This has important implications for the quality of ophthalmic diagnoses and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a deep convolutional neural network (CNN) for automated assessment of fundus image quality in retinopathy of prematurity (ROP). Design: Experimental study. Participants: Retinal fundus images were collected from preterm infants during routine ROP screenings. Methods: Six thousand one hundred thirty-nine retinal fundus images were collected from 9 academic institutions. Each image was graded for quality (acceptable quality [AQ], possibly acceptable quality [PAQ], or not acceptable quality [NAQ]) by 3 independent experts. Quality was defined as the ability to assess an image confidently for the presence of ROP. Of the 6139 images, NAQ, PAQ, and AQ images represented 5.6%, 43.6%, and 50.8% of the image set, respectively. Because of low representation of NAQ images in the data set, images labeled NAQ were grouped into the PAQ category, and a binary CNN classifier was trained using 5-fold cross-validation on 4000 images. A test set of 2109 images was held out for final model evaluation. Additionally, 30 images were ranked from worst to best quality by 6 experts via pairwise comparisons, and the CNN's ability to rank quality, regardless of quality classification, was assessed. Main Outcome Measures: The CNN performance was evaluated using area under the receiver operating characteristic curve (AUC). A Spearman's rank correlation was calculated to evaluate the overall ability of the CNN to rank images from worst to best quality as compared with experts. Results: The mean AUC for 5-fold cross-validation was 0.958 (standard deviation, 0.005) for the diagnosis of AQ versus PAQ images. The AUC was 0.965 for the test set. The Spearman's rank correlation coefficient on the set of 30 images was 0.90 as compared with the overall expert consensus ranking. Conclusions: This model accurately assessed retinal fundus image quality in a comparable manner with that of experts. This fully automated model has potential for application in clinical settings, telemedicine, and computer-based image analysis in ROP and for generalizability to other ophthalmic diseases.
AB - Purpose: Accurate image-based ophthalmic diagnosis relies on fundus image clarity. This has important implications for the quality of ophthalmic diagnoses and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a deep convolutional neural network (CNN) for automated assessment of fundus image quality in retinopathy of prematurity (ROP). Design: Experimental study. Participants: Retinal fundus images were collected from preterm infants during routine ROP screenings. Methods: Six thousand one hundred thirty-nine retinal fundus images were collected from 9 academic institutions. Each image was graded for quality (acceptable quality [AQ], possibly acceptable quality [PAQ], or not acceptable quality [NAQ]) by 3 independent experts. Quality was defined as the ability to assess an image confidently for the presence of ROP. Of the 6139 images, NAQ, PAQ, and AQ images represented 5.6%, 43.6%, and 50.8% of the image set, respectively. Because of low representation of NAQ images in the data set, images labeled NAQ were grouped into the PAQ category, and a binary CNN classifier was trained using 5-fold cross-validation on 4000 images. A test set of 2109 images was held out for final model evaluation. Additionally, 30 images were ranked from worst to best quality by 6 experts via pairwise comparisons, and the CNN's ability to rank quality, regardless of quality classification, was assessed. Main Outcome Measures: The CNN performance was evaluated using area under the receiver operating characteristic curve (AUC). A Spearman's rank correlation was calculated to evaluate the overall ability of the CNN to rank images from worst to best quality as compared with experts. Results: The mean AUC for 5-fold cross-validation was 0.958 (standard deviation, 0.005) for the diagnosis of AQ versus PAQ images. The AUC was 0.965 for the test set. The Spearman's rank correlation coefficient on the set of 30 images was 0.90 as compared with the overall expert consensus ranking. Conclusions: This model accurately assessed retinal fundus image quality in a comparable manner with that of experts. This fully automated model has potential for application in clinical settings, telemedicine, and computer-based image analysis in ROP and for generalizability to other ophthalmic diseases.
UR - http://www.scopus.com/inward/record.url?scp=85070432758&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85070432758&partnerID=8YFLogxK
U2 - 10.1016/j.oret.2019.01.015
DO - 10.1016/j.oret.2019.01.015
M3 - Article
C2 - 31044738
AN - SCOPUS:85070432758
SN - 2468-7219
VL - 3
SP - 444
EP - 450
JO - Ophthalmology Retina
JF - Ophthalmology Retina
IS - 5
ER -