Accuracy of retinopathy of prematurity diagnosis by retinal fellows

PURPOSE:: The purpose of this study was to measure the accuracy of retinopathy of prematurity (ROP) diagnosis by retinal fellows. METHODS:: An atlas of 804 retinal images was captured from 248 eyes of 67 premature infants with a wide-angle camera (RetCam-II, Clarity Medical Systems, Pleasanton, CA). Images were uploaded to a study Web site, from which an expert pediatric retinal specialist and 7 retinal fellows independently provided a diagnosis (no ROP, mild ROP, type 2 ROP, or treatment-requiring ROP) for each eye. The sensitivity and specificity of each retinal fellow were calculated and subsequently compared with a reference standard of diagnosis by an expert pediatric retinal specialist. RESULTS:: For detection of type 2 or worse ROP by fellows, mean (range) sensitivity was 0.751 (0.512-0.953), and specificity was 0.841 (0.707-0.976). For detection of treatment-requiring ROP, mean (range) sensitivity was 0.914 (0.667-1.000), and specificity was 0.871 (0.678-0.987). CONCLUSION:: In general, fellows showed high accuracy for detecting ROP. However, 3 of 7 fellows achieved <80% sensitivity for diagnosis of type 2 or worse ROP, and 2 of 7 achieved <90% sensitivity for diagnosis of treatment-requiring ROP. This could lead to undermanagement and undertreatment of clinically significant disease and raises potential concerns about the quality of ROP screening examinations performed by less-experienced examiners.