Objective: To measure accuracy and reliability of the computer-based Retinal Image Multiscale Analysis (RISA) system compared with those of recognized retinopathy of prematurity (ROP) experts, for plus disease diagnosis. Design: Evaluation of diagnostic test or technology. Participants: Eleven recognized ROP experts and the RISA image analysis system interpreted a set of 20 wide-angle retinal photographs for presence of plus disease. Methods: All experts used a secure Web site to review independently 20 images for presence of plus disease. Images were also analyzed by measuring individual computer-based system parameters (integrated curvature [IC], diameter, and tortuosity index) for arterioles and venules and by computing linear combinations and logical combinations of those parameters. Performance was compared with a reference standard, defined as the majority vote of experts. Main Outcome Measures: Diagnostic accuracy was measured by calculating sensitivity, specificity, and receiver operating characteristic area under the curve (AUC) for plus disease diagnosis by each expert, and by each computer-based system parameter, compared with the reference standard. Diagnostic agreement was measured by calculating the mean κ value of each expert compared with all other experts and the mean κ value of each computer-based system parameter compared with all experts. Results: Among the 11 experts, sensitivity ranged from 0.167 to 1.000, specificity ranged from 0.714 to 1.000, AUC ranged from 0.798 to 1.000, and mean κ compared with all other experts ranged from 0.288 to 0.689. Among individual computer system parameters, arteriolar IC had the highest diagnostic accuracy, with sensitivity of 1.000; specificity, 0.846; and AUC, 0.962. Arteriolar IC had the highest diagnostic agreement with experts, with a mean κ value of 0.578. Conclusions: A computer-based image analysis system has the potential to perform comparably to recognized ROP experts for plus disease diagnosis.
ASJC Scopus subject areas