Interrater agreement of nasal endoscopy in patients with a prior history of endoscopic sinus surgery

Background: Nasal endoscopy is an important part of the clinical evaluation of patients with chronic rhinosinusitis. However, the objectivity and interrater agreement of the procedure related findings have not been well studied, especially in patients who have previously had sinus surgery. Methods: Patients with a history of endoscopic sinus surgery for chronic rhinosinusitis were prospectively enrolled from a tertiary rhinology practice. Fourteen endoscopic nasal examinations were recorded using digital video capture software. Each patient also underwent computed tomography (CT) and completed the Sinonasal Outcome Test (SNOT-22). Blinded review of inflammatory and anatomic findings for each video was independently performed by 5 academic rhinologists at separate institutions. Comparisons were performed using the unweighted Fleiss' kappa statistic (K_f) and the prevalence- and bias-adjusted kappa (PABAK). Results: There were no significant correlations between age, Lund-Mackay score, or SNOT-22 score. Interrater agreement was variable across the characteristics studied. Mean PABAK was excellent for the assessment of polyps (K_f = 0.886); moderate for the assessments of middle turbinate (MT) integrity (K_f = 0.543), MT position (K_f = 0.443), maxillary sinus patency (K_f = 0.593), and ethmoid sinus patency (K_f = 0.429); fair for discharge (K_f = 0.314), synechiae (K_f = 0.257), and middle meatus patency (K_f = 0.229); and poor for MT mucosal changes (K_f = 0.148) and uncinate process (K_f = 0.126). Conclusion: This study was notable for variability in the interrater agreement among the inflammatory and anatomic attributes that were examined. Further standardization of nasal endoscopy with regard to interpretation may improve the reliability of this procedure in clinical practice.