Multicenter interrater and intrarater reliability in the endoscopic evaluation of velopharyngeal insufficiency

Objective: To explore interrater and intrarater reliability (R _inter and R_intra, respectively) of a standardized scale applied to nasoendoscopic assessment of velopharyngeal (VP) function, across multiple centers. Design: Multicenter blinded R_inter and R _intra study. Setting: Eight academic tertiary care centers. Participants: Sixteen otolaryngologists from 8 centers. Main Outcome Measures: Raters estimated lateral pharyngeal and palatal movement on nasoendoscopic tapes from 50 different patients. Raters were asked to (1) estimate gap size during phonation and (2) note the presence of the Passavant ridge, a midline palatal notch on the nasal surface of the soft palate, and aberrant pulsations. Primary outcome measures were R_inter and R_intra coefficients for estimated gap size, lateral wall, and palatal movement; κ coefficients for the Passavant ridge, a mid- line palatal notch on the nasal soft palate, and aberrant pulsations were also calculated. Results: The R_inter coefficients were 0.63 for estimated gap size, 0.41 for lateral wall movement, and 0.43 for palate movement; corresponding R_intra coefficients were 0.86, 0.79, and 0.83, respectively. Interrater k values for qualitative features were 0.10 for the Passavant ridge; 0.48 for a notch on the nasal surface of the soft palate, 0.56 for aberrant pulsations, and 0.39 for estimation of gap size. Conclusions: In these data, there was good R_intra and fair R _inter when using the Golding-Kushner scale for rating VP function based on nasoendoscopy. Estimates of VP gap size demonstrate higher reliability coefficients than total lateral wall, mean palate estimates, and categorical estimate of gap size. The reliability of rating qualitative characteristics (ie, the presence of the Passavant ridge, aberrant pulsations, and notch on the nasal surface of the soft palate) is variable.