TY - JOUR
T1 - Development and initial validation of an instrument for video-based assessment of technical skill in ERCP
AU - SVI study group and the U.S. Cooperative for Outcomes Research in Endoscopy
AU - Elmunzer, B. Joseph
AU - Walsh, Catharine M.
AU - Guiton, Gretchen
AU - Serrano, Jose
AU - Chak, Amitabh
AU - Edmundowicz, Steven
AU - Kwon, Richard S.
AU - Mullady, Daniel
AU - Papachristou, Georgios I.
AU - Elta, Grace
AU - Baron, Todd H.
AU - Yachimski, Patrick
AU - Fogel, Evan L.
AU - Draganov, Peter V.
AU - Taylor, Jason R.
AU - Scheiman, James
AU - Singh, Vikesh K.
AU - Varadarajulu, Shyam
AU - Willingham, Field F.
AU - Cote, Gregory A.
AU - Cotton, Peter B.
AU - Simon, Violette
AU - Spitzer, Rebecca
AU - Keswani, Rajesh
AU - Wani, Sachin
N1 - Publisher Copyright:
© 2021 American Society for Gastrointestinal Endoscopy
PY - 2021/4
Y1 - 2021/4
N2 - Background and Aims: The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool. Methods: Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability. Results: Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was.67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists’ skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis. Conclusions: Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
AB - Background and Aims: The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool. Methods: Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability. Results: Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was.67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists’ skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis. Conclusions: Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
UR - http://www.scopus.com/inward/record.url?scp=85095847288&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85095847288&partnerID=8YFLogxK
U2 - 10.1016/j.gie.2020.07.055
DO - 10.1016/j.gie.2020.07.055
M3 - Article
C2 - 32739484
AN - SCOPUS:85095847288
SN - 0016-5107
VL - 93
SP - 914
EP - 923
JO - Gastrointestinal endoscopy
JF - Gastrointestinal endoscopy
IS - 4
ER -