TY - JOUR
T1 - Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting
T2 - Validity and reliability
AU - Lentz, Gretchen M.
AU - Mandel, Lynn S.
AU - Lee, David
AU - Gardella, Carolyn
AU - Melville, Jennifer
AU - Goff, Barbara A.
N1 - Funding Information:
Supported by the Association of Professors of Gynecology and Obstetrics Abbott Medical Education Award, the American Board of Medical Examiners, Tap Pharmaceuticals, and US Surgical Corporation.
PY - 2001
Y1 - 2001
N2 - OBJECTIVE: Resident surgical skills are acquired mainly through observing and later performing procedures in the operating room. Evaluation of surgical skills has traditionally been done through subjective faculty evaluation, a technique that has poor reliability and unknown validity. Our goal was to develop specific surgical tasks, both laparoscopic and open abdominal, that could be objectively and reliably evaluated in a bench laboratory setting. STUDY DESIGN: The prospective development of a reliable and valid resident surgical skills test in a bench laboratory setting was our goal. A written test of surgical knowledge and 12 skills tests were administered to 36 residents. Laparoscopic bench tasks were simulated with the use of a box and camera with a video display. Six laparoscopic tasks were assessed, including placing pegs on a board, running the bowel simulation, and other tasks that involve hand-eye coordination and manual dexterity. Open abdominal skills simulated incision closure, suturing a vaginal cuff, knot tying, and using a tie on a passer. Residents were timed at each given station and were given a rating score by 2 examiners. RESULTS: Knowledge scores showed a significant improvement by residency level. Assessment of construct validity (the ability to discriminate among residency levels) demonstrated significant differences on the rating of overall performance and individual tasks by level (determined by 1-way analysis of variance). Interrater reliability (agreement between 2 raters) with the use of intraclass correlation was 0.79 for the total score. The cost to administer the bench laboratory test was less than $50 and required 30 hours of faculty time. CONCLUSION: The results of this study suggest that surgical bench laboratory tasks can assess residents' surgical skills with good reliability and validity on most tasks. Our previous study, which used an animal laboratory, was expensive, and the bench laboratory model may provide an alternative means to assess surgical skills.
AB - OBJECTIVE: Resident surgical skills are acquired mainly through observing and later performing procedures in the operating room. Evaluation of surgical skills has traditionally been done through subjective faculty evaluation, a technique that has poor reliability and unknown validity. Our goal was to develop specific surgical tasks, both laparoscopic and open abdominal, that could be objectively and reliably evaluated in a bench laboratory setting. STUDY DESIGN: The prospective development of a reliable and valid resident surgical skills test in a bench laboratory setting was our goal. A written test of surgical knowledge and 12 skills tests were administered to 36 residents. Laparoscopic bench tasks were simulated with the use of a box and camera with a video display. Six laparoscopic tasks were assessed, including placing pegs on a board, running the bowel simulation, and other tasks that involve hand-eye coordination and manual dexterity. Open abdominal skills simulated incision closure, suturing a vaginal cuff, knot tying, and using a tie on a passer. Residents were timed at each given station and were given a rating score by 2 examiners. RESULTS: Knowledge scores showed a significant improvement by residency level. Assessment of construct validity (the ability to discriminate among residency levels) demonstrated significant differences on the rating of overall performance and individual tasks by level (determined by 1-way analysis of variance). Interrater reliability (agreement between 2 raters) with the use of intraclass correlation was 0.79 for the total score. The cost to administer the bench laboratory test was less than $50 and required 30 hours of faculty time. CONCLUSION: The results of this study suggest that surgical bench laboratory tasks can assess residents' surgical skills with good reliability and validity on most tasks. Our previous study, which used an animal laboratory, was expensive, and the bench laboratory model may provide an alternative means to assess surgical skills.
KW - Inanimate laboratory
KW - Resident education
KW - Surgical skills
UR - http://www.scopus.com/inward/record.url?scp=0034972666&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0034972666&partnerID=8YFLogxK
U2 - 10.1067/mob.2001.114850
DO - 10.1067/mob.2001.114850
M3 - Article
C2 - 11408869
AN - SCOPUS:0034972666
SN - 0002-9378
VL - 184
SP - 1462
EP - 1470
JO - American journal of obstetrics and gynecology
JF - American journal of obstetrics and gynecology
IS - 7
ER -