Novel Educational Information Management Platform Improves the Surgical Skill Evaluation Process of Surgical Residents

Objective: We sought to increase compliance and timeliness of surgery resident operative evaluation, by providing faculty and residents with a Platform-linking evaluation to analytics and machine-learning-facilitated case logging. Design: We built a HIPAA-compliant web-based Platform for comprehensive management of resident education information, including resident operative performance evaluations. To assess evaluation timeliness, we compared the lag time for Platform-based evaluations to that of end-of-rotation evaluations. We also assessed evaluation compliance, based on a time threshold of 5 days for Platform evaluations and 2 weeks for end-of-rotation evaluations. Setting: University of Massachusetts, Baystate Medical Center, General Surgery Residency. Participants: Twenty three attendings and 43 residents for the Platform cohort; 15 services and 45 residents for the end-of-rotation cohort. Results: Three hundred and fifty-eight Platform evaluations were completed by 23 attendings for 43 residents for March through October 2017. Six hundred and ten end-of-rotation evaluations by 15 attendings for 45 residents were used for comparison (September 2015 through June 2017). Of Platform evaluations, 41.3% were completed within 24 hours of the operation (16.5% in 6 hours, 33.3% in 12 hours, and 62.2% in 48 hours), with 24.3% of evaluations completed within 3 hours after e-mail reminders. In the first 6 weeks (March 1 through April 12) 4.5 ± 3.7 evaluations were completed per week compared to 18.8 ± 5.8 in the last (September 18 through October 31). Evaluation lag times improved with the use of the Platform, both for median lag of 35 days earlier (1 ± 1.5 days Platform, 36 ± 28.2 days traditional, p < 0.0001) and a mean lag of 41 days earlier (3.0 ± 4.7 days Platform, 44.0 ± 32.6 days traditional, p < 0.0001). Conclusions: Our comprehensive Platform facilitated faculty compliance with evaluation requirements and timeliness of availability of performance information (often in near real time) for both residents and residency leadership. The added value of the Platform's integration of evaluations with resident and attending case logging may account for the rapidly increasing number of operative skill evaluations over the short time span since implementation.