Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students

Paul A. Hemmer, Gregory A. Dadekian, Christopher Terndrup, Louis N. Pangaro, Allison B. Weisbrod, Mark D. Corriere, Rechell Rodriguez, Patricia Short, William F. Kelly

Research output: Contribution to journalArticlepeer-review

19 Scopus citations


Background: Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty. Objective: We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees). Design: Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008–2009 academic year. Participants: The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives. Main Measures: Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients. Key Results: The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p < 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r = 0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p = 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r = 0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p = 0.21). Conclusions: On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.

Original languageEnglish (US)
Pages (from-to)1313-1318
Number of pages6
JournalJournal of general internal medicine
Issue number9
StatePublished - Sep 19 2015


  • Medical education–assessment methods
  • Medical education–assessment/evaluation
  • Medical education–faculty development

ASJC Scopus subject areas

  • Internal Medicine


Dive into the research topics of 'Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students'. Together they form a unique fingerprint.

Cite this