Susan D. Wolfsthal, MDUniversity of Maryland School of Medicine
Program Size: More than 101 residentsAcademic Setting: University-BasedClinical Setting: All
The overarching principle in developing our evaluation tools was to identify and map entrustable professional activities (EPA’s) created by the Internal Medicine community to the residents’ various rotations. To ensure faculty had the opportunity to observe resident skills in a variety of settings and rotations, our curriculum committee selected 24 EPA’s that were then mapped to all rotations and to the ACGME reporting milestones. The committee ensured that each EPA was appropriately represented across the spectrum of rotations and that each reporting milestone had an adequate number of EPA’s mapped to that milestone. Rather than using the curricular milestones, our curriculum committee chose the EPA’s as a more specific and goal directed measure of residency performance for our evaluation tools. Descriptive anchors were not provided for each EPA. Instead, faculty were trained to rate the resident as to the amount of supervision they required for each EPA. These nine supervision categories spanned from a very early learner requiring constant close supervision to a resident ready for independent practice who demonstrates high achievement or is aspirational. Clear descriptions of each supervisory class are provided at the top of each evaluation form. The 9-point numerical rating scale for each milestone is not revealed to the faculty. A general normative scale was added at the end of the form for faculty to rate the resident on the overall degree of supervision required and the overall performance compared to their peer group. Example provided is for GIM inpatient rotation; others available for all rotations.
Download the Tool
The faculty were offered workshops at the beginning of the process to review the concept of milestones and the structure of the new evaluation system. Suggestions were provided as to how to evaluate the milestones using direct observation. The slides were distributed to the entire clinical teaching faculty. Ongoing workshops are planned to reinforce concepts and teach skills to faculty members.
Low score notifications are set for each PGY level to provide real time information about residents scoring below expected milestones. After using the milestones for two years, tracking results provide critical markers in identifying residents who are either not meeting expected competency or those who are performing at a high level. The results have provided a cluster of residents for each of training year thus allowing a more clear distinction among the three classes of residents. By focusing on the EPA’s, the CCC has implemented targeted remediation plans that focus on behaviors that can be tracked over time.
For more information, please contact firstname.lastname@example.org.