245 - Improving the Quality of Faculty Feedback: A Multidimensional Approach

Poster Number: 245
Publication Number: 245.418

Kelly Murphy, Cohen Children's Medical Center, New Hyde Park, NY, United States; Stephen R. Barone, Cohen Children's Medical Center, New Hyde Park, NY, United States

Background: High quality (timely, accurate, actionable) faculty feedback is an integral part of resident growth and development. Written feedback from faculty varies in quality, with the least useful comments being general in nature and lacking actionable recommendations. We instituted a comprehensive evaluation and faculty development program to improve the quality of written feedback.

Objective: To improve the quality of written feedback provided to residents by faculty.

Design/Methods: Faculty complete electronic evaluations of residents after each rotation. These evaluations include numerical scores based on ACGME milestones and written comments. To evaluate the quality of written comments a novel assessment rubric was developed based on previously validated tools. This tool was used to assess the quality of evaluations of residents via a numerical score pre and post intervention. Comments were ranked 1 (low quality) to 4 (high quality). A score of 1 was assigned to comments which were all global, included no dimension-specific, examples or behavioral comments and no recommendations. Conversely to receive a score of 4, the evaluation needed to include both dimension-specific and example/behavioral comments as well as actionable recommendations. Evaluations with no comments received a 0. Individual and division scores were calculated and shared with division chiefs. Division chiefs subsequently developed action plans to improve written feedback.

Results: Pre-Intervention (6/1/20-4/30/2021), 631 evaluations were assessed. Of these, 1% received a score of 0, 12% received a score of 1, 39% received a score of 2, 30% received a score of 3 and 18% received a score of 4. Post-intervention (7/1/21-12/6/21) 191 evaluations were scored. Of these, 6% received a score of 0, 1% received a score of 1, 13% received a score of 2, 24% received a score of 3 and 56% received a score of 4. The average score across all divisions improved from 2.5 to 3.2 (pConclusion(s): A dedicated assessment system paired with division specific action plans and an infographic, improve the quality of written comments on faculty evaluations of residents.
Tips for Quality Narrative CommentsInfographic provided to faculty
Narrative Comment Evaluation RubricRubric used to score faculty evaluations of residents