294 - Resident Early Warning System (REWS): A System to Develop Timely Feedback for Pediatric Interns

Poster Number: 294
Publication Number: 294.421

Anna Brown, Akron Children's Hospital, Akron, OH, United States; Emily Kramer, Akron Children's Hospital, Brecksville, OH, United States; Kathryn Mansel, Akron Children's Hospital, Akron, OH, United States

Background: The Accreditation Council for Graduate Medical Education (ACGME) recognizes the importance of providing frequent and timely feedback from direct observation to support the development of pediatric trainees. Interns spend a significant amount of time on the inpatient unit under close senior resident supervision, providing an ideal opportunity to improve current feedback practices.

Objective: The Model for Improvement was used to develop a system to provide interns with timely milestone-based feedback from the senior residents providing direct observation. A secondary objective was to assess whether this tool could be used for early identification and, in turn, early interventions for residents requiring additional support to succeed on inpatient services.

Design/Methods: A panel consisting of Residency Program Director, Associate Program Director and Chief Resident developed the evaluation form. Four of the six ACGME core competencies were modified into a questionnaire utilizing novice-evaluator friendly language (Table 1).

Quality improvement methodology was used to assess and improve the project. Twice monthly evaluations were distributed to senior residents who worked more than four shifts with an intern. Time to first feedback was tracked in addition to intern performance. An annual survey was distributed to assess balancing measures and resident satisfaction. The Associate Residency Program Director was notified based on concerning criteria identified on intern feedback (Table 2).

Results: The time to first feedback for interns went from 29.6 to 14.17 days, demonstrating a statistically significant improvement (Figure 1). Based on survey results, only 7.7% of interns felt they received inconsistent feedback from senior residents (n = 10, response rate 45%) compared to 30% the year prior (n = 13, response rate 62%).

During the 2020 – 2021 academic year, three residents were identified by the existing evaluation system as requiring additional support or remediation. These residents were also identified by the novel REWS evaluation system, in addition to six residents who did not require intervention. All three residents requiring intervention triggered multiple categories of concern (Table 2).Conclusion(s): Timely feedback is important to the professional development of pediatric trainees. Implementation of a senior evaluation during inpatient pediatric rotations allowed for timely feedback to interns without compromising feedback quality.
Table 1Intern evaluation form for senior resident direct observation
Table 2Concerning evaluation criteria rubric to notify Associate Program Director