388 - Improving Timeliness of Resident Assessment by Pediatric Hospitalists using Incentivized Audit and Feedback

Poster Number: 388
Publication Number: 388.243

Laura Z. Lockwood, University of Colorado School of Medicine, Aurora, CO, United States; Kimberly O'Hara, University of Colorado School of Medicine/Children’s Hospital Colorado, Denver, CO, United States; Michael Tchou, Children's Hospital Colorado, Aurora, CO, United States; Jennifer Reese, University of Colorado School of Medicine, Aurora, CO, United States; Kathy Urban, University of Colorado School of Medicine, Aurora, CO, United States; Adam A. Rosenberg, --None--, Castle Rock, CO, United States

Background: The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to maintain 80% on-time completion of resident assessments by faculty. Our program historically obtained only 59% system wide. Exploration of key drivers revealed opportunities for high-impact improvement on the general inpatient wards, a clinical setting with frequent resident assessment. Audit and feedback using provider-level data have been effective elsewhere in clinical quality improvement but have not been applied to improving learner assessment.

Objective: To improve the percentage of on-time completion of resident assessments by pediatric hospitalists from 46% (July 2019 – April 2020) to 80% by June 2022

Design/Methods: Using the IHI Model for Improvement, we implemented a series of 3 PDSA cycles from May 2020 – January 2021 (Table 1). We developed interventions using multidisciplinary stakeholder input to address key drivers identified in the initial planning phase. We extracted monthly data for assessment timing from our electronic assessment system. We defined our primary outcome measures as 1) proportion of resident assessments with on-time completion by hospitalists and 2) average time from resident assessment assignment to completion by hospitalists. We defined on-time as 14 days from assignment to completion in alignment with ACGME requirements. We analyzed our data using descriptive statistics and t-tests for continuous variables. We created run/statistical process control charts for our primary outcome measures and assessed for special cause variation using Shewart’s rules.

Results: A similar number of assessments were assigned monthly to hospitalists in our baseline and study periods (28/month vs 29/month,
p =0.71). We observed special cause variation in the percentage of on-time completion (46% to 58%) and time from assignment to completion (36 to 14 days) with sustained centerline shifts in the desired direction. Improvement began following the faculty development workshops and sustained with implementation of the individualized dashboards & performance reviews (Figure 1).Conclusion(s):
Although shy of our goal, we observed meaningful, sustained improvement in timeliness of resident assessment by hospitalists. Future PDSA cycles will target timelier resident request of assessments in pursuit of our ultimate target of 80% on-time completion. Our outcomes suggest incentivized audit and feedback using provider-level data may be a powerful and feasible tool to improve timeliness of resident assessment.

Table 1. Interventions for primary PDSA cycles
Figure 1. Charts showing primary outcome data over time with evaluation for special cause variation