560 - Mixed Methods Evaluation of Implementation of an EPA-Based Workplace Based Assessment in Outpatient Pediatrics

Poster Number: 560
Publication Number: 560.115

Marguerite Costich, Columbia University, New York, NY, United States; Suzanne Friedman, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, United States; Marina Catallozzi, Columbia University Irving Med, NEW YORK, NY, United States

Background: Entrustable Professional Activities (EPAs) detail skills medical students need to perform as they transition to residency. Workplace Based Assessments (WBA) are tools that allow for an understanding of the developmental trajectory of EPAs and provision of formative feedback. Medical students spend little time in outpatient pediatrics during clinical training and may receive inadequate formative feedback on skills.  While WBAs have been implemented in other environments, there is limited literature on WBAs in the outpatient pediatric setting. This clinical context requires greater attention to provision of feedback given students’ limited exposure to well childcare.

Objective: Utilize mixed methods to understand how WBA implementation in the outpatient pediatric setting impacts frequency of faculty feedback and their perceptions of WBA use.

Design/Methods: Outpatient faculty were surveyed via quantitative methods prior to the introduction of the WBA regarding feedback frequency and satisfaction.  Faculty then underwent performance-driven training (PDT) prior to WBA implementation.  PDT focused on skills and comfort in faculty WBA use.  Six months following WBA implementation, faculty were surveyed again regarding quality and quantity of feedback delivered. Faculty also participated in individual semi-structured interviews exploring their perceptions of the WBA and its impact on feedback delivery.  Quantitative data were dichotomized and compared via chi-square tests and qualitative data was analyzed using thematic analysis, with acceptable inter-coder agreement (k=0.75) amongst the two team members.

Results: Sixteen (76%) faculty who work with students completed the pre-WBA implementation survey, 12 (57%) completed the post-WBA implementation survey and 8 faculty participated in semi-structured interviews. Faculty reported increased satisfaction with timeliness of feedback as well as a significant increase in provision of specific, task-oriented feedback following WBA implementation. (Table 1) Consistent with the domains of the interview guides, themes from the qualitative data reflected benefits of WBA, persistent barriers to provision of feedback, and suggestions for improvement of the WBA tool and feedback delivery. (Table 2)Conclusion(s): WBAs are feasible in the outpatient pediatric setting. Faculty engagement and training can help to ensure success of implementation and increase the frequency of in-time feedback. Findings will inform future faculty development and modifications to the WBA tool. Implementation of an EPA-based WBA in other settings may be beneficial to improve feedback delivery.
Table 1Faculty surveys results before and after WBA implementation
Table 2Faculty qualitative interview themes