141 - Test Failure or Failed Test: Evidence of Bias in the Step 2 Clinical Skills Exam

Poster Number: 141
Publication Number: 141.217

Jeffrey G. Edwards, Boston Combined Residency Program in Pediatrics (Boston Children's Hospital/Boston Medical Center), BROOKLINE, MA, United States; Evida Dennis-Heyward, Boston Children’s Hospital, Boston, MA, United States; Colin Sox, Boston University School of Medicine, Boston, MA, United States

Background: Racial disparities in Step 1 and Step 2 Clinical Knowledge examination scores have been demonstrated by other researchers. While the subjective nature of scoring the USLME Step 2 Clinical Skills examination (S2-CS) may have perpetuated racial bias, the impact of race on this recently discontinued exam has not been explored.

Objective: To fill this gap in the literature, a retrospective cohort study of applicants to one large pediatric residency program in the Northeastern United States between 2014 and 2019 was conducted.

Design/Methods: The primary outcome was failing to pass the S2-CS exam at the time of application to residency. The primary independent measure was a self-identified race/ethnicity that is underrepresented in medicine (URiM). Applications that did not include a self-reported race/ethnicity or S2-CS exam score were excluded. Chi-square and t-tests were used to assess the relationship between S2-CS failure and subject characteristics.

Results: Within the cohort of 8,550 subjects who met inclusion criteria,18.4% self-identified as URiM, 32.5% attended medical school outside of the United States (“IMS”), and 0.23% failed to pass the S2-CS exam. S2-CS failure was significantly more common among subjects who attended IMS (n=16, 0.57%) than those who attended medical school in the US (“USMS”. n=5, 0.09%; p  < 0.0005). Among all subjects, S2-CS failure was not significant different between URiM and non-URiM applicants (0.40% vs. 0.22% respectively, p < 0.22). Of the 20 subjects who failed S2-CS and reported speaking any language fluently, 60% spoke native English (n=12). After stratifying by medical school location, URiM subjects who attended USMSs were significantly more likely to fail S2-CS than non-URiM subjects from USMS (0.57% vs 0%, p < 0.0005). However, rates of S2-CS failure among IMS graduates did not significantly differ between URiM (0.16%) and non-URiM subjects (0.69%, p = 0.12). Among USMS subjects who failed S2-CS, 60% spoke English fluently, as did 60% of IMS subjects who failed S2-CS.Conclusion(s): This study contributes to the growing body of literature demonstrating the continued impact of systemic racism on the evaluation of URiM medical students. As state medical licensing boards and national medical education committees decide how to replace the recently discontinued S2-CS exam with another means of assessing medical trainees’ clinical competence, it is critical that health equity principles be incorporated into the new assessment system.
Jeffrey Edwards CV for APA Award SubmissionEdwards_Jeffrey_Biosketch_010522.pdf