153 - Setting Mastery Passing Standards for Simulated Neonatal Mask Ventilation Performance Using the Patient Safety Approach.

Poster Number: 153
Publication Number: 153.218

Beverley Robin, Rush Medical College of Rush University Medical Center, Chicago, IL, United States; Joseph B. deBettencourt, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, United States; Elizabeth L. Papautsky, University of Illinois at Chicago, Chicago, IL, United States; Yoon Soo Park, University of Illinois College of Medicine, Chicago, IL, United States; Rachel Yudkowsky, University of Illinois College of Medicine at Chicago, Chicago, IL, United States

Background: Neonatal mask ventilation (NMV) is a vital competency for neonatology fellows. Inconsistent NMV is documented, even for experienced healthcare providers. No performance (passing) standards for NMV exist, and most fellowship programs use direct supervision of an arbitrary number of procedures as an assessment method.

Objective: Set mastery performance standards for NMV using the published Patient Safety Approach.

Design/Methods: A 17-step NMV checklist was developed and reviewed by neonatologists (subject matter experts; SMEs). To set performance standards for NMV using the Patient Safety Approach, eight SMEs were asked to review the checklist and: 1) indicate whether each step is “essential” or “important but not essential” to patient outcome and/or safety, or “not important”, and 2) consider the neonatology fellow, well-prepared for independent practice on a patient and choose from three options for performance standards of NMV conducted on a mannequin. The options were: a) 100%; all steps performed correctly, b) x (%) of essential steps and x (%) of important but non-essential steps performed correctly, or c) x (%) of any category step performed correctly. Responses were summed and averaged separately for essential and non-essential steps.

Results: SMEs’ averaged judgements of the contribution of each checklist step to patient outcome and/or safety resulted in 16/17 of the steps considered essential. Four SMEs selected a), all steps must be performed correctly; two SMEs selected b), a percentage (100) of essential steps and a percentage (75; 0) of important, non-essential steps must be performed correctly; and two SMEs selected c) a percentage (80; 90) of any category step must be performed correctly. When averaged, the performance standard for NMV was 97% of essential steps and 93% of non-essential steps. To achieve a passing score of at least 97% for essential steps, a learner would need to correctly complete all (16/16) essential steps. Because only one step was considered important but not essential, to achieve a passing score for non-essential steps, a learner would need to correctly perform this step. Therefore, to achieve a passing score, a learner would be required to correctly perform all checklist steps (17/17).Conclusion(s): The high performance standard must be considered in the context of the simulated environment which underrepresents the clinical domain. This combined with the intricacies of the clinical environment contributes to degradation of performance on actual patients. Thus, to promote safe patient care, a mastery level of performance in the simulated environment is imperative.