302 - Quality of ED LP Procedure Note Documentation

Poster Number: 302
Publication Number: 302.347

Rachelle El Helou, Boston Children's Hospital, Boston, MA, United States; Amir Kimia, Boston Children's Hospital, Boston, MA, United States; Marvin B. Harper, Harvard Medical School, Boston, MA, United States; Assaf Landschaft, Boston Children's Hospital, Bergisch Gladbach, Nordrhein-Westfalen, Germany; Natasha Joglekar, Boston Children's Hospital, Boston, MA, United States; Al Ozonoff, Harvard Medical School, Boston, MA, United States; Jennifer Kupiec, Boston Children's Hospital, Boston, MA, United States; Mark L. Waltzman, Harvard Medical School, Boston, MA, United States; Mara Alexeev, Boston Children's Hospital, Brookline, MA, United States

Background: Procedure documentation provides important clinical, medico-legal, and potential research information as part of a patient’s medical record. The use of templates may improve completeness but may lack accuracy if default options are chosen.

Objective: 1) To assess the rate of procedure elements within Emergency Department (ED) documentation as a measure of completeness; and 2) to assess the accuracy of template-based documentation for elements that have a structured reference to medication orders and administration.

Design/Methods: We performed a retrospective cross-sectional study of all ED encounters for children age < 21 years who presented between 2017-2019 to a tertiary care pediatric center and had a lumbar puncture (LP) attempt documented.
We developed a Natural Language Processing (NLP) model to identify LPs from unstructured data (clinical note text), using a sample of manually reviewed notes from one year of data. We randomly selected 70% of the notes to train the model and 30% of notes for validation/testing. Comparing the NLP model classification to the manual classification we calculated the sensitivity and specificity of the model for identifying notes describing an LP attempt. We then applied the NLP model to all ED notes during the study period and reviewed the output manually.
We also used structured data in parallel (e.g., physician orders, laboratory results, billing codes) to flag cases for manual review.
For accuracy, we compared elements documented in the clinical narrative to computerized physician order entry (CPOE), a measure involving physicians, nurses and pharmacists per order.

Results: In the validation cohort, the NLP model identified 118/122 LPs identified by manual review: Sensitivity 96.6% (95% CI 93.4-99.9%) and specificity 97.9% (95% CI 96.4-99.3%). The NLP model was then applied to all 117,981 ED encounters during the study period. providing 2916 notes for manual review, of which we identified 1094 patient encounters notes with an LP attempted. n additional 713 cases flagged by structured data resulted in additional 19 cases, for a total of 1113 LP attempts (0.63% of encounters).
Completeness of procedure elements documented in the note (regardless of template use) are presented in Table 1.
For accuracy of template-based documentation we present comparison between documented elements and CPOE data in Table 2.Conclusion(s): Many procedure elements are missing in the ED clinical documentation. Template use may offer some benefit in terms of completeness, but default selections may provide inaccurate descriptions of some elements.
Table 1: Completeness of procedure elements documented
Table 2: Comparison between template documentation and CPOE data