Background

Electronic health record (EHR) notes are critical to medical decision making but have become unwieldy to manually sift through due to their many aims: communication, compliance, and billing. Consequently, clinicians spend more time navigating EHRs than with patients, leading to burnout.

Prior analysis focuses on static completed notes, and few have explored granular reading and writing activities. Clinicians describe time-varying purposes of EHR use: rapid sense-making of new patients and re-familiarization with existing patients. We introduce a framework to dynamically retrieve relevant notes at different points during a patient’s visit to streamline discovery of relevant history and satisfy changing information needs.

Methods

We analyze 48,192 note-writing sessions in the high-acuity emergency department (ED). We design a binary classification task of whether to retrieve a past note during the next session given the current writing context. Audit logs, which track reading and writing activity in the EHR, provide labels for machine learning. We encode patient and clinician information, note metadata, and a bag of words representation for note text. We train logistic regression models to predict the top ten most relevant notes for each session.

For quantitative evaluation, we measure precision@10, recall@10, and AUC. Additionally, two physicians conducted chart review for five random patients. Each clinician was given the patient's age, sex, and chief complaint, and assigned relevance for past notes. We compared their relevance judgments with the predicted relevant notes.

Results

Our learned models’ quantitative performance surpassed that of deterministic baselines and achieved an AUC of 0.963. For the qualitative evaluation, for four out of five visits, the top ranked predicted note surfaced during the first session of the patient’s visit was determined to be directly helpful to the patient’s current presentation by both clinicians. For three of four visits, showing only the top ranked predicted note was sufficient information that both clinicians needed to proceed with patient care.

Conclusion

We present a novel framework that updates retrieval suggestions dynamically as clinical context changes, thus helping clinicians find relevant information more quickly at different points during the patient’s visit. It also surfaces difficult-to-find information and suggests patterns a clinician may not have initially deemed relevant, prompting them to reassess a diagnosis they initially anchored on and evaluate potential biases. Demonstrating that our methods perform well in the demanding ED setting is a promising proof of concept that can translate to other clinical settings and data modalities.