Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index - 23/04/18

Doi : 10.1016/j.annemergmed.2017.08.005

Scott Levin, PhD ^a,^b,^⁎ , Matthew Toerper, BS ^a,^b, Eric Hamrock, MBA ^b, Jeremiah S. Hinson, MD, PhD ^a, Sean Barnes, PhD ^b,^c, Heather Gardner, RN ^a, Andrea Dugas, MD, PhD ^a, Bob Linton, MD ^a, Tom Kirsch, MD, MPH ^d, Gabor Kelen, MD ^a
^a Emergency Medicine, Johns Hopkins University, Baltimore, MD
^b StoCastic LLC, Baltimore, MD
^c Decision, Operations, and Information Technologies, University of Maryland, College Park, MD
^d National Center for Disaster Medicine and Public Health, Uniformed Services University, Bethesda, MD

^∗Corresponding Author.

Abstract

Study objective

Standards for emergency department (ED) triage in the United States rely heavily on subjective assessment and are limited in their ability to risk-stratify patients. This study seeks to evaluate an electronic triage system (e-triage) based on machine learning that predicts likelihood of acute outcomes enabling improved patient differentiation.

Methods

A multisite, retrospective, cross-sectional study of 172,726 ED visits from urban and community EDs was conducted. E-triage is composed of a random forest model applied to triage data (vital signs, chief complaint, and active medical history) that predicts the need for critical care, an emergency procedure, and inpatient hospitalization in parallel and translates risk to triage level designations. Predicted outcomes and secondary outcomes of elevated troponin and lactate levels were evaluated and compared with the Emergency Severity Index (ESI).

Results

E-triage predictions had an area under the curve ranging from 0.73 to 0.92 and demonstrated equivalent or improved identification of clinical patient outcomes compared with ESI at both EDs. E-triage provided rationale for risk-based differentiation of the more than 65% of ED visits triaged to ESI level 3. Matching the ESI patient distribution for comparisons, e-triage identified more than 10% (14,326 patients) of ESI level 3 patients requiring up triage who had substantially increased risk of critical care or emergency procedure (1.7% ESI level 3 versus 6.2% up triaged) and hospitalization (18.9% versus 45.4%) across EDs.

Conclusion

E-triage more accurately classifies ESI level 3 patients and highlights opportunities to use predictive analytics to support triage decisionmaking. Further prospective validation is needed.

Le texte complet de cet article est disponible en PDF.

Plan

Introduction

Materials and Methods

Setting and Selection of Participants

Methods of Measurement

Outcome Measures

Primary Data Analysis

Results

Limitations

Discussion

	Please see page 566 for the Editor’s Capsule Summary of this article.
	Supervising editors: Stephen Schenkel, MD, MPP; Robert L. Wears, MD, PhD^{^{† Dr. Wears died shortly before this article was accepted.}}
	Author contributions: SL, MT, AD, TK, and GK were responsible for the original development of the triage tool evaluated in this study. SL, MT, JH, and SB contributed to the study design. SL, MT, and SB analyzed the data and provided statistical advice. EH, JH, HG, and BL contributed to understanding of implications of the tool in practice, including insights into how the ESI is used at study sites. SL drafted the article and all authors contributed to its revision. SL takes responsibility for the paper as a whole.
	All authors attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
	Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org/). This study was supported by Agency for Healthcare Research and Quality award R21HS023641 and National Science Foundation (NSF) Engineering Directorate award SBIR 1621899. A patent application for electronic triage has been filed by Johns Hopkins University (JHU). Drs. Levin, Barnes, and Dugas and Mssrs. Toerper and Hamrock have been supported by an NSF Small Business Innovation Research award to commercialize electronic triage toward improving ED crowding. This award was granted to a JHU start-up company cofounded by Dr. Levin and Mr. Hamrock, with JHU as an equity partner.