A Natural Language Processing and Machine Learning Approach to Identification of Incidental Radiology Findings in Trauma Patients Discharged from the Emergency Department - 20/02/23
, Hugh D. Dorris, MD c, Michael T. Kane, MD d, e, Benjamin Mervak, MD, CIIP f, Jane H. Brice, MD, MPH g, Benjamin Gray, BS h, Carlton Moore, MD, MS c, dAbstract |
Study objective |
Patients undergoing diagnostic imaging studies in the emergency department (ED) commonly have incidental findings, which may represent unrecognized serious medical conditions, including cancer. Recognition of incidental findings frequently relies on manual review of textual radiology reports and can be overlooked in a busy clinical environment. Our study aimed to develop and validate a supervised machine learning model using natural language processing to automate the recognition of incidental findings in radiology reports of patients discharged from the ED.
Methods |
We performed a retrospective analysis of computed tomography (CT) reports from trauma patients discharged home across an integrated health system in 2019. Two independent annotators manually labeled CT reports for the presence of an incidental finding as a reference standard. We used regular expressions to derive and validate a random forest model using open-source and machine learning software. Final model performance was assessed across different ED types.
Results |
The study CT reports were divided into derivation (690 reports) and validation (282 reports) sets, with a prevalence of incidental findings of 22.3%, and 22.7%, respectively. The random forest model had an area under the curve of 0.88 (95% confidence interval [CI], 0.84 to 0.92) on the derivation set and 0.92 (95% CI, 0.88 to 0.96) on the validation set. The final model was found to have a sensitivity of 92.2%, a specificity of 79.4%, and a negative predictive value of 97.2%. Similarly, strong model performance was found when stratified to a dedicated trauma center, high-volume, and low-volume community EDs.
Conclusion |
Machine learning and natural language processing can classify incidental findings in CT reports of ED patients with high sensitivity and high negative predictive value across a broad range of ED settings. These findings suggest the utility of natural language processing in automating the review of free-text reports to identify incidental findings and may facilitate interventions to improve timely follow-up.
Le texte complet de cet article est disponible en PDF.Plan
| Please see page 263 for the Editor’s Capsule Summary of this article. |
|
| Supervising editor: Stephen Schenkel, MD, MPP. Specific detailed information about possible conflicts of interest for individual editors is available at editors. |
|
| Author contributions: CSE and CM designed the study, obtained research funding, and performed the statistical analysis. All authors participated in developing the annotation guide and/or performing annotation of CT reports. All authors contributed to manuscript drafting and revisions. CSE takes responsibility for the paper as a whole. |
|
| All authors attest to meeting the four ICMJE.org authorship criteria: (1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND (2) Drafting the work or revising it critically for important intellectual content; AND (3) Final approval of the version to be published; AND (4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. |
|
| Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have stated that they have no commercial, financial, or other relationships in any way related to the subject of this article. This project was supported by the North Carolina Translational and Clinical Sciences Institute Pilot Grant Program (2KR1342002). The project and natural language processing software used were supported by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through Grant Award Number UL1TR001111. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. |
|
| Part of this work is scheduled to be presented in oral abstract form at SAEM 2022. |
|
| Readers: click on the link to go directly to a survey in which you can provide SWVBW5J to Annals on this particular article. |
|
| A podcast for this article is available at www.annemergmed.com. |
Vol 81 - N° 3
P. 262-269 - mars 2023 Retour au numéro
Article précédent
- Derivation and Validation of a Brief Emergency Department-Based Prediction Tool for Posttraumatic Stress After Motor Vehicle Collision
- Christopher W. Jones, Xinming An, Yinyao Ji, Mochuan Liu, Donglin Zeng, Stacey L. House, Francesca L. Beaudoin, Jennifer S. Stevens, Thomas C. Neylan, Gari D. Clifford, Tanja Jovanovic, Sarah D. Linnstaedt, Laura T. Germine, Kenneth A. Bollen, Scott L. Rauch, John P. Haran, Alan B. Storrow, Christopher Lewandowski, Paul I. Musey, Phyllis L. Hendry, Sophia Sheikh, Brittany E. Punches, Michael S. Lyons, Michael C. Kurz, Robert A. Swor, Meghan E. McGrath, Lauren A. Hudak, Jose L. Pascual, Mark J. Seamon, Elizabeth M. Datner, Erica Harris, Anna M. Chang, Claire Pearson, David A. Peak, Roland C. Merchant, Robert M. Domeier, Niels K. Rathlev, Brian J. O'Neil, Paulina Sergot, Leon D. Sanchez, Steven E. Bruce, Mark W. Miller, Robert H. Pietrzak, Jutta Joormann, Deanna M. Barch, Diego A. Pizzagalli, John F. Sheridan, Jordan W. Smoller, Steven E. Harte, James M. Elliott, Karestan C. Koenen, Kerry J. Ressler, Ronald C. Kessler, Samuel A. McLean
- The Use of Automated Tools in the Electronic Health Record Can Enhance Clinical Care and Humanism in Medicine
- Diane Kuhn, Michael Weiner
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?