%0 Journal Article
%@ 2291-5222
%I JMIR Publications
%V 9
%N 10
%P e32301
%T Complete and Resilient Documentation for Operational Medical Environments Leveraging Mobile Hands-free Technology in a Systems Approach: Experimental Study
%A Woo,MinJae
%A Mishra,Prabodh
%A Lin,Ju
%A Kar,Snigdhaswin
%A Deas,Nicholas
%A Linduff,Caleb
%A Niu,Sufeng
%A Yang,Yuzhe
%A McClendon,Jerome
%A Smith,D Hudson
%A Shelton,Stephen L
%A Gainey,Christopher E
%A Gerard,William C
%A Smith,Melissa C
%A Griffin,Sarah F
%A Gimbel,Ronald W
%A Wang,Kuang-Ching
%+ Department of Public Health Sciences, Clemson University, 501 Edwards Hall, Clemson, SC, 29634, United States, 1 864 656 1969, rgimbel@clemson.edu
%K emergency medical services
%K prehospital documentation
%K speech recognition software
%K natural language processing
%K military medicine
%K documentation
%K development
%K challenge
%K paramedic
%K disruption
%K attention
%K medical information
%K audio
%K speech recognition
%K qualitative
%K simulation
%D 2021
%7 12.10.2021
%9 Original Paper
%J JMIR Mhealth Uhealth
%G English
%X Background: Prehospitalization documentation is a challenging task and prone to loss of information, as paramedics operate under disruptive environments requiring their constant attention to the patients. Objective: The aim of this study is to develop a mobile platform for hands-free prehospitalization documentation to assist first responders in operational medical environments by aggregating all existing solutions for noise resiliency and domain adaptation. Methods: The platform was built to extract meaningful medical information from the real-time audio streaming at the point of injury and transmit complete documentation to a field hospital prior to patient arrival. To this end, the state-of-the-art automatic speech recognition (ASR) solutions with the following modular improvements were thoroughly explored: noise-resilient ASR, multi-style training, customized lexicon, and speech enhancement. The development of the platform was strictly guided by qualitative research and simulation-based evaluation to address the relevant challenges through progressive improvements at every process step of the end-to-end solution. The primary performance metrics included medical word error rate (WER) in machine-transcribed text output and an F1 score calculated by comparing the autogenerated documentation to manual documentation by physicians. Results: The total number of 15,139 individual words necessary for completing the documentation were identified from all conversations that occurred during the physician-supervised simulation drills. The baseline model presented a suboptimal performance with a WER of 69.85% and an F1 score of 0.611. The noise-resilient ASR, multi-style training, and customized lexicon improved the overall performance; the finalized platform achieved a medical WER of 33.3% and an F1 score of 0.81 when compared to manual documentation. The speech enhancement degraded performance with medical WER increased from 33.3% to 46.33% and the corresponding F1 score decreased from 0.81 to 0.78. All changes in performance were statistically significant (P<.001). Conclusions: This study presented a fully functional mobile platform for hands-free prehospitalization documentation in operational medical environments and lessons learned from its implementation. 
%M 34636729
%R 10.2196/32301
%U https://mhealth.jmir.org/2021/10/e32301
%U https://doi.org/10.2196/32301
%U http://www.ncbi.nlm.nih.gov/pubmed/34636729