The data were collected over 3 months, from May 2, 2016 to August 2, 2016, during the field trial of the VISIT project at the refugee center in Bad Fallingbostel. The center is composed of two camps with a total capacity of 2000 persons. The German Red Cross (DRK) and the Johanniter-Unfallhilfe (JUH) run each camp, respectively. After study approval, the medical personnel in both camps were quickly trained to use the DPP technology. The study group consisted of two reception lines which were on duty every day, and during the study period a total of 495 data sets were collected. The Bundeswehr provided support for both camps with regard to medical personnel and medical documentation.

Medical personnel filled in the standardized protocol during medical examinations, and signs and symptoms, the clinical diagnosis, and sociodemographic characteristics (eg, year of birth, nationality and gender) were collected from the patients. The DPP technology was used consistently; however, personal information was not stored in the DPP system by paper design (no digital pattern) in order to protect patient privacy. Thus, only anonymous data were processed.

For each patient, one protocol (sheet of digital paper) was completed. Each protocol consisted of 277 fields into which data could be entered (either text or symbols, eg, check marks). Of course, for any given patient, many of the possible fields were left blank as they did not pertain to that specific patient.

At the end of each day, collected medical data were then transmitted, via a protected transfer, to central servers in Koblenz and Ulm, Germany. A data subset concerning syndromic surveillance was encrypted and transmitted to the Bundeswehr University in Munich and to the Deployment Health Surveillance Capability (DHSC) for further analysis by the specialists of the Bundeswehr Medical Service Headquarters as well as the DHSC. The DHSC provides epidemiological surveillance for the North Atlantic Treaty Organization (NATO) deployment areas and is a satellite branch of the NATO Centre of Excellence for Military Medicine (MILMED COE) in Budapest, Hungary.

Documentation time was the difference between first and last stroke on the paper, which includes clinical examination and translation. In the initial phase of the study, the medical documentation process was not uniform. In one medical treatment facility, only the documentation sheet accompanied the patient. This resulted in entries being produced by different pens on one documentation sheet with multiple XML-files. As for all the other documentation processes, a unique digital pen and a documentation sheet accompanied the patient, resulting in a single XML-file per person.

The data set was developed by the DHSC and the Bundeswehr Medical Service Headquarters. The implemented data set is a subset of the German national data set regarding anything emergency department–related that is relevant for primary health care [19]. This collection was complemented by adapting the syndrome-specific parameter, resulting in a total of 64 sets of data. For further statistical analysis, the diagnosis was matched to the 10^th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) code, according to the World Health Organization (WHO) list.

The VISIT protocol sheets are available upon request from the authors.

With the goal of optimizing the processing time, we asked which fields had the strongest effect on total examination time for a patient. For this analysis, we grouped the 277 fields into 70 categories, pooling fields covering different aspects of the same topic into one category. For example, the category “symptoms eye/ear” consists of five fields covering symptoms from conjunctival hemorrhaging to otalgia (see upper right corner of section 4 in Figure 1). Then, a Mann-Whitney U test was conducted to compare the documentation time of reports where a specific category was completed (at least one of the fields filled out) with those where this specific category was empty.

For a more detailed analysis, we also asked which fields required the most time for completion. This analysis is possible since, for each field, the time stamps of the first and the last entry into the field are stored. This is not corrected for cases where the examiner goes back and forth between fields. If, for example, the examiner starts filling out field A, then makes entries into field B, and then finally goes back to field A and completes it, then the time stamp would count the whole time from the first change in field A to the last. In this sense, our analysis captures those fields which are the most complicated to fill out.

Finally, the design of the documentation sheet determines the speed at which the form can be completed. As the digital pen stores a timestamp for each entry, the sequence of documentation can be analyzed. The total time of documentation for each sheet was set at 1, so each timestamp was normalized and expressed as a value between 0 and 1. The fields of the documentation sheet were then grouped into nine sections (Figure 1). For each sheet, the normalized time of the first entry into each section was noted, thus, the order in which the sections were worked on can be analyzed. For example, the median of these normalized times for a section expresses the time point (relative to the total time needed for the full medical examination) where half of the examiners have started on that section.

The XML files were imported into Microsoft Excel (version 15.29) and prepared for further analysis. Statistical analysis was performed using standard statistical software (SPSS version 24 for Windows), depicted as box-and-whisker plots. The effect of different pens on time of documentation was analyzed with the Mann-Whitney U test.

Prior to the investigation, ethical clearance was obtained from the Ethics Commission of the Ludwig-Maximilians-University of Munich (143-16). Verbal informed consent was received from the refugees.

Altogether, 495 data sets (XML files of individual documentation sheets, each documenting one patient) were generated. However, after an initial phase of training personnel and optimizing the documentation process, the procedure was adapted. During the initial phase, digital pens were available at each station. This ended up causing multiple entries per sheet being made with different digital pens, which caused problems with respect to merging data sets and data integrity. This technical problem was solved when the digital paper and pen accompanied the patient during the whole medical examination process. As a result, 74 data sets, generated during this initial phase with more than one digital pen ID, were excluded from further analysis, leaving 421 data sets to be analyzed.

The total time of documentation (time difference between first and last stroke on a documentation sheet) ranged between about 90 seconds and over 2 hours, with a median of 41:41 minutes and an interquartile range (IQR) of 29:40 minutes (mean 45:02; SD 22:28 min). The distribution of these times is skewed to the right, and the Kolmogorov-Smirnov test rejects the null hypothesis of the normal distribution (P=.03). The median time of documentation for individual pens varied from 31:44 min to 48:20 min, and the number of completed forms per pen ranged between 28-62. A summary of documentation times per pen is presented in Figure 2; however, the time of documentation for the different pens showed no significant difference, according to the Kruskal-Wallis test (P=.54).

Overall, 21 out of 70 categories have a significant effect on documentation time. The data are shown in Table 1. The documentation of vital signs (eg, blood pressure, blood oxygenation, heart rate, temperature, and blood glucose) had the highest impact on documentation time, with 5 out of the 21 significant categories belonging to this group. Additionally, anything connected to the calculation of the Glasgow Coma Scale also had a significant effect on overall documentation time. Interestingly, sheets where eye, ear or dermatological symptoms were documented had a significantly shorter completion time. A Mann-Whitney U test was conducted to compare the documentation time of reports with entries in a category and reports without entries in the same category. Significant effects were considered where P<.05.

For each field, the time needed to fill it was calculated. Table 2 shows the nine fields with the highest median times (4 seconds or more) for completion. The large free-text field called clinical findings, or therapy, or measures, needed the most time (median 8:06 min; IQR 23:56 min). The next four most time-consuming fields were also free-text fields: patient history, proceeding, diagnosis and discharge.

The data presented in Figure 3 shows the sequence of documentation. Almost every medical examiner started with the first section (patient personal data and manner of transfer to the medical center). Section 2 (patient vital signs and medical history) usually came next. Following that, the diagnostic sections 3, 5, 6, and 7 were started more or less interchangeably and at the same time. Interestingly, half of the medical examiners made their first entry into section 4 (symptoms) only after the other diagnostic sections, namely at roughly the same time as the concluding sections 8 (diagnosis with ICD code; this was often the last section to be started) and 9 (further proceeding, discharge or transfer). It should be noted, however, that these times (except for section 1) displayed large variances as seen in Figure 3.

Data are shown in Table 3. The categories more frequently completed (i.e. more than 90%) were patient data (99.8%), receiving facility (99.5%), time of admission (99.3%), proceeding date (99.0%), status (98.1%), first ICD-10 code (97.1%), patient history (96.2%), nationality (96.0%) and self-admission (95.2%). 17 of the 70 categories had no entry in all the analyzed data sets; 11 had only one entry.

Taking handwritten notes is one of the least technical ways to collect medical data. Generations of medical doctors have been trained to use paper-based documentation and to note their clinical findings [20], however, the enormous medicolegal need for documentation poses a considerable challenge [21]. Additionally, paper-based documentation should be phased out and instead be digitized for long-term storage, integration into hospital information systems, and for further analysis with respect to quality management [22].

The speed of handwriting is limited and slows down the documentation process even further, as the use of other devices (eg, smartphones, typewriting or keyboard-based typing on personal computers) increases [23]. Therefore, forms are needed to ensure both reliable data collection as well as to speed up the process of filling them in through structured protocols. To overcome the obstacles of paper-based documentation, new technologies are emerging to support the documentation process. Speech recognition [24,25], automated data recording [26,27], and DPP technology [14] are examples of this. Handwriting recognition is improving and being implemented in standard office software suites, but it can still lead to errors [28,29]. The same holds true for speech recognition, although further training is needed as well as a relatively silent ambience around the user [23]. Voice recognition technology often shows no advantage over manual transcription [30]; however, the rise of artificial intelligence (AI) or deep machine learning in combination with voice recognition is promising new technologies to further speed up the documentation process.

In the case of mass casualties during a crisis, it is crucial to increase the speed of medical documentation without losing quality. Processing speed and data quality should be as high as possible to cope with a large number of patient casualties [31]. In this context, valid data for health surveillance are of utmost importance [11], but the recorded information is also of interest for evaluating complex interventions [32]. The data imported into automated health surveillance systems should be as simple as possible, and in the best case it should be an automatic byproduct of the legally required patient documentation. DPP technology has proven to be a good combination of both paper-based records and electronic documentation [14,33].

No statistical difference was identified between the digital pens used. Thus, both reception lines worked equally effectively. Apart from the first week, use of the technology was very reliable. The pen strokes from the 421 collected data sets allowed a reconstruction of the sequence in which the fields were filled in the form. The graphical representation of the sequence showed that the design of the report seems to be appropriate. As stated above, the diagnostic sections were recorded earlier than the final diagnosis field, including the ICD-10 code that was often the last field to be worked on. A request submitted to the medical documentation team revealed that matching the diagnosis to the ICD-10 code required additional time, and thus the ICD-10 code was added as the last step of medical documentation. This personal observation is backed up by the fact that the step to match the diagnosis with the ICD code had a significant impact on overall documentation time. Additionally, the diagnosis and proceedings fields were among the five fields with the longest documentation time.

Another time-consuming block involved everything related to the examination of the patient. The clinical findings, or therapy, or measures free-text field required the most time for documentation per field. Furthermore, the recording of vital signs had the highest impact on documentation time. A total of 98% of the clinical findings, or therapy, or measures field, and approximately 60% of the significant vital sign fields, were filled in. Thus, the optimization of this step has great time-saving potential.

The fields with the most data entries are assumed to be more relevant. By contrast, fields with little or no data entry must be considered less important and are thus probably dispensable in further studies. The recorded pen strokes clearly showed that data sets were recorded during the patient examination process and not afterwards. It has been shown in previous studies that the completeness of data sets is superior to any kind of retrospective documentation [34,35]. Furthermore, total documentation time is reduced using DPP technology compared to paper-based documentation, where secondary data entry in a computer system is necessary and errors may occur. The time required to transfer all the data sets from a pen is less than 10 min. Given that 30 min per form are needed for secondary data entry, the time saved is estimated to be 210 hours in total, which is approximately 26 days of work, or more than one month, for one documentation assistant. Thus, DPP technology has a return on investment that will increase the longer this technology is used. Additionally, the time-saving effect in epidemiological surveillance is crucial, as with DPP technology more surveillance reports can be generated in less time compared to traditional paper-based systems [32].

The strength of this study lies in its description of how DPP technology can be used to analyze the speed of medical documentation, as well as the behavior of the medical documentation team [36]. In particular, it was possible to conduct detailed analysis of the documentation sequence and speed with regard to the form used. Furthermore, the study highlighted the possibility of training medical personnel very quickly to use DPP technology. However, the main limitation of this study lies in the limited timeframe of data capture. It could be seen as a longitudinal cross-sectional study with the inherent weaknesses of this study design, but to acquire more valid data, it is necessary to analyze more data sets. Furthermore, it may be of interest to see whether a learning curve is detectable as users adapt to this new technology. A cohort design is likely to produce more reliable and precise data regarding these questions.

We conclude that medical documentation using DPP technology leads to an increase in documentation speed and quality. Further analysis of more data sets will allow optimization of the documentation form used. DPP technology is an effective tool for the medical documentation process in refugee camps.