In 2012, an institution-wide project, the “Happinovation” (Happy innovation), was initiated to enhance patient and provider happiness through process and hardware innovations. The “Happy Emergency Room Team” was formed as a satellite team for the overall project. The team focused on visualizing the ED process for providers and patients.

Specifically, 2 subprojects were implemented. One was a visualization project for providers—an electronic dashboard to develop visualization of ED performance status on wall-mounted monitors and PCs. The other was a visualization project for patients and their families—a wall-mounted electronic dashboard, kiosks, and tablets.

The multidisciplinary Happy Emergency Room Team included 6 physicians, 4 nurses, 1 administrator, 2 quality improvement team members, 2 consultants, and 2 designers. While hospital staff provided inputs, consultants and designers tried to see things from patients’ perspective, thus balancing the conclusion. Several rounds of discussions and debates took place before the first dashboard principle and design feature was created.

The team had agreed on the following 3 dashboard principles that define the characteristics of the dashboard.

The team adopted the following 3 design features for the dashboard:

Initially, multiple measures were suggested for the dashboard. Measures were chosen and categorized based on a “conceptual model of ED crowding.” The model consisted of 3 factors, among which the throughput factors, which reflect the internal process of ED care, were mostly demonstrated with the dashboard [13]. Among input factors, which are components that contribute to the demand for ED services, patient severity and measure of visits were included. Cautions on infection information were also included since they were well correlated with severity and patient allocation within the ED and hospital. Throughput factors were divided into 2 sections: structures and functions. Output factors, such as boarding and discharge data, were also included. Boarding pertains to the inability to admit a patient to a ward due to the lack of inpatient beds even through the patient is determined to be admitted.

We used a Windows server as the ED visualization platform to support various devices in the ED, including the dashboard and kiosks for patients and mobile devices. Our entire platform was developed and deployed on 2 Windows 2008 servers, each with a 1200-GB hard drive and 20-GB memory, and 2 4-core Intel Xeon 2.4 GHz processors. The servers queried the measurements mentioned above from the electronic medical records and picture archiving and communication system servers. A Windows communication foundation was used as a visualization tool (Figure 2).

We developed and updated the dashboard over 5 years. During the observation, the ED had gone through a Middle East Respiratory Syndrome outbreak, followed by a major structural and functional renovation [26]. The ED dashboard adopted such changes and evolved through the process.

Inclusion criteria were those currently working as ED physicians and triage or charge nurses and those using the dashboard. Participants were recruited from January 1 to February 10, 2018.

Participants responded to 20 questions on a 5-point Likert scale after completing a consent form. The first 10 items of the questionnaire were derived from the system usability scale (SUS) to investigate the usability of the dashboard [27]. The last 10 items were derived from the situation awareness index (SAI), which we composed based on the Situation Awareness Rating Technique (SART) [28]. The SAI aimed to assess whether ED physicians and nurses were using the dashboard to help them recognize the situation. The SAI includes the constructs of the SART as a whole, and small modifications were made to adjust it to fit the dashboard.

After completing the questionnaire, the participants received about US $8 as compensation for participating in the survey.

The SUS scores were interpreted using an adjective rating scale [29].

The SAI score was calculated using the following formula:

The SAI scores were also examined using descriptive analyses.

The semicircle-shaped indicator in the upper central area represents the complexity of the ED and presents the expected mean length of stay of a current patient in the ED. The semicircle borrows the scheme of the traffic light so that the user intuitively grasps the current situation of the ED.

A semitransparent colored square represents each section of the ED and matches the geographical layout presented in Figure 1. The squares are not visible on the actual dashboard. Small icons, such as that shown in the red circle, indicate patients, and they intuitively inform the user about the patient’s journey through the color corresponding to the patient’s process on the left.

The left side of the dashboard presents a summary of the patient process. This information allows the physician to set up a patient diagnosis strategy and the nurse to determine the order of various tests. The central area of the dashboard reflects the geographical layout of the ED. Here, individual patients’ specific information is displayed, such as real-time clinical processes. The right side of the dashboard presents the number of patients by zone, thereby enabling efficient distribution of medical labor (Figure 3).

Major difference between versions 1 and 2 of the dashboard is that the latter reflects a geographical change. In 2015, the ED of the study site underwent a Middle East Respiratory Syndrome outbreak, and a respiratory isolation area was established and operated. This structural change to the ED was reflected on the dashboard (Figure 4).

In version 3, a revision was made to improve usability. We updated the dashboard to change the overall color coding. By doing so, we were able to reflect the users’ suggestions, such as “difficulty to identify bed status” (Figure 4).

A major change in version 4 is the improvement of patient-specific information.

The circle next to the patient indicates the mapping of the Korean Triage and Acuity Scale score from 1 to 5, presented from red to green. Additionally, the rectangle next to the patient reflects “infection caution,” such as “air caution” and “blood caution” (Figure 4).

A total of 52 participants were recruited; 25 were physicians and 27 were nurses. In the physician group, 17 were males and 8 were females; in the nurse group, only 2 were males and 25 were females. The participants’ years of experience also varied; 8 had worked for less than 3 years, 10 had worked for 3-5 years, 19 had worked for 5-10 years, and 15 had worked for over 10 years (Table 1).

The SUS score of the ED dashboard system was 67.6 points (Table 2). The SUS score showed a slight difference between the 2 groups; the physician and nurse groups had scores of 67.5 and 67.69 points, respectively. We could interpret our result as follows: acceptability of the dashboard was “marginally high,” and the adjective rating was “OK-to-Good” [29]. These findings indicate that the participants used it very frequently. They also felt that this system was easy to learn and use.

The overall SAI score was 3.87 points, and the score of the physician group (3.95) was higher than that of the nurse group (3.80). The top 5 rated items were “concentration support” (4.15), “complexity representation” (4.02 points), “variability representation” (3.96 points), “information quality provided” (3.94 points), and “familiarity of dashboard” (3.94 points). However, the score for “division of attention” was 2.25 points (Table 3).