Emergency response apps, commonly smartphone based, are increasingly being used to identify and dispatch volunteer first responders (VFRs) to the location of a medical emergency [1]. Automated dispatch algorithms generally rely on a simple estimated time of arrival (ETA) calculation based on the locations of the VFRs and the incident as well as the known modes of transport. A key aspect lacking in these algorithms is a consideration of the likelihood of response; for instance, given a set of potential VFRs with equivalent ETAs, which subset should be alerted to maximize the likelihood of response? The automated dispatch of VFRs to medical emergencies is suboptimal owing to a large percentage of alerts wasted on VFRs with shorter ETA but a low likelihood of response. This results in delays until a volunteer who will actually respond can be identified and dispatched. Using actual demographic and response data taken from a 12-month study of 112 VFRs alerted to respond to opioid overdose emergencies, we applied a series of analytical methods and advanced classification models to learn and predict volunteer response behaviors. Our findings can be used to improve dispatch algorithms in VFR networks to optimize dispatch decisions and increase the likelihood of timely emergency responses.

A medical emergency is an acute injury or illness that can result in death or long-term health complications [2]. Some common medical emergencies include out-of-hospital cardiac arrest (OHCA), severe trauma, opioid overdose, and anaphylaxis. OHCA is a leading cause of death worldwide [3], with a poor survival rate (only 5.6% in adults) [4]. Major trauma is the sixth leading cause of death worldwide [5,6]. Opioid overdose is a severe public health problem that has been consistently rising for the past 20 years and in the United States is the leading cause of accidental death [7]. The incidence of anaphylaxis ranges from 1.5 to 7.9 per 100,000 population per year in Europe [8,9].

The immediate provision of first aid is crucial in lowering mortality and improving long-term prognosis, particularly in regard to OHCA [10-12] and opioid overdose events [13]. Emergency medical services (EMS) are the primary first aid provider [14,15], but EMS response times vary significantly among countries and geographies [16,17]. Interventions to achieve faster response times include the deployment of automatic external defibrillators (AEDs) in public places [18-21] and the establishment of local networks of VFRs [22-30]. Recently, there was a concerted effort to use smartphone apps for faster emergency response, such as PulsePoint, HelpAround, Heartrunner, and UnityPhilly. An extensive review of emergency response apps can be found in the study by Gaziel-Yablowitz and Schwartz [31].

An emergency response community (ERC) [32], a subtype of a VFR network, is a social network of patients who are prescribed to carry life-saving medication for themselves and can potentially help other patients who are without their medication in a medical emergency. Two projects that apply the ERC approach are the subjects of recent field studies: EPIMADA, which focused on patients at risk of anaphylaxis and their parents [33]; and UnityPhilly, which focuses on people who have experienced an opioid overdose [34].

Once a person becomes a volunteer, they are expected to respond if available when a relevant event occurs. However, the actual rates of response to emergency alerts are far from 100%. Brooks et al [35] reported a response rate of 23% among PulsePoint volunteers. In a recent study, the willingness of cardiopulmonary resuscitation (CPR)–trained bystanders to respond to an OHCA event was 46.6% [36]. Another study analyzed barriers to receiving notifications and reported that 32% of the responders who were sent notifications did not receive the notification because, for example, they were away from their device (21%), their device was switched off (8%), or their device was out of network range (4%) [35]. Stress levels among responders varied for different medical conditions, different locations, and different demographic groups [37]. Younger age, higher education level, shorter time since the last CPR training, and cardiac arrest event in a public location were good predictors of bystanders’ greater willingness to perform CPR. The main reasons for not performing CPR were panic, the perception of bystanders that they are not able to perform CPR correctly, and a fear of hurting the patient [38]. Familial experiences of receiving CPR were associated with an increase in responders’ willingness to perform CPR [39]. The UnityPhilly study, which established a network of volunteers to provide naloxone to those experiencing an opioid overdose, reported that 17% of the alerted volunteers accepted the alert, and 11.9% of the alerted volunteers arrived at the scene [34].

Profiling is “the process of generating profiles from obtained data, associated to one or multiple subjects” [49]. Profiling of people is widely used in several areas, such as targeted advertising [50], donation solicitation [51], and volunteer recruitment [52]. Elsner et al [22,49] proposed to use the profiling of volunteers in dispatch algorithms to enhance the prediction of the volunteers’ position, trajectory, and constraints. In this study, we used classification techniques to generate different behavioral profiles of volunteers that serve as independent variables for predicting responses to alerts.

The challenge of improving volunteer dispatch speed and response rates is recognized in fields ranging from food rescue operations [53] to OHCA response in which the optimization of the responder network is now taking center stage [54,55]. Studies such as the one by Gregers et al [56] have attempted to determine the optimal number of responders to dispatch, yet such studies base response viability solely on current ETA with no consideration of responder history or other characteristics that could improve responsiveness. Currently used dispatch algorithms that select volunteers based on their ETA without considering the likelihood of response may be suboptimal owing to a large percentage of alerts wasted on VFRs with shorter ETA but a low likelihood of response. We build on prior work on VFR optimization by presenting a novel approach for predicting whether a VFR will respond to, or ignore, a given alert. As such, the enhanced algorithm reduces the time that the system unnecessarily spends waiting for a response from volunteers who are likely to ignore the alert. The amount of time wasted depends on the specific dispatch algorithm; for example, in UnityPhilly trials, the system waited 2 minutes before dispatching a substitute volunteer. A faster dispatch of substitute volunteers has the potential to reduce the response time of the VFR network as a whole and improve its effectiveness. However, overdispatch of more VFRs than necessary to secure an effective emergency response can have a negative impact on future willingness to respond [48].

We used data from the UnityPhilly study that piloted a smartphone-based app for requesting and providing ERC assistance to those suspected of experiencing an opioid overdose in the neighborhood of Kensington, PA, over 12 months from March 1, 2019, to February 28, 2020. Kensington has Philadelphia’s highest concentration of overdose deaths and is also home to Prevention Point Philadelphia, which is a city-sanctioned syringe exchange program that also distributes naloxone and provides naloxone training. Recruitment occurred via face-to-face screening at Prevention Point’s drop-in center, Prevention Point’s substance use disorder treatment van, street intercepts, and chain referrals from enrolled participants. The inclusion criteria for participants were that they lived, worked, or used drugs within 4 zip codes around the Kensington neighborhood (19122, 19125, 19133, and 19134); possessed a smartphone with a data plan; were willing to have location and movements tracked via an app; were willing to carry naloxone; and were aged ≥18 years. Sampling purposefully targeted a mix of members of the Kensington community who used opioids nonmedically in the past 30 days and those who reported no nonmedical opioid use in the past 30 days. The study recruited 112 volunteers who were almost equally divided between people who reported opioid use in the past 30 days at baseline (n=57, 50.9%) and community members, that is, people who reported no opioid use at baseline (n=55, 49.1%).

At a research storefront in Kensington, the study enrollment procedure included obtaining written informed consent, the recording of contact information, structured baseline interviews, app installation and training, and naloxone distribution and training. During the informed consent procedure, participants agreed to participate in a baseline interview, monthly follow-up interviews, and brief surveys after overdose incidents. Project staff installed the app on the participant’s smartphone and provided app training, which included watching an animated training video explaining app use and practicing using the app to send and receive alerts with project staff. Naloxone training included recognizing the signs of opioid overdose, practicing rescue breathing on a CPR dummy, and demonstrating how to administer intranasal naloxone. All participants received a kit containing 2 doses of intranasal naloxone. The UnityPhilly app enabled them to report opioid overdose events and to receive notifications about opioid overdose events reported by other members in their proximity. Participants received US $25 in cash for the baseline interview and US $5 for each completed follow-up monthly interview or incident survey. No compensation was offered or given for the use of the app to signal or respond to overdose incidents. More details about the study are available in prior publications [34].

The data used for this analysis consist of 4 components (Textbox 1 and Figure 2).

Of the 112 volunteers recruited to UnityPhilly, 27 (24.1%) were completely inactive as either signaler or responder (ie, they did not send or respond to a single alert). Of the remaining 85 volunteers, 80 (94%) received at least 1 alert and were defined as responders, and 52 (61%) who signaled at least 1 event were defined as signalers (many volunteers served in both roles). Figure 3 presents the distribution of responders and signalers.

Events that were canceled by the signaler for any reason were considered false alarms. For this analysis, we excluded these events because we were not able to distinguish between alarms ignored by the responder and alarms that were canceled before the responder had a chance to respond. Figure 4 describes the sample.

We used alerts as a unit of analysis.

We used multiple analytical methods to classify the behavior of each volunteer identified as being in the proximity of an overdose event. We integrated data on specific volunteers and events into the dispatch algorithm in such a way that for each dispatched volunteer who is most likely to ignore the alert, an additional volunteer is dispatched right away (if available), until the maximum number of volunteers to be dispatched is reached, or no more volunteers are available. Volunteers for whom the algorithm predicts a low probability of response are still dispatched and thus are given the chance to respond. Figure 5 depicts this process.

We tested 4 models, based on different configurations of variables, to predict whether a given responder is likely to respond to a given event (Textbox 2 and Table 1).

The classification analysis for all models was conducted using four classification algorithms suitable for binary classification: (1) the J48 decision tree algorithm, which is an extension of the C4.5 algorithm, implemented in Weka software (University of Waikato) used in the research; (2) random forest; (3) neural network (multilayer perceptron); and (4) logistic regression. The J48 algorithm creates univariate decision trees for classification and provides effective alternatives to other classification methods. The choice of the best classification model is based on the combination of different evaluation metrics. The main interest was to identify the model that succeeds in correctly classifying any answer class.

We used 4 evaluation metrics: accuracy, F-score, precision, and recall. Accuracy is the overall percentage of correctly classified instances. The F-score is the harmonic mean of the recall and precision metrics and can take values ranging between 0 (none of the instances were correctly classified) and 1 (all instances were correctly classified). Precision is the percentage of true positively classified instances out of all positively classified instances. Recall is the percentage of positively classified instances out of all positive instances. The best way to explain the trends found in this analysis is to explain the differences in the recall metric among the different classification algorithms and among the different classes.

Because of the relatively small overall number of cases in the data set, we did not use a percentage split for the training and test sets for models 1 to 3; instead, we used a cross-validation option with 10 folds. Model 4 includes the additional synthetic dichotomous variable called frequent responder that reflects the previous behavior of the responder. The variable is dynamically updated; therefore, a responder can change their behavior several times throughout the research period—from being active to inactive or vice versa. As the frequent responder variable cannot be treated as an independent sequence of values and behavioral patterns that must be preserved, there is no option to use cross-validation for classification analysis. For this reason, we split the data set into a training set with 66.9% (664/993) of the data and a test set with 33.1% (329/993) of the data.

All 4 algorithms were used for a binary classification task in a baseline analysis that included only the events and alerts data (model 1 in Table 1). The obtained results provide the baseline for comparison with the additional data related to the responder’s previous experience data (models 2, 3, and 4 in Table 1). We claim that building a model that considers the responders’ behavioral characteristics can improve the use of the dispatch algorithm. In this kind of analysis, precision in predicting nonresponse is more important than precision in predicting response because in the former case a mistake will delay the dispatch of a substitute responder, whereas in the latter case a mistake will result in the dispatch of too many volunteers.

The comparison between all classification techniques and all evaluation metrics for the 4 models is presented in Multimedia Appendix 1.

All study procedures were approved by the Drexel University Institutional Review Board and registered with ClinicalTrials.gov (NCT03305497). Study enrollment included written informed consent. All data used for this research were deidentified. Participants received US $25 in cash for the baseline interview and US $5 for each completed follow-up monthly interview or incident survey. No compensation was offered or given for use of the app to signal or respond to overdose incidents.

Table 2 presents the characteristics of overdose events.

Table 3 presents the distribution and correlation of the responders’ characteristics. Cramér V was used for categorical variables, and Spearman ρ was used for ordinal variables. ANOVA tests for age differences among the different subgroups of categorical or ordinal variables did not reveal any significant differences at the 5% significance level.

Significant correlations were found between gender and homelessness (P<.001) as well as between history of witnessing an opioid overdose and history of administering naloxone (P<.001).

Textbox 3 and Figure 6 present how the alerted volunteers responded (true alarms only; n=993). Responders could change their decision.

Figure 7 presents the ability of each model to predict the responder’s behavior. To compare model 4 with the other models, all models were tested using the test set of alerts (n=329).

For the test set, model 4 provided the best classification accuracy both overall and for ignored alerts. Model 3 provided the same classification accuracy for ignored alerts, slightly lower accuracy overall, and lower accuracy for answered alerts. Model 2 provided the best classification accuracy for ignored alerts; however, its accuracy was lower overall and significantly lower for answered alerts. Model 1’s classification accuracy was the lowest.

Figure 8 presents the ability of models 1 to 3 to classify the responder’s behavior, using the full data set (n=993).

For the full set, model 3 provided the best classification accuracy. Model 2 had similar accuracy for ignored events and lower accuracy both overall and for answered events. Model 1’s classification accuracy was the lowest. Model 4 was not tested with the full set because the construction of the frequent responder variable requires training.

Figure 9 presents the J48 decision tree for model 4 for the test set.

The analysis of the classification tree reveals 5 possible routes to the response result: infrequent responders aged >54 years, frequent responders who administered naloxone <20 times, male frequent responders who administered naloxone 21 to 40 times, fully employed female frequent responders who administered naloxone 21 to 40 times, and unemployed female frequent responders who administered naloxone 21 to 40 times in situations where the distance to the scene was <272 meters.

We have to remember that the overall accuracy is not very high and that there are false-positive and false-negative statistical errors in the classification output. A false-positive error occurs when the ignored alert is classified as a responded alert, and a false-negative error occurs when the responded alert is classified as an ignored alert.

Substitute responders (responders who were not initially selected by the algorithm) were used in 73.4% (138/188) of the events. Substitute responders received 33.6% (334/993) of the alerts. Figure 10 presents the lengths of the delays (in min) before substitute responders were dispatched.

Table 4 presents the analysis of differences between alerts that were ignored and alerts that resulted in some responses (en route, no go, or on scene).

Significant differences between responded alerts and ignored alerts were found for the following variables: gender (higher response rate by male volunteers; P=.05), naloxone carriage adherence (P<.001), employment (higher response rate by volunteers who were unemployed; P<.001), age (slightly higher average age among volunteers who responded; P=.003), the number of previous alerts (higher among volunteers who responded; P=.003), previous false alerts (higher among volunteers who responded; P=.003), previous false alerts by the same signaler (lower among volunteers who responded; P=.02), previous responses (higher among volunteers who responded; P<.001), and previous responses to false alerts (higher among volunteers who responded; P<.001).

To the best of our knowledge, this is the first study that integrates the predictions of VFRs’ response behavior into dispatch algorithms. We found that volunteers’ past response behavior is the most influential predictor of future response behavior. Our findings suggest that the behavior-based approach can be applied to VFR dispatch to achieve a better response rate of the network as a whole.

Model 1 used events and alerts data only and completely ignored any volunteer-related data. The ability of this model to predict volunteers’ behavior was extremely low—only approximately 0.9% (3/329) of the responded alerts were classified correctly. This model would lead to the dispatch of all available volunteers, resulting in burnout, a “diffusion of responsibility,” and low willingness to respond. We conclude that this model is unacceptable.

Model 2 assumed that the volunteers are completely anonymous and that the algorithm knows only their previous behavior in the VFR network. Using these data as well as events and alerts data, model 2 correctly predicted 84.2% (277/329) of the cases in the test set in which responders ignored an alert. Lower prediction accuracy for the full data set (738/993, 74.3%) can be explained by the inclusion of early events for which the model did not have enough data about previous behavior. The ability of model 2 to predict that a volunteer will respond to an event was lower (153/329, 46.5% for the test set and 570/993, 57.4% for the full data set). On the one hand, this model is expected to improve the response rate of the network, but, on the other hand, in half of the cases in which a volunteer responds, another volunteer would be dispatched unnecessarily. We conclude that this model should be used only if the responders are completely anonymous.

Model 3 used events and alerts data, responders’ demographics, their prior experience of witnessing an opioid overdose, their prior experience in administering naloxone, and their naloxone carriage adherence before joining the VFR network. This model’s ability to predict that a volunteer will ignore an alert was similar to that of model 2, but its ability to predict that a volunteer will respond to an event was higher (202/329, 61.4% in the test set and 698/993, 70.3% in the full data set). A closer look at the decision tree of this model reveals that the most influential variables were related to the volunteer’s experience before joining the network: naloxone carriage adherence and the provision of naloxone to those experiencing an overdose. We conclude that this model can be used if data about a volunteer’s app use behavior in the network are not currently available (eg, during the period between recruiting the volunteer and until they receive enough alerts).

Model 4 used all available data, including the frequency of events and alerts, volunteers’ demographics, their prior overdose witnessing and naloxone provision experience before joining the VFR network, and their response behavior in the VFR network (according to the frequent responder indicator recalculated after each event). The ability of this model to predict that a volunteer will ignore an alert was similar to that of model 3, but its ability to predict that a volunteer will respond to an event was higher (225/329, 68.4% in the test set). The decision tree presented in Figure 9 reveals that the frequent responder indicator was the most influential variable. We conclude that model 4 achieves the best prediction accuracy and should be preferred whenever the necessary data are available.

The dispatch of substitute responders is relevant whenever the initial subset of closest volunteers do not provide a response and is a common approach in VFR systems. However, valuable time is lost until nonresponse is identified. Although existing algorithms are based on technical variables such as distance from the scene and the ETA, this study introduces a completely new variable: volunteers’ behavior and their probability to respond to a specific alert. The demonstrated importance of considering multiple factors in volunteer demographics and behavioral characteristics and the insights from the models we have tested are applicable wherever volunteer dispatch optimization is important. Such challenges are found in areas as diverse as food rescue operations [53], OHCA response [54,56], and mass casualty events [57]. Following our approach, these domains and more may find value in testing different sets of demographic and behavioral factors. Predicting the response behavior of candidate responders in advance of dispatch can allow any VFR system to choose the best possible response candidates based not only on ETA or location but also on the probability of actual response. The potential time savings depend on the network-specific period of time until a nonresponsive volunteer is considered unavailable, and a substitute responder is dispatched. The longer this time period, the greater the potential savings provided by a predictive dispatch algorithm.

The data used in our algorithm can be divided into four categories (Table 1): (1) event characteristics, (2) past responder behavior, (3) demographics, and (4) certain parameters specific to a medical condition relevant to the VFR network. The first 3 categories are directly generalizable because most responder mobilization apps collect and store these data, which, based on our findings, can be harvested for improved dispatch algorithms. The fourth data category includes factors that may differ depending on the medical condition relevant to the VFR network.

Herein, we provide a brief discussion of the factors that affect volunteers’ decisions to respond. A full analysis of these factors is beyond the scope of this research and should be pursued using a larger sample that may provide generalizability.

Experience, including the experience gained both before and after joining the VFR community, was found to be the most influential factor in volunteers’ willingness to respond. In model 3, naloxone carriage adherence and experience in the provision of naloxone were the most influential factors, whereas in model 4, the frequent responder indicator was the most influential factor. Significant differences were found between responded alerts and ignored alerts for the following variables: naloxone carriage adherence, previous alerts, previous false alerts, previous false alerts by the same signaler, previous responses, and previous responses to false alerts.

Part-time employment led to lower willingness to respond.

The average age of volunteers who responded to alerts was a little higher than the average age of volunteers who ignored alerts. Model 4 revealed that age is an important factor for volunteers who are not frequent responders: volunteers aged >54 years are expected to respond.

Male volunteers had a higher willingness to respond, but this difference had borderline significance (P=.05). The results of model 4 were consistent with this difference.

VFR dispatch has evolved in the last decades, thanks to technological advances and a better understanding of VFR network management. In a pretechnology era, VFRs (eg, volunteer firefighters) were alerted by sirens or other means rather than individually dispatched. Once pagers and SMS text messaging technology became available, the first generation of individual dispatch based on usual location was implemented (eg, the study by Zijlstra et al [43]). Further technological advances, including smartphone apps and GPS, enabled the second generation of individual dispatch based on current location [34,36,46,47] and the integration of VFRs’ feedback into the algorithm [34]. The integration of the probability to respond based on event characteristics as well as VFRs’ demographic data and previous behavior into the dispatch algorithm constitutes the third generation of individual dispatch, making this one of the first studies to harness the power of predictive analytics for VFR dispatch.

A relatively small sample for a specific condition (opioid overdose) and a specific emergency intervention (the provision of naloxone) was used. The sample has specific socioeconomic characteristics: it included a significant proportion of people experiencing homelessness and those who were unemployed (volunteers may have lower motivation to help owing to these destabilizing factors), as well as a significant proportion of people dependent on drugs (volunteers may have lower response rates when intoxicated). The setting was very specific: a large number of outdoor opioid overdoses within a relatively small geographic area. The responders were aware that there were many trained bystanders nearby, and this could have led to the “diffusion of responsibility” phenomenon and reduced the willingness to respond. No randomization or control group was used.

The proposed approach should be tested with a larger sample and for different conditions and interventions. A randomized study comparing the outcomes of the proposed dispatch algorithm with those of a regular location-based dispatch algorithm should be considered.

Machine learning techniques should be considered to calculate the frequent responder indicator. Future studies should examine whether the probability that a specific responder will respond to a specific event can be used instead of a binary indicator.

Further research is necessary on whether the proposed approach may have implications for multisided networks dispatching nonemergency services, such as ride sharing and package delivery.

In this research, we proposed a way to improve dispatch algorithms in VFR networks based on the individual characteristics of the volunteers and their behavior. We have shown that even in a relatively small sample, a classification model can predict with fair accuracy whether a specific volunteer will respond to a specific event or ignore it. Such prediction may improve the dispatchers’ decision-making process and enable the dispatch of substitute responders without delay.

Our findings can help VFR network administrators in their continual efforts to improve the response rates and response times of their networks and to save lives.