This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
The solution to the growing problem of rural residents lacking health care access may be found in the use of telemedicine and mobile health (mHealth). Using mHealth or telemedicine allows patients from rural or remote areas to have better access to health care.
The objective of this study was to understand factors influencing the choice of communication medium for receiving care, through the analysis of mHealth versus telemedicine encounters with a virtual urgent clinic.
We conducted a postdeployment evaluation of a new virtual health care service, Virtual Urgent Clinic, which uses mHealth and telemedicine modalities to provide patient care. We used a multinomial logistic model to test the significance and predictive power of a set of features in determining patients’ preferred method of telecare encounters—a nominal outcome variable of two levels (mHealth and telemedicine).
Postdeployment, 1403 encounters were recorded, of which 1228 (87.53%) were completed with mHealth and 175 (12.47%) were telemedicine encounters. Patients’ sex (
We studied factors influencing patients’ choice of communication medium, either mHealth or telemedicine, for a virtual care clinic. Sex and geographic location, as well as their chief concern, were strong predictors of patients’ choice of communication medium for their urgent care needs. This study suggests providing the option of mHealth or telemedicine to patients, and suggesting which medium would be a better fit for the patient based on their characteristics.
In the United States, approximately 19.3% of the population live in rural areas. With only 9% of the nation’s physicians practicing in such communities, the lack of health care providers in rural areas tends to be an intractable problem [
The solution to the growing problem of rural residents lacking health care access may be found in the use of telemedicine and mobile health (mHealth). In telemedicine, the doctor-patient interaction is conducted by live video consultation [
Telemedicine is used in rural areas to educate patients, deliver teaching programs, and facilitate administrative meetings [
mHealth is an innovative way to deliver care. mHealth is used for remote monitoring and treating chronic diseases, to raise awareness, and for behavioral modification [
These two modalities, telemedicine and mHealth, improve access to care: telemedicine enables physician intervention, and mHealth promotes patients’ participation [
We conducted a postdeployment evaluation of a new virtual health care service, Virtual Urgent Clinic (VUC), which uses mHealth and telemedicine modalities to provide patient care. VUC is a 24-hour-a-day, 7-day-a-week, on-demand service aimed at helping individuals with urgent medical needs to consult with a physician regarding their medical condition. The service was primarily designed to offer services regardless of the time of day or location of the patient in a more convenient form than the traditional in-person urgent care clinics. We obtained institutional review board approval from the University of North Carolina at Chapel Hill to conduct this research.
VUC is cloud-based platform offered through a public website. Individuals with urgent medical needs can use VUC, despite their location, as long as they have access to a phone or a computer equipped with a microphone and camera with internet connection. Inclusion criteria for this study were individuals with a medical need who were over the age of 2 years. Exclusion criteria were patients under the age of 2 years, patients with no access to a phone or a computer with microphone and camera with internet connection.
Individuals were required to create an account through the VUC website prior to scheduling a consultation. During the registration process, each individual had to fill out a short form providing basic demographic information. A secure link was sent to the individual’s email address for activation of the account. Once the account was activated, the individual indicated whom the e-visit was for and the intended provider type (eg, family physician). The website provided information regarding conditions not treatable through VUC, medications that VUC physicians could not prescribe, and important information regarding children under the age of 3 years. Once the individual verified having read this information, they were asked to fill out a series of short forms on the reason for the visit, their medical history, choice of pharmacy, choice of provider, payment, and confirmation. The cost of a VUC visit was a flat fee of US $49.
After the encounter, patients were asked to voluntarily participate in a short patient satisfaction survey. The survey aimed to solicit patients’ assessment of the encounter based on 4 criteria: (1) overall experience, (2) physician rating, (3) if they gave a fair or poor rating of the overall experience, their reason for the rating, and (4) open-ended patient comments.
The primary outcomes were two predictive models that projected the users’ medium of choice given their demographics and chief concern. Secondary outcomes were encounter duration and satisfaction levels per encounter medium.
The dependent variable was encounter medium (mHealth, telemedicine). Independent variables were sex (female, male), age range (<18, 19-34, 35-49, ≥50 years), setting (urban, rural), insurance status (insured, uninsured), encounter time range (6 AM-12 PM, 12 PM-5 PM, 5 PM-12 AM, 12 AM-6 PM), day of the-week (weekday, weekend), top 20 chief concerns (
We included the top 20 chief concerns, which made up 68.57% (962/1403) of the total encounters, as a predictor instead of including all 148 concerns; we classified the remaining 128 encounters as others. The rationale behind this is that an excessive number of levels with a small number of data points would have added unnecessary complexity to the models.
We used multinomial logistic regression to build and compare the two models based on the predictive power of two sets of features in determining patients’ preferred method of telecare (mHealth and telemedicine) encounters. We selected the first set of independent variables to represent the demographics and socioeconomic status of the patient population. The additional feature, chief concern, captures patients’ self-reported reason for the telecare visit.
For model selection purposes, we used the step function in R version 3.6.0 (R Foundation) to eliminate the least significant predictors. The process started with the full model, where all predictors were included; it ceased when the current model reached its maximum performance measured by the Akaike information criterion (AIC) [
To measure the features’ predictive performance, we inferred the odds ratio (OR) by exponentiating the models’ coefficients. However, due to the lack of a simple and intuitive explanation of OR outcomes, we decided to follow previous research by interpreting OR as the risk ratio—the relative probability of an event happening in one group compared with another group [
To evaluate the models’ prediction accuracy, we performed cross-validation with 70% of the original dataset training data and using 30% as the testing set. In addition, we measured the models’ efficiency and effectiveness using two common performance metrics: AIC and the simulated McFadden pseudo-
We used several R packages for advanced analysis and model building: nnet for modelling the multinomial logistic regression function; mfx for calculating the relative risk ratio; and stargazer for rendering the summary statistics. We generated visualizations using Tableau version 9.0 (Tableau Software).
Postdeployment, 1403 encounters were recorded, of which 87.53% (1228) were completed with mHealth, and 175 (12.5%) were telemedicine encounters (
Demographics of Virtual Urgent Clinic users.
Characteristics | Type of encounter | |||
mHealth, n (%) | Telemedicine, n (%) | Total, n (%) | ||
Number of encounters | 1228 (87.53) | 175 (12.47) | 1403 (100.00) | |
Male | 269 (82.01) | 59 (17.99) | 328 (23.38) | |
Female | 959 (89.21) | 116 (10.79) | 1075 (76.62) | |
2-18 | 115 (83.94) | 22 (16.06) | 137 (9.76) | |
19-34 | 434 (87.85) | 60 (12.15) | 494 (35.22) | |
35-49 | 465 (88.24) | 62 (11.76) | 527 (37.56) | |
≥50 | 214 (87.35) | 31 (12.65) | 245 (17.46) | |
Rural | 569 (92.22) | 48 (7.78) | 617 (44.04) | |
Urban | 657 (83.80) | 127 (16.20) | 784 (55.96) | |
Insured | 556 (91.15) | 54 (8.85) | 610 (43.48) | |
Uninsured | 672 (84.74) | 121 (15.26) | 793 (56.52) |
Odds ratio and significance (
Predictor | Odds ratio | |
Sex: male | 1.662 | .004 |
Setting: urban | 2.014 | <.001 |
Insurance status: uninsured | 1.42 | .06 |
Constant | 0.064 | <.001 |
aReference group: telemedicine.
Among the 6 predictors, sex and setting were the most predictive determinants of patients’ preferred method of telecare delivery, with significantly small
Among the 20 chief concerns, 6 were significant predictors of patients’ preferred medium of telecare encounter (
Odds ratio and significance (
Predictor | Odds ratio | |
Urinary tract infection | 0.11 | <.001 |
Ear pain | 0.256 | .06 |
Pink eye | 2.39 | .05 |
Rash | 2.325 | .01 |
Sinus infection | 0.5 | .04 |
Vaginal discharge | 0 | <.001 |
Constant | 0.168 | <.001 |
aReference group: telemedicine.
Evaluation metrics of multinomial logistic regression models.
Model | Akaike information criterion | McFadden |
Cross-validation prediction accuracy, % |
Model I: demographics | 1027.153 | 0.035 | 86.22 |
Model II: chief concerns | 1030.168 | 0.064 | 86.22 |
The AIC of both models performed similarly, indicating that the two models were of similar complexity [
The cross-validation yielded a prediction accuracy of 86.22% (363 instances were correctly predicted out of the 421 data points in the testing set) for both models.
Pearson chi-square test returned a strong indication of dependency between chief concern and encounter mediums, with a close-to-zero
We analyzed the top 10 chief concerns of the two encounter methods, the results of which confirmed the difference between mHealth and telemedicine users’ primary reasons for seeking virtual urgent care. We observed a few extreme cases: for instance, urinary tract infection the most common concern among the mHealth users (n=147, 12.0% of a total of 1228 mHealth encounters), was absent from the telemedicine users’ top 10 list (
Top 10 chief concerns in mobile health (mHealth) encounters (n=1228).
Chief concerns | Encounter medium: mHealth, n (%) | Sex: female, n (%) | Setting: rural, n (%) |
Urinary tract infection | 147 (11.98) | 147 (100.0) | 62 (42.2) |
Sinus infection | 129 (10.51) | 113 (87.6) | 62 (48.1) |
Sore throat | 116 (9.45) | 94 (81.0) | 58 (50.0) |
Cough | 82 (6.68) | 53 (65) | 49 (60) |
Ear pain | 42 (3.42) | 27 (64) | 23 (55) |
Rash | 37 (3.02) | 23 (62) | 24 (65) |
Fever | 32 (2.61) | 22 (69) | 16 (50) |
Nasal congestion | 31 (2.53) | 24 (77) | 19 (61) |
Cold | 30 (2.44) | 25 (83) | 12 (40) |
Animal or insect bite or scratch | 28 (2.28) | 18 (64) | 9 (32) |
The average duration of telemedicine encounters was 5.46 minutes, which is 5.4 percentage points higher than the mean duration of mHealth encounters (5.18 minutes). A Welch 2-sample
mHealth encounter duration had a range of 1 to 15 minutes, where 70.93% (871/1228) of the total encounters fell within the 1- to 5-minute range. Telemedicine encounters had a similar range of 0 to 16 minutes. Encounters lasting longer than 10 minutes accounted for 12.6% (22/175) of all telemedicine calls, double the 6.03% (74/1228) of mHealth encounters. In addition, 14.9% (26/175) of telemedicine calls lasted less than 1 minute, in contrast to the absence of mHealth calls of this length, as shown
Distribution of encounter durations by encounter methods.
Self-reported overall experience satisfaction ratings. mHealth: mobile health.
For participants in all 1403 encounters, 204 (14.54%) responded to the satisfaction survey. High satisfaction levels were reported among both the mHealth and telemedicine groups. Of mHealth patients, 91.1% (154/169) were satisfied by their encounter compared with 89% (31/35) of telemedicine patients. A higher proportion of telemedicine patients (4/35, 11.4%) than mHealth patients (15/169, 8.9%) rated their experience as fair or poor (
We looked further into the telemedicine and mHealth users’ self-reported preferences for alternative care-seeking options. Patients were asked after their VUC consultation “if VUC was not available, which medical service would you have used?” The analysis revealed an almost identical distribution of users among the 5 options (
Alternative care-seeking choices of mobile health (mHealth) and telemedicine users.
To our knowledge, this is the first study to comprehensively assess the effectiveness of providing patients with medium choice (phone call vs video call) of either mHealth or telemedicine to consult with physicians for urgent care needs. We leveraged a data science approach, namely, data analytics, to predict what factors informed patients’ choice of an mHealth or telemedicine medium. We analyzed the top 20 chief concerns in both groups to gain insight into the potential association between concern and choice of medium. Then, we analyzed the duration of encounters, self-reported alternative care-seeking options, and users’ responses to satisfaction surveys between both groups.
We proposed a model to predict the preferred choice of care delivery for patients. Patients’ sex and geographic location (rural or urban) significantly predicted their choice of care between mHealth and telemedicine. Patients from an urban area were twice as likely as users from rural regions to choose telemedicine over mHealth. Similarly, male patients were 1.6 times more likely than female patients with identical features to use telemedicine than mHealth. We conclude that male users from urban regions are the most likely to choose telemedicine over mHealth.
Patients’ chief concern significantly correlated with their choice of medium, where chief concern strongly correlated with mHealth or telemedicine. The duration of encounters was similar between both mediums, around the 5-minute mark. Overall, telemedicine encounters had a notable difference in range, from less than 1 minute up to 16 minutes. A possible justification for telemedicine encounters to last less than 1 minute needs to be studied in the future.
We observed that patients were satisfied with their choice of medium, as well as the service provided, which suggests that providers should offer the option of mHealth or telemedicine to their patients and allow them to choose. We recommend considering patients’ sex and setting as predictive factors to provide suggestions on which communication medium would best fit patients based on their characteristics. Patient satisfaction was high in both groups, with higher dissatisfaction among telemedicine users, which may be attributed to the quality of the video or audio feed. There was no significant difference between the groups in terms of their self-reported responses to alternative care-seeking options.
A strength of this research is the ability to alleviate the demand on in-person urgent care clinics and emergency rooms by providing a virtual clinic where patients can be seen and treated. Since VUC is an on-demand and cloud-based service, there was no purposive sampling, which allows the findings of this study to be more generalizable. The digital nature of the service may introduce bias to the sample population; however, this study focused on two digital interventions and, therefore, if any bias was introduced, it should not have influenced the study findings. Another strength is the convenience of providing both mHealth and telemedicine options to patients within the same platform without further setup. The response rate of the voluntary survey was adequate given that we provided no incentive to participate.
One limitation of this study is the lower number of telemedicine encounters relative to mHealth encounters, which can be attributed to several factors, such as personal preference, time of the call, access to a Web camera, and internet connection speed. Another limitation is the absence of information regarding the reason for telemedicine encounters ending in less than 1 minute. This study can be further strengthened by capturing patient outcomes after the consultation visit by looking at 30-day hospitalization rates to assess the quality of care for each medium, which is a future direction of this research.
We studied factors influencing patients’ choice of communication medium, either mHealth or telemedicine, for a virtual care clinic. Patients’ preference for mHealth or telemedicine was significantly influenced by their sex and geographic location, as well as their chief concern. Despite other preferences, patients were highly satisfied by their choice of communication medium. This study showed that providing the option of mHealth or telemedicine to patients suggests which medium would be a better fit for patients based on their characteristics.
Top 20 chief concerns.
Akaike information criterion
mobile health
odds ratio
Virtual Urgent Clinic
None declared.