This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.
The global health emergency generated by the COVID-19 pandemic is posing an unprecedented challenge to health care workers, who are facing heavy workloads under psychologically difficult situations. Mental mobile Health (mHealth) interventions are now being widely deployed due to their attractive implementation features, despite the lack of evidence about their efficacy in this specific population and context.
The aim of this trial is to evaluate the effectiveness of a psychoeducational, mindfulness-based mHealth intervention to reduce mental health problems in health care workers during the COVID-19 pandemic.
We conducted a blinded, parallel-group, controlled trial in Spain. Health care workers providing face-to-face health care to patients with COVID-19 were randomly assigned (1:1) to receive the PsyCovidApp intervention (an app targeting emotional skills, healthy lifestyle behavior, burnout, and social support) or a control app (general recommendations about mental health care) for 2 weeks. The participants were blinded to their group allocation. Data were collected telephonically at baseline and after 2 weeks by trained health psychologists. The primary outcome was a composite of depression, anxiety, and stress (overall score on the Depression Anxiety Stress Scale-21 [DASS-21]). Secondary outcomes were insomnia (Insomnia Severity Index), burnout (Maslach Burnout Inventory Human Services Survey), posttraumatic stress (Davidson Trauma Scale), self-efficacy (General Self-Efficacy Scale), and DASS-21 individual scale scores. Differences between groups were analyzed using general linear modeling according to an intention-to-treat protocol. Additionally, we measured the usability of the PsyCovidApp (System Usability Scale). The outcome data collectors and trial statisticians were unaware of the treatment allocation.
Between May 14 and July 25, 2020, 482 health care workers were recruited and randomly assigned to PsyCovidApp (n=248) or the control app (n=234). At 2 weeks, complete outcome data were available for 436/482 participants (90.5%). No significant differences were observed between the groups at 2 weeks in the primary outcome (standardized mean difference –0.04; 95% CI –0.11 to 0.04;
In health care workers assisting patients with COVID-19 in Spain, PsyCovidApp, compared with a control app, reduced mental health problems at 2 weeks only among health care workers receiving psychotherapy or psychotropic medications.
ClinicalTrials.gov NCT04393818; https://clinicaltrials.gov/ct2/show/NCT04393818.
Since the declaration of the COVID-19 pandemic, the disease has spread globally, with almost 55 million known cases and a death toll of over 1.3 million people [
Health services worldwide are being urged to implement strategies to mitigate the severe psychological consequences experienced by health care workers. Among the different types of strategies considered, mobile health (mHealth) interventions are receiving special attention [
This blinded, individually randomized, parallel-group, controlled trial aimed to evaluate the effectiveness of PsyCovidApp (a self-managed and self-guided psychoeducational mobile-based intervention with no therapist support) to reduce symptoms of depression, anxiety, stress and other mental health problems in health care workers during the COVID-19 pandemic in Spain.
We conducted a blinded, individually randomized, parallel-group, controlled trial in Spain. Because the ultimate goal of the study was to inform decisions about rolling up the intervention to make it available to all health care workers in Spain, we used a pragmatic approach. Pragmatic trials are ideally suited to inform choices between treatments because as opposed to exploratory trials (which typically examine treatment benefits under ideal conditions using carefully defined subjects), pragmatic trials enable measurement of the effectiveness of interventions under real conditions in a sample of participants to whom the treatment would be applied [
The target population was male and female health care workers aged >18 years who had provided health care to patients with COVID-19 during the viral outbreak in Spain. We included health care workers from any medical specialty (pneumology, internal medicine, emergency, primary care, etc) and role (physicians, nurses, nurse assistants, etc) with access to a smartphone. We included health care workers who had provided direct, face-to-face health care to patients with a diagnosis of infection with COVID-19. We excluded health care workers who were not able to download and activate the app used to deliver the intervention during the next 10 days following the baseline assessment.
Following a purposive sampling method, we sent invitations to health care workers to participate in the trial through direct contact via email and telephone to key stakeholders (115 hospital and home care centers, 138 professional associations, and 8 scientific societies and trade unions) and by social media. Health care workers who were willing to participate registered their interest by completing a web-based questionnaire, consenting to be contacted telephonically. A team of 23 health psychologists who had previously received a 2-hour training session (to ensure homogeneity in recruitment, questionnaire administration, and data entry methods) contacted the registered health care workers by telephone to confirm the eligibility criteria and obtain informed consent (audio-recorded). The recruitment period spanned 10 weeks, from May 14 to July 25, 2020. Participants were enrolled on a rolling “first-come-first-served” basis until target sample sizes were met. To incentivize trial participation, we offered participation certificates to all health care workers completing the postintervention assessment.
Participants were randomly assigned (1:1) to receive the PsyCovidApp intervention or the control app over 2 weeks by a designated researcher (MAF, who was not involved in data collection or analysis) using a computer-generated sequence of random numbers created by internet relay chat (IRC). Randomization was not stratified. Health care workers were blinded to group allocation (as both groups received an app). The outcome data collectors and trial statisticians were unaware of the treatment allocation.
Immediately after obtaining informed consent, a team of psychologists conducted a psychological (preintervention) evaluation via telephone interview and instructed participants on how to download the Clinicovery App (Apploading, Inc). Clinicovery is the app that was used to deliver either the contents of the PsyCovidApp intervention or the control contents. Within 48 hours after participants successfully downloaded and activated the app (user activation was used as a checkpoint to ensure the participants could successfully access the app), a member of our research team loaded the contents to the app according to the group the participants had been allocated to. During the following 14 days, all health care workers had access to the content of their assigned group (PsyCovidApp intervention or control). The PsyCovidApp intervention was developed by a group of psychologists (MJSR, EG, CS, RJ, MEGB), psychiatrists (JGC, MGT), and experts in healthy lifestyle promotion (AMYJ, MBV), informed by findings from an exploratory qualitative study involving in-depth interviews with 9 health care workers seeking psychological support as a result of their professional activity during the COVID-19 pandemic (unpublished results). PsyCovidApp was specifically designed to prevent and mitigate the most frequent mental problems suffered by health care workers who are dealing with the COVID-19 emergency (depression, anxiety, posttraumatic stress, and burnout). A detailed description of the intervention is available elsewhere [
Participants in the Control App group had access through the Clinicovery app to brief written information about the mental health care of health care workers during the COVID-19 pandemic (adapted from a set of materials developed by the Spanish Society of Psychiatry; the contents are available in
After 2 weeks, the apps in both groups were disabled, and a postintervention psychological assessment was undertaken. The follow-up was undertaken via telephone between 24 hours and 10 days after the intervention concluded, and it included the same questionnaires used in the first evaluation and the System Usability Scale (SUS) [
The primary outcome was an overall index of depression, anxiety, and stress (overall score of the Spanish version of the Depression, Anxiety, and Stress Scale [DASS-21] instrument [
Secondary outcome measures were the difference between the intervention and control groups in the mean scores of the following instruments:
Davidson Trauma Scale (DTS) [
Maslach Burnout Inventory - Human Services Survey (MBI-HSS) [
Insomnia Severity Index (ISI) [
General Self-Efficacy Scale (GSE) [
Individual subscales of the DASS-21 instrument: depression (α=.90), anxiety (α=.88), and stress (α=.88) [
Usability of PsyCovidApp at postintervention: SUS [
The sample size and power calculations have been described previously [
The analyses followed the agreed statistical analysis plan, published before database lock [
We conducted three prespecified subgroup analyses to examine the impact of the PsyCovidApp intervention on the primary and secondary outcomes based on the following baseline characteristics: use of psychotropic medications (yes vs no), use of psychotherapy (yes vs no), and symptomatology of depression, anxiety, and stress (yes vs no, based on baseline DASS-21 median overall score). We conducted statistical tests for interaction (including an interaction term in the models) to determine whether chance was an unlikely explanation for the apparent subgroup effects identified. We used the Instrument to Assess the Credibility of Effect Modification Analyses (ICEMAN) [
The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Between May 14 and July 25, 2020, 684 health care workers submitted an expression of interest in enrolling in the PsyCovidApp trial. 482 eligible participants provided informed consent and were randomly assigned to the PsyCovidApp intervention group (n=248) or the Control App group (n=234;
The baseline characteristics were balanced between the groups (
Trial profile. Multiple imputation was used to facilitate the overall sample analysis; all randomized participants contributed to the statistical analysis.
Baseline demographic characteristics of the intention-to-treat population (N=482).
Characteristic | PsyCovidApp group (n=248) | Control App group (n=234) | Total (N=482) | |
|
||||
|
Mean (SD) | 42.07 (11.0) | 40.62 (9.6) | 41.37 (10.4) |
|
Median (IQR; range) | 42 (34-51; 23-63) | 41 (32-47; 23-61) | 41.5 (33-49; 23-63) |
|
<36, n (%) | 75 (30.2) | 82 (35) | 157 (32.6) |
|
36-45, n (%) | 79 (31.9) | 81 (34.6) | 160 (33.2) |
|
46-55, n (%) | 60 (24.2) | 52 (22.2) | 112 (23.2) |
|
>55, n (%) | 34 (13.7) | 19 (8.1) | 53 (11) |
|
||||
|
Male | 38 (15.3) | 43 (18.4) | 81 (16.8) |
|
Female | 210 (84.7) | 191 (81.6) | 401 (83.2) |
|
||||
|
Physician | 76 (30.6) | 77 (32.9) | 153 (31.7) |
|
Nurse | 87 (35.1) | 74 (31.6) | 161 (33.4) |
|
Nurse assistant | 77 (31) | 70 (29.9) | 147 (30.5) |
|
Other | 8 (3.2) | 13 (5.6) | 22 (4.6) |
|
||||
|
Primary care | 35 (14.1) | 26 (11.1) | 61 (12.7) |
|
Internal medicine | 48 (19.4) | 50 (21.4) | 98 (20.3) |
|
Intensive care unit | 40 (16.1) | 41 (17.5) | 81 (16.8) |
|
Hospital emergencies unit | 31 (12.5) | 48 (20.5) | 79 (16.4) |
|
Home care | 19 (7.7) | 10 (4.3) | 29 (6) |
|
Infection unit | 16 (6.5) | 15 (6.4) | 31 (6.4) |
|
Other hospital unit | 59 (23.8) | 44 (18.8) | 103 (21.4) |
|
||||
|
<2 | 9 (3.6) | 4 (1.7) | 13 (2.7) |
|
2-4 | 20 (8.1) | 14 (6) | 34 (7.1) |
|
>4 | 219 (88.3) | 216 (92.3) | 435 (90.2) |
|
||||
|
Yes | 31 (12.5) | 34 (14.5) | 65 (13.5) |
|
No | 214 (86.3) | 195 (83.3) | 409 (84.9) |
|
Unknown | 3 (1.2) | 5 (2.1) | 8 (1.7) |
|
||||
|
Inadequate measures | 84 (33.9) | 77 (32.9) | 161 (33.4) |
|
Adequate measures | 163 (65.7) | 157 (67.1) | 320 (66.4) |
|
||||
|
Inadequate information | 101 (40.7) | 116 (49.6) | 217 (45) |
|
Adequate information | 147 (59.3) | 117 (50) | 264 (54.8) |
|
||||
|
No | 207 (83.5) | 196 (83.8) | 403 (83.6) |
|
Yes | 41 (16.5) | 38 (16.2) | 79 (16.4) |
|
||||
|
No | 227 (91.5) | 212 (90.6) | 439 (91.1) |
|
Yes | 21 (8.5) | 22 (9.4) | 43 (8.9) |
By the time of recruitment, most participants (435/482, 90.2%) had been providing health care to patients with COVID-19 for more than 4 weeks, and 65/482 (13.5%) had received a diagnosis of COVID-19 infection. Approximately one-third (161/482, 33.4%) perceived that the measures offered to protect them from COVID-19 had been inadequate, and 217/482 (45%) perceived that they had received inadequate information about the procedures to provide health care to patients with COVID-19. Of the 482 participants, 79 (16.4%) were using psychotropic medications, and 43 (8.9%) were receiving psychotherapy.
In relation to their mental health, 206 of the 482 participants (42.7%) presented symptoms of depression, 250 (51.9%) had symptoms of anxiety, 292 (60.6%) had symptoms of stress, 194 (40.2%) had symptoms of posttraumatic stress, and 128 (26.6%) had symptoms of insomnia (
Baseline clinical characteristics of the intention-to-treat population (N=482).
Characteristic | PsyCovidApp group (n=248) | Control App group (n=234) | Total (N=482) | ||
|
|||||
|
DASS-21a overall score, mean (SD) | 5.8 (3.9) | 6.1 (3.8) | 6.0 (3.8) | |
|
|
||||
|
|
No symptoms (<5 points) | 143 (57.7) | 133 (56.8) | 276 (57.3) |
|
|
Mild (5-6 points) | 26 (10.5) | 32 (13.7) | 58 (12) |
|
|
Moderate (7-10 points) | 55 (22.2) | 48 (20.5) | 103 (21.4) |
|
|
Severe (11-13 points) | 15 (6) | 11 (4.7) | 26 (5.4) |
|
|
Extremely severe (>13 points) | 9 (3.6) | 10 (4.3) | 19 (3.9) |
|
|
||||
|
|
No symptoms (<4 points) | 121 (48.8) | 111 (47.4) | 232 (48.1) |
|
|
Mild (4 points) | 32 (12.9) | 25 (10.7) | 57 (11.8) |
|
|
Moderate (5-7 points) | 40 (16.1) | 42 (17.9) | 82 (17) |
|
|
Severe (8-9 points) | 24 (9.7) | 20 (8.5) | 44 (9.1) |
|
|
Extremely severe (>9 points) | 31 (12.5) | 36 (15.4) | 67 (13.9) |
|
|
||||
|
|
No symptoms (<8 points) | 104 (41.9) | 86 (36.8) | 190 (39.4) |
|
|
Mild (8-9 points) | 28 (11.3) | 32 (13.7) | 60 (12.4) |
|
|
Moderate (10-12 points) | 56 (22.6) | 58 (24.8) | 114 (23.7) |
|
|
Severe (13-16 points) | 43 (17.3) | 45 (19.2) | 88 (18.3) |
|
|
Extremely severe (>16 points) | 17 (6.9) | 13 (5.6) | 30 (6.2) |
|
|||||
|
No (<40 points) | 150 (60.5) | 138 (59) | 288 (59.8) | |
|
Yes (≥40 points) | 98 (39.5) | 96 (41) | 194 (40.2) | |
|
|||||
|
|
||||
|
|
Low (0-16 points) | 95 (38.3) | 89 (38) | 184 (38.2) |
|
|
Moderate (17-26 points) | 61 (24.6) | 50 (21.4) | 111 (23) |
|
|
High (>27 points) | 92 (37.1) | 95 (40.6) | 187 (38.8) |
|
|
||||
|
|
High (>39 points) | 144 (58.1) | 135 (57.8) | 279 (57.9) |
|
|
Moderate (32-38 points) | 65 (26.2) | 54 (23.1) | 119 (24.7) |
|
|
Low (0-31 points) | 39 (15.7) | 45 (19.2) | 84 (17.4) |
|
|
||||
|
|
Low (0-6 points) | 163 (65.7) | 154 (65.8) | 317 (65.8) |
|
|
Moderate (17-12 points) | 50 (20.2) | 31 (13.2) | 81 (16.8) |
|
|
High (>13 points) | 35 (14.1) | 49 (20.9) | 84 (17.4) |
|
|||||
|
Not clinically significant (0-7 points) | 102 (41.1) | 87 (37.2) | 189 (39.2) | |
|
Subthreshold insomnia (8-14 points) | 89 (35.9) | 76 (32.5) | 165 (34.2) | |
|
Clinical insomnia (moderate severity) (15-21 points) | 49 (19.8) | 61 (26.1) | 110 (22.8) | |
|
Clinical insomnia (severe) (22–28) | 8 (3.2) | 10 (4.3) | 18 (3.7) | |
|
|||||
|
Mean score out of 40 points (SD) | 32.4 (4.7) | 32 (4.7) | 32 (4.7) |
aDASS-21: Depression, Anxiety, and Stress Scale-21.
bDTS, Davidson Trauma Scale.
cMBI-HSS: Maslach Burnout Inventory - Human Services Survey.
dISI: Insomnia Severity Index.
eGSE: General Self-Efficacy Scale.
At 2 weeks, 27 of the 248 participants (10.9%) in the PsyCovidApp group and 19 of the 234 participants (8.1%) in the Control App group were lost to follow-up because they decided to withdraw from the study at the time of the postintervention psychological assessment (6 in the intervention group and 6 in the control group) or because we were unable to reach them for the telephonic postintervention psychological assessment (21 in the intervention group and 13 in the control group). None of the participants were deemed to be associated with reported adverse events or death.
Primary and secondary outcome data were available for 436 of the 482 participants (90.5%): 221 of 248 (89.1%) health care workers in the PsyCovidApp intervention group versus 215 of 234 (91.9%) in the Control App group. For the primary outcome, scale scores were lower at 2 weeks than at baseline in the PsyCovidApp and Control App groups (
Changes in median DASS-21 scores over time, with the raw data plot of the median DASS-21 scores. Baseline scores were recorded before randomization.
The effect sizes for all outcomes are shown in
Descriptive summaries of the primary and secondary outcome measures at baseline and 2 weeks for the PsyCovidApp and Control App groups.
Measure | At baseline | At 2 weeks | |||||||||||||||||
|
|
PsyCovidApp group (n=248) | Control App group (n=234) | Overall (N=482) | PsyCovidApp group (n=221) | Control App group (n=215) | Completers at follow-up (n=436) | ||||||||||||
|
|
Mean (SD) | Range | Mean (SD) | Range | Mean (SD) | Range | Mean (SD) | Range | Mean (SD) | Range | Mean (SD) | Range | ||||||
|
|||||||||||||||||||
|
DASS-21a overall score | 5.84 (3.85) | 0-17.33 | 6.14 (3.77) | 0-16.33 | 5.99 (3.81) | 0-17.33 | 3.83 (3.21) | 0-16.33 | 4.27 (3.47) | 0-15.33 | 4.05 (3.35) | 0-16.33 | ||||||
|
|||||||||||||||||||
|
DASS-21 depression subscale | 4.46 (4.13) | 0-18 | 4.58 (4.02) | 0-15 | 4.51 (4.07) | 0-18 | 2.97 (3.49) | 0-17 | 3.05 (3.65) | 0-18 | 3.01 (3.56) | 0-18 | ||||||
|
DASS-21 anxiety subscale | 4.35 (3.86) | 0-16 | 4.70 (4.25) | 0-18 | 4.52 (4.06) | 0-18 | 2.21 (2.43) | 0-15 | 2.84 (3.36) | 0-17 | 2.64 (3.13) | 0-17 | ||||||
|
DASS-21 stress subscale | 8.75 (5.07) | 0-21 | 9.15 (4.63) | 0-19 | 8.94 (4.86) | 0-21 | 6.11 (4.50) | 0-21 | 6.94 (4.68) | 0-21 | 6.51 (4.60) | 0-21 | ||||||
|
Posttraumatic stress (DTSb) | 34.57 (23.47) | 0-117 | 36.91 (23.18) | 0-100 | 35.71 (23.33) | 0-117 | 24.91 (20.41) | 0-96 | 26.36 (21.02) | 0-91 | 25.62 (20.70) | 0-96 | ||||||
|
Burnout (MBI-HSSc) emotional exhaustion subscale | 23.27 (12.20) | 0-54 | 23.57 (12.34) | 0-54 | 23.41 (12.26) | 0-54 | 19.43 (12.25) | 0-51 | 19.67 (12.91) | 0-54 | 19.54 (12.57) | 0-54 | ||||||
|
Burnout (MBI-HSS) professional accomplishment subscaled | 39.69 (6.43) | 10-48 | 39.59 (6.62) | 15-48 | 39.64 (6.52) | 10-48 | 40.33 (6.31) | 13-48 | 39.54 (6.93) | 15-48 | 39.94 (6.63) | 13-48 | ||||||
|
Burnout (MBI-HSS) depersonalization subscale | 4.69 (5.08) | 0-29 | 5.24 (5.41) | 0-23 | 4.95 (5.25) | 0-29 | 4.51 (4.96) | 0-29 | 4.78 (5.25) | 0-23 | 4.64 (5.10) | 0-29 | ||||||
|
Insomnia (ISIe) | 9.80 (6.19) | 0-26 | 10.16 (6.53) | 0-27 | 9.98 (6.36) | 0-27 | 8.07 (6.18) | 0-28 | 8.44 (6.68) | 0-23 | 8.25 (6.43) | 0-28 | ||||||
|
Self-efficacy (GSEf)d | 32.42 (4.71) | 19-40 | 32.00 (4.73) | 18-40 | 32.21 (4.72) | 18-40 | 33.22 (4.65) | 18-40 | 32.54 (4.88) | 17-40 | 32.88 (4.77) | 17-40 |
aDASS-21: Depression, Anxiety, and Stress Scale-21.
bDTS: Davidson Trauma Scale.
cMBI-HSS: Maslach Burnout Inventory - Human Services Survey.
dScale scores are reversed to homogeneously convey a similar treatment effect.
eISI: Insomnia Severity Index.
fGSE: General Self-Efficacy Scale.
Comparison of outcome measures between the PsyCovidApp and the Control App groups at 2 weeks. Data are adjusted standardized between-group mean differences with 95% CIs in parentheses.
Measure | Sample of completers at follow-up (n=436) | Overall samplea (N=482) | ||
|
Adjusted standardized between-group mean differences (95% CI) | Adjusted standardized between-group mean differences (95% CI) | ||
DASS-21b overall score | –0.04 (-0.11 to 0.04) | .16 | –0.04 (–0.11 to 0.04) | .15 |
DASS-21 depression subscale | 0.00 (–0.07 to 0.08) | .47 | 0.00 (–0.07 to 0.08) | .47 |
DASS-21 anxiety subscale | –0.04 (–0.12 to 0.04) | .15 | –0.04 (–0.12 to 0.04) | .17 |
DASS-21 stress subscale | –0.06 (–0.14 to 0.01) | .06 | –0.06 (–0.14 to 0.01) | .05 |
Posttraumatic stress (DTSc) | 0.00 (–0.06 to 0.06) | .47 | 0.00 (–0.06 to 0.07) | .47 |
Burnout (MBI-HSSd) emotional exhaustion subscale | 0.01 (–0.06 to 0.08) | .38 | 0.01 (–0.06 to 0.08) | .39 |
Burnout (MBI-HSS) professional accomplishment subscalee | –0.04 (–0.12 to 0.03) | .13 | –0.05 (–0.12 to 0.03) | .12 |
Burnout (MBI-HSS) depersonalization subscale | 0.01 (–0.06 to 0.09) | .36 | 0.01 (–0.06 to 0.09) | .36 |
Insomnia (ISIf) | 0.01 (–0.05 to 0.07) | .38 | 0.01 (–0.05 to 0.07) | .38 |
Self-efficacy (GSEg)e | –0.02 (–0.10 to 0.05) | .26 | –0.02 (–0.01 to 0.05) | .27 |
aOverall sample, derived by multiple imputation (50 imputations).
bDASS-21: Depression, Anxiety, and Stress Scale-21.
cDTS: Davidson Trauma Scale.
dMBI-HSS, Maslach Burnout Inventory - Human Services Survey.
eScale scores reversed to homogeneously convey a similar treatment effect.
fISI: Insomnia Severity Index.
gGSE: General Self-Efficacy Scale.
The impact of the intervention on prespecified subgroups of health care workers is presented in
In the subgroup of participants receiving psychotherapy (n=43) (
No statistically significant differences (
Standardized mean differences for primary and secondary outcomes in healthcare workers reporting the use of psychotropic medications at baseline. Forest plot of standardized group differences between PsyCovidApp and Control App groups for all outcomes, whereby an effect lower than 0 favored the PsyCovidApp group. Error bars show 95% Confidence Intervals (CIs). DASS-21, Depression, Anxiety, and Stress Scale. *
Standardized mean differences for primary and secondary outcomes in health care workers reporting the use of psychotherapy at baseline. Forest plot of standardized group differences between the PsyCovidApp and Control App groups for all outcomes, whereby an effect lower than 0 favored the PsyCovidApp group. Error bars show 95% CIs. DASS-21: Depression, Anxiety, and Stress Scale. *
The usability of the PsyCovidApp intervention is described in
Usability of the PsyCovidApp intervention by the PsyCovidApp group (n=221) measured with the System Usability Scale (the theoretical score range is 0-4 for single items; higher scores are indicative of higher usability).
Item | Mean score (SD) |
I would like to use this App frequently | 3.00 (1.02) |
The App is not unnecessarily complex | 3.39 (1.03) |
The App is easy to use | 3.56 (0.88) |
No need for the support of a technical person to use this App | 3.73 (0.81) |
The various functions in this App are well integrated | 3.40 (0.83) |
Not too much inconsistency in this App | 3.61 (0.81) |
Most people would learn to use this App very quickly | 3.43 (0.82) |
The App is not very cumbersome to use | 3.70 (0.70) |
Confidence using the App | 3.35 (0.95) |
No need to learn a lot of things to use the App | 3.69 (0.84) |
Overall usability scorea | 87.21 (12.65) |
aOverall System Usability Scale theoretical score range: 0-100 (higher scores are indicative of higher usability).
The sensitivity analysis of all outcomes on a complete case basis (ie, without imputation or adjustment for baseline predictors of missingness) is shown in
The global health emergency generated by the COVID-19 pandemic is posing an unprecedented challenge to frontline health care workers, who are facing high levels of workload under psychologically difficult situations with scarce resources and support. To our knowledge, this is the first randomized controlled trial to date to assess the efficacy of a mental mHealth intervention for frontline health care workers fighting the health emergency generated by the COVID-19 pandemic. Our analysis showed that at 2 weeks, PsyCovidApp only produced significant improvements in the primary and secondary outcomes of health care workers who were receiving psychotherapy or psychotropic medications.
Although the results of our trial indicate that the PsyCovidApp intervention was not effective in comparison with the control app in the overall health care worker population, we cannot rule out the possibility that the intervention produced beneficial effects that our trial was not able to detect for various reasons, including the choice of an active comparator and the level of use of the intervention. Concerning the choice of an active comparator, most of the outcomes at 2 weeks improved similarly in both the intervention and control group. The improvements in the Control App group could be attributed to the natural progression of the disease in a context of decreasing levels of external stressors (as the impact of the first wave of COVID-19 in Spain was starting to decrease by the time the trial was initiated). However, it is also plausible that the intervention in the control group (which consisted of a similar app but with access limited to general information and contents) also had a positive effect. The impact of the control app may have been enhanced by the Hawthorne effect [
It is plausible that the intervention did not produce the desired effects because of the short trial duration (ie, too short for the intervention to produce the intended benefits). During the time of the study, health care workers in Spain were overwhelmed with heavy workloads, and it is likely that a large proportion struggled to find time to use PsyCovidApp during the 2-week intervention period. Suboptimal use of mental health apps is indeed a widely acknowledged challenge: user retention rate for smartphone apps in the general population is low, and approximately 25% of users abandon apps after one use [
In any case, as it stands, the trial showed that the PsyCovidApp intervention did not produce significant improvements in the primary and secondary outcomes in the overall population when compared with a control app. It could be interpreted that the PsyCovidApp intervention was not effective in improving mental health outcomes in the short term in this specific population and context. It could be argued that considering all the issues health care workers are required to deal with, providing psychological aid only through a mHealth intervention may not be sufficient to produce significant improvements.
In our subgroup analyses, we observed that the intervention did not produce significant effects among those health care workers using the intervention in absence of additional mental help. This finding is consistent with findings from a recent systematic review, which showed a lack of effect of mental mHealth interventions when used as a standalone therapy [
The fact that female health care workers were overrepresented in our trial (83% in our trial vs 68% overall in Spain [
PsyCovidApp presented a high usability level, with an overall score of 87.2 points—clearly above the threshold of 68 points used to determine high usability [
In terms of future research needs, it is worth noting that PsyCovidApp was based on a group of software features related to the intervention (eg, learning and in situ use) and communication (eg, prompting) deployed in smartphone interventions that mostly mimic more traditional mobile phone and mHealth solutions. More innovative use of the capabilities of smartphones, such as sensing, alternative delivery paradigms, and advanced analytics, could have produced a more beneficial effect. The possibilities of current smartphone technology have only just been tapped, and further research is needed to explore them fully, as are studies to rigorously analyze the empirical effectiveness of these systems.
A process evaluation is now underway, which will shed light on the mechanisms and contexts in which the intervention did or did not work. In this process evaluation, we will retrospectively investigate the “reach” of interventions (the extent to which the study participants came into contact with the intervention and how they did so). Although a recent meta-analysis of a range of mental health apps concluded that age does not impact treatment effect [
The study has several limitations. First, the 2-week follow-up period may not have been sufficient to detect clinically meaningful differences in mental health. A longer period of time may be needed to produce the desired positive effects. There were two main reasons for this short follow-up period: (1) according to available literature [
The strengths of this study include the pragmatic design, large sample size, and high follow-up rates. Moreover, the trial participants, outcome assessors, and data analysts of the research were blinded to the intervention allocation to reduce biases in the evaluation of the effects of the intervention. A common limitation of previous mHealth trials is that researchers do not have control of the proportion of participants having actual access to their interventions. In our trial, we ensured that all participants successfully downloaded and activated the app before their enrollment in the trial, which is a novel and important strength.
For the first time, the PsyCovidApp trial studied the impact of a cognitive behavioral therapy and mindfulness-based mHealth intervention specifically designed to protect the mental health of health care workers fighting on the front lines of the COVID-19 pandemic. No significant differences were observed between the intervention and control groups at 2 weeks in the primary outcome and in the rest of the outcomes. However, significant improvements were observed among health care workers who were consuming psychotropic medications or receiving psychotherapy in the primary outcome, as well as in posttraumatic stress, insomnia, anxiety, and stress. PsyCovidApp may therefore improve mental health among health care workers who are already using other effective interventions, such as psychotherapy or pharmacological treatments.
Deidentified data collected for the study, including individual participant data and a data dictionary defining each field in the set, will be made available to others upon request to the corresponding author, following a signed data access agreement.
Content of the PsyCovidApp intervention.
Content of the control app.
Subgroup analyses: comparison of outcome measures between the PsyCovidApp and the Control App groups at 2 weeks in prespecified subgroups (high DASS-21 baseline scores; users of psychotherapy; consumers of psychotropic medications).
Number of health care workers recruited by Spanish region.
Trial results reported in terms of adjusted mean between-group differences (overall sample).
Instrument to Assess the Credibility of Effect Modification Analyses (ICEMAN) used in the study.
CONSORT-EHEALTH checklist (v 1.6.1).
Depression, Anxiety, and Stress Scale
Davidson Trauma Scale
General Self-Efficacy Scale
Balearic Islands Health Research Institute
Insomnia Severity Index
Maslach Burnout Inventory - Human Services Survey
mobile health
System Usability Scale
This study was funded by the Balearic Islands Health Research Institute (IdISBa) through the Regional Government of the Balearic Islands, Spain (grant code: COVID-19/06). IRC was supported by a Miguel Servet Fellowship (CP17/00019) from the Instituto de Salud Carlos III (Spanish Ministry of Sciences, Innovation and Universities). MJSR was supported by a FOLIUM - FUTURMed Fellowship from IdISBa, co-founded by ITS-2017 and PO FSE 2014-2020 from the Balearic Islands. We thank our psychologist team (Marian Pérez, Elena Bonet, Toni Riera, Andrea Seguí, Beatriz González, Yasmina Castaño, María Barceló, Catalina Calafat, Miriam Kefauver, Desirée Guillem, Omar Vial, Adela Marmy, Inés Forteza-Rey, Patricia López, Ana Serapio, Catalina Moragues, Alberto Ávila and Ana Martí) for conducting the psychological assessments of the health care workers. We also thank Cristian Sánchez and Daniel Gallego for their support with data management and with technical assistance for health care workers who experienced difficulties in setting up the study Apps. We also thank Massimo de Faveri, Mercedes Martínez, and Alfonso Morillas, from Apploading, Inc, for making the Clinicovery app available for use in this study. Finally, we thank all the health care workers who participated in this study.
IRC, MJSR, MAF, and JGC designed the study. MJSR, AMYJ, MBV, MEGB, MGT, CS, RJ, EG, and JGC developed the contents of the PsyCovidApp intervention under the coordination of MJSR. MAF transferred the intervention contents to the Clinicovery app. MJSR and the psychologist team (see
MJSR, AMYJ, MBV, MEGB, MGT, CS, RJ, EG, and JGC developed the contents of the intervention. All other authors have no conflicts to declare.