This study aimed to evaluate the effectiveness of a smartphone app–based intervention program compared with a standard intervention program for caregivers in primary health care institutions. The usability and satisfaction of the app were a focus of this study as well. The hypothesis was that an app-based intervention program would improve caregivers’ mental health and decrease their burden.

A randomized controlled trial (RCT) was conducted. Participants were randomly assigned to either the experimental group or the control group (Figure 1). The RCT was registered in the International Standard Randomized Controlled Trial Number registry (ISRCTN: 14818443; May 24, 2019).

The study aimed to show differences between an app-based intervention and standard intervention with a standardized effect size (Cohen d) of 0.33 or larger. A standardized effect of 0.33 can be considered the lowest limit of a moderate clinical effect [39] and is based on a meta-analysis of well-being intervention research [40] and a recent RCT [41]. The total population of caregivers was uncertain, so the sample size was calculated considering an α risk of .05 and a β risk of .20. The minimum number of participants required to obtain evidential data of the results was 108 subjects, with 54 subjects each in the experimental and control groups. An estimated 30% loss to follow-up was taken into account for each group and a difference greater than or equal to 10 was recognized as statistically significant.

Primary health care is the first point of access to the public health system in Catalonia, Spain. The rest of the services in the system can be accessed by referral from primary care, except for medical emergencies, which can be accessed directly in the event of urgent need. The primary care center is the place for on-site care where people have to go when they have a health problem or want to prevent an illness. It provides diagnosis and care for the main health problem; health and social care; health promotion services; preventive, curative, and rehabilitative care; home care service; urgent or continuous care; and sexual health care. Primary care services are part of the basic common core portfolio, which are services fully covered by public funding.

The eligible caregivers were recruited by nurses from 7 primary health care centers in central and southern Catalonia. Each nurse invited caregivers they knew who met the criteria to participate in the study. The inclusion criteria for the participants were as follows: (1) primary or secondary nonprofessional caregiver of someone with a chronic disease; (2) over 18 years of age; (3) minimum of 4 months of experience as a caregiver; (4) knowledge of Spanish or Catalan (the app was available in both languages and the user could choose his/her preference); (5) user of a mobile device and the WhatsApp mobile app; (6) access to a mobile device with an Android operating system and internet access; and (7) signature of informed consent.

The online randomization procedure was carried out on an individual basis. Caregivers who agreed to take part in the study—who had a phone that supported the app and met the inclusion criteria—were randomly assigned. The randomization was stratified by the primary health center attended (7 centers in total), gender, age, and perceived level of well-being. A computer program allocated the participants using a generated randomization list. Given the nature of the study, it was not possible to blind the caregivers and the professionals.

Both study groups received the same standard intervention for caregivers by the nursing staff in the primary health care centers. Every year, the nurses assess the burden of the caregivers using the validated Zarit Burden Interview tool. If they identify a high level of burden, they refer the caregiver for psychological intervention. There is no specific protocol or intervention to promote positive attitudes toward caregiving.

The study was performed from September 2019 to November 2019. For the intervention group, primary outcome data were collected through the app, and secondary outcome data were collected by nurses through ad hoc questionnaires. There were three time points for data collection—at baseline (when the caregiver agreed to participate), and at 1 month and 3 months after baseline.

Descriptive statistics, including means for continuous variables and proportions for categorical variables, were used to summarize the characteristics of the participants. Analyses were conducted using an intention-to-treat analysis. Bivariate analysis was performed to calculate the mean change between the baseline and follow-up (at 1 month and 3 months). Next, to estimate the difference between the two groups, we calculated the difference between the mean change in the intervention group and the mean change in the control group. Due to the nonnormal distribution of the sample, nonparametric tests were used (Kolmogorov-Smirnov test, P<.001). To compare differences between the groups, the Mann-Whitney U test and Wilcoxon test were used. A P value ≤0.05 was considered significant. Cohen d analysis was performed to measure the effect size. Dropouts were not analyzed. Statistical analysis was performed using SPSS Statistics software (version 25 for Mac; IBM Corp).

If more than two items were missing from either of the instruments, the total score was not calculated and the data were considered missing. Missing data from the PMHQ and ZBI-7 were handled through average imputation of answered items, as long as no more than 40% of the items were missing. If more than 40% of the items were missing, the overall score was not calculated and the data were considered missing.

CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth) [45,46] was used to guide the design and implementation of the RCT. The researchers, who are experienced with clinical trials, monitored the study design, study protocols, patient recruitment, blinding, subject dropouts, and patient information confidentiality. The scrupulous study design ensured quality management and high external validity.

The study was approved by the Ethics Committee of Institut d’Atenció Primaria Jordi Gol (reference number: PI18/207). Caregivers were informed about the content, purpose, and procedure of the study. Written informed consent was given. Patients were reassured that their withdrawal would not prevent them from receiving the care that they would normally receive. The study was conducted in accordance with the principles of the Declaration of Helsinki, revised and updated, and followed Spain’s best practice guidelines (Buena Práctica Clínica). No negative impact was expected on the participants of the study. Data confidentiality was protected under the Spanish law governing the protection of personal data (Ley Orgánica 3/2018 de Protección de Datos de Carácter Personal). The participants were identified by research codes, and research information remained confidential.

A total of 152 caregivers were assessed for eligibility by the nurses in the primary health centers, and 131 agreed to participate and met the inclusion criteria. During the allocation into the intervention group, 4 caregivers were excluded because the app did not work properly on their smartphones. An additional 2 caregivers dropped out of the study because of the death of the person being cared for. Finally, 13 participants were lost to follow-up, including 7 participants in the intervention group who did not complete the app activities).

In all, 113 participants completed the trial: 56 in the intervention group and 57 in the control group. Thus, 79% (56/71) of the sample in the intervention group finished the intervention program. The loss to follow-up rate for the study was 13.7% (18/131) (Figure 3).

The primary outcome was assessed using the PMHQ and ZBI-7. One of the main differences between the groups was related to the higher baseline PMHQ scores in the control group than in the intervention group. In fact, F2–Prosocial attitude, F3–Self-control, and F5–Problem solving and self-actualization are the factors behind this difference. The scores between the groups on the other factors were similar (Table 4).

Assessments were conducted at baseline, and at 1 month and 3 months following the app intervention with the intervention group. Immediately after the intervention (Table 5), there were no statistically significant differences in changes in the PMHQ (P=.24) or ZBI-7 (P=.24) scores between the groups. There were statistically significant—but not clinically relevant—differences in the mean change in F1–Personal satisfaction (0.96; P=.03; d=–0.00) between the groups. Each group obtained statistically significant differences in ZBI-7 scores. While ZBI-7 scores decreased in the intervention group, they increased in the control group. However, there were no clinically relevant differences in the ZBI-7 scores between the groups.

The comparison of the mean changes at the 3-month follow-up (Table 6) revealed statistically significant differences in PMHQ scores between the intervention and control groups (11.43; P<.001; d=0.82). F5–Problem solving and self-actualization stood out for its clinically relevant mean difference of change (5.69; P<.001; d=0.71). However, F2–Prosocial attitude was the factor with the largest effect size (mean difference of change of 2.47; P<.001; d=1.18). F1–Personal satisfaction and F6–Interpersonal relationship skills had smaller effect sizes, but there were statistically significant differences between the groups (F1: 1.36; P=.05; d=0.36; and F6: 1.39; P=.04; d=0.25). F3–Self-control had a statistically significant difference and a moderate effect size (0.76; P=.03; d=0.50). F4–Autonomy did not show clinically relevant differences between the groups (–0.25; P=.99). ZBI-7 scores showed statistically significant differences with moderate clinically relevant results (–2.03; P<.001; d=–0.68).

The level of satisfaction with the app was high. Questions 1 to 3 on the ad hoc questionnaire related to the operating system and had an average score of 93.9%. Regarding the organization of the app (questions 4 and 5), users did not report many difficulties in using the app. Users liked the daily phrases, claiming that they helped to improve their mental health (Q6: 43/49, 87.8%). Also, 100% (49/49) of users rated the activities as being easy (Q7), and 91.8% (45/49) of users found the TIVA character to encourage them to continue using the app (Q8). The majority of caregivers (46/49, 93.9%) responded that they would recommend the app and 27 of 49 (56.3%) users indicated that they felt the intervention should last longer (Table 7).

This study aimed to asses the effectiveness of a digital intervention program to promote positive mental health among nonprofessional caregivers and to evaluate the usability and satisfaction of the app-based intervention program. The results demonstrated that the implementation of an app-based intervention program for caregivers significantly contributed to enhancing their positive mental health. In fact, results showed that the program produced a larger effect on the caregivers’ lives after 3 months of the intervention, which suggests that it is an effective long-term program. In addition, the participants in the study reported a high satisfaction rate with the app.

The intervention program seemed to be effective in relation to the differences in F1–Personal satisfaction scores between the groups at 1-month follow-up. Although the other factors did not reflect any significant differences between the groups, F4–Autonomy and F6–Interpersonal relationship skills obtained a positive mean difference of change, which reflects greater improvement of the scores in the intervention group. In addition to these program results, the level of burden of the control group seemed to increase after 1 month; in contrast, the ZBI-7 scores decreased in the intervention group, although the mean difference of change was not statistically significant.

The results seemed to be more satisfactory after 3 months of the intervention. There were significant differences between the groups on all factors, with the exception of F4–Autonomy, which was the only factor with scores that decreased from 1-month to 3-month follow-up.

The factor with the biggest increase after 3 months of intervention was F5–Problem solving and self-actualization. The mean change in the intervention group increased. These improvements could be related to the increase in confidence and caregiving information promoted by the program. A recent review about caregiver programs highlighted the importance of interventions that include program resolution strategies [47]. These strategies include the need for interventions to be easy and provide a connection with health care providers. Both elements are included in these app-based programs.

The app-based intervention program was also shown to be effective by comparing the scores of the intervention group on F2–Prosocial attitude, F3–Self-control, and F6–Interpersonal relationship skills with those of the control group. Programs involving group social support have already been tested and demonstrate their efficacy in caregivers. However, a few studies demonstrate that online strategies could promote those elements. In fact, a meta-analysis performed in 2017 highlighted the need to promote this type of program, as there are many isolated caregivers who cannot access on-site caregiver support programs [48]. The results of our study provide the first step in promoting this kind of program to foster prosocial attitudes through ICT strategies and with the support of primary health care nurses.

Another outcome of the study relates to F1–Personal satisfaction. In fact, this factor slightly increased in the intervention group after the first month, and decreased after 3 months. Therefore, interventions related to this factor are effective to prevent a further decrease over time. The research team believes that this result may be related to the messages of coaching and motivation that the app included, which were designed by the research team, including psychologists and nurses. Those motivational phrases were created with the aim of recognizing the caregiver’s task and its value.

In this section, it is important to highlight that after the app-based intervention program ended, users had the option of keeping the app on their smartphones, which would continue to display the question “Hello, how are you today?” every day. Furthermore, if they used the app, they could see the final evolution of TIVA and access the website [35]. Although not formally registered, this element seemed to satisfy the caregivers, who had made their request to the nurses responsible for data collection. A previous study shows that caregiver support is strongly related to their need for more care-related information and more information on available resources [49]. These needs could be related to our outcomes and to increased effectiveness in the intervention group after 3 months.

Results related to the ZBI-7 showed that after 3 months, the intervention group had a statistically significant decrease in caregiver burden, with a moderate size effect between the groups. However, researchers consider that these results are inconclusive considering the lack of information recorded in this study about the evolution of the person being cared for, which could have varied considerably after 3 months of intervention. Evidence shows how caregiver burden is directly associated with the caregiver’s duties, and further studies should evaluate the evolution of the characteristics of the caregiving situation [50,51].

The secondary outcome related to the app-based intervention program was satisfactory in terms of user acceptance. Users found the app easy to use, which was one of the goals of the app’s development. The literature highlights the need for these types of online programs to have an easy operating system to prevent dropouts [52,53]. According to informal feedback from caregivers and nurses, the character of the program, TIVA, was one of the reasons for the high degree of loyalty of the intervention program users to the app. In fact, many caregivers expressed several positive emotions after witnessing TIVA’s evolution every day.

Similar to applications designed for other types of users, new technologies designed to care for caregivers can increase satisfaction and motivation by empowering them, which promotes healthy changes in their behaviors and improves their quality of life [53]. Currently available applications have worked on aspects related more to the care of the chronic patient than to the well-being of the caregiver. With this application, and as recommended by other authors, behavioral interventions have been performed using “persuasive health technologies,” which are becoming much more promising approaches that encourage healthy behaviors [54]. Interventions that improve positive mental health should be an integral part of emotional support for caregivers and used in conjunction with advice from health professionals, improving their relationship without overloading face-to-face care. Most of these programs were conducted in a traditional face-to-face format. Nursing has the responsibility of initiating changes with the use of new technology and, as shown with the TIVA program, mHealth must be used as an indispensable working tool to reach all users, including those who cannot be reached in person [55].

Several studies emphasize the need to implement online intervention programs with gamification [21,52,53]. During the TIVA app’s creation, several options were considered to implement gamification in order to ensure the caregivers’ involvement during the 28 days of the intervention program. A game-based option was not considered, as it would have required more of the caregiver’s time and thus posed a risk of increasing dropouts.

On the other hand, the program allowed the self-management of time when planning the daily operation of the program. In this way, adherence was facilitated and the risk of dropouts was also reduced, as was also shown in another study [56].

Although the increase in the use of new technologies opens up a new space in the promotion of health for patients and caregivers, we identified in consonance with other authors that most of the existing digital support programs were more commercial than functional and it is necessary to promote studies that provide scientific evidence on the impact of these applications in the health field [57].

Therefore, the TIVA app seems to have been a good choice for this app-based intervention program. However, we can only express qualitative data on this outcome, so this should be assessed in further studies.

A number of important issues remain to be addressed by future research. First, the basic characteristics of the participants showed some differences. There was an 8-year difference in the mean ages of the groups, which was related to individuals’ knowledge of how to use a smartphone. Further studies may need to compare the same age ranges with a minimum sample size for each group. This was not possible in our case, as the sample was not large enough to do this comparison. A similar problem occurred in relation to the level of education: the intervention group appeared to have a higher level of education than the control group. This could also have led to a bias in the results given that previous studies have linked the level of education to better coping strategies and level of resilience [58-60]. Despite these differences in sociodemographic characteristics, both groups obtained similar scores on characteristics of well-being and the person being cared for, which strengthens the validity of the results. Nevertheless, further studies with larger sample sizes should be undertaken to avoid possible biases.

Second, the level of positive mental health measured using the PMHQ appeared to be different at baseline for the intervention and control groups. The intervention group had a significantly lower level of positive mental health. This could have resulted in a bias in our study, as it might have been easier to show an improvement in the intervention group than in the control group. However, user satisfaction seems to be consistent in the results. Nevertheless, we believe that further studies need to be conducted to consider the baseline level of positive mental health, as well as the type of caregiver, as criteria to be included during the randomization process to verify the results.

Third, another limitation of the study is related to the type of caregiver. There was a higher percentage of secondary caregivers in the intervention group than in the control group. However, as the analysis was conducted with a consideration of pre- and postintervention scores, significant differences are valid.

Fourth, at the baseline of the study, caregiver burden was assessed using the ZBI-7. At the same time, data related to the characteristics of the person being cared for and information about the caregiving tasks (including the time per week dedicated to caregiving and caregiving duties) were recorded. However, while the ZBI-7 was also assessed after 1 month and after 3 months, the rest of the information was not. We believe that results related to caregiver burden should be carefully considered, and further studies should include an assessment of changes of caregivers’ duties and characteristics of the person cared for throughout the study.

Fifth, the app-based intervention program includes the features described in the “Methods,” which includes a total of 20 activities. Those activities, developed by the research team, were created with consideration of the 10 recommendations from Lluch-Canut’s model for promoting positive mental health [36,37]. Each activity cannot be linked directly to a factor from the model, as each activity promotes more than one factor. This is the reason why a redesign of the activities/features cannot be linked directly to a factor. Based on the results of this study, we consider, as a future line of research, a qualitative evaluation of the actual design of the app-based intervention program with the research team, stakeholders, and caregivers who participated in this study in the intervention group.

The app-based intervention program analyzed in this study can be considered effective, with a high degree of user satisfaction. The development of a 28-day program with interventions based on Lluch-Canut’s positive mental health framework was successful in terms of user satisfaction and usability. The effectiveness of the program as it relates to positive mental health was demonstrated with positive results. However, we conclude that the program’s effectiveness should be verified by further studies where the baseline level of positive mental health and type of caregiver are included as randomization criteria.

This app-based intervention program showed increased positive mental health levels after 1 month and 3 months. The factors with the greatest long-term effect were F2–Prosocial attitude, F3–Self-control, and F5–Problem solving and self-actualization. However, activities related to F4–Autonomy should be revised, as the effectiveness was only maintained after 1 month and not after 3 months.

The app’s user satisfaction was encouraging. Users considered the app’s operational system and activities to be appropriate. The character created especially for the app, TIVA, with a programmed evolution after each activity, was considered to be a fundamental element to encourage users to continue with the app-based intervention program.

The results of the study encourage the promotion of app-based intervention programs for caregivers and endorse their effectiveness and user satisfaction. We believe that a further study involving stakeholders and participants should be considered to evaluate the adequacy of the activities in the actual app-based program in order to address the need to redesign activities/features of the app-based intervention program to increase the program’s effectiveness.