Background: Associations between higher levels of patient engagement and better health outcomes have been found in face-to-face interventions; studies on such associations with mobile health (mHealth) interventions have been limited and the results are inconclusive.
Objective: The objective of this study is to investigate the relationship between patient engagement in an mHealth intervention and depressive symptoms using repeated measures of both patient engagement and patient outcomes at 4 time points.
Methods: Data were drawn from a randomized controlled trial (RCT) of an mHealth intervention aimed at reducing depressive symptoms among people living with HIV and elevated depressive symptoms. We examined the association between patient engagement and depressive symptoms in the intervention group (n=150) where participants received an adapted cognitive-behavioral stress management (CBSM) course and physical activity promotion on their WeChat social media app. Depressive symptoms were repeatedly measured using the Patient Health Questionnaire (PHQ-9) at baseline and 1 month, 2 months, and 3 months. Patient engagement was correspondingly measured by the completion rate, frequency of items completed, and time spent on the program at 1 month, 2 months, and 3 months. Latent growth curve models (LGCMs) were used to explore the relationship between patient engagement and depressive symptoms at multiple time points in the intervention.
Results: The mean PHQ-9 scores were 10.2 (SD 4.5), 7.7 (SD 4.8), 6.5 (SD 4.7), and 6.7 (SD 4.1) at baseline, 1 month, 2 months, and 3 months, respectively. The mean completion rates were 50.6% (SD 31.8%), 51.5% (SD 32.2%), and 50.8% (SD 33.7%) at 1, 2, and 3 months, respectively; the average frequencies of items completed were 18.0 (SD 14.6), 32.6 (SD 24.8), and 47.5 (SD 37.2) at 1, 2, and 3 months, respectively, and the mean times spent on the program were 32.7 (SD 66.7), 65.4 (SD 120.8), and 96.4 (SD 180.4) minutes at 1, 2, and 3 months, respectively. LGCMs showed good model fit and indicated that a higher completion rate (β at 3 months=–2.184, P=.048) and a greater frequency of items completed (β at 3 months=–0.018, P=.04) were associated with fewer depressive symptoms at 3 months. Although not significant, similar trends were found in the abovementioned relationships at 1 and 2 months. There was no significant relationship between time spent on the program and depressive symptoms.
Conclusions: This study revealed a positive association between patient engagement and health outcomes at 3 months of an mHealth intervention using LGCMs and repeated measures data. The results underscore the importance of improving patient engagement in mHealth interventions to improve patient-centered health outcomes.
Trial Registration: Chinese Clinical Trial Registry ChiCTR-IPR-17012606; https://tinyurl.com/yxb64mef
International Registered Report Identifier (IRRID): RR2-10.1186/s12889-018-5693-1
Mobile health (mHealth) has increasingly become a promising tool to deliver treatments for a range of psychosocial disorders [- ]. Studies have shown that mHealth interventions are effective in reducing depressive symptoms among people living with HIV and elevated depressive symptoms [ - ]. In comparison with traditional face-to-face interventions, mHealth interventions have the potential to implement personalized and cost-effective interventions, provide real-time feedback, and overcome geographical and logistical barriers [ ].
Despite the growing success of mHealth interventions, the relationship between patient engagement and intervention outcomes is still unclear. Patient engagement is defined as the extent to which participants actively interact with intervention content . In face-to-face randomized controlled trials (RCTs), patient engagement has been found to be an essential predictor of intervention outcomes, with higher levels of patient engagement predicting better health outcomes of participants [ ]. However, such a relationship has not yet been confirmed in mHealth interventions [ ]. Findings regarding patient engagement and intervention outcomes in mHealth interventions are mixed. Some studies found a relationship similar to that observed in face-to-face interventions [ - ], while others found no significant relationship [ - ].
Understanding the relationship between patient engagement and health outcomes in mHealth interventions is critical for developing and implementing effective mHealth programs . In most mHealth studies, patient engagement was only assessed at a single point in time and changes in patient outcomes were typically evaluated between pre- and postintervention [ ]. There is a lack of examination of dose-response relationships between patient engagement and treatment outcomes during the entire course of the interventions. It is unclear whether such a dose-response relationship exists in mHealth interventions, and if yes, whether such a relationship occurs early in the intervention as demonstrated in some face-to-face interventions [ ] or appears later in the intervention. Existing mHealth studies have varying intervention durations, which may lead to differential results of the dose-response relationship. For example, for the same purpose of reducing depressive symptoms among the general population, one mHealth intervention lasted 4 weeks whereas another was 24 weeks in duration [ , ]. With different lengths of treatment, it might be difficult to compare the dose-response relationship in these mHealth interventions, especially when the relationship was only evaluated pre- and postintervention. More studies are needed to explore the potentially changing relationship between patient engagement and health outcomes during the course of mHealth interventions. Longitudinal analyses with repeated measures of both patient engagement and treatment outcomes are likely to improve our understanding of the dose-response relationship throughout the course of an intervention.
To fill the gaps in the existing literature, this study aimed to explore the relationship between patient engagement and depressive symptoms during the 3-month course of an RCT of an mHealth program for people living with HIV and elevated depressive symptoms, the Run4Love intervention . We used 3 measures of patient engagement to capture multiple aspects of this construct in the study, as suggested by a recent systematic review on adherence reporting in online RCTs [ ]. Using 4 time points of outcome measures and the corresponding patient engagement measures, we examined the levels of and changes in patient engagement and depressive symptoms at 1, 2, and 3 months and the relationship between patient engagement and depressive symptoms at 1, 2, and 3 months.
This study is a secondary analysis of data from the intervention group of a 3-month RCT (Chinese Clinical Trial Registry [ChiCTR-IPR-17012606]) that was conducted to examine the efficacy of a WeChat-based intervention on reducing depressive symptoms among people living with HIV and elevated depressive symptoms . WeChat, similar to WhatsApp, is a widely used social media platform for instant communication in China with 1.1 billion active users [ ]. Participants were recruited from the outpatient department of a large hospital designated for HIV treatment in 2017 in Guangzhou, the third largest city in China, with a population of more than 13 million [ ]. A total of 300 people living with HIV who had elevated depressive symptoms (Center for Epidemiological Studies Depression Scale [CES-D] score ≥16) and used WeChat were randomized into the intervention or wait-list control group in a 1:1 ratio. The institutional review board of Sun Yat-sen University approved the intervention protocol, and the detailed recruitment and randomization procedure were described elsewhere [ ].
In addition to routine care, the Run4Love intervention consisted of two major components, an adapted cognitive-behavioral stress management (CBSM) course and physical activities promotion. The adapted CBSM course included content covering coping skills, muscle relaxation, and meditation tutorials. The physical activities promotion consisted of information on the benefits of regular exercise, exercise guidance, and healthy dietary suggestions. During the 3-month intervention, multimedia items were delivered to participants to support the adapted CBSM course and physical activities promotion via WeChat in the form of short articles, audio recordings, and posters almost once a day (5 to 6 items per week). Out of 65 items in total, 29 were short articles with an average of 1300 words, requiring about 5 minutes to read; 24 were posters with motivational messages, requiring about 30 seconds to read; and 12 were audio recordings in Chinese requiring 5 to 10 minutes to listen to. The WeChat-based Run4Love platform with extended functions was used to automatically deliver intervention items and collect real-time data on patient engagement (eg, time spent on an article, audio recording, or poster). The intervention design has been detailed elsewhere .
During the intervention, we used several measures to promote engagement: (1) automatically sending feedback on each participant’s compliance via WeChat on a weekly basis (eg, number of items completed and not completed); (2) giving verbal encouragement and financial incentives (up to US $2) via WeChat on a weekly basis based on the level of compliance; and (3) conducting motivational interviews by phone to sustain participation and help overcome barriers of compliance at 1 week, 1 month, and 2 months after baseline.
Depressive symptoms were assessed by the Chinese version of the Patient Health Questionnaire (PHQ-9) at baseline, 1 month, 2 months, and 3 months . Compared with the 20-item measurement of CES-D, PHQ-9 is shorter but has demonstrated both good reliability and validity in the Chinese populations [ , ]. The scale consists of 9 items such as “Little interest or pleasure in doing things” and “Feeling down, depressed, or hopeless” to measure the depressive status of the participants over the previous 2 weeks. Each item was rated on a 4-point Likert scale ranging from 0 (not at all) to 3 (nearly every day), providing a 0 to 27 total severity score, with higher scores indicating increased depressive symptoms [ ]. A good reliability of PHQ-9 was shown in the study, and the Cronbach alphas were .814, .861, .872, and .852 at baseline, 1 month, 2 months, and 3 months, respectively.
Three cumulative measures, including completion rate, frequency of items completed, and time spent on the program, were used to assess patient engagement in this study. Completion rate was the ratio of the number of items a patient completed to the number of items assigned, where the items that had been clicked were considered completed. Frequency of items completed was assessed not only by the number of items read or listened to by the participants, but also by the number of times the items were repeatedly read or listened to. Compared with the completion rate, frequency of items completed captured the repeat aspect of patient engagement. For example, if the same item was read or listened to twice by a participant, frequency was counted as 2 and the frequency of items completed for the participant was the accumulative frequency for all the items. The time spent on the program was measured by the overall time a participant spent on the Run4Love program, including reading or listening to the delivered items during the course of the intervention. We repeatedly assessed these 3 measures of patient engagement in conjunction with measurements of depressive symptoms that were evaluated at 1, 2, and 3 months after baseline, as has been recommended in previous studies . Specifically, all 3 measures at 1, 2, and 3 months were used to assess the average patient engagement from baseline until 1, 2, and 3 months after baseline.
Sociodemographic characteristics and HIV-related variables were collected at baseline including age, gender, sexual orientation, education, marital status, employment status, income, and duration of HIV infection (years). Income was measured by asking the participants whether they had adequate income to cover their daily expenses (adequate or inadequate).
Descriptive statistics on depressive symptoms, 3 measures of patient engagement, and baseline characteristics were provided. Mean and standard deviation were used to describe continuous variables with normal distribution, median and interquartile range (IQR) for continuous variables that did not follow a normal distribution, and frequency and percentage for categorical variables. Latent growth curve models (LGCMs) were constructed to examine the time-varying associations between patient engagement and depressive symptoms using repeated measure data at 4 time points . Maximum likelihood estimation with robust standard errors (MLR) was used to handle missing data and obtain parameter estimates [ ]. Descriptive analyses were conducted in SPSS Statistics version 20.0 (IBM Corporation), and LGCMs were conducted in Mplus version 7 (Muthen & Muthen).
The analyses of LGCMs were conducted in two steps . First, we developed an unconditional LGCM to estimate the growth trajectory of depressive symptoms across time. The initial level of depressive symptoms was represented as the intercept, and the average rate of change was represented as the slope in the model. For the intercept factor, all loadings were fixed to 1. For the slope factor, the first and second loadings were fixed to 0 and 1, and the remaining loadings were freely estimated from the data to test whether there was a linear or nonlinear growth trajectory (eg, linear growth pattern: 0, 1, 2, 3; nonlinear growth pattern: the third and fourth loadings significantly differ from 2 and 3, respectively).
Second, we developed 3 conditional LGCMs to examine the effects of patient engagement on depressive symptoms based on the 3 measures of patient engagement. The conditional model was extended from the unconditional model to incorporate one measure of patient engagement and related baseline characteristics as covariates. Since no baseline characteristics were found to significantly associate with the 3 measures of patient engagement or the outcome of depressive symptoms in this study, we included variables such as education, income, and duration of HIV infection that were associated with depressive symptoms among people living with HIV in the literature as control variables in the conditional models [- ]. Associations between patient engagement and depressive symptoms were treated as time-varying effects, which were specified by regressing depressive symptoms (eg, PHQ-9 at 1 month) on the corresponding patient engagement variable (eg, completion rate at 1 month). Patient engagement, therefore, could be considered as a time-specific predictor of depressive symptoms after controlling for the influences of the underlying growth trajectory of depressive symptoms and baseline characteristics [ ].
Model fit of LGCM was evaluated using chi-square tests and indices including the standardized root mean of residual (SRMR), confirmatory fit index (CFI), Tucker-Lewis index (TLI), and root mean-squared error of approximation (RMSEA). The criteria used to determine adequate model fit included: CFI≥0.95, TLI≥0.95, SRMR≤0.08, and RMSEA≤0.08 [- ].
As shown in, the mean age of participants was 28.0 (SD 5.8) years and the average duration of HIV infection was 2.4 (SD 2.3) years. The participants were predominately male (142/150, 94.7%), nonheterosexual (130/150, 86.7%), well-educated (98/150, 65.3%, with at least some college education), single (132/150, 88.0%), employed (123/150, 82.0%), and had adequate income to cover daily expenses (129/150, 86.0%).
|Age in years, mean (SD)||28.0 (5.8)|
|Gender, n (%)|
|Sexual orientation, n (%)|
|Homosexual, bisexual, uncertain||130 (86.7)|
|Education, n (%)|
|Below or at high school level||52 (34.7)|
|Above high school level||98 (65.3)|
|Marital status, n (%)|
|Single, divorced, widowed||132 (88.0)|
|Employment status, n (%)|
|Income, n (%)|
|Adequate to cover daily expenses||129 (86.0)|
|Inadequate to cover daily expenses||21 (14.0)|
|Duration of HIV infection in years, mean (SD)||2.4 (2.3)|
Change in Depressive Symptoms and Patient Engagement Over Time
Of the participants, 87.3% (131/150), 76.0% (114/150), and 92.0% (138/150) completed the online questionnaire for depressive symptoms at 1, 2, and 3 months, respectively, after baseline. Compared with the depressive symptom score at baseline (PHQ-9: mean 10.2 [SD 4.5]), all follow-up assessments reported a decrease in depressive symptoms with mean values of 7.7 (SD 4.8), 6.5 (SD 4.7), and 6.7 (SD 4.1) at 1, 2, and 3 months, respectively ().
Nearly all participants (149/150) read or listened to at least one item during the 3-month intervention. A moderate level of patient engagement was found during the intervention process. The completion rate remained around 50.0% (33/65) during the course of the intervention, with a mean of 50.6% (SD 31.8%), 51.5% (SD 32.2%), and 50.8% (SD 33.7%) at 1, 2, and 3 months, respectively. The mean frequency of items completed was 18.0 (SD 14.6), 32.6 (SD 24.8), and 47.5 (SD 37.2) at 1, 2, and 3 months, respectively. The mean time spent on the program was 32.7 (SD 66.7), 65.4 (SD 120.8), and 96.4 (SD 180.4) minutes at 1, 2, and 3 months, respectively. When comparing the mean and median scores in, we found that time spent on the program was positively skewed, with means much larger than the medians at 1, 2, and 3 months.
|Baseline||1 month||2 months||3 months|
|Mean (SD)||Median (IQR)||Mean (SD)||Median (IQR)||Mean (SD)||Median (IQR)||Mean (SD)||Median (IQR)|
|PHQ-9b||10.2 (4.5)||10.0 (7.0-13.0)||7.7 (4.8)||8.0 (4.0-11.0)||6.5 (4.7)||6.0 (3.0-9.0)||6.7 (4.1)||7.0 (3.0-9.0)|
|Completion rate (%)||—c||—||50.6 (31.8)||54.2 (16.7-81.8)||51.5 (32.2)||53.0 (20.6-81.7)||50.8 (33.7)||47.6 (16.9-85.3)|
|Frequency of items completed||—||—||18.0 (14.6)||17.0 (5.8-26.0)||32.6 (24.8)||29.5 (10.0-51.3)||47.5 (37.2)||41.0 (11.0-78.3)|
|Time spent on program (min)||—||—||32.7 (66.7)||9.7 (2.5-42.1)||65.4 (120.8)||22.0 (5.3-80.3)||96.4 (180.4)||31.3 (6.7-117.2)|
aFor PHQ-9, the sample sizes were 150, 131, 114, and 138 at baseline, 1 month, 2 months, and 3 months, respectively. For patient engagement, the sample size remained 150 across the 3-month intervention.
bPHQ-9: Patient Health Questionnaire.
Unconditional Latent Growth Curve Model
Results of the unconditional model indicated that depressive symptoms decreased during the 3-month intervention, and the largest reduction occurred at 1 month after baseline. This model had good model fit (). The mean intercept was 10.157 (P<.001), indicating a mean PHQ-9 score of 10.157 at baseline. The mean slope was –2.336 (P<.001), and the factor loadings on the slope were 0, 1, 1.565, and 1.425 at baseline, 1 month, 2 months, and 3 months, respectively. These results indicated a nonlinear pattern of the reduction of depressive symptoms: a rapid decrease of PHQ-9 score (2.336 points) occurred at the first month, and then the magnitude of reduction flattened out. Variances of the intercept and slope were 15.069 (P=.003) and 5.564 (P=.12), indicating significant individual differences in depressive symptoms at baseline, but no individual differences in the rates of change over time. Covariance between the intercept and slope was –4.381 (P=.25), suggesting that the rate of change in depressive symptoms was independent of its initial levels ( ).
|Growth parameters||Estimate||SE||P value|
|Variance of intercept||15.069||5.100||.003|
|Variance of slope||5.564||3.614||.12|
|Covariance of intercept and slope||–4.381||3.799||.25|
aPHQ-9: Patient Health Questionnaire.
Conditional Latent Growth Curve Models
Three conditional LGCMs were constructed, each using one measure of patient engagement as a time-varying variable. Results of the conditional models indicated that higher completion rate and greater frequency of items completed were significantly associated with lower depressive symptoms at 3 months, but there was no significant relationship between time spent on the program and depressive symptoms over the course of the intervention. All conditional models reported good model fit (- ).
Based on the literature, baseline characteristics of education, income to cover daily expenses, and duration of HIV infection were included as control variables. As shown in, education and duration of HIV infection did not have significant effect either on the initial levels (intercept) or the rate of change (slope) of depressive symptoms in all conditional models. Income had a significant effect on the intercept in all conditional models, suggesting that participants with adequate income to cover daily expenses had a lower level of depressive symptoms at baseline. The time-varying variable, patient engagement, had a significant effect on depressive symptoms at 3 months after baseline (completion rate: β3=–2.184, P=.048; frequency of items completed: β3=–0.018, P=.04), whereas the effect of patient engagement, regardless of the measurement, was not significant on depressive symptoms at 1 or 2 months. Specifically, participants who had higher completion rates or greater frequency of items completed were more likely to report fewer depressive symptoms at 3 months after controlling for the underlying growth trajectory of depressive symptoms and baseline characteristics. By contrast, the time spent on the program was not significantly associated with depressive symptoms throughout the intervention process (β1=0.004, P=.29; β2=–0.001, P=.74; β3=–0.002, P=.16), suggesting that the amount of time patients spent on the intervention program was not related to the changes in depressive symptoms.
|Covariate parameters||Completion rate, estimate (SE)||Frequency of items completed, estimate (SE)||Time spent on program, estimate (SE)|
|Education→intercept||–0.893 (0.858)||–0.914 (0.849)||–0.909 (0.843)|
|Income→intercept||–2.329b (1.179)||–2.306b (1.175)||–2.309b (1.169)|
|Duration of HIV infection→intercept||0.064 (0.165)||0.058 (0.157)||0.061 (0.156)|
|Education→slope||–0.257 (0.650)||–0.274 (0.632)||–0.239 (0.628)|
|Income→slope||1.382 (0.860)||1.298 (0.899)||1.399 (0.898)|
|Duration of HIV infection→slope||0.103 (0.155)||0.116 (0.147)||0.092 (0.139)|
|Measures of patient engagement|
|Engagement1→PHQ-91a||–1.221 (0.881)||–0.025 (0.019)||0.004 (0.004)|
|Engagement2→PHQ-92||–1.813 (1.158)||–0.026 (0.014)||0.001 (0.002)|
|Engagement3→PHQ-93||–2.184b (0.945)||–0.018b (0.009)||–0.002 (–0.082)|
aPHQ-9: Patient Health Questionnaire.
This study was among the first to explore dose-response relationship between patient engagement and depressive symptoms in a social media–based mHealth intervention among people living with HIV using longitudinal data. Three measures of patient engagement were used to assess different aspects of the construct. By investigating the time-varying relationship between patient engagement and health outcomes, our study provided a better understanding of the dose-response relationship in mHealth interventions and the time of occurrence of this association. Results of LGCMs revealed that two measures of patient engagement, completion rate and frequency of items completed, were significantly associated with reduced depressive symptoms, and these relationships occurred at 3 months of the intervention but not at 1 or 2 months. However, time spent on the program was not significantly related to depressive symptoms throughout the intervention.
First and foremost, this study provides new evidence on the dose-response relationship in mHealth interventions as the literature in this regard was limited and inconclusive . Potential reasons for the ambivalent relationship in previous mHealth intervention studies could include inadequate research design or implementation, inadequate intervention durations for significant treatment effect, low levels of patient engagement, high dropout rate, and/or inappropriate or inadequate measurements of patient engagement or data analysis methods [ , , ]. This study, however, shows that an evidence-based mHealth intervention with rigorous design and implementation could produce significant treatment effects, in particular, for participants with better engagement.
Our data suggests that the significant relationship of patient engagement and health outcomes did not occur early (1 or 2 months) in the intervention, but only at 3 months. Few studies have explored the dose-response relationship across the span of an intervention, especially in mHealth programs. To our knowledge, only one mHealth study intended to explore the patient engagement–outcome relationship using multiple measures of patient engagement . However, the web-based iCBT intervention conducted in a community setting in Dublin, Ireland, in 2014 only measured pre- and postintervention depressive symptoms [ ]. The methodology of repeated-measures analysis of variance used in the study could only reveal that patients with significant improvement in depressive symptoms had higher levels of patient engagement during the 8-week intervention [ ]. Although the study further found that patients tended to engage in the online program more in the first half of the program (1 to 4 weeks) than in the latter half (5 to 8 weeks) [ ], the design and findings of the study could not identify when the linkage between patient engagement and outcome occurred during the course of the intervention.
Compared with mHealth interventions, face-to-face studies have focused more on the relationship between patient engagement and health outcomes. One face-to-face intervention investigated the dose-response relationship in the first 5 weeks of the intervention using both weekly engagement measures and depressive symptoms outcomes among people with major depressive disorder . However, the study used repeated measures regression analysis that could only conclude that there was a significant dose-response relationship during the 5-week intervention but not in which week(s) the relationship started to appear. Another face-to-face study conducted among patients with diabetes in Netherlands investigated the dose-response relationship between patient engagement and depressive symptoms during an 8-week intervention [ ]. Patient engagement and depressive symptoms were repeatedly measured at 2, 4, and 8 weeks [ ]. Although the study used multilevel analysis that was able to identify both the dose-response relationship and timing of the occurrence of any relationship, the study did not find a dose-response relationship in any of the time points [ ]. One potential explanation for the nonsignificant dose-response relationship might be that individuals with chronic diseases and depressive symptoms need a longer time of exposure to an intervention to obtain the beneficial outcomes. Likewise, our study did not find a dose-response relationship at 1 or 2 months of the intervention but only at 3 months of the intervention among people living with HIV and depressive symptoms. Such a relationship among individuals with chronic diseases and psychological disorders needs to be further explored in future studies. Although the nonsignificant dose-response relationship at 1 or 2 months seemed to be contradictory to the finding that most reduction of depressive symptoms occurred at 1 month, reasons for the reduction of depressive symptoms could be many. Besides patient engagement, other factors such as social support might also play a role in the reduction of depressive symptoms. For example, several phone calls made by research staff to motivate participants to continue the intervention at 1 week and 1 and 2 months after baseline may provide social support needed by people living with HIV and elevated depressive symptoms.
Identifying the time point of effective dose-response relationship between patient engagement and treatment outcomes is critical for optimizing the design, pace, and duration of interventions. Unnecessarily lengthy interventions waste resources and lead to high dropout rates , whereas interventions with inadequate duration may not generate intended outcomes [ ]. Repeatedly examining the relationship between patient engagement and health outcomes allows us to identify the optimal time for intervention. This study demonstrates such measurement in an mHealth intervention.
In addition to identifying the timing of occurrence of the dose-relationship in the intervention, this study also found the differential relationships between different measures of patient engagement and health outcomes. Compared with face-to-face interventions, mHealth interventions have more measurement tools to capture different aspects of patient engagement . These different measurements may not have similar impacts on patient outcomes and may explain the conflicting results regarding the dose-response relationships in mHealth interventions [ ].
In this study, participants with higher completion rates reported fewer depressive symptoms at 3 months. On average, participants completed 50.8% (33/65) of the program items (ie, short articles, audio recordings, and posters) at 3 months, which was similar to the completion rates reported in other online interventions for depressive symptoms [, ]. A recent meta-analysis found a significant association between higher completion rates and reduced depressive symptoms in online programs [ ]. However, some more recent internet-based RCTs did not find such a relationship [ , ]. One potential reason for the nonsignificant relationship was low levels of patient engagement in these mHealth interventions [ ]. Therefore, adherence to the program needs to be underscored in mHealth interventions, especially considering that most of the existing mHealth interventions have relatively low completion rates [ , ].
Frequency of items completed was another patient engagement measure we used in the study, and it was also positively associated with patients’ outcomes. Such a measure of patient engagement has rarely been reported in the existing mHealth interventions. At 3 months of the intervention, participants had read or listened to the items 47.5 times on average. As the average number of items completed by participants was 33, each item was repeatedly read or listened to for approximately 1.5 times (47.5/33). This measure captured an important aspect of patient engagement as it reflected to what extent participants were willing to repeatedly engage with the intervention items. Based on this frequency data, we subsequently chose items that had been most read or listened to and re-sent these items to the patients in a booster session 3 months after the intervention . In a separate study, we found a continued intervention effect with further reductions of patients’ depressive symptoms at 3 and 6 months after the intervention [ ].
In contrast, the third patient engagement measure, time spent on the program, was found not to relate to the reduced depressive symptoms, which was consistent with previous studies [, ]. Time spent on the program may not reflect the actual levels of patient engagement as it is likely to be influenced by a variety of factors such as reading speed, cognitive capacity, and familiarity with the program [ ]. Participants might keep the program on without actively engaging. For example, patients might fall asleep when listening to a relaxation audio recording; yet time stayed on the page was still counted as time spent on the program. In this study, there was a noticeable difference (31.3 vs 96.7 minutes) between the median and mean time spent on the platform at 3 months of the intervention, indicating a positively skewed distribution of patients’ time spent online, suggesting few patients spent extra-long time on the platform. Future interventions should incorporate a time-out function to better capture the actual patient engagement.
Several limitations in this study should be noted. First, although the sample was drawn from a large hospital designated for HIV treatment in Guangzhou with more than 10,000 HIV seropositive patients on treatment, participants were mostly from an urban setting and the majority of them were nonheterosexual young men. Thus, the generalizability of the findings to other people living with HIV or other populations may be limited. Second, although depressive symptoms are the major outcome in this study, more types of outcome measures such as other psychological indicators (eg, anxiety) and/or biomarkers (eg, hair or salivary cortisol concentration) can be incorporated in future studies to provide further evidence on dose-response relationship between patient engagement and intervention outcomes. Third, while measures of patient engagement were collected objectively through the Run4Love platform, depressive symptoms were self-reported and might suffer from social desirability and recall bias. Fourth, there might be measurement biases in assessing patient engagement. For example, when clicked, a material was readily considered as completed. Nevertheless, this definition of completion was widely used in mHealth interventions, and the relevant patient engagement measures served as reliable and efficient metrics to assess dose-response relationship . Additionally, time spent on the program might not capture the real patient engagement as it was not able to differentiate between active and inactive engagement and time on distraction or interruption could not be measured. Finally, we did not differentiate patient engagement in different formats of the intervention items (ie, short articles, audio recordings, and posters) nor did we examine the relationship between the content of the Run4Love intervention and patient engagement in this study, as it goes beyond the purposes of this study. We call for future research to further explore the relationships between patient engagement and the content of the intervention as well as the potential differences in patient engagement by different types of multimedia items in mHealth interventions.
This study indicated that higher completion rate or greater frequency of accessing items in an mHealth intervention contributed to reduced depressive symptoms among people living with HIV and elevated depressive symptoms at 3 months in the intervention. Given the observed positive impact of patient engagement on intervention outcomes, adherence to mHealth interventions should be evaluated and strengthened as a critical and integral strategy to improve treatment effects. Approaches for promoting patient engagement include enhancing patients’ initial motivation in program participation, education on the importance of active participation, frequent and effective reminders, financial and verbal incentives, and engaging patients in goal setting (eg, physical activities) [- ]. Future studies with repeated assessments of patient engagement and outcomes throughout the intervention process have the potential to analyze the time-varying dose-response relationship. Such studies would optimize the intervention duration and maximize intervention outcomes through improving the design and implementation of mHealth interventions.
In conclusion, this study revealed a positive association between patient engagement and reduction in depressive symptoms at 3 months of an mHealth intervention using latent growth curve models and repeated measure data from 4 time points. The results underscore the importance of enhancing patient engagement in mHealth interventions to improve health outcomes of the participants.
This study was supported by the National Natural Science Foundation of China (Grant No 71573290) and China Medical Board open competition funding (Grant No. 17-271). The funding sources had no role in the study design, data collection, statistical analyses, and writing of the manuscript.
YG and YZ had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. YG, YAH, and YZ were responsible for the concept and design of the study. YZ, YL, MZ, HZ, CZ, SW, CP, and YH were responsible for the acquisition, analysis, or interpretation of data. YZ and YG drafted the manuscript. YG, YZ, YAH, AMW, YH, and RTHH completed the critical revision of the manuscript for important intellectual content. LL, WC, CL, and YH were responsible for administrative, technical, and material support, and YG supervised the study.
Conflicts of Interest
- McManus RJ, Mant J, Bray EP, Holder R, Jones MI, Greenfield S, et al. Telemonitoring and self-management in the control of hypertension (TASMINH2): a randomised controlled trial. Lancet 2010 Jul 17;376(9736):163-172. [CrossRef] [Medline]
- Sawesi S, Rashrash M, Phalakornkule K, Carpenter JS, Jones JF. The Impact of Information Technology on Patient Engagement and Health Behavior Change: A Systematic Review of the Literature. JMIR Med Inform 2016;4(1):e1 [FREE Full text] [CrossRef] [Medline]
- Ben-Zeev D, Scherer EA, Gottlieb JD, Rotondi AJ, Brunette MF, Achtyes ED, et al. mHealth for schizophrenia: patient engagement with a mobile phone intervention following hospital discharge. JMIR Ment Health 2016;3(3):e34 [FREE Full text] [CrossRef] [Medline]
- Cook DJ, Manning DM, Holland DE, Prinsen SK, Rudzik SD, Roger VL, et al. Patient engagement and reported outcomes in surgical recovery: effectiveness of an e-health platform. J Am Coll Surg 2013 Oct;217(4):648-655. [CrossRef] [Medline]
- van Luenen S, Garnefski N, Spinhoven P, Kraaij V. Guided internet-based intervention for people with HIV and depressive symptoms: a randomised controlled trial in the Netherlands. Lancet HIV 2018 Dec;5(9):e488-e497. [CrossRef] [Medline]
- Cheng LJ, Kumar PA, Wong SN, Lau Y. Technology-delivered psychotherapeutic interventions in improving depressive symptoms among people with HIV/AIDS: a systematic review and meta-analysis of randomised controlled trials. AIDS Behav 2020 Jun;24(6):1663-1675. [CrossRef] [Medline]
- Karyotaki E. Internet-based interventions for people with HIV and depression. Lancet HIV 2018 Dec;5(9):e474-e475. [CrossRef] [Medline]
- Guo Y, Hong YA, Cai W, Li L, Hao Y, Qiao J, et al. Effect of a WeChat-based intervention (Run4Love) on depressive symptoms among people living with HIV in China: a randomized controlled trial. J Med Internet Res 2020 Feb 09;22(2):e16715 [FREE Full text] [CrossRef] [Medline]
- Rathbone AL, Prescott J. The use of mobile apps and SMS messaging as physical and mental health interventions: systematic review. J Med Internet Res 2017 Aug 24;19(8):e295 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
- Mausbach BT, Moore R, Roesch S, Cardenas V, Patterson TL. The relationship between homework compliance and therapy outcomes: an updated meta-analysis. Cognit Ther Res 2010 Oct;34(5):429-438 [FREE Full text] [CrossRef] [Medline]
- Beintner I, Vollert B, Zarski A, Bolinski F, Musiat P, Görlich D, et al. Adherence reporting in randomized controlled trials examining manualized multisession online interventions: systematic review of practices and proposal for reporting standards. J Med Internet Res 2019 Aug 15;21(8):e14181 [FREE Full text] [CrossRef] [Medline]
- Enrique A, Palacios JE, Ryan H, Richards D. Exploring the relationship between usage and outcomes of an internet-based intervention for individuals with depressive symptoms: secondary analysis of data from a randomized controlled trial. J Med Internet Res 2019 Aug 01;21(8):e12775 [FREE Full text] [CrossRef] [Medline]
- Couper MP, Alexander GL, Zhang N, Little RJA, Maddy N, Nowak MA, et al. Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res 2010;12(4):e52 [FREE Full text] [CrossRef] [Medline]
- Mattila E, Lappalainen R, Välkkynen P, Sairanen E, Lappalainen P, Karhunen L, et al. Usage and dose response of a mobile acceptance and commitment therapy app: secondary analysis of the intervention arm of a randomized controlled trial. JMIR Mhealth Uhealth 2016 Jul 28;4(3):e90 [FREE Full text] [CrossRef] [Medline]
- Carroll KM, Ball SA, Martino S, Nich C, Babuscio TA, Nuro KF, et al. Computer-assisted delivery of cognitive-behavioral therapy for addiction: a randomized trial of CBT4CBT. Am J Psychiatry 2008 Jul;165(7):881-888 [FREE Full text] [CrossRef] [Medline]
- Hadgkiss EJ, Jelinek GA, Taylor KL, Marck CH, van der Meer DM, Pereira NG, et al. Engagement in a program promoting lifestyle modification is associated with better patient-reported outcomes for people with MS. Neurol Sci 2015 Jun;36(6):845-852 [FREE Full text] [CrossRef] [Medline]
- Van Gemert-Pijnen JE, Kelders SM, Bohlmeijer ET. Understanding the usage of content in a mental health intervention for depression: an analysis of log data. J Med Internet Res 2014;16(1):e27 [FREE Full text] [CrossRef] [Medline]
- Geramita EM, Herbeck Belnap B, Abebe KZ, Rothenberger SD, Rotondi AJ, Rollman BL. The association between increased levels of patient engagement with an internet support group and improved mental health outcomes at 6-month follow-up: post-hoc analyses from a randomized controlled trial. J Med Internet Res 2018 Jul 17;20(7):e10402 [FREE Full text] [CrossRef] [Medline]
- Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, et al. The association between adherence and outcome in an Internet intervention for depression. J Affect Disord 2018 Dec 15;229:443-449. [CrossRef] [Medline]
- Donkin L, Hickie IB, Christensen H, Naismith SL, Neal B, Cockayne NL, et al. Rethinking the dose-response relationship between usage and outcome in an online intervention for depression: randomized controlled trial. J Med Internet Res 2013 Oct;15(10):e231 [FREE Full text] [CrossRef] [Medline]
- Conklin LR, Strunk DR. A session-to-session examination of homework engagement in cognitive therapy for depression: do patients experience immediate benefits? Behav Res Ther 2015 Sep;72:56-62 [FREE Full text] [CrossRef] [Medline]
- van Straten A, Cuijpers P, Smits N. Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J Med Internet Res 2008;10(1):e7 [FREE Full text] [CrossRef] [Medline]
- Guo Y, Hong YA, Qiao J, Xu Z, Zhang H, Zeng C, et al. Run4Love, a mHealth (WeChat-based) intervention to improve mental health of people living with HIV: a randomized controlled trial protocol. BMC Public Health 2018 Dec 26;18(1):793 [FREE Full text] [CrossRef] [Medline]
- WeChat. Wikipedia. URL: https://zh.wikipedia.org/wiki/%E5%BE%AE%E4%BF%A1 [accessed 2020-05-30]
- Guangzhou Population 2020. World Population Review. URL: https://worldpopulationreview.com/world-cities/guangzhou-population/ [accessed 2020-05-30]
- Wang W, Bian Q, Zhao Y, Li X, Wang W, Du J, et al. Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population. Gen Hosp Psychiatry 2014;36(5):539-544. [CrossRef] [Medline]
- Sun XY, Li YX, Yu CQ, Li LM. [Reliability and validity of depression scales of Chinese version: a systematic review]. Zhonghua Liu Xing Bing Xue Za Zhi 2017 Jan 10;38(1):110-116. [CrossRef] [Medline]
- Chen S, Fang Y, Chiu H, Fan H, Jin T, Conwell Y. Validation of the nine-item Patient Health Questionnaire to screen for major depression in a Chinese primary care population. Asia Pac Psychiatry 2013 Jun;5(2):61-68. [CrossRef] [Medline]
- Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline]
- Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012;14(6):e152 [FREE Full text] [CrossRef] [Medline]
- Bollen KA, Curran PJ. Latent Curve Models: A Structural Equation Perspective. Hoboken: John Wiley & Sons; 2006.
- Chen W, Shiu C, Yang JP, Simoni JM, Fredriksen-Goldsen KI, Lee TS, et al. Antiretroviral therapy (ART) side effect impacted on quality of life, and depressive symptomatology: a mixed-method study. J AIDS Clin Res 2013 Jun 29;4:218 [FREE Full text] [CrossRef] [Medline]
- Remien RH, Exner T, Kertzner RM, Ehrhardt AA, Rotheram-Borus MJ, Johnson MO, NIMH Healthy Living Project Trial Group. Depressive symptomatology among HIV-positive women in the era of HAART: a stress and coping model. Am J Community Psychol 2006 Dec;38(3-4):275-285. [CrossRef] [Medline]
- van Servellen G, Aguirre M, Sarna L, Brecht M. Differential predictors of emotional distress in HIV-infected men and women. West J Nurs Res 2002 Feb;24(1):49-72. [CrossRef] [Medline]
- Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociol Method Res 2016 Jun 29;21(2):230-258. [CrossRef]
- Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equation Modeling 1999 Jan;6(1):1-55. [CrossRef]
- Bentler PM. Fit indexes, lagrange multipliers, constraint changes and incomplete data in structural models. Multivariate Behav Res 1990 Apr 01;25(2):163-172. [CrossRef] [Medline]
- Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
- Snippe E, Schroevers MJ, Annika Tovote K, Sanderman R, Emmelkamp PMG, Fleer J. Patients' outcome expectations matter in psychological interventions for patients with diabetes and comorbid depressive symptoms. Cognit Ther Res 2015;39(3):307-317 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol Med 2006 Dec;36(12):1737-1746. [CrossRef] [Medline]
- Chu BC, Kendall PC. Positive association of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. J Consult Clin Psychol 2004 Oct;72(5):821-829. [CrossRef] [Medline]
- Kazantzis N, Whittington C, Zelencich L, Kyrios M, Norton PJ, Hofmann SG. Quantity and quality of homework compliance: a meta-analysis of relations with outcome in cognitive behavior therapy. Behav Ther 2016 Sep;47(5):755-772. [CrossRef] [Medline]
- Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009;38(4):222-234 [FREE Full text] [CrossRef] [Medline]
- Carroll SL, Embuldeniya G, Abelson J, McGillion M, Berkesse A, Healey JS. Questioning patient engagement: research scientists' perceptions of the challenges of patient engagement in a cardiovascular research network. Patient Prefer Adherence 2017;11:1573-1583 [FREE Full text] [CrossRef] [Medline]
- Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005;7(2):e16 [FREE Full text] [CrossRef] [Medline]
- Loh A, Leonhart R, Wills CE, Simon D, Härter M. The impact of patient participation on adherence and clinical outcome in primary care of depression. Patient Educ Couns 2007 Jan;65(1):69-78. [CrossRef] [Medline]
|CBSM: cognitive-behavioral stress management course|
|CES-D: Center for Epidemiological Studies Depression Scale|
|CFI: confirmatory fit index|
|IQR: interquartile range|
|LGCM: latent growth curve model|
|mHealth: mobile Health|
|MLR: maximum likelihood estimation with robust standard error|
|PHQ-9: Patient Health Questionnaire|
|RCT: randomized controlled trial|
|RMSEA: root mean-squared error of approximation|
|SRMR: standardized root mean of residual|
|TLI: Tucker-Lewis index|
Edited by G Eysenbach; submitted 10.07.20; peer-reviewed by E White, S van Luenen; comments to author 19.08.20; revised version received 30.08.20; accepted 22.09.20; published 29.10.20Copyright
©Yu Zeng, Yan Guo, Linghua Li, Y Alicia Hong, Yiran Li, Mengting Zhu, Chengbo Zeng, Hanxi Zhang, Weiping Cai, Cong Liu, Shaomin Wu, Peilian Chi, Aliza Monroe-Wise, Yuantao Hao, Rainbow Tin Hung Ho. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 29.10.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.