This study is a secondary analysis of data from the intervention group of a 3-month RCT (Chinese Clinical Trial Registry [ChiCTR-IPR-17012606]) that was conducted to examine the efficacy of a WeChat-based intervention on reducing depressive symptoms among people living with HIV and elevated depressive symptoms [8]. WeChat, similar to WhatsApp, is a widely used social media platform for instant communication in China with 1.1 billion active users [25]. Participants were recruited from the outpatient department of a large hospital designated for HIV treatment in 2017 in Guangzhou, the third largest city in China, with a population of more than 13 million [26]. A total of 300 people living with HIV who had elevated depressive symptoms (Center for Epidemiological Studies Depression Scale [CES-D] score ≥16) and used WeChat were randomized into the intervention or wait-list control group in a 1:1 ratio. The institutional review board of Sun Yat-sen University approved the intervention protocol, and the detailed recruitment and randomization procedure were described elsewhere [24].

In addition to routine care, the Run4Love intervention consisted of two major components, an adapted cognitive-behavioral stress management (CBSM) course and physical activities promotion. The adapted CBSM course included content covering coping skills, muscle relaxation, and meditation tutorials. The physical activities promotion consisted of information on the benefits of regular exercise, exercise guidance, and healthy dietary suggestions. During the 3-month intervention, multimedia items were delivered to participants to support the adapted CBSM course and physical activities promotion via WeChat in the form of short articles, audio recordings, and posters almost once a day (5 to 6 items per week). Out of 65 items in total, 29 were short articles with an average of 1300 words, requiring about 5 minutes to read; 24 were posters with motivational messages, requiring about 30 seconds to read; and 12 were audio recordings in Chinese requiring 5 to 10 minutes to listen to. The WeChat-based Run4Love platform with extended functions was used to automatically deliver intervention items and collect real-time data on patient engagement (eg, time spent on an article, audio recording, or poster). The intervention design has been detailed elsewhere [24].

During the intervention, we used several measures to promote engagement: (1) automatically sending feedback on each participant’s compliance via WeChat on a weekly basis (eg, number of items completed and not completed); (2) giving verbal encouragement and financial incentives (up to US $2) via WeChat on a weekly basis based on the level of compliance; and (3) conducting motivational interviews by phone to sustain participation and help overcome barriers of compliance at 1 week, 1 month, and 2 months after baseline.

Descriptive statistics on depressive symptoms, 3 measures of patient engagement, and baseline characteristics were provided. Mean and standard deviation were used to describe continuous variables with normal distribution, median and interquartile range (IQR) for continuous variables that did not follow a normal distribution, and frequency and percentage for categorical variables. Latent growth curve models (LGCMs) were constructed to examine the time-varying associations between patient engagement and depressive symptoms using repeated measure data at 4 time points [32]. Maximum likelihood estimation with robust standard errors (MLR) was used to handle missing data and obtain parameter estimates [32]. Descriptive analyses were conducted in SPSS Statistics version 20.0 (IBM Corporation), and LGCMs were conducted in Mplus version 7 (Muthen & Muthen).

The analyses of LGCMs were conducted in two steps [32]. First, we developed an unconditional LGCM to estimate the growth trajectory of depressive symptoms across time. The initial level of depressive symptoms was represented as the intercept, and the average rate of change was represented as the slope in the model. For the intercept factor, all loadings were fixed to 1. For the slope factor, the first and second loadings were fixed to 0 and 1, and the remaining loadings were freely estimated from the data to test whether there was a linear or nonlinear growth trajectory (eg, linear growth pattern: 0, 1, 2, 3; nonlinear growth pattern: the third and fourth loadings significantly differ from 2 and 3, respectively).

Second, we developed 3 conditional LGCMs to examine the effects of patient engagement on depressive symptoms based on the 3 measures of patient engagement. The conditional model was extended from the unconditional model to incorporate one measure of patient engagement and related baseline characteristics as covariates. Since no baseline characteristics were found to significantly associate with the 3 measures of patient engagement or the outcome of depressive symptoms in this study, we included variables such as education, income, and duration of HIV infection that were associated with depressive symptoms among people living with HIV in the literature as control variables in the conditional models [33-35]. Associations between patient engagement and depressive symptoms were treated as time-varying effects, which were specified by regressing depressive symptoms (eg, PHQ-9 at 1 month) on the corresponding patient engagement variable (eg, completion rate at 1 month). Patient engagement, therefore, could be considered as a time-specific predictor of depressive symptoms after controlling for the influences of the underlying growth trajectory of depressive symptoms and baseline characteristics [32].

Model fit of LGCM was evaluated using chi-square tests and indices including the standardized root mean of residual (SRMR), confirmatory fit index (CFI), Tucker-Lewis index (TLI), and root mean-squared error of approximation (RMSEA). The criteria used to determine adequate model fit included: CFI≥0.95, TLI≥0.95, SRMR≤0.08, and RMSEA≤0.08 [36-38].

As shown in Table 1, the mean age of participants was 28.0 (SD 5.8) years and the average duration of HIV infection was 2.4 (SD 2.3) years. The participants were predominately male (142/150, 94.7%), nonheterosexual (130/150, 86.7%), well-educated (98/150, 65.3%, with at least some college education), single (132/150, 88.0%), employed (123/150, 82.0%), and had adequate income to cover daily expenses (129/150, 86.0%).

Of the participants, 87.3% (131/150), 76.0% (114/150), and 92.0% (138/150) completed the online questionnaire for depressive symptoms at 1, 2, and 3 months, respectively, after baseline. Compared with the depressive symptom score at baseline (PHQ-9: mean 10.2 [SD 4.5]), all follow-up assessments reported a decrease in depressive symptoms with mean values of 7.7 (SD 4.8), 6.5 (SD 4.7), and 6.7 (SD 4.1) at 1, 2, and 3 months, respectively (Table 2).

Nearly all participants (149/150) read or listened to at least one item during the 3-month intervention. A moderate level of patient engagement was found during the intervention process. The completion rate remained around 50.0% (33/65) during the course of the intervention, with a mean of 50.6% (SD 31.8%), 51.5% (SD 32.2%), and 50.8% (SD 33.7%) at 1, 2, and 3 months, respectively. The mean frequency of items completed was 18.0 (SD 14.6), 32.6 (SD 24.8), and 47.5 (SD 37.2) at 1, 2, and 3 months, respectively. The mean time spent on the program was 32.7 (SD 66.7), 65.4 (SD 120.8), and 96.4 (SD 180.4) minutes at 1, 2, and 3 months, respectively. When comparing the mean and median scores in Table 2, we found that time spent on the program was positively skewed, with means much larger than the medians at 1, 2, and 3 months.

Results of the unconditional model indicated that depressive symptoms decreased during the 3-month intervention, and the largest reduction occurred at 1 month after baseline. This model had good model fit (Figure 1). The mean intercept was 10.157 (P<.001), indicating a mean PHQ-9 score of 10.157 at baseline. The mean slope was –2.336 (P<.001), and the factor loadings on the slope were 0, 1, 1.565, and 1.425 at baseline, 1 month, 2 months, and 3 months, respectively. These results indicated a nonlinear pattern of the reduction of depressive symptoms: a rapid decrease of PHQ-9 score (2.336 points) occurred at the first month, and then the magnitude of reduction flattened out. Variances of the intercept and slope were 15.069 (P=.003) and 5.564 (P=.12), indicating significant individual differences in depressive symptoms at baseline, but no individual differences in the rates of change over time. Covariance between the intercept and slope was –4.381 (P=.25), suggesting that the rate of change in depressive symptoms was independent of its initial levels (Table 3).

Three conditional LGCMs were constructed, each using one measure of patient engagement as a time-varying variable. Results of the conditional models indicated that higher completion rate and greater frequency of items completed were significantly associated with lower depressive symptoms at 3 months, but there was no significant relationship between time spent on the program and depressive symptoms over the course of the intervention. All conditional models reported good model fit (Figures 2-4).

Based on the literature, baseline characteristics of education, income to cover daily expenses, and duration of HIV infection were included as control variables. As shown in Table 4, education and duration of HIV infection did not have significant effect either on the initial levels (intercept) or the rate of change (slope) of depressive symptoms in all conditional models. Income had a significant effect on the intercept in all conditional models, suggesting that participants with adequate income to cover daily expenses had a lower level of depressive symptoms at baseline. The time-varying variable, patient engagement, had a significant effect on depressive symptoms at 3 months after baseline (completion rate: β3=–2.184, P=.048; frequency of items completed: β3=–0.018, P=.04), whereas the effect of patient engagement, regardless of the measurement, was not significant on depressive symptoms at 1 or 2 months. Specifically, participants who had higher completion rates or greater frequency of items completed were more likely to report fewer depressive symptoms at 3 months after controlling for the underlying growth trajectory of depressive symptoms and baseline characteristics. By contrast, the time spent on the program was not significantly associated with depressive symptoms throughout the intervention process (β1=0.004, P=.29; β2=–0.001, P=.74; β3=–0.002, P=.16), suggesting that the amount of time patients spent on the intervention program was not related to the changes in depressive symptoms.

This study was among the first to explore dose-response relationship between patient engagement and depressive symptoms in a social media–based mHealth intervention among people living with HIV using longitudinal data. Three measures of patient engagement were used to assess different aspects of the construct. By investigating the time-varying relationship between patient engagement and health outcomes, our study provided a better understanding of the dose-response relationship in mHealth interventions and the time of occurrence of this association. Results of LGCMs revealed that two measures of patient engagement, completion rate and frequency of items completed, were significantly associated with reduced depressive symptoms, and these relationships occurred at 3 months of the intervention but not at 1 or 2 months. However, time spent on the program was not significantly related to depressive symptoms throughout the intervention.

First and foremost, this study provides new evidence on the dose-response relationship in mHealth interventions as the literature in this regard was limited and inconclusive [39]. Potential reasons for the ambivalent relationship in previous mHealth intervention studies could include inadequate research design or implementation, inadequate intervention durations for significant treatment effect, low levels of patient engagement, high dropout rate, and/or inappropriate or inadequate measurements of patient engagement or data analysis methods [12,20,39]. This study, however, shows that an evidence-based mHealth intervention with rigorous design and implementation could produce significant treatment effects, in particular, for participants with better engagement.

Our data suggests that the significant relationship of patient engagement and health outcomes did not occur early (1 or 2 months) in the intervention, but only at 3 months. Few studies have explored the dose-response relationship across the span of an intervention, especially in mHealth programs. To our knowledge, only one mHealth study intended to explore the patient engagement–outcome relationship using multiple measures of patient engagement [13]. However, the web-based iCBT intervention conducted in a community setting in Dublin, Ireland, in 2014 only measured pre- and postintervention depressive symptoms [13]. The methodology of repeated-measures analysis of variance used in the study could only reveal that patients with significant improvement in depressive symptoms had higher levels of patient engagement during the 8-week intervention [13]. Although the study further found that patients tended to engage in the online program more in the first half of the program (1 to 4 weeks) than in the latter half (5 to 8 weeks) [13], the design and findings of the study could not identify when the linkage between patient engagement and outcome occurred during the course of the intervention.

Compared with mHealth interventions, face-to-face studies have focused more on the relationship between patient engagement and health outcomes. One face-to-face intervention investigated the dose-response relationship in the first 5 weeks of the intervention using both weekly engagement measures and depressive symptoms outcomes among people with major depressive disorder [22]. However, the study used repeated measures regression analysis that could only conclude that there was a significant dose-response relationship during the 5-week intervention but not in which week(s) the relationship started to appear. Another face-to-face study conducted among patients with diabetes in Netherlands investigated the dose-response relationship between patient engagement and depressive symptoms during an 8-week intervention [40]. Patient engagement and depressive symptoms were repeatedly measured at 2, 4, and 8 weeks [40]. Although the study used multilevel analysis that was able to identify both the dose-response relationship and timing of the occurrence of any relationship, the study did not find a dose-response relationship in any of the time points [40]. One potential explanation for the nonsignificant dose-response relationship might be that individuals with chronic diseases and depressive symptoms need a longer time of exposure to an intervention to obtain the beneficial outcomes. Likewise, our study did not find a dose-response relationship at 1 or 2 months of the intervention but only at 3 months of the intervention among people living with HIV and depressive symptoms. Such a relationship among individuals with chronic diseases and psychological disorders needs to be further explored in future studies. Although the nonsignificant dose-response relationship at 1 or 2 months seemed to be contradictory to the finding that most reduction of depressive symptoms occurred at 1 month, reasons for the reduction of depressive symptoms could be many. Besides patient engagement, other factors such as social support might also play a role in the reduction of depressive symptoms. For example, several phone calls made by research staff to motivate participants to continue the intervention at 1 week and 1 and 2 months after baseline may provide social support needed by people living with HIV and elevated depressive symptoms.

Identifying the time point of effective dose-response relationship between patient engagement and treatment outcomes is critical for optimizing the design, pace, and duration of interventions. Unnecessarily lengthy interventions waste resources and lead to high dropout rates [41], whereas interventions with inadequate duration may not generate intended outcomes [42]. Repeatedly examining the relationship between patient engagement and health outcomes allows us to identify the optimal time for intervention. This study demonstrates such measurement in an mHealth intervention.

In addition to identifying the timing of occurrence of the dose-relationship in the intervention, this study also found the differential relationships between different measures of patient engagement and health outcomes. Compared with face-to-face interventions, mHealth interventions have more measurement tools to capture different aspects of patient engagement [12]. These different measurements may not have similar impacts on patient outcomes and may explain the conflicting results regarding the dose-response relationships in mHealth interventions [12].

In this study, participants with higher completion rates reported fewer depressive symptoms at 3 months. On average, participants completed 50.8% (33/65) of the program items (ie, short articles, audio recordings, and posters) at 3 months, which was similar to the completion rates reported in other online interventions for depressive symptoms [10,13]. A recent meta-analysis found a significant association between higher completion rates and reduced depressive symptoms in online programs [43]. However, some more recent internet-based RCTs did not find such a relationship [19,20]. One potential reason for the nonsignificant relationship was low levels of patient engagement in these mHealth interventions [10]. Therefore, adherence to the program needs to be underscored in mHealth interventions, especially considering that most of the existing mHealth interventions have relatively low completion rates [10,13].

Frequency of items completed was another patient engagement measure we used in the study, and it was also positively associated with patients’ outcomes. Such a measure of patient engagement has rarely been reported in the existing mHealth interventions. At 3 months of the intervention, participants had read or listened to the items 47.5 times on average. As the average number of items completed by participants was 33, each item was repeatedly read or listened to for approximately 1.5 times (47.5/33). This measure captured an important aspect of patient engagement as it reflected to what extent participants were willing to repeatedly engage with the intervention items. Based on this frequency data, we subsequently chose items that had been most read or listened to and re-sent these items to the patients in a booster session 3 months after the intervention [8]. In a separate study, we found a continued intervention effect with further reductions of patients’ depressive symptoms at 3 and 6 months after the intervention [8].

In contrast, the third patient engagement measure, time spent on the program, was found not to relate to the reduced depressive symptoms, which was consistent with previous studies [21,44]. Time spent on the program may not reflect the actual levels of patient engagement as it is likely to be influenced by a variety of factors such as reading speed, cognitive capacity, and familiarity with the program [39]. Participants might keep the program on without actively engaging. For example, patients might fall asleep when listening to a relaxation audio recording; yet time stayed on the page was still counted as time spent on the program. In this study, there was a noticeable difference (31.3 vs 96.7 minutes) between the median and mean time spent on the platform at 3 months of the intervention, indicating a positively skewed distribution of patients’ time spent online, suggesting few patients spent extra-long time on the platform. Future interventions should incorporate a time-out function to better capture the actual patient engagement.

Several limitations in this study should be noted. First, although the sample was drawn from a large hospital designated for HIV treatment in Guangzhou with more than 10,000 HIV seropositive patients on treatment, participants were mostly from an urban setting and the majority of them were nonheterosexual young men. Thus, the generalizability of the findings to other people living with HIV or other populations may be limited. Second, although depressive symptoms are the major outcome in this study, more types of outcome measures such as other psychological indicators (eg, anxiety) and/or biomarkers (eg, hair or salivary cortisol concentration) can be incorporated in future studies to provide further evidence on dose-response relationship between patient engagement and intervention outcomes. Third, while measures of patient engagement were collected objectively through the Run4Love platform, depressive symptoms were self-reported and might suffer from social desirability and recall bias. Fourth, there might be measurement biases in assessing patient engagement. For example, when clicked, a material was readily considered as completed. Nevertheless, this definition of completion was widely used in mHealth interventions, and the relevant patient engagement measures served as reliable and efficient metrics to assess dose-response relationship [12]. Additionally, time spent on the program might not capture the real patient engagement as it was not able to differentiate between active and inactive engagement and time on distraction or interruption could not be measured. Finally, we did not differentiate patient engagement in different formats of the intervention items (ie, short articles, audio recordings, and posters) nor did we examine the relationship between the content of the Run4Love intervention and patient engagement in this study, as it goes beyond the purposes of this study. We call for future research to further explore the relationships between patient engagement and the content of the intervention as well as the potential differences in patient engagement by different types of multimedia items in mHealth interventions.

This study indicated that higher completion rate or greater frequency of accessing items in an mHealth intervention contributed to reduced depressive symptoms among people living with HIV and elevated depressive symptoms at 3 months in the intervention. Given the observed positive impact of patient engagement on intervention outcomes, adherence to mHealth interventions should be evaluated and strengthened as a critical and integral strategy to improve treatment effects. Approaches for promoting patient engagement include enhancing patients’ initial motivation in program participation, education on the importance of active participation, frequent and effective reminders, financial and verbal incentives, and engaging patients in goal setting (eg, physical activities) [45-47]. Future studies with repeated assessments of patient engagement and outcomes throughout the intervention process have the potential to analyze the time-varying dose-response relationship. Such studies would optimize the intervention duration and maximize intervention outcomes through improving the design and implementation of mHealth interventions.

In conclusion, this study revealed a positive association between patient engagement and reduction in depressive symptoms at 3 months of an mHealth intervention using latent growth curve models and repeated measure data from 4 time points. The results underscore the importance of enhancing patient engagement in mHealth interventions to improve health outcomes of the participants.