This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Associations between higher levels of patient engagement and better health outcomes have been found in face-to-face interventions; studies on such associations with mobile health (mHealth) interventions have been limited and the results are inconclusive.
The objective of this study is to investigate the relationship between patient engagement in an mHealth intervention and depressive symptoms using repeated measures of both patient engagement and patient outcomes at 4 time points.
Data were drawn from a randomized controlled trial (RCT) of an mHealth intervention aimed at reducing depressive symptoms among people living with HIV and elevated depressive symptoms. We examined the association between patient engagement and depressive symptoms in the intervention group (n=150) where participants received an adapted cognitive-behavioral stress management (CBSM) course and physical activity promotion on their WeChat social media app. Depressive symptoms were repeatedly measured using the Patient Health Questionnaire (PHQ-9) at baseline and 1 month, 2 months, and 3 months. Patient engagement was correspondingly measured by the completion rate, frequency of items completed, and time spent on the program at 1 month, 2 months, and 3 months. Latent growth curve models (LGCMs) were used to explore the relationship between patient engagement and depressive symptoms at multiple time points in the intervention.
The mean PHQ-9 scores were 10.2 (SD 4.5), 7.7 (SD 4.8), 6.5 (SD 4.7), and 6.7 (SD 4.1) at baseline, 1 month, 2 months, and 3 months, respectively. The mean completion rates were 50.6% (SD 31.8%), 51.5% (SD 32.2%), and 50.8% (SD 33.7%) at 1, 2, and 3 months, respectively; the average frequencies of items completed were 18.0 (SD 14.6), 32.6 (SD 24.8), and 47.5 (SD 37.2) at 1, 2, and 3 months, respectively, and the mean times spent on the program were 32.7 (SD 66.7), 65.4 (SD 120.8), and 96.4 (SD 180.4) minutes at 1, 2, and 3 months, respectively. LGCMs showed good model fit and indicated that a higher completion rate (β at 3 months=–2.184,
This study revealed a positive association between patient engagement and health outcomes at 3 months of an mHealth intervention using LGCMs and repeated measures data. The results underscore the importance of improving patient engagement in mHealth interventions to improve patient-centered health outcomes.
Chinese Clinical Trial Registry ChiCTR-IPR-17012606; https://tinyurl.com/yxb64mef
RR2-10.1186/s12889-018-5693-1
Mobile health (mHealth) has increasingly become a promising tool to deliver treatments for a range of psychosocial disorders [
Despite the growing success of mHealth interventions, the relationship between patient engagement and intervention outcomes is still unclear. Patient engagement is defined as the extent to which participants actively interact with intervention content [
Understanding the relationship between patient engagement and health outcomes in mHealth interventions is critical for developing and implementing effective mHealth programs [
To fill the gaps in the existing literature, this study aimed to explore the relationship between patient engagement and depressive symptoms during the 3-month course of an RCT of an mHealth program for people living with HIV and elevated depressive symptoms, the Run4Love intervention [
This study is a secondary analysis of data from the intervention group of a 3-month RCT (Chinese Clinical Trial Registry [ChiCTR-IPR-17012606]) that was conducted to examine the efficacy of a WeChat-based intervention on reducing depressive symptoms among people living with HIV and elevated depressive symptoms [
In addition to routine care, the Run4Love intervention consisted of two major components, an adapted cognitive-behavioral stress management (CBSM) course and physical activities promotion. The adapted CBSM course included content covering coping skills, muscle relaxation, and meditation tutorials. The physical activities promotion consisted of information on the benefits of regular exercise, exercise guidance, and healthy dietary suggestions. During the 3-month intervention, multimedia items were delivered to participants to support the adapted CBSM course and physical activities promotion via WeChat in the form of short articles, audio recordings, and posters almost once a day (5 to 6 items per week). Out of 65 items in total, 29 were short articles with an average of 1300 words, requiring about 5 minutes to read; 24 were posters with motivational messages, requiring about 30 seconds to read; and 12 were audio recordings in Chinese requiring 5 to 10 minutes to listen to. The WeChat-based Run4Love platform with extended functions was used to automatically deliver intervention items and collect real-time data on patient engagement (eg, time spent on an article, audio recording, or poster). The intervention design has been detailed elsewhere [
During the intervention, we used several measures to promote engagement: (1) automatically sending feedback on each participant’s compliance via WeChat on a weekly basis (eg, number of items completed and not completed); (2) giving verbal encouragement and financial incentives (up to US $2) via WeChat on a weekly basis based on the level of compliance; and (3) conducting motivational interviews by phone to sustain participation and help overcome barriers of compliance at 1 week, 1 month, and 2 months after baseline.
Depressive symptoms were assessed by the Chinese version of the Patient Health Questionnaire (PHQ-9) at baseline, 1 month, 2 months, and 3 months [
Three cumulative measures, including completion rate, frequency of items completed, and time spent on the program, were used to assess patient engagement in this study. Completion rate was the ratio of the number of items a patient completed to the number of items assigned, where the items that had been clicked were considered completed. Frequency of items completed was assessed not only by the number of items read or listened to by the participants, but also by the number of times the items were repeatedly read or listened to. Compared with the completion rate, frequency of items completed captured the repeat aspect of patient engagement. For example, if the same item was read or listened to twice by a participant, frequency was counted as 2 and the frequency of items completed for the participant was the accumulative frequency for all the items. The time spent on the program was measured by the overall time a participant spent on the Run4Love program, including reading or listening to the delivered items during the course of the intervention. We repeatedly assessed these 3 measures of patient engagement in conjunction with measurements of depressive symptoms that were evaluated at 1, 2, and 3 months after baseline, as has been recommended in previous studies [
Sociodemographic characteristics and HIV-related variables were collected at baseline including age, gender, sexual orientation, education, marital status, employment status, income, and duration of HIV infection (years). Income was measured by asking the participants whether they had adequate income to cover their daily expenses (adequate or inadequate).
Descriptive statistics on depressive symptoms, 3 measures of patient engagement, and baseline characteristics were provided. Mean and standard deviation were used to describe continuous variables with normal distribution, median and interquartile range (IQR) for continuous variables that did not follow a normal distribution, and frequency and percentage for categorical variables. Latent growth curve models (LGCMs) were constructed to examine the time-varying associations between patient engagement and depressive symptoms using repeated measure data at 4 time points [
The analyses of LGCMs were conducted in two steps [
Second, we developed 3 conditional LGCMs to examine the effects of patient engagement on depressive symptoms based on the 3 measures of patient engagement. The conditional model was extended from the unconditional model to incorporate one measure of patient engagement and related baseline characteristics as covariates. Since no baseline characteristics were found to significantly associate with the 3 measures of patient engagement or the outcome of depressive symptoms in this study, we included variables such as education, income, and duration of HIV infection that were associated with depressive symptoms among people living with HIV in the literature as control variables in the conditional models [
Model fit of LGCM was evaluated using chi-square tests and indices including the standardized root mean of residual (SRMR), confirmatory fit index (CFI), Tucker-Lewis index (TLI), and root mean-squared error of approximation (RMSEA). The criteria used to determine adequate model fit included: CFI≥0.95, TLI≥0.95, SRMR≤0.08, and RMSEA≤0.08 [
As shown in
Baseline characteristics of people living with HIV and elevated depressive symptoms in the intervention group (n=150).
Variables | Value | |
Age in years, mean (SD) | 28.0 (5.8) | |
|
|
|
|
Male | 142 (94.7) |
|
Female | 8 (5.3) |
|
|
|
|
Heterosexual | 20 (13.3) |
|
Homosexual, bisexual, uncertain | 130 (86.7) |
|
|
|
|
Below or at high school level | 52 (34.7) |
|
Above high school level | 98 (65.3) |
|
|
|
|
Single, divorced, widowed | 132 (88.0) |
|
Married | 18 (12.0) |
|
|
|
|
Unemployed | 27 (18.0) |
|
Employed | 123 (82.0) |
|
|
|
|
Adequate to cover daily expenses | 129 (86.0) |
|
Inadequate to cover daily expenses | 21 (14.0) |
Duration of HIV infection in years, mean (SD) | 2.4 (2.3) |
Of the participants, 87.3% (131/150), 76.0% (114/150), and 92.0% (138/150) completed the online questionnaire for depressive symptoms at 1, 2, and 3 months, respectively, after baseline. Compared with the depressive symptom score at baseline (PHQ-9: mean 10.2 [SD 4.5]), all follow-up assessments reported a decrease in depressive symptoms with mean values of 7.7 (SD 4.8), 6.5 (SD 4.7), and 6.7 (SD 4.1) at 1, 2, and 3 months, respectively (
Nearly all participants (149/150) read or listened to at least one item during the 3-month intervention. A moderate level of patient engagement was found during the intervention process. The completion rate remained around 50.0% (33/65) during the course of the intervention, with a mean of 50.6% (SD 31.8%), 51.5% (SD 32.2%), and 50.8% (SD 33.7%) at 1, 2, and 3 months, respectively. The mean frequency of items completed was 18.0 (SD 14.6), 32.6 (SD 24.8), and 47.5 (SD 37.2) at 1, 2, and 3 months, respectively. The mean time spent on the program was 32.7 (SD 66.7), 65.4 (SD 120.8), and 96.4 (SD 180.4) minutes at 1, 2, and 3 months, respectively. When comparing the mean and median scores in
Change in depressive symptoms and patient engagement across the 3-month interventiona.
Variables | |||||||||
|
Baseline | 1 month | 2 months | 3 months | |||||
|
Mean (SD) | Median (IQR) | Mean (SD) | Median (IQR) | Mean (SD) | Median (IQR) | Mean (SD) | Median (IQR) | |
PHQ-9b | 10.2 (4.5) | 10.0 (7.0-13.0) | 7.7 (4.8) | 8.0 (4.0-11.0) | 6.5 (4.7) | 6.0 (3.0-9.0) | 6.7 (4.1) | 7.0 (3.0-9.0) | |
Completion rate (%) | —c | — | 50.6 (31.8) | 54.2 (16.7-81.8) | 51.5 (32.2) | 53.0 (20.6-81.7) | 50.8 (33.7) | 47.6 (16.9-85.3) | |
Frequency of items completed | — | — | 18.0 (14.6) | 17.0 (5.8-26.0) | 32.6 (24.8) | 29.5 (10.0-51.3) | 47.5 (37.2) | 41.0 (11.0-78.3) | |
Time spent on program (min) | — | — | 32.7 (66.7) | 9.7 (2.5-42.1) | 65.4 (120.8) | 22.0 (5.3-80.3) | 96.4 (180.4) | 31.3 (6.7-117.2) |
aFor PHQ-9, the sample sizes were 150, 131, 114, and 138 at baseline, 1 month, 2 months, and 3 months, respectively. For patient engagement, the sample size remained 150 across the 3-month intervention.
bPHQ-9: Patient Health Questionnaire.
cNot available.
Results of the unconditional model indicated that depressive symptoms decreased during the 3-month intervention, and the largest reduction occurred at 1 month after baseline. This model had good model fit (
Unconditional latent growth curve model for depressive symptoms (n=150). Model fit statistic:
Parameter estimates for unconditional latent growth curve model for depressive symptoms (n=150).
Growth parameters | Estimate | SE | |
Slope0→PHQ-90a | 0.000 | 0.000 | —b |
Slope1→PHQ-91 | 1.000 | 0.000 | — |
Slope2→PHQ-92 | 1.565 | 0.196 | <.001 |
Slope3→PHQ-93 | 1.425 | 0.225 | <.001 |
Mean intercept | 10.157 | 0.370 | <.001 |
Mean slope | –2.336 | 0.411 | <.001 |
Variance of intercept | 15.069 | 5.100 | .003 |
Variance of slope | 5.564 | 3.614 | .12 |
Covariance of intercept and slope | –4.381 | 3.799 | .25 |
aPHQ-9: Patient Health Questionnaire.
bNot available.
Three conditional LGCMs were constructed, each using one measure of patient engagement as a time-varying variable. Results of the conditional models indicated that higher completion rate and greater frequency of items completed were significantly associated with lower depressive symptoms at 3 months, but there was no significant relationship between time spent on the program and depressive symptoms over the course of the intervention. All conditional models reported good model fit (
Conditional latent growth curve model for depressive symptoms with completion rate as the measure of patient engagement (n=150). Model fit statistic:
Conditional latent growth curve model for depressive symptoms with frequency of items completed as the measure of patient engagement (n=150). Model fit statistic:
Conditional latent growth curve model for depressive symptoms with the time spent on the program as the measure of patient engagement (n=150). Model fit statistic:
Based on the literature, baseline characteristics of education, income to cover daily expenses, and duration of HIV infection were included as control variables. As shown in
Parameter estimates for the three conditional latent growth curve models (n=150).
Covariate parameters | Completion rate, estimate (SE) | Frequency of items completed, estimate (SE) | Time spent on program, estimate (SE) | |
|
||||
|
Education→intercept | –0.893 (0.858) | –0.914 (0.849) | –0.909 (0.843) |
|
Income→intercept | –2.329b (1.179) | –2.306b (1.175) | –2.309b (1.169) |
|
Duration of HIV infection→intercept | 0.064 (0.165) | 0.058 (0.157) | 0.061 (0.156) |
|
Education→slope | –0.257 (0.650) | –0.274 (0.632) | –0.239 (0.628) |
|
Income→slope | 1.382 (0.860) | 1.298 (0.899) | 1.399 (0.898) |
|
Duration of HIV infection→slope | 0.103 (0.155) | 0.116 (0.147) | 0.092 (0.139) |
|
||||
|
Engagement1→PHQ-91a | –1.221 (0.881) | –0.025 (0.019) | 0.004 (0.004) |
|
Engagement2→PHQ-92 | –1.813 (1.158) | –0.026 (0.014) | 0.001 (0.002) |
|
Engagement3→PHQ-93 | –2.184b (0.945) | –0.018b (0.009) | –0.002 (–0.082) |
aPHQ-9: Patient Health Questionnaire.
b
This study was among the first to explore dose-response relationship between patient engagement and depressive symptoms in a social media–based mHealth intervention among people living with HIV using longitudinal data. Three measures of patient engagement were used to assess different aspects of the construct. By investigating the time-varying relationship between patient engagement and health outcomes, our study provided a better understanding of the dose-response relationship in mHealth interventions and the time of occurrence of this association. Results of LGCMs revealed that two measures of patient engagement, completion rate and frequency of items completed, were significantly associated with reduced depressive symptoms, and these relationships occurred at 3 months of the intervention but not at 1 or 2 months. However, time spent on the program was not significantly related to depressive symptoms throughout the intervention.
First and foremost, this study provides new evidence on the dose-response relationship in mHealth interventions as the literature in this regard was limited and inconclusive [
Our data suggests that the significant relationship of patient engagement and health outcomes did not occur early (1 or 2 months) in the intervention, but only at 3 months. Few studies have explored the dose-response relationship across the span of an intervention, especially in mHealth programs. To our knowledge, only one mHealth study intended to explore the patient engagement–outcome relationship using multiple measures of patient engagement [
Compared with mHealth interventions, face-to-face studies have focused more on the relationship between patient engagement and health outcomes. One face-to-face intervention investigated the dose-response relationship in the first 5 weeks of the intervention using both weekly engagement measures and depressive symptoms outcomes among people with major depressive disorder [
Identifying the time point of effective dose-response relationship between patient engagement and treatment outcomes is critical for optimizing the design, pace, and duration of interventions. Unnecessarily lengthy interventions waste resources and lead to high dropout rates [
In addition to identifying the timing of occurrence of the dose-relationship in the intervention, this study also found the differential relationships between different measures of patient engagement and health outcomes. Compared with face-to-face interventions, mHealth interventions have more measurement tools to capture different aspects of patient engagement [
In this study, participants with higher completion rates reported fewer depressive symptoms at 3 months. On average, participants completed 50.8% (33/65) of the program items (ie, short articles, audio recordings, and posters) at 3 months, which was similar to the completion rates reported in other online interventions for depressive symptoms [
Frequency of items completed was another patient engagement measure we used in the study, and it was also positively associated with patients’ outcomes. Such a measure of patient engagement has rarely been reported in the existing mHealth interventions. At 3 months of the intervention, participants had read or listened to the items 47.5 times on average. As the average number of items completed by participants was 33, each item was repeatedly read or listened to for approximately 1.5 times (47.5/33). This measure captured an important aspect of patient engagement as it reflected to what extent participants were willing to repeatedly engage with the intervention items. Based on this frequency data, we subsequently chose items that had been most read or listened to and re-sent these items to the patients in a booster session 3 months after the intervention [
In contrast, the third patient engagement measure, time spent on the program, was found not to relate to the reduced depressive symptoms, which was consistent with previous studies [
Several limitations in this study should be noted. First, although the sample was drawn from a large hospital designated for HIV treatment in Guangzhou with more than 10,000 HIV seropositive patients on treatment, participants were mostly from an urban setting and the majority of them were nonheterosexual young men. Thus, the generalizability of the findings to other people living with HIV or other populations may be limited. Second, although depressive symptoms are the major outcome in this study, more types of outcome measures such as other psychological indicators (eg, anxiety) and/or biomarkers (eg, hair or salivary cortisol concentration) can be incorporated in future studies to provide further evidence on dose-response relationship between patient engagement and intervention outcomes. Third, while measures of patient engagement were collected objectively through the Run4Love platform, depressive symptoms were self-reported and might suffer from social desirability and recall bias. Fourth, there might be measurement biases in assessing patient engagement. For example, when clicked, a material was readily considered as completed. Nevertheless, this definition of completion was widely used in mHealth interventions, and the relevant patient engagement measures served as reliable and efficient metrics to assess dose-response relationship [
This study indicated that higher completion rate or greater frequency of accessing items in an mHealth intervention contributed to reduced depressive symptoms among people living with HIV and elevated depressive symptoms at 3 months in the intervention. Given the observed positive impact of patient engagement on intervention outcomes, adherence to mHealth interventions should be evaluated and strengthened as a critical and integral strategy to improve treatment effects. Approaches for promoting patient engagement include enhancing patients’ initial motivation in program participation, education on the importance of active participation, frequent and effective reminders, financial and verbal incentives, and engaging patients in goal setting (eg, physical activities) [
In conclusion, this study revealed a positive association between patient engagement and reduction in depressive symptoms at 3 months of an mHealth intervention using latent growth curve models and repeated measure data from 4 time points. The results underscore the importance of enhancing patient engagement in mHealth interventions to improve health outcomes of the participants.
cognitive-behavioral stress management course
Center for Epidemiological Studies Depression Scale
confirmatory fit index
interquartile range
latent growth curve model
mobile Health
maximum likelihood estimation with robust standard error
Patient Health Questionnaire
randomized controlled trial
root mean-squared error of approximation
standardized root mean of residual
Tucker-Lewis index
This study was supported by the National Natural Science Foundation of China (Grant No 71573290) and China Medical Board open competition funding (Grant No. 17-271). The funding sources had no role in the study design, data collection, statistical analyses, and writing of the manuscript.
YG and YZ had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. YG, YAH, and YZ were responsible for the concept and design of the study. YZ, YL, MZ, HZ, CZ, SW, CP, and YH were responsible for the acquisition, analysis, or interpretation of data. YZ and YG drafted the manuscript. YG, YZ, YAH, AMW, YH, and RTHH completed the critical revision of the manuscript for important intellectual content. LL, WC, CL, and YH were responsible for administrative, technical, and material support, and YG supervised the study.
None declared.