Evidence of Efficacy of the My Personal Health Guide Mobile Phone App on Antiretroviral Therapy Adherence Among Young African American Men Who Have Sex With Men at 1 Month: Randomized Controlled Trial

Introduction

African American men who have sex with men (AAMSM) have disproportionate HIV infection rates, retention in care [1], mortality, and rates of viral nonsuppression [2-8], an outcome that relies on antiretroviral therapy (ART) adherence. Additionally, health literacy is a factor associated with ART adherence [9] that is lower among African American people than White people [10]. Therefore, interventions that bolster health literacy might improve healthy HIV-related behaviors like ART adherence, potentially resulting in personal and public health benefits.

Relational agents are computerized images that a person may react to as if they are in a relationship. A relational agent can be designed to promote healthy behavioral change. We developed My Personal Health Guide, an innovative theory-driven relational agent–based app with improvement of ART adherence as one of its objectives [11]. During iterative app development focus groups with young AAMSM living with HIV, participants reported universal acceptability for the African American realistic avatar we developed that spoke about, taught, explained, and encouraged behavior to promote healthy HIV-related outcomes [11]. Responding to meeting their adherence-related needs and interests, the app includes clear and simple language, an audiovisual format, and gamification. A pilot study performed in Chicago provided evidence that My Personal Health Guide has the potential to improve adherence [12]. Here, we present preliminary short-term follow-up results from a randomized controlled trial (RCT) of My Personal Health Guide that demonstrate evidence of improvement of ART adherence among young AAMSM living with HIV. We hypothesized that those randomized to the My Personal Health Guide app would have greater ART adherence compared to controls at 1-month follow-up post randomization. Evidence of short-term improvement in ART adherence provides scientific rationale for further research of this realistic avatar and relational agent approach as an adherence intervention and informs efforts to refine the approach to produce longer-term effects.

Methods

Intervention

My Personal Health Guide features a realistic talking avatar with supportive functions that were designed to inform users of HIV-related health information, encourage participants to take their medication, and facilitate adherence-related behavioral skills with reminders and push notifications to take medication and promotion of using weekly pill boxes. It also includes recordings of peers and health care providers offering personal and motivational messages. The app provides a private space, in one’s own mobile phone, for hearing and seeing information relevant to healthy living with HIV. Users may replay information, which can be especially helpful to people who are uncomfortable asking questions of their health care provider, including asking for a repeat of what was said or an explanation in simple terms.

Design, Recruitment, Eligibility, and Enrollment

This RCT was performed among young AAMSM living with HIV. Participants were recruited throughout the United States between February 2020 and September 2023, predominantly through social media (eg, the dating app Jack’d and Facebook advertisements) and also by word of mouth, provider referral, and fliers in health care settings of the 3 enrollment sites: University of Illinois at Chicago, Emory University, and University of Mississippi Medical Center. Those who were eligible were assigned male sex at birth, currently identified as a man, African American or Black, aged 18-34 years, had at least one male sexual partner in their lifetime, English speaking, owned a smartphone, were initiating or prescribed oral ART, and had a history of nonoptimal ART adherence defined as having a detectable viral load within the past 4 weeks, self-reported nonoptimal ART adherence within the past 30 days, or were referred by a provider for adherence concerns. Those who were on injectable ART, participating in another HIV treatment study, or did not have a stable mailing address at the time of enrollment were ineligible for participation. This study was determined to be minimal risk, with no adverse events reported.

Data Collection, Sample Size, and Randomization

Figure 1 summarizes participant interactions with the study through the first month of follow-up. A structured, interviewer-administered baseline questionnaire included collection of demographics, baseline knowledge of HIV (items taught in the app), self-reported adherence, incarceration history, housing stability, substance use history, self-efficacy (HIV Treatment Adherence Self-Efficacy Scale [HIV-ASES]) [13], literacy (adapted Rapid Estimate of Adult Literacy in Medicine-Short Form [REALM-SF]) [14], perceived social support (Multidimensional Scale of Perceived Social Support [MSPSS]) [15], and depression symptomatology (Patient Health Questionnaire for Depression [PHQ-9]) [16]. If a participant exhibited moderately severe or severe depressive symptomatology, they were encouraged to reach out to their provider for mental health resources. One month after the baseline interview, participants were scheduled for the app orientation and download that included explaining how to download, setting up a password, briefly pointing out features, and encouraging app use. Additionally, a check-in call was made monthly during 6 months of follow-up that asked troubleshooting questions about the app and collected self-reported ART adherence. Participants received compensation of US $40 for the baseline interview, US $75 for an appointment to download the app, and US $10 for responding to each monthly check-in call. Sample size calculations were carried out using Repeated Measures and Sample Size (RMASS2), the power calculation software for repeated measures design that allows attrition over time. The study was designed with a proposed sample size of 250 at the end of the study duration, or 295 at baseline, which would provide 80% statistical power to detect a 10% group difference in HIV care retention at the end of the study, considering a 15% attrition rate. Permuted randomization in blocks of 8 was used to allocate research participants in a 1:1 ratio to the intervention or control arm, and allocation concealment from the principal investigator, research staff, and research participants was carried out by the senior statistician. Participants randomized to the intervention arm received the My Personal Health Guide app, and those randomized to the attention control arm received a food safety app, which featured an avatar and functions related to food and nutrition, narrated illustrated stories about food safety, an educational game about sugar content in food, and information promoting oral health. The principal investigator and research participants were blinded after assignment to the intervention. Research coordinators oriented the participants to their respective app after randomization.

‎

Outcome Measurement

ART adherence was assessed with Wilson’s 3-item self-reported adherence measurement, which has been validated against electronic drug monitoring for HIV medication (Cronbach α=0.83) [17]. These three questions include (1) “In the last 30 days, on how many days did you miss at least one dose of any of your HIV medication?” (with a response range from 0 to 30 days); (2) “In the last 30 days, how often did you take your HIV medication in the way you were supposed to?” (with response options including Never, Rarely, Sometimes, Usually, Almost Always, and Always); and (3) “In the last 30 days, how good a job did you do at taking your HIV medications in the way you were supposed to?” (with response options including Very Poor, Poor, Fair, Good, Very Good, and Excellent). Answers were averaged together and converted to a percentage score on a 100-point scale. Consistent with other studies [18,19], 80% was defined as the cut point for nonoptimal ART adherence. HIV viral load was the intended primary outcome for this research. However, the trial was disturbed by the COVID-19 pandemic, and significant difficulty occurred collecting viral load from this population. Therefore, self-reported ART adherence was used.

Statistical Analysis

The primary statistical goal for this analysis was to determine the efficacy of the My Personal Health Guide app on ART adherence at 1-month follow-up. Participant demographic and study characteristics are presented as means and SDs for continuous variables and as absolute numbers (n) and proportions (%) for categorical variables stratified by randomization arm. The success of randomization was assessed using 2-tailed t tests for continuous baseline characteristics or chi-square tests for categorical characteristics. Baseline characteristics between individuals that completed both the baseline meeting and 1-month follow-up versus those who had missing data at 1-month follow-up were compared to ensure comparability between the samples. Bivariate comparisons were made between participant characteristics and ART adherence at baseline (≥80% vs <80%) using chi-square tests for categorical data, Cochran-Armitage tests for ordinal data, and independent t tests for continuous data. Logistic regression models with the backward model selection method were used to evaluate the effect of the My Personal Health Guide app, adjusted for significant factors that were associated with ART adherence at month 1, with a P≤.05 considered statistically significant. Baseline adherence was adjusted for a priori. The baseline characteristics that were considered in the logistic regression models included age, ethnicity, remote participation, residence, education, employment status, sexual orientation, currently in a committed relationship, health literacy (REALM-SF), HIV-ASES, depression symptomatology (PHQ-9), perceived social support (MSPSS), ever incarcerated, housing insecurity, and substance use seriousness (serious includes cocaine, methamphetamines, crack, heroin, and opiates; less serious includes alcohol, marijuana, ecstasy, speedballs, and gamma-hydroxybutyrate; none includes no use of substances or alcohol) at baseline. Adjusted model results are presented as odds ratios (ORs) and 95% CI.

Ethical Considerations

This research was reviewed and approved by the institutional review boards of the University of Illinois Chicago, Emory University, and the University of Mississippi Medical Center (protocol #2019-1184). The research involving human data was conducted in accordance with institutional guidelines of the University of Illinois Chicago, Emory University, and the University of Mississippi Medical Center. All participants were administered written informed consent. All data were deidentified prior to analysis and dissemination. Participants were compensated up to US $125 by the 1-month follow-up by their preferred method of check, cash, or e-code.

Results

Sample Characteristics

We attempted to contact the 911 men that met preliminary eligibility criteria, responded to recruitment methods, and provided contact information. Among those, 471 were either not responsive or not interested in the study, 44 were enrolled in another HIV treatment study, and 5 did not have a stable mailing address in order to receive study materials. Five participants were erroneously enrolled in the study due to either being unable to verify having a current HIV prescription (n=3) or having been previously enrolled in the study (n=2). A total of 386 men were consented. Among these, the baseline questionnaire was completed by 315 (81.6%), and 253 (65.5%) had the study apps installed on their mobile phones 1 month later (Figure 1), including 124 (49%) randomized to My Personal Health Guide and 129 (51%) randomized to the Food Safety attention control app. After 1 month, 93 participants did not respond to contact attempts to complete the month 1 check-in call and were excluded from the current analysis. The remaining 160 participants completed the first check-in phone call, and of those, 131 had complete self-reported adherence data at month 1.

The mean age of the 253 participants at baseline was 29.4 (SD 3.7; range18-35) years; most (n=221, 87.4%) were 25-34 years of age. Few (n=18, 7%) identified as Hispanic. Most participated remotely (n=202, 79.8%). Most participants were enrolled by the University of Illinois at Chicago (n=131, 51.8%) and Emory University (n=106, 41.9%), and 16 (6%) were enrolled by the University of Mississippi Medical Center. Over half (n=145, 57.3%) of participants resided in the Southern United States, but all US regions were represented in the sample. Nearly half (n=175, 42.3%) had some college education, and over half (n=146, 57.7%) reported being employed. Three-quarters (n=180, 71.1%) identified as homosexual or gay and were not currently in a committed relationship with a male partner (n=198, 78.3%). Over one-third (n=96, 38%) had less than optimal literacy (adapted REALM-SF). Over half reported mild-severe depressive symptomatology, with 26% (n=65) reporting mild (PHQ-9=5.0-9.9), 14% (n=35) reporting moderate (PHQ-9=10.0-14.9), 9% (n=23) reporting moderately severe (PHQ-9=15.0-19.9), and 4% (n=11) reporting severe (PHQ-9>20.0) depressive symptomatology. The median HIV-ASES score was 104 (IQR 89-114). Although most (n=150, 59.3%) reported high perceived social support (MSPSS>5.0), 9% (n=23) had low (MSPSS<3.0) perceived social support, and 32% (n=80) had moderate (MSPSS=3.0-5.0) perceived social support. Approximately one-quarter experienced housing insecurity in the past 6 months (n=61, 24%), and nearly one-third were ever incarcerated (n=77, 30%). Some (n=39, 15%) reported serious substance use in the past 2 weeks. Nearly three-quarters (n=178, 70.4%) reported less serious substance use, including alcohol, marijuana, ecstasy, speedballs, or gamma-hydroxybutyrate. Most (n=180, 71.1%) self-reported being ≥80% adherent to ART at baseline. Group comparisons for all baseline characteristics and ART adherence at baseline (Table 1) revealed that randomizations were successful.

Primary Outcome

Among the 253 participants who were randomized and downloaded an app, 131 (51.8%) completed the 1-month follow-up for ART adherence, and 122 (48.2%) did not. A significantly greater proportion of those randomized to the Food Safety app were missing 1-month follow-up data compared to the My Personal Health Guide app (74/129, 57.4% vs 48/124, 38.7%; P=.005; Table 2). Additionally, remote participation, employment, substance use, and ART adherence at baseline differed by complete 1-month follow-up data and were therefore considered during multivariable model construction. Bivariate analysis (Table 3) revealed the mean HIV-ASES score (mean 103, SD 15.8 vs mean 86.4, SD 28.5; P=.02) was significantly higher in those who had ≥80% adherence to ART at 1 month as compared to those who had <80% adherence at 1 month. The mean depression symptomatology scale (PHQ-9) was moderately significantly lower among those who were ≥80% adherent to ART at 1 month as compared to those who were <80% adherent to ART at 1 month (5.70, SD 4.96 vs 9.00, SD 7.43; P=.07). There was also a significantly higher prevalence of being in a committed relationship with a male partner among those who had ≥80% adherence to ART at 1 month as compared to those who had <80% adherence at 1 month (25/111, 22.5% vs 0/20, 0%; P=.01).

In multivariable analysis (Table 4), the odds of being ≥80% adherent to ART at 1-month follow-up were almost 4 times greater (OR 3.97, 95% CI 1.26-12.55) among participants randomized to the My Personal Health Guide app as compared to the Food Safety app after adjusting for other characteristics that were also associated with ART adherence at 1 month, including HIV-ASES and having a history of ever being incarcerated. Every 1-point increase in the HIV-ASES increased the odds of ≥80% ART adherence by 3% (OR 1.03, 95% CI 1.00-1.06). A history of ever being incarcerated reduced the odds of ≥80% ART adherence (OR 0.31, 95% CI 0.09-1.06). ART adherence at baseline was marginally associated with ≥80% ART at 1-month follow-up (OR 2.97, 95% CI 0.88-9.99).

Discussion

Principal Findings

Participants randomized to receive My Personal Health Guide demonstrated nearly 4 times greater odds of being ≥80% adherent to ART compared to participants who received the Food Safety control app at 1-month follow-up. This is a substantial magnitude, and although these are short-term follow-up results, these data provide strong evidence that a relational agent approach to improving medication adherence is promising.

Our study is the first of its kind in the field of HIV medication adherence. We are aware of only 2 other published RCTs of a relational agent for behavior change. Rubin et al [20] performed an RCT of a relational agent to deliver a brief alcohol intervention and referral to treatment to veterans within VA primary care clinics. Their approach involved 2 meetings with the relational agent, who requested a commitment to change. They reported that their relational agent elicited concern about the consequences of drinking and bolstered motivation to change. Although they did not observe accelerated decline in drinking during 3 months with their intervention compared to their control group, they noted that their intervention group was more likely to receive referrals for follow-up and that this approach allowed for more screening and brief intervention without increasing the burden on clinic staff. Another study by Prochaska et al [21] studied a smartphone intervention called Woebot for substance use disorders that used a therapeutic relational agent delivering cognitive behavioral therapy. At 8 weeks, they reported reduced substance use occasions compared to a waitlist group. These studies demonstrate the potential for a relational agent to influence behavior and underscore the scientific rationale for their application to the field of medication adherence. Relational mobile health interventions may be particularly beneficial for younger (18-24 years) AAMSM because they are more likely to still be developing medication-taking routines and may be especially receptive to interactive mobile technologies that can bolster daily ART adherence (Hightow-Weidman et al [22]). Given that these studies and ours are RCTs, they provide evidence for consideration of relational agents in the design of behavioral interventions, especially involving conditions where stigma about diagnosis or treatment could create hesitancy with face-to-face health care provider education or counseling.

In addition to our study demonstrating a large magnitude efficacy, there are several other strengths and considerations. First, our study began shortly before the COVID-19 pandemic spread throughout the United States, disrupting in-person research with lockdowns and other restrictions. This led to a nearly entirely remote rather than facility-based administration and may have made the meeting with research staff for orientation to the app less interactive than it might have been otherwise. The remote approach could have diminished the participant’s feeling of accountability to the study or facilitated participation by reducing the barrier of travel to a study activity. Second, unlike medication or immunization, our app intervention involves a passive approach. In other words, patients are encouraged to use it, but it is their choice how much of the app they experience and how often they open it. The app was designed to have new functions and experiences occur with repeated use to encourage return to the app. However, we have no control to ensure regular app exposure. Despite these issues, we observed a significant effect in achieving the adherence goal. This suggests that the effect might increase further and benefit more patients with additional app refinement and involvement of health care providers or case managers in implementation who could refer the app to their patients and check in with them at future appointments. Third, our study sample was diverse, including a large proportion who experienced housing insecurity, screened for moderate or severe depression, and reported a history of incarceration. Additionally, our sample included an overrepresentation of men living in the Southern United States, a high HIV incidence region where many experience structural barriers to HIV care [23] and where a low-cost, scalable intervention like ours could contribute to achieving higher prevalence of viral suppression and help end the HIV epidemic. Finally, our study also demonstrated an independent effect of self-efficacy on being ≥80 adherent. This finding is consistent with other studies, such as those by Dworkin et al [24], Barclay et al [25], and Waldrop-Valverde et al [26]. However, this is the first study to report this specifically in a population of young AAMSM living with HIV.

Limitations

Limitations of this study included that many participants were recruited using a social media dating app and the responsiveness of participants to the 1-month call was moderate, both of which may limit generalizability. Notably, we struggled to recruit younger (18-24 years) AAMSM; therefore, we recommend future studies use a multimodal recruitment approach. Additionally, missingness of self-reported ART adherence was high, and we observed a higher proportion of those randomized to the control with missing data at month 1. Study procedures were disrupted by the COVID-19 pandemic, necessitating the change from a facility-based trial to online. While this pivot allowed for further recruitment and retention, it compromised the collection of viral load data from participants’ clinics, which was initially an outcome of interest. This change also impacted the collection of dried blood spots at the 6-month follow-up, which was initially intended to be conducted on-site and changed to be self-administered in the participant’s home to accommodate COVID-19 stay-at-home orders. This resulted in a high proportion of missingness. Additionally, the laboratory methodology for the threshold of detection changed during the study period and used a higher threshold (<839 copies/mL) than the clinical definition of undetectable viral load (<200 copies/mL). Similarly, the usage of Wisepill devices to monitor ART adherence was low, so self-report was used as the primary outcome of interest. Using self-reported ART adherence also had limitations. Self-report can overestimate individual medication-taking behavior and therefore may introduce recall and social desirability bias. However, self-reported adherence is often used to assess ART adherence in both routine care and research settings and has been correlated with viral load and other objective markers of adherence [26]. Last, these findings are limited to the first month of observation. Additional follow-up analysis from this trial is planned to determine evidence of long-term effect.

Conclusion

These findings provide evidence that My Personal Health Guide has short-term efficacy in improving ART adherence among young AAMSM living with HIV and contribute to the emerging field of technology-based behavioral interventions promoting health using relational agents. We recommend further research on the inclusion of relational agents in behavioral research, especially in populations affected by stigma and nonoptimal health literacy, where this nonhuman supportive and educational approach may be complementary to health care systems.