Background: The poor engagement of men with HIV care is attributed to a number of factors: fear of stigma, masculine representations, concerns related to confidentiality, and the time commitment needed to visit public health clinics. Digital technologies are emerging as an approach to support the engagement of men with care.
Objective: This study aims to deliver a usable and engaging tablet-based app, called EPIC-HIV 2 (Empowering People through Informed Choices for HIV 2), to support men in making informed decisions about engaging with HIV care in rural KwaZulu Natal, South Africa.
Methods: We employed a mixed methods, iterative, and three-phased design that was guided by self-determination theory (SDT), a person-based approach, and human-computer interaction techniques. We reviewed related literature and conducted secondary analyses of existing data to identify barriers and facilitators to linkage to care and inform content development and design principles and used focus group discussions with members of the community advisory board and general community to evaluate a PowerPoint prototype of the app; used observations and guided questions with a convenience sample of potential users from the intervention community to iteratively test and refine a functioning interactive app; and conducted qualitative interviews and satisfaction surveys with actual users to evaluate acceptability.
Results: Phase 1 identified supply- and demand-side barriers to linkage to care. Specifically, clinics were feminized spaces unattractive to men with high social costs of attendance. Men did not feel vulnerable to HIV, preferred traditional medicine, and were afraid of the consequences of being HIV positive. Thus, the app needed to allow men to identify the long-term health benefits to themselves and their families of starting antiretroviral therapy early and remaining on it, and these benefits typically outweigh the social costs of attending and being seen at a clinic. SDT led to content design that emphasized long-term benefits but at the same time supported the need for autonomy, competence, and relatedness and informed decision making. Phase 2 indicated that we needed to use simpler text and more images to help users understand and navigate the app. Phase 3 indicated that the app was acceptable and likely to encourage men to link to care.
Conclusions: We found that iteratively developing the app with potential users using local narratives ensured that EPIC-HIV 2 is usable, engaging, and acceptable. Although the app encouraged men to link to HIV care, it was insufficient as a stand-alone intervention for men in our sample to exercise their full autonomy to link to HIV care without other factors such as it being convenient to initiate treatment, individual experiences of HIV, and support. Combining tailored digital interventions with other interventions to address a range of barriers to HIV care, especially supply-side barriers, should be considered in the future to close the present linkage gap in the HIV treatment cascade.
Poor Engagement of Men With HIV Care
Poor engagement of men with HIV care is well described [- ]. This results in the persistence of unacceptably high HIV-related mortality among men and new HIV cases in young women [ ]. In rural KwaZulu-Natal, South Africa, despite freely available antiretrovirals (ARVs) in the public sector, men are 25% more likely than women to die of HIV-related illnesses, and 70% of them had never sought care [ ]. Fear of stigma, cultural constructions of masculinity that do not involve caring for health, concerns related to confidentiality, and time commitment needed to attend public health clinics have been cited as key reasons for men’s reluctance to get tested and start HIV treatment [ , ]. Novel, scalable interventions that improve the uptake of HIV testing and care among men are needed to improve individual health and reduce HIV transmission risk [ , ].
The Potential of Digital Technologies in Closing Gaps in HIV Care
Digital technologies such as mobile phones are emerging as one of the approaches to close the gaps across the HIV care cascade [, ], especially in light of the expanding access to smartphones in areas most affected by HIV/AIDS. A recent Pew internet survey found that 33% of respondents from sub-Saharan Africa owned a smartphone, and the number is expected to double by 2025 [ ]. Digital technologies are particularly suitable as interventions to help prevent and manage HIV and AIDS, as they have the potential to access and impact hard-to-reach populations, including those who typically feel stigmatized within health care settings [ ]. Daher et al [ ] conducted a systematic review of digital innovations for HIV or sexually transmitted infections and found them to be effective in improving clinic attendance, adherence to antiretroviral therapy (ART), and turnaround time from testing to treatment. This suggests that digital interventions have considerable potential to support engagement in the HIV care continuum.
Challenges to the Development of Digital Interventions
There are many challenges to the development of digital interventions: most importantly, ensuring that the digital platform is usable and engaging for end users  and delivering the intended messages. Designing digital interventions for resource-constrained settings such as rural South Africa has added challenges, as there are varying levels of education, health, and digital literacy [ ]. Furthermore, ensuring that the intervention supports individual decision making can be challenging. There are limited data on how to develop digital interventions for HIV care in resource-constrained settings that are usable and engaging and support informed individual decision making.
As part of a larger, cluster randomized controlled clinical trial to increase home-based HIV testing and linkage to care in men , we developed 2 interactive tablet-based apps (to integrate with the trial) called EPIC-HIV 1 and 2 (Empowering People through Informed choices for HIV 1 and 2). Both are guided by self-determination theory (SDT) to support informed decision making and make explicit and usually implicit processes of decision making. EPIC-HIV 1 is offered at the point of HIV test to help men make an informed choice about testing and if necessary linkage to care. EPIC-HIV 2 is offered to men who have tested HIV positive but have not linked to care within a specified time frame. It aims to encourage men to link to care and stay in care by supporting autonomy, feelings of relatedness, and competence in linking to care by using positive examples from other men living with HIV.
The primary aim of EPIC-HIV 2 is to support informed decision making by making explicit decisions that are usually implicit and thus, encourage men to link to care, to initiate ART early, and to introduce the potential benefits of staying on ART. Therefore, it is aimed at shifting the motivation of link to care from unmotivated or a motivated to internally motivated because of the benefits of ART. SDT addresses factors that either facilitate or undermine motivation. According to SDT, human motivation is based on the satisfaction of 3 inherent psychological needs: (1) autonomy, (2) competence, and (3) relatedness. Autonomy requires an individual to make personal decisions and act in a way that corresponds to their identity, belief systems, and values [, ]; competence refers to an individual’s perceived ability to perform an act [ ]; and relatedness refers to the ability of an individual to connect to others and feel cared for [ ]. SDT proposes that events or conditions that enhance a person’s sense of autonomy, competence, and relatedness support internal motivation, which, in turn, can facilitate the adoption of new behaviors to be internally motivated and sustained [ , ].
In the case of HIV, SDT has been used to understand ART adherence, treatment motivation , and overall health behavior among individuals living with HIV [ , ]. Applying evidence-based theories to the development process of interventions helps to direct attention to design characteristics that may otherwise be ignored and indicate conditions under which interventions are more likely to be effective [ ]. EPIC-HIV 2 development was guided by SDT to increase men’s internal motivation to engage with HIV care. In this paper, we provide an overview of the approach we adopted for designing and developing EPIC-HIV 2. Using one component of the app as a detailed case study, we describe the design rationale and components of the app and report on its acceptability to users.
As mentioned earlier, EPIC-HIV is being evaluated in a home-based intervention to test and start (HITS), a cluster randomized controlled trial designed to evaluate the effect of 2 interventions—EPIC-HIV (male-targeted HIV-specific decision support app) and microincentives on increasing home-based HIV testing and linkage to care among men, with the ultimate aim of reducing HIV-related mortality in men and hence HIV incidence in young women in rural South Africa . The details of the study are described elsewhere [ ] and is registered on the National Institute of Health trials (identifier NCT03757104). In summary, the trial uses the Africa Health Research Institute (AHRI) HIV surveillance platform, which visits households annually to conduct surveys using Android-based tablets and offer home-based HIV testing. EPIC-HIV 1 was administered at the point of HIV test offer during the fieldworker surveillance activity to encourage men to test for HIV. If a participant was diagnosed with HIV but did not present in the local Department of Health clinic within a month of the positive HIV test, a study fieldworker revisited the participant at home to offer EPIC-HIV 2. Both apps were installed on the field worker’s tablet. During the home visit, the field worker handed over the tablet to the participant together with earphones to allow the participant to independently explore the app. EPIC-HIV 2 uses a mixture of audio, text, video, still photos, and graphics and has different pathways through 3 interactive modules. This paper focuses on the design and development of EPIC-HIV 2.
This was a mixed methods study using a person-centered approach and human-computer interaction (HCI) techniques. The study was conducted in 3 phases. Phase 1 focused on user requirements and early prototyping. It included identifying barriers and facilitators to HIV linkage to care through literature review and secondary analysis of existing qualitative data from AHRI. We then developed a PowerPoint prototype of the app based on SDT and drawing on established behavior change techniques , which was discussed with members of the Community Advisory Board (CAB; special body representing members of the intervention community that act as a bridge between AHRI and the community, safeguarding the rights of the study participants) and general members of the community in focus group discussions (FGDs). Together, these methods were used to inform app content requirements and conceptual design. In phase 2, based on the findings from phase 1, we developed a functioning, interactive app. We iteratively tested and refined the app with potential users from the intervention community using observations and guided surveys. Finally, in phase 3, we evaluated the acceptability of the app with actual users (participants who received the EPIC-HIV intervention in the HITS trial). This process is illustrated in .
This study was conducted in uMkhanyakude district in KwaZulu-Natal, South Africa. The area is predominantly rural and poor, with high levels of unemployment  and HIV prevalence [ ]. Previous research has shown poor linkage and low ART uptake among men in this area [ , ] mainly because of fear of stigma [ , ].
Recruitment and Sample
All participants were recruited from the AHRI HIV surveillance community. Each phase employed different recruitment and sampling methods; these are described in detail under each phase.
Ethical approval for the study was granted by the Biomedical Research Ethics Committee of the University of KwaZulu-Natal (BREC 398/16).
Below, we provide an overview of each phase, outlining its main aim, study procedures, and participants.
Phase 1: Requirements and Conceptual Design (App Development, September 2016 to December 2017)
The aim is to (1) gather the app content requirements and formulate design principles and features that the app should include by identifying modifiable barriers to linkage to care, (2) develop an initial prototype, and (3) evaluate it with potential users for overall impressions, appropriateness, and relevance to our setting.
First, we drew on systematic reviews conducted on barriers and facilitators to testing and linkage to care and searched for additional studies in South Africa or elsewhere in sub-Saharan Africa ([ , - ]). Second, we conducted a thematic secondary analysis from an AHRI study that explored community experiences and perceptions of tuberculosis and HIV transmission, treatment, and prevention [ ] ( ). The thematic coding frame also identified supply- and demand-side barriers to linkage to care, focusing on men’s perspectives. The findings from the literature review and secondary analysis were summarized as barriers to linkage, and those that were potentially modifiable were identified ( ). Third, we identified specific behavior change techniques from the Behavior Change Taxonomy [ ] that could support feelings of autonomy, relatedness, and competence and thus were likely to shift motivations for linkage to care, support informed decision making, and make explicit processes that are usually implicit (such as avoiding the clinic). Finally, we developed a PowerPoint prototype of the app and evaluated it with focus groups using an FGD guide (semistructured questions, assessing the content and design and overall app impressions). Informed consent was obtained from all participants before beginning each focus group. FGDs were conducted in the AHRI offices in isiZulu (native language), audio recorded, transcribed, translated, summarized by TZ, and discussed with the development team (MS, PM, SW, TM, and TZ). They were analyzed using thematic analysis [ ] to draw out key themes for design.
To obtain diverse views, we invited a purposive sample of men aged 18 years and above from the CAB and general community. We sought members representing periurban and rural communities in the surveillance area. In general, we aimed for men of different ages and occupying various positions in the community. For example, we recruited men from the taxi industry, church, liquor business, and formal and informal occupations, those who were unemployed, and those in the traditional health sector. The AHRI public engagement (PE) department assisted with the recruitment of participants. Through PE, the research team explained the study to the CAB, and the members who were willing to participate and provide informed consent were phoned for further engagement and to arrange a suitable time for the FGD. Of the 10 CAB members, 7 were included in the study. To recruit community members, a research assistant in the study approached men meeting the study inclusion criteria and explained the study, and those who were willing to participate provided their telephone numbers to be contacted to arrange a time for the FGD. Of 11 men who were approached, 4 participated in the study. We conducted 2 FGDs, one for the CAB and one for the community members.
Phase 2: Iterative Design and Usability Evaluations (App Refinement, January to June 2018)
The main aim of this phase is to iteratively design and test the app to refine it to be usable and engaging in our setting.
We conducted 3 cycles of usability evaluations to assess participants’ ability to successfully navigate the app, comprehend the educational content, and determine whether they found the app to be engaging and relevant. The first cycle was to establish usability requirements, the second cycle was used to evaluate the design changes made as a result of the first evaluation, and the last cycle was used to refine the understanding of the user requirements. User-centered evaluations were conducted with potential users using observations and guided surveys that consisted of a mix of open and closed questions to assess comprehension of the content and record usability issues. The questions were adjusted to suit the objectives of each cycle of evaluation with 2 questions that remained consistent to benchmark the design changes (). Evaluations were conducted in isiZulu, and we recorded the audio and tablet screen during the session. Data files were translated and transcribed, summarized, and discussed with the HCI expert and technological partner after each evaluation cycle. On the basis of the findings, the app was tailored to ensure that participants understood the messages correctly and to improve the user experience.
We used pop-up user testing  to recruit a convenience sample of men aged 18 years and above from the intervention community. We sought participants who represented a broad spectrum of technological and educational literacy, involving at least six participants in each cycle to get quick feedback on the app. The team drove (in an AHRI-marked vehicle) to different locations in the area and asked men (novice users) to participate in this study. Participants gave verbal consent and were asked to confirm if they were aged 18 years and above; they were not required to disclose their HIV status or any personal information.
Phase 3: Acceptability Testing (October 2018 to March 2019)
The aim of this phase is to evaluate the acceptability and perceived value of the resulting app with actual users (participants who received the EPIC-HIV intervention in the HITS trial).
We conducted satisfaction surveys and individual in-depth interviews. An adapted Client Satisfaction Questionnaire (CSQ-8)  was used to assess satisfaction with the app (see for the adapted questionnaire). The CSQ-8 has 8 items: quality of app, app met needs, kind of service received from the app, recommend app to a friend, amount of help received from the app, effectiveness of the app for dealing with the health problem, overall satisfaction, and willingness to use the app again. These items are assessed on a 4-point Likert scale of 1 to 4 with individually specified anchors, and 4 is consistently the positive assessment. For this study, we added 2 questions to assess the relevance of the app for this setting and user friendliness. The CSQ-8 has been used to evaluate technology-based intervention and has demonstrated high consistency [ ]. The 10 items were administered on a REDCap [ ] project by a fieldworker. Proportions were used to describe the acceptability. Statistical analyses were conducted using STATA software (version 15.1; StataCorp LLC). We complemented the satisfaction surveys with in-depth interviews with a purposive sample of participants who received EPIC-HIV 2. The interview questions explored why participants had not been linked to care after the HIV diagnosis and their views of EPIC-HIV 2 and the impact it had on them. Interviews were conducted at participants’ homes in isiZulu and lasted approximately an hour. The audio files were transcribed, translated, and analyzed thematically with themes guided by SDT. NVivo was used to code and manage the transcripts. Separate informed consent was obtained for participating in the satisfaction surveys and in-depth interviews.
|Survey item||Response, n (%)|
|How would you rate the quality of the EPIC-HIVa app you just listened to?|
|To what extent has the app helped in meeting your needs with regards to information about HIV treatment?|
|Only a few||1 (7)|
|Did you get the kind of HIV management information you wanted or expected?|
|Yes, definitely||14 (100)|
|Yes, generally||0 (0)|
|No, not really||0 (0)|
|No, definitely||0 (0)|
|Did the information you just listened to appeal to your conscience to go for HIV treatment and take ARVsb if you have not done so?|
|Yes, definitely||13 (93)|
|Yes, generally||1 (7)|
|No, not really||0 (0)|
|No, definitely||0 (0)|
|Do you feel empowered to make informed choices regarding your health with the information from the app?|
|Yes, definitely||12 (86)|
|Yes, I think so||2 (14)|
|No, I don\'t think so||0 (0)|
|No, definitely not||0 (0)|
|If a friend were in need of HIV treatment and management information, would you recommend the app to him or her?|
|Yes, definitely||14 (100)|
|Yes, I think so||0 (0)|
|No, I don\'t think so||0 (0)|
|No, definitely not||0 (0)|
|How satisfied are you with the amount of HIV management information you received from the app?|
|Very satisfied||12 (86)|
|Mostly satisfied||2 (14)|
|Mostly dissatisfied||0 (0)|
|Quite dissatisfied||0 (0)|
|How would you rate the simplicity and user-friendliness of the app?|
|In general, how satisfied are you with the app?|
|Very satisfied||14 (100)|
|Mostly satisfied||0 (0)|
|Mostly dissatisfied||0 (0)|
|Quite dissatisfied||0 (0)|
|If you were to seek information about HIV management again, would you consider EPIC-HIV app?|
|Yes, definitely||14 (100)|
|Yes, I think so||0 (0)|
|No, I don\'t think so||0 (0)|
|No, definitely not||0 (0)|
aEPIC-HIV: Empowering People through Informed Choices for HIV 2.
All men aged 15 years and above who received EPIC-HIV 2 between October 2018 and January 2019, as part of the intervention in the HITS trial, were asked to consent and complete the satisfaction surveys. For the individual in-depth interviews, we purposively selected 14 men from the 28 who had received EPIC-HIV 2 between April 2018 and January 2019 (which meant that they had tested positive for HIV but did not link to care within a month).
Each phase is presented separately with a particular focus on the iterative development.shows the key takeaway findings from the 3 phases.
Key takeaway findings for the different phases.
- Identified key learning points to incorporate in the content to persuade men to attend clinic and start antiretroviral therapy
- Long-term health benefits of starting antiretroviral therapy early and remaining on it
- Men in our setting did not understand the seesaw metaphor
- The click worked better than swipe
- Two options were better than 3 options
- App was usable and acceptable in our setting
- It encouraged men to link to HIV care
Phase 1: App Development
In the literature review and analyses of the earlier study, we identified barriers and facilitators to HIV testing and linkage to care among men, particularly relevant to our setting (). Through this process, we identified 4 key learning points (shown in ) to be incorporated into the content of EPIC-HIV 2 with the aim of supporting informed decision making by making explicit decisions that are usually implicit and demonstrate that attending a clinic to start ART can be managed and that the long-term benefits of starting ART early and remaining on ART outweigh the costs. Drawing on SDT, we developed the app content and design to map to the psychological requirements for autonomy, relatedness, and competence (shown in ). To increase the likely personal identification with the messages and potential feelings of relatedness, the content was provided in the form of personal testimonies or experiences from local men. This was informed by evidence that suggests experiential information helps users make sense of what various outcomes might be like (in imagined futures) and increases their awareness of personal health risk and the likelihood of a response to the intervention [ ].
A total of 11 men aged 34 to 66 years participated in 2 focus groups to evaluate the first (paper) prototype. They provided feedback on content and design (organization of the app, appropriateness and appeal of language and images, acceptability and relevance, and overall app impressions). On the basis of the feedback from the FGDs, the team modified the initial conceptual design of the app and created a revised plan for the app features and content.
The final realized product (EPIC-HIV 2) has 3 main modules (sections), and to further support the psychological requirements for autonomy, the user has the option to engage with 1 or all 3 modules (and ; ). All 3 modules cover the 4 key learning points ( ) but present the information in different ways to support different learning styles, social identities, and barriers to engaging with care: (1) Jabu’s man-to-man advice—a taxi driver talking about how he views HIV as an uninvited guest and gives his personal testimony on how he controls HIV; (2) healthy and strong with HIV—4 male characters who give their experiential information on how they managed going to the clinic, how they disclosed their HIV status, and how they started ART, and stayed on ART; and (3) the facts are your shield—gives a broader picture of ART and how it fights HIV in the body by explaining the commonly used medical terminology in HIV care. There is a start button, and a male voice over that introduces EPIC-HIV 2 and gives instructions for navigating the app. As mentioned earlier, the app uses a mixture of audio, text, video, still photos, and graphics, and there is a continue button that the user needs to click to move to the next page ( provides an example of the navigation in the Jabu module).
The final conceptual design (realized in the product) starts with an introduction and instructions on how to use the app, and users are then able to choose between “Jabu’s man to man advice” or “healthy and strong with HIV” modules. The “shield” module was not offered initially because it has more focused material and is expected to appeal to fewer men (and to make the number of choices manageable). After users complete the initial module, they are returned to the menu and are able to choose one of the remaining modules or to end.shows all the possible pathways through the app. Each module was developed and tested individually, as described in phase 2, and once finalized, all the different modules were integrated into a single app that was evaluated for functionality and comprehension.
Phase 2: Iterative Design and Usability Testing (January to June 2018)
Decision Support Tool
The main outcome of phase 1 was an in-depth understanding of user needs relevant to EPIC-HIV 2. We subsequently refined the prototypes using a standard app development toolkit to deliver testable interactive demonstrator prototypes on an Android tablet computer. In this paper, we illustrate the approach to iterative design and usability testing by focusing on the development of the decision support tool (a component within Jabu).
The decision support tool aimed to help men weigh up and reflect on the costs and benefits of attending a clinic. The user was presented with different statements that either support or delay going to the clinic. To facilitate engagement, we wanted to make this section interactive. Initially, we applied a seesaw metaphor for users to attempt to render usually implicit decisions as explicit and interpret whether they were ready to attend a clinic within 2 weeks. This was done by adding the arguments to one side or the other of the seesaw to illustrate how the arguments stacked up in decision making. For example, if the user agrees that they want to see their children and grandchildren grow up, that would be an argument that weighs in favor of engaging with HIV care, whereas if they agree that they do not wish to be seen at a clinic by one of their neighbors, that would be an argument that weighs against engaging with care. In making design decisions about the clinic decision support tool, we prototyped and tested 2 versions: drag-and-drop with illustrations and a Likert scale without illustrations, both employing the seesaw metaphor. These are illustrated in(loose translation: “ngifuna ukubona abazukulu bami”—I want to see my grandchildren; “qiniso”—true; “amanga”—false; “ngiyavumelana”—I agree; “ngiphakathi nendawo”—neutral; “angivumelani”—I disagree) and .
The main objective of the first evaluation was to refine the user requirements. The key objectives and results of this evaluation are shown in.
In total, 12 men participated in this evaluation, which was conducted as outlined earlier. In addition to the questions related to the key objectives outlined in, we also asked participants what the main messages were and whether there were any statements that should be added (ie, other considerations in deciding whether to engage in HIV care) and whether there were other ways the design could be improved. Overall, participants found it difficult to use the drag-and-drop option and did not understand the seesaw metaphor. Clicking was intuitive, and most participants preferred the images and text and the 2 options (agree or disagree). There was a consensus that the key message of this section was going to the clinic to start ART.
Key objectives and results of evaluation 1.
To test which interaction style participants preferred and found easier to use between a drag-and-drop version (dragging each statement into a bucket labeled agree or disagree) or a Likert scale version (where people clicked in a box labeled agree, neutral or disagree”)
- Participants found it difficult to use the drag-and-drop version, particularly more aged participants and participants with lower digital literacy levels
- A majority of participants needed to be shown how to drag-and-drop, despite the illustration at the beginning of the statements
- In contrast, clicking was intuitive; almost all the participants started clicking when they took the tablet
To determine whether the neutral option was valued by participants
- Some participants preferred drag-and-drop because it had 2 options (agree or disagree): “drag—I found it easy with the scale I was not completely sure of what was happening. With the drag option there was only two options which made it easy to select” [P6]
- Another participant reported that he found the neutral option confusing
To test whether participants preferred each option presented as text only or as text with an illustrative picture
- Most participants preferred text with an illustrative image to help them select the statements over the text only with many participants stating that images made it easier to understand the statements: “the image makes it clear what the statement says” [P7]
- Some found the images to facilitate engagement: “the images capture your attention” [P6]
To test whether participants were able to correctly interpret a seesaw metaphor, in which each response was stacked up on the end of a seesaw depending on whether it weighed toward or against attending a clinic (correct interpretation of the metaphor was that the end that was more heavily weighted and hence down was the preferred outcome)
- Participants did not understand the seesaw metaphor
- Many expressed confusion on how to interpret the results: “not sure how it should look like on the seesaw. When is it good, when it is up or down?” [P1]
To assess whether it was clear how to agree or disagree with a statement and whether it was clear how their selections related to the outcome of the seesaw
- It was not clear for some participants on how to agree or disagree with the statements; 2 participants (2/8) in the Likert scale option selected agree for all the statements
On the basis of the findings, the following design changes and decisions were made (as illustrated inand ):
- As the app was intended for one-off use, after seeing that clicking was intuitive and users struggled with the drag-and-drop, we decided to use clicking to select statements.
- We limited the options to agree and disagree, as some participants did not find value in the neutral option.
- To support users to make informed choices regarding attending a clinic for care, we used text and image as users expressed that images made it easier to understand the statements.
- It was judged that the seesaw metaphor was not applicable for this exercise, as many participants struggled to understand what it meant if one side was more heavily weighted than the other. We removed the seesaw, summarized the statements, and explained what it means if they were skewed on one side. Furthermore, we labeled the right side as supporting going to the clinic in 2 weeks and left as delaying going to the clinic.
- No further changes were judged necessary to address this question.
The main objective of the second evaluation is to validate the design changes made, particularly the summary screen shown in. A total of 7 men participated in this evaluation cycle. We stopped them at the end of each section to ask guided questions: (1) Does the text only make the choices you selected clearer or do you think text and image would make it clearer? (2) What do you think is the key message of this section? (3) Would you prefer the agree and disagree buttons to be color coded?
Results of Evaluation 2
The design of the selection tool was received positively by all participants. All participants were selective on the statements they agreed or disagreed with as per their individual experiences, and they were able to interpret what it means if statements were weighted on one side (). However, 2 participants mentioned that the statements were not clear or applicable to them. This could have been because the user was not affected by or living with HIV (the researcher did not ask the participant’s HIV status).
Of 7 participants, 4 preferred the text only in the summary page, validating the first evaluation findings that images were only important at the selection point. In addition, 6 participants reiterated that the key message of this section was about the importance of going to the clinic to start ART, with 1 participant adding that one should go to the clinic early and not wait until they are sick. Moreover, 5 participants said they would prefer the agree or disagree buttons to be color coded, with 4 participants suggesting that agree should be green and disagree, red:
colours help, maybe green and red so that they can see that they have made the right or wrong choice.
One participant stated that different colors would be particularly useful for people who cannot read.
On the basis of the preference for color coding and the association of green with right and red with wrong, we decided to use neutral colors that are not associated with right or wrong to help users make an individual choice as opposed to fearing being right or wrong ().
This evaluation was conducted as part of the full Jabu section testing, mainly to assess understanding of the content as well as to check whether users find the presentation of content interesting and to validate the changes made as a result of evaluation 2. A total of 8 men participated in this cycle.
This evaluation did not reveal any usability issues, indicating that the changes made to the designs were effective. Participants were paying more attention before deciding whether they agreed or disagreed, and they listened to the audio while looking at the screen. Overall, participants reported finding the decision support tool engaging and useful. For example, when asked which part of the app they thought would be most relevant to their friends or other men, 3 participants referred to the clinic decision support tool. In addition, 1 participant, when asked if he learned anything new as a result of using the app, mentioned that he learned how to use a tablet, suggesting that there was incidental value (not directly related to the primary aim) in introducing such an app to people.
Phase 3: Acceptability Testing (October 2018 to March 2019)
User Satisfaction Surveys
A total of 15 participants received EPIC-HIV 2 between October 2018 and January 2019, and 14 completed the user satisfaction surveys. Their mean age was 35.6 years (SD 13.38299), and a majority (8/14, 57%) had high school education, and half of them were unemployed. User satisfaction ratings were high (). Participants reported that they were generally satisfied with the app and found it to meet their needs for information on HIV treatment. All participants said that they received the kind of HIV management information that they wanted or expected and that they would definitely consider the app when seeking information about HIV management as well as recommending the app to friends in need of similar help. In addition, 86% (12/14) of participants were very satisfied with the amount of information that they received from the app. Overall, 79% (11/14) of the participants were very satisfied with the app by rating the simplicity and user friendliness of the app as excellent. The app has proven to empower the participants in managing their health; 86% (12/14) of participants definitely agreed.
A total of 14 men (aged 35-53 years) who had tested positive for HIV and had not linked to care within a month of receiving their test results received EPIC-HIV 2 between April 2018 and January 2019 agreed to be interviewed; 11 of them had linked to HIV care at the time of the interview. A majority of participants reported that the app offered relevant information about HIV management, boosted their confidence, and encouraged them to start and stay on treatment. However, this was not enough for all participants to link to HIV care. We report the experiences of men who used EPIC-HIV 2 using the SDT framework: autonomy, competence, and relatedness.
Complexities of the HIV Care Pathway: Examining Men’s Autonomy to Link to HIV Treatment
Participants described the app as motivating them to start ART by providing them with information on how to manage HIV to live a healthy life:
Mmm... I can say I have listened to the App from the beginning up to the end. Yes, I have really seen a need...I have seen it as a very important thing. I think that has encouraged me because I have never been to the clinic previously, but I went to clinic after that.
[Participant 2, linked to care]
For some, this was a catalyst to link to care (even if they did not act on it immediately):
I was supposed to visit the clinic a long time ago, but I didn’t because of work...I was thinking about that when I listened to EPIC [app]...you must be open as the EPIC was saying that you mustn’t fear that people will laugh at you. I took a decision because...this is life and we are living within a community...I realised that in order to be assisted, I must go to AHRI nurses at the clinic.
[Participant 5, not linked to care]
However, for some participants, the app alone was insufficient to motivate them to go to the clinic for HIV treatment. Other factors reduced the logistic barriers to linkage that nudged them to link. For some participants, it was the availability of our research nurses at local clinics, with the:
referral letter... that was going to make things easy for me...
[Participant 4 linked to care]
For others, such as participant 1, it was experiences such as the death of his best friend that motivated him to link to care after using the app:
Er...I can just say that there is nothing new I have learnt (from EPIC)...I started treatment because I've lived with people who had been using it (ARVs) before I knew my health status. One of my best friends passed away after he defaulted from taking treatment. After I found out that I am HIV positive I was motivated, and I said I will try by all means to take treatment accordingly. So, I can say I have learnt a lot from him about the risk of delaying...
[Participant 1, linked to care]
This suggests that for men to exercise autonomy to link to care, in addition to the app, external factors such as clinic operation hours, support, and individual experiences of HIV need to be considered.
“I Am Not Giving Up on Life”: Men’s Competency and Decision to Manage HIV
Several of the participants reflected on how the positive narratives in the app encouraged them to take charge of their lives and manage HIV to improve their health outcomes and “avoid infecting other people with diseases” by describing “how to take the pills on time” and “comply with treatment.”
Some participants also reported that app messaging lessened their fears (particularly fear of disclosure) and motivated them to adopt positive attitudes to advance their health. The quotation below illustrates how some participants felt about their fears:
...I had that feeling of fear to say, oh my Lord how could I do this thing since I am afraid as it will be revealed? Eh, I also gained advices because they start with allowing you to listen from the messages that say once you know your status, you can get help so it’s up to you whether you do it or not. But I chose to be assisted because I am not giving up on life.
[Participant 4, linked to care]
Others also drew attention to the message of EPIC-HIV 2 that it gave them tips on how to discuss disclosure with their sexual partners:
Er...like disclosing your health status to your partner so that she can check for HIV as well for both of you to take treatment, yes, I think it’s good not to hide from your partner:
[Participant 2, linked to care]
Overall, for many participants, the app supported their competence to link to care through the positive messaging that gave hope, alleviated their fears, and reinforced the need for them to manage their health effectively to live a healthy life while minimizing the risk of transmitting HIV.
Relatedness to Characters in the App: Men’s Perceptions of HIV Treatment and Management
Participants reflected how the narratives of different characters in the app reverberated with their present realities and how the app stories shaped their understanding of the importance of HIV treatment and management for them to live a healthy life:
...They [app characters] have mentioned the fact that once you have defaulted from taking treatment you will be very sick…the characters I have seen on the app were fit. They were sharing their experiences as to how they are taking their ARVs...
[Participant 6, linked to care]
They [app characters] were sharing simple information, something that is said even on TV by Ministers that you need not to be bedridden until you go to the clinic.
[Participant 3, linked to care]
Furthermore, a participant who had previously defaulted treatment had this to say:
The part I have seen as most essential one for me it’s where the characters were talking about the risk of defaulting from taking treatment...It motivated me to decide to be re-initiated on treatment...It is really necessary...because characters on the Epic were talking about things they had experienced.
[Participant 2, linked to care]
In sum, the empathetic nature of the personal testimonies of various characters in the app resonated with participants, and they felt cared for. Some of them described the relevance of the information shared through the app and were able to initiate treatment and took the decision to manage HIV to improve their health outcomes.
We found that it was possible to successfully apply a multiphased iterative development process to create a theoretically informed, interactive, tablet-based app to support men in making informed decisions about engaging with HIV care in a low-income, rural South African setting. Men in our setting who had been missing from the HIV care cascade found the app to be acceptable and reported that the stories in the app resonated with their realities and encouraged them to link to care. Previous work has established that digital interventions are acceptable and effective in improving clinic attendance, ART adherence, and turnaround time from testing to treatment [, , ]. Our results underscore the value of using a person-based approach to integrate evidence-based content and design to ensure that the app is relatable and addresses the local perception of HIV care. Using HCI techniques ensured that the app was simple, usable, and engaging for end users with varying levels of education, health, and digital literacy. For example, user testing established that the seesaw metaphor was difficult for participants to understand, so it was removed between the first and the second iterations of app testing. In addition, supporting different learning styles and motivations through tailoring of content provided a greater level of engagement for participants. Previous studies have shown that interactive digital interventions that deliver tailored content that address the specific challenges for individual users can be highly engaging and likely to be understood [ ]. Furthermore, using digital technologies could help people learn new skills (using a tablet) and facilitate a sense of accomplishment that can improve their competence and possibly improve engagement.
Drawing on behavior change theory, such as SDT, ensured that EPIC-HIV 2 supported individual decision making. There is an increasing need to apply evidence-based theories to enhance the development of digital interventions and improve their efficacy [, , ]. Our findings corroborate previous studies that have used the SDT framework (autonomy, competence, and relatedness) to understand ART adherence and treatment motivation and overall health behavior among men living with HIV [ , ]. However, the app was insufficient as a stand-alone intervention for men in our sample to exercise their full autonomy to link to HIV care without other factors such as it being convenient to initiate treatment, individual experiences of HIV, and support. Combining tailored digital interventions with other interventions to address a range of barriers to HIV care, especially supply-side barriers, should be considered in the future to close the present linkage gap in the HIV treatment cascade.
Study Strengths and Limitations
Using a mixed methods approach to understand usability and acceptability and the potential of the app to encourage men to link to care are strengths of the study. However, the generalizability of study findings outside our area may be limited because of the locally tailored design of the EPIC-HIV 2 app, the sampling method, sample size, and specific study sites. In addition, the possibility of social desirability bias cannot be excluded as some participants might have found it easier to say positive things about the app and not necessarily translate to the actual initiation of ART. Furthermore, we cannot establish what the outcome would have been if we had made different design decisions or applied a different theory of behavior change. The decision to develop a tablet-based app was fitting for the specific context of use, but more widespread use of such an app would probably be achieved by implementing it on mobile phones for independent use; this remains an area for future work.
Using a multidisciplinary approach and drawing on evidence-based theories to develop digital interventions can ensure that the resultant products are acceptable and engage in a wide range of users. Our work aims to pave the way for a greater focus on mixed methods and person-centered approaches to the development of digital interventions for HIV care.
The authors would like to acknowledge the study participants for their time. They wish to appreciate Velaphi Gumede, Sithembiso Luthuli, Siyabonga Nxumalo, and Jaco Dreyer for participant recruitment, data collection, and data management support. This study was supported by a Partnership Research Fund from the Engineering and Physical Sciences Research Council Interdisciplinary Research Collaboration Early-warning Sensing Systems for Infectious Diseases (i-sense) EP/K031953/1. The HITS trial is funded by the National Institutes for Health R01AI124389. The AHRI’s Demographic Surveillance Information System and Population Intervention Program is funded by the Wellcome Trust (201433/Z/16/Z) and the South Africa Population Research Infrastructure Network (funded by the South African Department of Science and Technology and hosted by the South African Medical Research Council). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding bodies.
FT, MS, SW, and JS are the investigators of the HITS trial and developed the EPIC-HIV intervention design and protocol in collaboration with AB and PM. PM, AB, AZ, TM, TZ, SW, MS, OA, and FT were involved in the development of EPIC-HIV. SW, TZ, MS, OA, and JS designed the process evaluation. KD provided statistical support. TM wrote the first draft of the manuscript with support from AB and OA and coordinated the study. All authors contributed to the manuscript. All authors have read and approved the final manuscript.
Conflicts of Interest
Literature review and secondary analysis.PDF File (Adobe PDF File), 243 KB
User evaluation questions and guides.PDF File (Adobe PDF File), 108 KB
Self-determination theory app content and design.PDF File (Adobe PDF File), 221 KB
Example of navigation pathway in the Jabu module.PNG File , 286 KB
- Chikovore J, Gillespie N, McGrath N, Orne-Gliemann J, Zuma T, ANRS 12249 TasP Study Group. Men, masculinity, and engagement with treatment as prevention in KwaZulu-Natal, South Africa. AIDS Care 2016;28(Suppl 3):74-82 [FREE Full text] [CrossRef] [Medline]
- Govindasamy D, Ford N, Kranzer K. Risk factors, barriers and facilitators for linkage to antiretroviral therapy care: a systematic review. AIDS 2012 Oct 23;26(16):2059-2067. [CrossRef] [Medline]
- Kranzer K, Lewis JJ, Ford N, Zeinecker J, Orrell C, Lawn SD, et al. Treatment interruption in a primary care antiretroviral therapy program in South Africa: cohort analysis of trends and risk factors. J Acquir Immune Defic Syndr 2010 Nov;55(3):e17-e23 [FREE Full text] [CrossRef] [Medline]
- Tsai AC, Siedner MJ. The missing men: HIV treatment scale-up and life expectancy in sub-Saharan Africa. PLoS Med 2015 Nov;12(11):e1001906 [FREE Full text] [CrossRef] [Medline]
- Adeagbo O, Herbst C, Blandford A, McKendry R, Estcourt C, Seeley J, et al. Exploring people's candidacy for mobile health-supported HIV testing and care services in rural KwaZulu-Natal, South Africa: qualitative study. J Med Internet Res 2019 Nov 18;21(11):e15681 [FREE Full text] [CrossRef] [Medline]
- Cornell M, Schomaker M, Garone DB, Giddy J, Hoffmann CJ, Lessells R, International Epidemiologic Databases to Evaluate AIDS Southern Africa Collaboration. Gender differences in survival among adult patients starting antiretroviral therapy in South Africa: a multicentre cohort study. PLoS Med 2012;9(9):e1001304 [FREE Full text] [CrossRef] [Medline]
- Bor J, Rosen S, Chimbindi N, Haber N, Herbst K, Mutevedzi T, et al. Mass HIV treatment and sex disparities in life expectancy: demographic surveillance in rural South Africa. PLoS Med 2015 Nov;12(11):e1001905; discussion e1001905 [FREE Full text] [CrossRef] [Medline]
- Ahmed S, Autrey J, Katz IT, Fox MP, Rosen S, Onoya D, et al. Why do people living with HIV not initiate treatment? A systematic review of qualitative evidence from low- and middle-income countries. Soc Sci Med 2018 Sep;213:72-84 [FREE Full text] [CrossRef] [Medline]
- Tanser F, Bärnighausen T, Grapsa E, Zaidi J, Newell M. High coverage of ART associated with decline in risk of HIV acquisition in rural KwaZulu-Natal, South Africa. Science 2013 Feb 22;339(6122):966-971 [FREE Full text] [CrossRef] [Medline]
- Rodger A, Cambiano V, Bruun T, Vernazza P, Collins S, Degen O, PARTNER Study Group. Risk of HIV transmission through condomless sex in serodifferent gay couples with the HIV-positive partner taking suppressive antiretroviral therapy (PARTNER): final results of a multicentre, prospective, observational study. Lancet 2019 Jun 15;393(10189):2428-2438 [FREE Full text] [CrossRef] [Medline]
- Simoni JM, Kutner BA, Horvath KJ. Opportunities and challenges of digital technology for HIV treatment and prevention. Curr HIV/AIDS Rep 2015 Dec;12(4):437-440 [FREE Full text] [CrossRef] [Medline]
- Trujillo D, Turner C, Le V, Wilson E, Arayasirikul S. Digital HIV care navigation for young people living with HIV in San Francisco, California: feasibility and acceptability study. JMIR Mhealth Uhealth 2020 Jan 10;8(1):e16838 [FREE Full text] [CrossRef] [Medline]
- Silver L, Johnson C. Internet Connectivity Seen as Having Positive Impact on Life in Sub-Saharan Africa. Pew Research Center. 2018. URL: https://www.pewresearch.org/global/2018/10/09/internet-connectivity-seen-as-having-positive-impact-on-life-in-sub-saharan-africa/ [accessed 2020-11-05]
- Pellowski JA, Kalichman SC. Recent advances (2011-2012) in technology-delivered interventions for people living with HIV. Curr HIV/AIDS Rep 2012 Dec;9(4):326-334 [FREE Full text] [CrossRef] [Medline]
- Daher J, Vijh R, Linthwaite B, Dave S, Kim J, Dheda K, et al. Do digital innovations for HIV and sexually transmitted infections work? Results from a systematic review (1996-2017). BMJ Open 2017 Nov 3;7(11):e017604 [FREE Full text] [CrossRef] [Medline]
- Boulos M, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online 2011 Apr 5;10:24 [FREE Full text] [CrossRef] [Medline]
- Medhi I, Gautama S, Toyama K. A Comparison of Mobile Money-transfer UIs for Non-literate and Semi-literate Users. In: Conference on Human Factors in Computing Systems. 2009 Presented at: CHI'09; April 4-9, 2009; Boston MA USA. [CrossRef]
- Mathenjwa T, Kim H, Zuma T, Shahmanesh M, Seeley J, Matthews P, et al. Home-based intervention to test and start (HITS) protocol: a cluster-randomized controlled trial to reduce HIV-related mortality in men and HIV incidence in women through increased coverage of HIV treatment. BMC Public Health 2019 Jul 19;19(1):969. [CrossRef] [Medline]
- Johnson VD. Promoting behavior change: making healthy choices in wellness and healing choices in illness - use of self-determination theory in nursing practice. Nurs Clin North Am 2007 Jun;42(2):229-41, vi. [CrossRef] [Medline]
- Deci EL, Ryan RM. The 'What' and 'Why' of goal pursuits: human needs and the self-determination of behavior. Psychol Inq 2000 Oct;11(4):227-268. [CrossRef]
- Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000 Jan;55(1):68-78. [CrossRef] [Medline]
- Quinlivan EB, Messer LC, Roytburd K, Blickman A. Unmet core needs for self-determination in HIV-infected women of color in medical care. AIDS Care 2017 May;29(5):603-611 [FREE Full text] [CrossRef] [Medline]
- Houston E, McKirnan DJ, Cervone D, Johnson MS, Sandfort TG. Assessing treatment motivation among patients receiving antiretroviral therapy: a multidimensional approach. Psychol Health 2012;27(6):674-687 [FREE Full text] [CrossRef] [Medline]
- Igreja I, Zuroff DC, Koestner R, Saltaris C, Brouillette M, Lalonde R. Applying self‐determination theory to the prediction of distress and well‐being in gay men with HIV and AIDS1. J Appl Social Pyschol 2000 Apr;30(4):686-706. [CrossRef]
- Moller AC, Merchant G, Conroy DE, West R, Hekler E, Kugler KC, et al. Applying and advancing behavior change theories and techniques in the context of a digital health revolution: proposals for more effectively realizing untapped potential. J Behav Med 2017 Feb;40(1):85-98. [CrossRef] [Medline]
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95. [CrossRef] [Medline]
- Herbst K, Law M, Geldsetzer P, Tanser F, Harling G, Bärnighausen T. Innovations in health and demographic surveillance systems to establish the causal impacts of HIV policies. Curr Opin HIV AIDS 2015 Nov;10(6):483-494 [FREE Full text] [CrossRef] [Medline]
- Vandormael A, de Oliveira T, Tanser F, Bärnighausen T, Herbeck JT. High percentage of undiagnosed HIV cases within a hyperendemic South African community: a population-based study. J Epidemiol Community Health 2018 Feb;72(2):168-172. [CrossRef] [Medline]
- Barnighausen T, Herbst A, Tanser F, Newell M, Bor J. Unequal Benefits From ART: A Growing Male Disadvantage in Life Expectancy in Rural South Africa. In: The Annual Conference on Retroviruses and Opportunistic Infections. 2014 Presented at: CROI'14; March 3-6, 2014; Boston, Massachusetts.
- Baisley K, Seeley J, Siedner M, Koole K, Matthews P, Tanser F, et al. Findings from home-based HIV testing and facilitated linkage after scale-up of test and treat in rural South Africa: young people still missing. HIV Med 2019 Nov;20(10):704-708. [CrossRef] [Medline]
- Cox V, Campbell A, Raphahlelo N, McIntyre J, Rebe K. Towards a male-centered model of care: understanding male preferences and barriers to HIV care in Khayelitsha, South Africa. 2013 Presented at: 7th IAS Conference on HIV Pathogenesis, Treatment and Prevention; 2013; Kuala Lampur.
- Groh K, Audet CM, Baptista A, Sidat M, Vergara A, Vermund SH, et al. Barriers to antiretroviral therapy adherence in rural Mozambique. BMC Public Health 2011 Aug 16;11:650 [FREE Full text] [CrossRef] [Medline]
- Heestermans T, Browne JL, Aitken SC, Vervoort SC, Klipstein-Grobusch K. Determinants of adherence to antiretroviral therapy among HIV-positive adults in sub-Saharan Africa: a systematic review. BMJ Glob Health 2016;1(4):e000125 [FREE Full text] [CrossRef] [Medline]
- Fitzgerald M, Collumbien M, Hosegood V. 'No one can ask me 'Why do you take that stuff?': men's experiences of antiretroviral treatment in South Africa. AIDS Care 2010 Mar;22(3):355-360. [CrossRef] [Medline]
- Cornell M, Cox V, Wilkinson L. Public health blindness towards men in HIV programmes in Africa. Trop Med Int Health 2015 Dec;20(12):1634-1635 [FREE Full text] [CrossRef] [Medline]
- Bhagwanjee A, Govender K, Akintola O, Petersen I, George G, Johnstone L, et al. Patterns of disclosure and antiretroviral treatment adherence in a South African mining workplace programme and implications for HIV prevention. Afr J AIDS Res 2011;10(Suppl 1):357-368. [CrossRef] [Medline]
- Mills EJ, Beyrer C, Birungi J, Dybul MR. Engaging men in prevention and care for HIV/AIDS in Africa. PLoS Med 2012 Feb;9(2):e1001167 [FREE Full text] [CrossRef] [Medline]
- Kumwenda MK. The influence of masculinity on HIVST community intervention: a qualitative evaluation of empirical evidence from Blantyre, Malawi. Malawi-Liverpool Welcome Trust, Behaviour and Health, Blantyre, Malawi. URL: https://hivstar.lshtm.ac.uk/files/2016/12/The-influence-of-masculinity-on-HIVST-community-intervention.pdf [accessed 2020-11-06]
- Marson KG, Tapia K, Kohler P, McGrath CJ, John-Stewart GC, Richardson BA, et al. Male, mobile, and moneyed: loss to follow-up vs transfer of care in an urban African antiretroviral treatment clinic. PLoS One 2013;8(10):e78900 [FREE Full text] [CrossRef] [Medline]
- Chikovore J, Hart G, Kumwenda M, Chipungu GA, Corbett L. 'For a mere cough, men must just chew Conjex, gain strength, and continue working': the provider construction and tuberculosis care-seeking implications in Blantyre, Malawi. Glob Health Action 2015;8:26292 [FREE Full text] [CrossRef] [Medline]
- Hosegood V, Richter L, Clarke L. '...I Should Maintain a Healthy Life Now and Not Just Live as I Please...': men's health and fatherhood in rural South Africa. Am J Mens Health 2016 Nov;10(6):NP39-NP50 [FREE Full text] [CrossRef] [Medline]
- Siu GE, Wight D, Seeley J. 'Dented' and 'resuscitated' masculinities: the impact of HIV diagnosis and/or enrolment on antiretroviral treatment on masculine identities in rural eastern Uganda. Sahara J 2014;11:211-221 [FREE Full text] [CrossRef] [Medline]
- Boullé C, Kouanfack C, Laborde-Balen G, Boyer S, Aghokeng AF, Carrieri MP, Stratall ANRS 12110ESTHER Study Group. Gender differences in adherence and response to antiretroviral treatment in the Stratall trial in rural district hospitals in Cameroon. J Acquir Immune Defic Syndr 2015 Jul 01;69(3):355-364. [CrossRef] [Medline]
- Ngwenya N, Gumede D, Shahmanesh M, McGrath N, Grant A, Seeley J. Community perceptions of the socio-economic structural context influencing HIV and TB risk, prevention and treatment in a high prevalence area in the era of antiretroviral therapy. Afr J AIDS Res 2018 Mar;17(1):72-81 [FREE Full text] [CrossRef] [Medline]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
- User Research Doing Pop-up Research. Government of UK. 2017. URL: https://www.gov.uk/service-manual/user-research/doing-pop-up-research [accessed 2020-11-05]
- Brooke J. SUS-a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation In Industry. Boca Raton, Florida: CRC Press; 1996.
- Cordova D, Alers-Rojas F, Lua FM, Bauermeister J, Nurenberg R, Ovadje L, et al. The usability and acceptability of an adolescent mHealth HIV/STI and drug abuse preventive intervention in primary care. Behav Med 2018;44(1):36-47 [FREE Full text] [CrossRef] [Medline]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
- Entwistle VA, France EF, Wyke S, Jepson R, Hunt K, Ziebland S, et al. How information about other people's personal experiences can help with healthcare decision-making: a qualitative study. Patient Educ Couns 2011 Dec;85(3):e291-e298. [CrossRef] [Medline]
- Lustria ML, Cortese J, Gerend MA, Schmitt K, Kung YM, McLaughlin C. A model of tailoring effects: a randomized controlled trial examining the mechanisms of tailoring in a web-based STD screening intervention. Health Psychol 2016 Nov;35(11):1214-1224. [CrossRef] [Medline]
- Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res 2017 Jun 29;19(6):e232 [FREE Full text] [CrossRef] [Medline]
|AHRI: Africa Health Research Institute|
|ART: antiretroviral therapy|
|CAB: community advisory board|
|CSQ: Client Satisfaction Questionnaire|
|EPIC-HIV: Empowering people through informed choices for HIV|
|FGD: focus group discussion|
|HCI: human-computer interaction|
|HITS: home-based intervention to test and start|
|PE: public engagement|
|SDT: self-determination theory|
Edited by G Eysenbach; submitted 22.12.19; peer-reviewed by A Farnham, F Holl, A Rung, C Lancaster; comments to author 07.07.20; revised version received 17.09.20; accepted 22.09.20; published 24.11.20Copyright
©Thulile Mathenjwa, Oluwafemi Adeagbo, Thembelihle Zuma, Keabetswe Dikgale, Anya Zeitlin, Philippa Matthews, Janet Seeley, Sally Wyke, Frank Tanser, Maryam Shahmanesh, Ann Blandford. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 24.11.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.