Background: HIV disproportionately affects men who have sex with men (MSM). In Malaysia, where stigma and discrimination toward MSM are high, including in health care settings, mobile health (mHealth) platforms have the potential to open new frontiers in HIV prevention.
Objective: We developed an innovative, clinic-integrated smartphone app called JomPrEP, which provides a virtual platform for Malaysian MSM to engage in HIV prevention services. In collaboration with the local clinics in Malaysia, JomPrEP offers a range of HIV prevention (ie, HIV testing and pre-exposure prophylaxis [PrEP]) and other support services (eg, referral to mental health support) without having to interface face to face with clinicians. This study evaluated the usability and acceptability of JomPrEP to deliver HIV prevention services for MSM in Malaysia.
Methods: In total, 50 PrEP-naive MSM without HIV in Greater Kuala Lumpur, Malaysia, were recruited between March and April 2022. Participants used JomPrEP for a month and completed a postuse survey. The usability of the app and its features were assessed using self-report and objective measures (eg, app analytics, clinic dashboard). Acceptability was evaluated using the System Usability Scale (SUS).
Results: The participants’ mean age was 27.9 (SD 5.3) years. Participants used JomPrEP for an average of 8 (SD 5.0) times during 30 days of testing, with each session lasting an average of 28 (SD 38.9) minutes. Of the 50 participants, 42 (84%) ordered an HIV self-testing (HIVST) kit using the app, of whom 18 (42%) ordered an HIVST more than once. Almost all participants (46/50, 92%) initiated PrEP using the app (same-day PrEP initiation: 30/46, 65%); of these, 16/46 (35%) participants chose PrEP e-consultation via the app (vs in-person consultation). Regarding PrEP dispensing, 18/46 (39%) participants chose to receive their PrEP via mail delivery (vs pharmacy pickup). The app was rated as having high acceptability with a mean score of 73.8 (SD 10.1) on the SUS.
Conclusions: JomPrEP was found to be a highly feasible and acceptable tool for MSM in Malaysia to access HIV prevention services quickly and conveniently. A broader, randomized controlled trial is warranted to evaluate its efficacy on HIV prevention outcomes among MSM in Malaysia.
Trial Registration: ClinicalTrials.gov NCT05052411; https://clinicaltrials.gov/ct2/show/NCT05052411
International Registered Report Identifier (IRRID): RR2-10.2196/43318
Men who have sex with men (MSM) are disproportionately affected by HIV in Malaysia and accounted for 63% of new HIV diagnoses in 2021, a proportion that has been increasing over the past decade [, ]. This pattern requires implementation of more effective HIV prevention in MSM, yet in Malaysia, as in many low- and middle-income countries (LMICs), MSM often do not adequately access evidence-based HIV prevention (and treatment). Gaps in prevention and treatment are due, in part, to high levels of social stigma and discrimination against MSM. In Malaysia, these factors are heightened further because same-sex sexual behaviors are criminalized [ - ]. Other factors also contribute to low uptake of services, including sexual networks that have evolved through social networking apps. Transmission potential is heightened by behavioral or biological factors, including condomless sex, multiple concurrent sexual partners, substance use, and mental health problems (eg, depression, anxiety) that act synergistically to increase the HIV risk in this group [ - ].
Routine HIV testing and expanded use of pre-exposure prophylaxis (PrEP) would drastically reduce the population-level burden of HIV [- ]. Uptake of these evidence-based tools, however, is suboptimal among Malaysian MSM. For example, recent data suggest that 55% of MSM reported not having tested for HIV in the past 6 months, and approximately 30% reported they had never been tested [ - ]. Additionally, only 18.3% of MSM with indications for PrEP reported ever using it, despite high awareness and willingness to use PrEP [ , ]. This low uptake is partly explained by the need to maintain meaningful engagement with the health care system to access these services. Yet, it is often difficult for MSM in Malaysia to find culturally appropriate health care services due to known barriers, such as discomfort and distrust associated with disclosing sexual behavior to providers for fear of ramifications [ ]. As such, there is a need for innovative strategies to improve access to HIV prevention services for Malaysian MSM.
Mobile health (mHealth), particularly smartphone apps, holds great promise for HIV prevention [, - ], especially when linked to accessible HIV testing and PrEP. App-based interventions can help overcome multilevel barriers, given their ability to anonymously reach and engage populations that are disenfranchised from existing prevention efforts and offer “real-time” delivery and rapid scalability of programs at relatively low implementation costs. In Malaysia, smartphone use among MSM is nearly universal, and MSM report a strong preference for app-based HIV prevention programs [ , ]. Although app-based interventions are evolving and promote the HIV prevention continuum, most, if not all, are limited to high-income countries and none provide comprehensive HIV prevention services [ ]. Furthermore, many of these emerging apps deploy an online-to-offline (O2O) strategy [ , ], where clients eventually must be seen in person. In these cases, the anonymity afforded through the online experience ends during the clinical encounter where individuals must be linked to their testing results and medication prescriptions. An important innovation in the traditional O2O and clinic-based models would be to keep the entire care process in the virtual space (online). However, such strategies have yet to be developed and assessed.
To address this unmet need, we developed JomPrEP (where “Jom” means “let us” in Bahasa Malaysia), a clinic-integrated smartphone app, designed to provide HIV prevention services for MSM in Malaysia. The development of JomPrEP has been described previously . In brief, it is adapted from HealthMindr, an app previously demonstrated to increase HIV testing and PrEP uptake among MSM in the United States [ ]. In collaboration with local Malaysian clinics, JomPrEP offers a virtual platform for Malaysian MSM to access a range of HIV prevention (ie, HIV testing and PrEP) and other support services (eg, referral to mental health support) without having to interface face to face with clinicians. It includes several on-demand features, including scheduling and managing appointments in person or through e-consultation, communicating with the clinical team (ie, chat), home-based testing, accessing test results, ordering health products, discrete door-to-door delivery, timely notifications, a points-based reward system for completing activities within the app, and a multimedia resource center. Here, we report findings from beta testing of the recently developed JomPrEP app to evaluate its usability and acceptability.
Study Design and Settings
We conducted beta testing of JomPrEP to assess its usability and acceptability among MSM living in the Greater Kuala Lumpur region, Malaysia. Beta testing of an app helps to identify any final areas for improvement . We hypothesized that beta testing for 30 days of observation (N=50) would allow us to evaluate the app's design, functionality, and usability. The sample size of 50 was determined based on the pragmatics of recruitment and the need to examine feasibility [ - ].
We partnered with the Centre of Excellence for Research in AIDS (CERiA) at the University of Malaya, Kuala Lumpur, Malaysia, to conduct this study. Working closely with several other local and international institutions, CERiA conducts innovative and interdisciplinary research that combines epidemiological, biomedical, and sociobehavioral approaches, focusing on the implementation of HIV prevention and treatment. As part of JomPrEP integration with existing clinics, we partnered with 2 local clinics—the Red Clinic (private clinic) and the Community Health Care Clinic (nongovernmental organization [NGO]-based clinic)—to provide clinical services (eg, HIV testing, sexually transmitted infection [STI] testing, PrEP services) virtually via the app.
Study Participants and Recruitment
Eligibility criteria included (1) being 18 years or older; (2) identifying as a cis-gender man; (3) self-reporting an HIV-negative or HIV status unknown at screening; (4) not having used PrEP previously (ie, PrEP naive); (5) self-reporting evidence of being at risk for HIV acquisition, as defined by the World Health Organization PrEP clinical guidelines ; (6) owning a smartphone; and (7) currently residing in the Greater Kuala Lumpur region.
In total, 50 participants were recruited between March and April 2022 using in-person and online recruitment strategies. For in-person recruitment, flyers were distributed to potential participants as well as posted at local partner organizations (eg, clinics, lesbian, gay, bisexual, transgender [LGBT]–friendly community-based organizations). Additionally, we used various general and MSM-specific social media platforms as venues for participant recruitment. These included placing advertisements in geosocial networking (GSN) apps popular among MSM in Malaysia (ie, Hornet) as well as posting study flyers on Malaysian MSM–focused Facebook pages. Interested individuals who clicked on an advertisement were directed to the study website , where they were presented with a brief description of the study and web-based screening.
After meeting enrollment criteria, eligible participants were asked to provide electronic informed consent for study participation, followed by undergoing a baseline assessment. Study staff then assisted enrolled participants with downloading JomPrEP and provided them with brief instructions on the purpose of the app and an overview of how to use it. To restrict access to JomPrEP to the study participants, participants were provided with a single-use registration code needed to gain access to the app. Upon downloading the app, participants were asked to complete an onboarding process, which included creating log-in credentials. They were then redirected to the JomPrEP landing screen (home screen), which contains several icons representing key app functions (). Screenshots of the app are available in . Participants were requested to keep and use the app for 30 days and encouraged to use all app features. On day 30, participants completed a posttest survey and were asked to provide a synthesis of issues regarding the app (ie, exit interviews). For exit interviews, 20 (40%) participants were randomly sampled and interviews were conducted until data saturation was reached. The 1-on-1 sessions were conducted online via licensed videoconferencing software. Participants were given the choice of turning on or off their cameras and were asked to use a pseudonym/nickname.
Participants received point-based rewards (known as JomPrEP points [JPP]) for completing specific activities or meeting milestones via the app (eg, 100 JPP for baseline and follow-up assessment each, 50 for completing app onboarding, 20 for an HIVST in-app order, 50 for completing a lab test, 50 for completing an e-consultation for PrEP, 30 for an in-app PrEP order, 50 for optimal PrEP adherence; maximum points that could be earned: 740 or US $18.50). Participants were allowed to redeem points for cash at any point during the study period (10 JPP=RM 1, or US $0.24).
|Customizable Home page|
|Messages (ie, chat)|
|JomPrEP Clinic Dashboard|
aHIVST: HIV self-testing.
bPrEP: pre-exposure prophylaxis.
cReceive PrEP at the first doctor visit (no need to wait for lab results).
dGet PrEP after lab results are complete (need a follow-up doctor visit to review lab results).
eApplies only to those who choose traditional PrEP.
fIncludes a referral letter for mental health.
The Institutional Review Board at the University of Connecticut approved this study (H22-0049), with an institutional reliance agreement with the University of Malaya. Eligible participants provided electronic informed consent for study participation.
All assessments (ie, baseline and follow-up) were conducted virtually and self-administered using Qualtrics. We collected participant demographic and baseline characteristics, including age, ethnicity, educational status, relationship status, income, housing status, depressive symptoms , substance use and sexual history, HIV/STI-testing practices, and past use of PrEP and postexposure prophylaxis (PEP).
JomPrEP App Evaluation
After 30 days of use, participants were asked to assess the app’s features, usability, design, content, and functionality using 2 Likert scales. JomPrEP acceptability was assessed using the System Usability Scale (SUS) , a validated measure that assesses the subjective usability of an app. Scores range from 0 to 100, with scores of ≥50 indicating that the app is acceptable [ ]. We also collected app analytics, such as the number of log-ins, session duration, pages visited, and frequency and duration of use of app components, to determine usability. Additionally, data on the uptake of HIV testing and PrEP, mental health screening and referral to mental health support services, and use of the HIVST kit for those who placed in-app orders were extracted from the web-based JomPrEP Clinic Dashboard.
Finally, we conducted exit interviews with 20 (40%) participants to obtain feedback on app functionality, technical performance, errors and software bugs encountered, overall experience using the app, feedback for further refinement, and subjective impact of the app on HIV testing and PrEP uptake. One-on-one interviews were conducted by research staff virtually using videoconferencing technology. Interviews were recorded and transcribed for analysis.
All quantitative data were managed and analyzed using IBM SPSS Statistics version 28. Means for continuous variables and frequencies for categorical variables were calculated to describe the participants at baseline. App usability and acceptability were based on descriptive statistics from the app analytics and acceptability measure. For example, evaluation responses are reported as the percentage of users who completed the posttest survey. SUS results are reported as an aggregate score, with a score of ≥50 indicating that the app is acceptable [, ]; the percentage of participants with scores ≥50 is also reported. Descriptive statistics of app analytics were used to examine app engagement and are reported as the mean with the range for time and action measurements. For qualitative data, all the exit interviews were audio-recorded, transcribed, and analyzed. The comments and issues were grouped and categorized according to common themes relative to specific app functions by 3 coders (including 2 senior coders) and agreed upon by all authors. Dedoose version 9.0.54 was used throughout to assist in data management and analysis.
The mean age of the 50 participants was 27.9 years (range 21-45 years), with most being single (36/50, 72%), Malay (26/50, 52%), university graduates (34/50, 68%) and living in a house/apartment with other people (36/50, 72%). Almost all participants reported having been tested for HIV (49/50, 98%), and 39/50 (78%) participants had done so in the past 6 months. Of all 50 participants, 26 (52%) reported using HIVST and only 5 (10%) had used PrEP previously. Regarding sexual behaviors in the past 6 months, 47/50 (94%) participants reported anal sex with another man, while only 16/50 (32%) participants reported consistent condom use, 4/50 (8%) reported having engaged in sexualized drug use, and 9/50 (18%) reported having engaged in group sex ().
|Age (years), mean (SD)||27.9 (5.3)|
|Ethnicity (Malaya), n (%)|
|University graduatea, n (%)|
|Relationship status, n (%)|
|Monthly income (RM/US $), mean (SD)||3553.40 (2985.90)/837.97 (704.14)|
|Living status, n (%)|
|Living with others||36 (72)|
|Tested for HIV (past 6 months), n (%)|
|Ever used an HIVSTb kit, n (%)|
|Previously diagnosed with STIc, n (%)|
|Ever used PrEPd, n (%)|
|Ever used PEPe, n (%)|
|Perceived HIV risk, n (%)|
|Ever injected drugs, n (%)|
|Engaged in anal sex (past 6 months), n (%)|
|HIV serodiscordant relationship (past 6 months), n (%)|
|Consistent condom use (past 6 months), n (%)|
|Engaged in group sex (past 6 months), n (%)|
|Engaged in sexualized drug usef (past 6 months), n (%)|
aIncludes college, university, and professional degrees.
aHIVST: HIV self-testing.
cSTI: sexually transmitted infections (eg, gonorrhea, chlamydia, syphilis).
dPrEP: preexposure prophylaxis.
ePEP: postexposure prophylaxis.
fUse of psychoactive substances (eg, amphetamines, 3,4-methylene dioxymethamphetamine [MDMA]) before or during sexual activity.
Uptake of HIV Prevention Services
During the 30-day beta-testing phase of JomPrEP, 42/50 (84%) participants ordered an HIVST kit using the app. Almost all (46/50, 92%) participants used the app to get on PrEP. Specifically, 30/46 (65%) participants chose same-day PrEP versus traditional PrEP, and the majority of them picked up the PrEP medication at the pharmacy (28/46, 61%). Additionally, 44/50 (88%) participants used the online assessment tool to screen for depression, and 39/44 (89%) of them met the criteria for moderate-to-severe depressive symptoms  and were provided with a referral letter ( ).
|Ordered HIVSTb kit||42 (84)|
|Verified HIVST resultsc||40 (95)|
|Linked to PrEP services (n=46, 92%)|
|Traditionald PrEP delivery||16 (35)|
|Same-daye PrEP delivery||30 (65)|
|Completed phlebotomy||46 (100)|
|Completed e-consultationf||16 (35)|
|Completed in-person consultation||30 (65)|
|Picked-up PrEP medication at pharmacy||28 (61)|
|PrEP medication delivered at home||18 (39)|
|Mental health screening||44 (88)|
aPrEP: preexposure prophylaxis.
bHIVST: HIV self-testing.
cHIVST result verified by providing an image of the result via the app.
dReceive PrEP after lab results are complete (need a follow-up doctor visit to review lab results).
eReceive PrEP at the first doctor visit (no need to wait for lab results).
fApplies only to those who choose same-day PrEP.
JomPrEP App Evaluation
During the beta-testing phase, 29/50 (58%) participants were Android users, while the remainder (21/50, 42%) were iOS users. Usability measures by participants included app use, with an average of 8 (SD 5.0, range 2-18) unique visits over 30 days, with an average duration of 28 (SD 38.9) minutes per session. The app had a mean of 34.9 (SD 14.7) daily users, with 939.3 (SD 597.9) daily page views, 63.4 (SD 28.5) daily sessions on average, and consistent returning visits (eg, >10: 29/50, 58%; 6-10: 22/50, 44%).
The mean acceptability score was 73.8 (SD 10.1) on the SUS, well above the minimum criteria (≥50) set for the acceptability of the app [, ], with all participants reporting acceptability scores of >50. Almost all participants reported that they were satisfied with JomPrEP (46/50, 92%) and that the app was useful in addressing their HIV prevention needs (49/50, 98%); see .
When participants were asked about future app use, most said they were likely to continue using the app as part of their HIV prevention plan (42/50, 84%), would download the app if publicly available (43/50, 86%), and would recommend the app to their friends or colleagues (50/50, 100%). Most participants felt confident in in-app security (43/50, 86%), including autologout after 5 minutes of inactivity (44/50, 88%), an email and password log-in (42/50, 84%), a 4-digit personal identification number (36/50, 72%), and the app name and icon not associated with HIV (32/50, 64%); see.
Participants found JomPrEP to be easy to use and felt confident that they would be able to learn how to use it quickly and without technical assistance ().
|Activity (“How satisfied are you with the following features of the JomPrEP app?”)||Participants, n (%)a|
|Ordering an HIVSTb kit||47 (94)|
|Ordering PrEPc medication||46 (92)|
|Reward system (earning and redeeming points)||43 (86)|
|Completing the mental health screener||42 (84)|
|Chat with clinical or research staff||42 (84)|
|Booking appointments (blood draw, consultation with doctor)||42 (84)|
|Keeping track of upcoming and past appointments||42 (84)|
|Tracking order status (ie, HIVST, PrEP)||40 (80)|
|MoodTracker (track mood daily)||40 (80)|
|Online consultation with the doctor (e-consultation)||40 (80)|
|Reviewing laboratory test results||38 (76)|
|Resources/News Center||36 (72)|
|Notifications from the app||36 (72)|
|MedManager (receive medication reminders, track medication use)||30 (60)|
aVery satisfied and extremely satisfied (not included: not at all satisfied, slightly satisfied, moderately satisfied).
bHIVST: HIV self-testing.
cPrEP: preexposure prophylaxis.
|Activity (“How much do you agree that the use of the JomPrEP app…”)||Participants, n (%)a|
|Assisted in getting tested for HIV||50 (100)|
|Assisted in getting started on PrEPb||50 (100)|
|Made access to medical records easier (ie, test results, appointments)||49 (98)|
|Made access to HIV testing much easier||49 (98)|
|Helped to understand the risk of getting HIV||46 (92)|
|Motivated to get on PrEP||46 (92)|
|Motivated to get tested for STIsc||46 (92)|
|Helped to understand whether PrEP would be a good fit||46 (92)|
|Helped to get in touch with the clinic staff (via chat messages)||44 (88)|
|Helped to get the latest information about HIV||43 (86)|
|Helped to understand mental health needs||42 (84)|
|Made access to PrEP much easier||40 (80)|
aAgree and strongly agree (not included: strongly disagree, disagree, neither agree nor disagree).
bPrEP: preexposure prophylaxis.
cSTI: sexually transmitted infection.
|Activity (“How easy or hard was it to do the following tasks on the JomPrEP app?”)||Participants, n (%)a|
|Ordering an HIVSTb kit||49 (98)|
|Ordering PrEPc medication||47 (94)|
|Booking appointments (blood draw, consultation with doctor)||46 (92)|
|Reward system (earning and redeeming points)||44 (88)|
|Completing a mental health screener||43 (86)|
|Customizing profile page (eg, avatar, password, address, name)||43 (86)|
|Tracking the order status (ie, HIVST, PrEP)||42 (84)|
|Creating an account (onboarding process)||40 (80)|
|Reviewing laboratory test results||40 (80)|
|MoodTracker (track mood daily)||40 (80)|
|Chat with clinical or research staff||38 (76)|
|Find relevant information about HIV prevention||35 (70)|
|Online consultation with a doctor (e-consultation)||33 (66)|
|MedManager (receive medication reminders, track medication use)||25 (50)|
aVery easy (not included: very difficult, difficult, neutral).
bHIVST: HIV self-testing.
cPrEP: preexposure prophylaxis.
In follow-up exit interviews (n=20, 40%), participants indicated a high level of acceptability for the content, interface, and features of JomPrEP. Participants found the app to be user friendly, easy to navigate, and with a good layout. Participants also appreciated the ability to earn reward points for using specific app features, facilitating user engagement and retention.
…it's straightforward, it's user friendly, it's easy to use…I think the critical part for me is actually the ease of use of the app.
Because of convenience. It’s like having a mini-doctor. It’s much easier for you to get tested, instead of going to a clinic and stuff.
The point and rewards system are very interesting and attractive.
Participants noted that ordering HIVST kits via the app was straightforward and that the multimedia instructions helped them use HIVST kits and interpret test results.
I tried self-test kits from other sources, comparing the experience using this and also ordering elsewhere, I think JomPrEP was very prompt, the delivery was, I think, the next day and self-test kit was easy to use.
The feature I used most in the app is the one that allowed me to order a self-testing kit. It’s super convenient because first, order placement is very easy to do and the second one because the delivery is very fast…And they have very clear instructions on how to do it and get the result. Instructions to do the screening at home are very clear. And to upload the result is easy as well.
Furthermore, participants endorsed that the app helped them initiate PrEP use and maintain optimal adherence by facilitating a safe and stigma-free virtual platform to access PrEP services. Participants commented on the relevant information presented in the app, and many noted that they “didn’t know anything about [PrEP] until [they] used the app.”
There’s a lot of information that makes me want to take PrEP more, because of the useful information and why I need to take the PrEP. This app is very like a one-stop center to take the PrEP…inside, you can book a consultation, view your result and you can directly order the PrEP. So, this helped me more easily to get the PrEP.
I have been thinking of getting PrEP but didn’t know much about it (process, the cost, etc). The app makes everything more transparent. And when you take it, there’s a reminder every day so you can set up the clock, so you don’t forget.
When I first came here, I am not originally from here, I was trying to find PrEP, and I had a hard time finding it. When I used this app, it made things much easier. I don’t need to worry if the clinic is judgmental.
Participants also provided suggestions to improve the app and specific feedback on additional resources and features that they found interesting and helpful. For example, participants suggested that the app include an option to make an appointment with a mental health counselors and support groups and the ability to connect with other JomPrEP users through private messaging or discussion forums. Participants shared occasional issues with lagging app response time, difficulty setting up reminder notifications, and missing notifications. A few participants indicated that the test result feature of the app was a little challenging to use and required multiple clicks to view the results. A few participants also noted that some of the information in the app is repetitive or is not updated frequently. One participant recommended that the app allow the users to make the app more discreet (eg, the ability to change app icons).
Participants indicated that they would continue to use the JomPrEP app after the final version is released to the public.
Yes, definitely. I would use it because it's easier to put my appointment and view my lab results. I don't have to have it in a hardcopy form, easily accessible to my smartphone, and I could easily order my HIV self-testing kit as well.
I will continue to use it. And I think the JomPrEP app is very useful for me in terms of ordering the PrEP and booking e-consultations.
Using innovative tools, such as mHealth, in public health programming and the health care system can help bridge gaps in the adoption of needed health and prevention services, particularly among underserved populations [- ]. In this study, we sought to investigate the usability and acceptability of JomPrEP, a clinic-integrated smartphone app, as an additional platform to promote routine HIV testing and PrEP uptake among MSM in Malaysia. Our findings demonstrated that Malaysian MSM will use a smartphone app to virtually access HIV prevention services and that such an app is acceptable to this at-risk group, as indicated by the participants’ empiric use of the app.
Comparison With Prior Work
Prior studies have demonstrated several apps for HIV prevention and treatment efforts to be promising and cost-effective strategies to reach and engage stigmatized and hidden populations, such as MSM [, - ]. In Malaysia, the use of mobile technology over the past decades has grown markedly, particularly among MSM, with a mobile phone penetration rate of 97.5% and an internet penetration rate of 71.1% [ , , ]. Importantly, our beta testing of JomPrEP revealed that an overwhelming majority of men used the app to receive HIV prevention services: ordering an HIVST kit (84%) and getting on PrEP (94%). Participants reported that they were satisfied and comfortable using JomPrEP and would recommend it to friends or colleagues. These findings indicate the potential utility of JomPrEP for Malaysian MSM to promote HIV prevention services.
One of the key innovations on JomPrEP includes incorporating on-demand features, such as home-based HIVST, e-consultations, and discrete door-to-door delivery, to provide a scalable model for remote HIV prevention services delivery in the LMIC setting. Although the users would still be required to visit a laboratory for clinical testing, this would not require them to be face-to-face with their clinicians. Moreover, the platform allows users to self-assess their HIV risk, consult online with clinicians from the participant clinics, and have their medication delivered to their preferred location, thus minimizing the need for in-person interactions with the clinician. This represents a significant and much-needed innovation over traditional clinic-based and O2O models of HIV service delivery to keep at-risk individuals wedded into the virtual clinical ecosystem and boost the uptake of clinical services [, ]. This is particularly important in LMIC settings, such as Malaysia, as the virtual platform allows users to bypass barriers to care for marginalized populations and feel safer and less vulnerable to potential legal or social harm (eg, by reducing face-to-face interactions with providers).
Prior research has documented low user retention and a lack of sustained use after adoption as key challenges to the effectiveness of existing app-based interventions . Obtaining high engagement and retention is necessary to maintain the integrity and long-term sustainability of effective mHealth interventions [ ]. Strategies to integrate other features that do not include individual input, such as passive data collection using inputs from their smartphones or unobtrusive wearable devices, may strengthen the features of the app.
Results from our beta testing, however, revealed that MSM were actively engaged in the app and that retention was excellent through the beta testing. Although it is possible that the perfect retention rate could be because of the shorter follow-up time, it is likely that the incorporation of additional components, such as the ability to customize profiles, personalized messages, and gaming elements (ie, the ability to “level up,” earn and redeem points), may have allowed for enhanced user engagement. In recent years, the utility of gamification features (eg, challenges, tasks, rewards, badges, leaderboards) in nontraditional gaming contexts has increased significantly, thus providing opportunities for greater user engagement in mHealth interventions [, ]. As confirmed in the exit interviews, the overall high engagement and usage of key features of JomPrEP suggest that an app-based intervention, such as JomPrEP, has a high degree of feasibility to ensure equitable access to HIV testing and PrEP services for MSM in Malaysia.
Although there was consensus on the usability and acceptability of JomPrEP, with no significant differences between different subgroups of Malaysian MSM, the app would benefit from continued refinement to address some of the shortcomings identified by men. For example, most participants who completed screening for depressive symptoms (88%) received referral letters to seek care offline (ie, outside the app). Given the focus of JomPrEP to offer holistic HIV prevention services within the online ecosystem, it would be important for the app to incorporate online consultation with mental health counselors and linkage to support groups via the app. Additionally, as part of the continued effort to ensure the safety and security of users, it would be important that JomPrEP incorporate added security measures, including 2-factor authentication and a discreet app icon (DAI). The availability of a DAI allows users to replace the default JomPrEP app log on their phone with another symbol (of their choice). This will help protect users when there is a possibility that someone may accidentally look at users’ phones and recognize that they have an app that might link them to the HIV or lesbian, gay, bisexual, transgender, queer, and others (LGBTQ+) community. A study conducted with MSM of Malaysia also highlighted the importance of privacy and confidentiality features in the mobile apps targeted for HIV prevention and treatment to minimize harm and safeguard users' privacy and confidentiality . Furthermore, it is important that the app be available for users outside of Kuala Lumpur, the capital city, and be linked to both private and government clinics/hospitals. This will ensure widespread implementation of JomPrEP to scale-up HIV prevention services for MSM across Malaysia.
The results of this study should be viewed in the context of the limitations. First, this pilot study included a small sample size and short-term follow-up (ie, 30-days) and used a single-arm design that is commonly used in beta testing; therefore, it was not powered or designed to evaluate efficacy. Second, our participants were subject to selection bias across several dimensions. For example, we recruited men using Facebook or a dating app (ie, Hornet) who may have been more comfortable using mobile apps than other men (ie, hidden MSM). Moreover, participants were already engaged, at least in part, due to their prior high levels of HIV testing. Participants were enrolled in the Greater Kuala Lumpur area only, potentially limiting the generalizability of the findings. Third, social desirability bias may have led participants to speak more positively about their app experience during the survey and exit interviews. This was in part observed by participants who responded favorably to app features, but our usability testing had not confirmed they used the specific feature. Finally, HIVST kits and PrEP services were free to the participants, which may have led to an overestimation of the actual uptake of HIV testing and PrEP services.
Further research is warranted to examine the implementation of JomPrEP in a more real-world setting. Regardless of these limitations, we believe that our findings carry important implications for efforts to improve the uptake of HIV testing and PrEP services among Malaysian MSM using an app-based intervention.
Overall, the JomPrEP app represents a feasible and acceptable tool for Malaysian MSM to access HIV prevention services. Importantly, it incorporates several on-demand features to support the remote delivery of HIV prevention services, thus representing a significant innovation on traditional clinic-based and O2O service delivery models [, ]. The reported outcomes are promising and indicate the benefits of systematically implementing this platform to foster HIV prevention efforts in LMICs, such as Malaysia, where MSM are disenfranchised from existing prevention efforts [ - ]. A large-scale randomized controlled trial is warranted to establish the efficacy of JomPrEP among this at-risk group.
This work was supported by a career development and research award from the National Institute on Drug Abuse (K01 DA051346) and a research award from the Fogarty International Center (R21TW011665) for Roman Shrestha.
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.
Conflicts of Interest
JomPrEP app screenshots.PDF File (Adobe PDF File), 776 KB
- Ministry of Health Malaysia. Global AIDS Monitoring 2020: Malaysia HIV/AIDS Progress Report. URL: https://www.moh.gov.my/moh/resources/Penerbitan/Laporan/Umum/Laporan_Global_AIDS_Monitoring_2020_new.pdf [accessed 2023-01-29]
- Ministry of Health Malaysia. Global AIDS Monitoring: Country Progress Report - Malaysia. URL: https://www.moh.gov.my/moh/resources/Penerbitan/Laporan/Umum/20211130_MYS_country_report_2021.pdf [accessed 2023-01-29]
- Earnshaw VA, Jin H, Wickersham JA, Kamarulzaman A, John J, Lim SH, et al. Stigma toward men who have sex with men among future healthcare providers in Malaysia: would more interpersonal contact reduce prejudice? AIDS Behav 2016 Jan 1;20(1):98-106 [FREE Full text] [CrossRef] [Medline]
- Ni Z, Shrestha R, Earnshaw VA, Tee YC, Altice FL, Azwa I, et al. Exploring Malaysian physicians' intention to discriminate against gay, bisexual, and other men who have sex with men patients. LGBT Health 2022 Oct 14:Online ahead of print. [CrossRef] [Medline]
- United Nations Office on Drugs and Crime. World Drug Report 2016. URL: https://www.unodc.org/wdr2016/ [accessed 2023-01-30]
- Chooi W, Mohd Zaharim N, Desrosiers A, Ahmad I, Yasin MAM, Syed Jaapar SZ, et al. Early initiation of amphetamine-type stimulants (ATS) use associated with lowered cognitive performance among individuals with co-occurring opioid and ATS use disorders in Malaysia. J Psychoactive Drugs 2017 Jun 29;49(4):326-332 [FREE Full text] [CrossRef] [Medline]
- Desrosiers A, Chooi W, Zaharim NM, Ahmad I, Mohd Yasin MA, Syed Jaapar SZ, et al. Emerging drug use trends in Kelantan, Malaysia. J Psychoactive Drugs 2016 May 25;48(3):218-226. [CrossRef] [Medline]
- Singh D, Chawarski MC, Schottenfeld R, Vicknasingam B. Substance abuse and the HIV situation in Malaysia. J Food Drug Anal 2013 Dec;21(4):S46-S51 [FREE Full text] [CrossRef] [Medline]
- Kanter J, Koh C, Razali K, Tai R, Izenberg J, Rajan L, et al. Risk behaviour and HIV prevalence among men who have sex with men in a multiethnic society: a venue-based study in Kuala Lumpur, Malaysia. Int J STD AIDS 2011 Jan 01;22(1):30-37. [CrossRef] [Medline]
- Lim SH, Akbar M, Wickersham JA, Kamarulzaman A, Altice FL. The management of methamphetamine use in sexual settings among men who have sex with men in Malaysia. Int J Drug Policy 2018 May;55:256-262 [FREE Full text] [CrossRef] [Medline]
- Nemoto T, Xie H, Iwamoto M, Piroth K, Hazratzai M, Teh YK. HIV risk behaviors and psychological well-being among men who have sex with men (MSM) in Kuala Lumpur, Malaysia. AIDS Educ Prev 2022 Oct;34(5):427-439. [CrossRef] [Medline]
- Ng RX, Guadamuz TE, Akbar M, Kamarulzaman A, Lim SH. Association of co-occurring psychosocial health conditions and HIV infection among MSM in Malaysia: implication of a syndemic effect. Int J STD AIDS 2020 May 16;31(6):568-578. [CrossRef] [Medline]
- Sullivan PS, Lyons MS, Czarnogorski M, Branson BM. Routine screening for HIV infection in medical care settings: a decade of progress and next opportunities. Public Health Rep 2016 Jan 01;131(1_suppl):1-4. [CrossRef]
- Patel D, Williams WO, Heitgerd J, Taylor-Aidoo N, DiNenno EA. Estimating gains in HIV testing by expanding HIV screening at routine checkups. Am J Public Health 2021 Aug;111(8):1530-1533. [CrossRef]
- Chen Y, Fang C, Shih M, Lin K, Chang S, Wu Z, et al. Routine HIV testing and outcomes: a population-based cohort study in Taiwan. Am J Prev Med 2022 Mar;62(2):234-242 [FREE Full text] [CrossRef] [Medline]
- Estcourt C, Yeung A, Nandwani R, Goldberg D, Cullen B, Steedman N, et al. Population-level effectiveness of a national HIV preexposure prophylaxis programme in MSM. AIDS 2021 Mar 15;35(4):665-673 [FREE Full text] [CrossRef] [Medline]
- Shrestha R, Alias H, Wong LP, Altice FL, Lim SH. Using individual stated-preferences to optimize HIV self-testing service delivery among men who have sex with men (MSM) in Malaysia: results from a conjoint-based analysis. BMC Public Health 2020 Nov 25;20(1):1777 [FREE Full text] [CrossRef] [Medline]
- Shrestha R, Maviglia F, Altice FL, DiDomizio E, Khati A, Mistler C, et al. Mobile health technology use and the acceptability of an mHealth platform for HIV prevention among men who have sex with men in Malaysia: cross-sectional respondent-driven sampling survey. J Med Internet Res 2022 Jul 25;24(7):e36917 [FREE Full text] [CrossRef] [Medline]
- Shrestha R, Lim SH, Altice FL, Copenhaver M, Wickersham JA, Saifi R, et al. Use of smartphone to seek sexual health information online among Malaysian men who have sex with men (MSM): implications for mHealth intervention to increase HIV testing and reduce HIV risks. J Community Health 2020 Mar 2;45(1):10-19 [FREE Full text] [CrossRef] [Medline]
- Lim SH, Mburu G, Bourne A, Pang J, Wickersham JA, Wei CKT, et al. Willingness to use pre-exposure prophylaxis for HIV prevention among men who have sex with men in Malaysia: findings from an online survey. PLoS One 2017 Sep 13;12(9):e0182838 [FREE Full text] [CrossRef] [Medline]
- Eger WH, Adaralegbe A, Khati A, Azwa I, Wickersham JA, Osborne S, et al. Exploring drivers of pre-exposure prophylaxis uptake among gay, bisexual, and other men who have sex with men in Malaysia. Int J STD AIDS 2022 Aug 30;33(9):821-828. [CrossRef] [Medline]
- Lim SH, Alias H, Kien JKW, Akbar M, Kamarulzaman A, Wong LP. A qualitative study of HIV "test-and-treat" experience among men who have sex with men in Malaysia. AIDS Educ Prev 2019 Jun;31(3):193-205. [CrossRef] [Medline]
- Sullivan PS, Hightow-Weidman L. Mobile apps for HIV prevention: how do they contribute to our epidemic response for adolescents and young adults? Mhealth 2021 Apr;7:36-36 [FREE Full text] [CrossRef] [Medline]
- Kawichai S, Songtaweesin WN, Wongharn P, Phanuphak N, Cressey TR, Moonwong J, et al. A mobile phone app to support adherence to daily HIV pre-exposure prophylaxis engagement among young men who have sex with men and transgender women aged 15 to 19 years in Thailand: pilot randomized controlled trial. JMIR Mhealth Uhealth 2022 Apr 21;10(4):e25561 [FREE Full text] [CrossRef] [Medline]
- Sullivan PS, Driggers R, Stekler JD, Siegler A, Goldenberg T, McDougal SJ, et al. Usability and acceptability of a mobile comprehensive HIV prevention app for men who have sex with men: a pilot study. JMIR Mhealth Uhealth 2017 Mar 09;5(3):e26 [FREE Full text] [CrossRef] [Medline]
- Goedel WC, Mitchell JW, Krebs P, Duncan DT. Willingness to use mobile phone apps for HIV prevention among men who have sex with men in London: web-based survey. JMIR Mhealth Uhealth 2017 Oct 11;5(10):e153 [FREE Full text] [CrossRef] [Medline]
- Nelson K, Perry NS, Horvath KJ, Smith LR. A systematic review of mHealth interventions for HIV prevention and treatment among gay, bisexual, and other men who have sex with men. Transl Behav Med 2020 Oct 12;10(5):1211-1220 [FREE Full text] [CrossRef] [Medline]
- Anand T, Nitpolprasert C, Phanuphak N. Online-to-offline models in HIV service delivery. Curr Opin HIV AIDS 2017;12(5):447-457. [CrossRef]
- Anand T, Nitpolprasert C, Trachunthong D, Kerr SJ, Janyam S, Linjongrat D, et al. A novel online-to-offline (O2O) model for pre-exposure prophylaxis and HIV testing scale up. J Int AIDS Soc 2017 Jan 01;20(1):21326. [CrossRef]
- Shrestha R, Wickersham JA, Khati A, Azwa I, Ni Z, Kamarulzaman A, et al. Clinic-integrated mobile health intervention ("JomPrEP" app) to improve uptake of HIV testing and pre-exposure prophylaxis among men who have sex with men in Malaysia: protocol for an intervention development and multiphase trial. JMIR Res Protoc 2022 Dec 21;11(12):e43318 [FREE Full text] [CrossRef] [Medline]
- Naeem F, Syed Y, Xiang S, Shokraneh F, Munshi T, Yang M, et al. Development, testing and reporting of mobile apps for psycho-social interventions: lessons from the pharmaceuticals. J Med Diagn Meth 2015;4(4):191. [CrossRef]
- Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res 2011 May;45(5):626-629 [FREE Full text] [CrossRef] [Medline]
- Wuest J, Merritt-Gray M, Dubé N, Hodgins MJ, Malcolm J, Majerovich JA, et al. The process, outcomes, and challenges of feasibility studies conducted in partnership with stakeholders: a health intervention for women survivors of intimate partner violence. Res Nurs Health 2015 Mar 12;38(1):82-96 [FREE Full text] [CrossRef] [Medline]
- Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. WHO Implementation Tool for Pre-exposure Prophylaxis (PrEP) of HIV Infection. URL: https://apps.who.int/iris/handle/10665/255890 [accessed 2023-01-30]
- JomPrEP. URL: https://jomprep.com/# [accessed 2023-01-30]
- Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
- Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum-Comput Interact 2008 Jul 30;24(6):574-594. [CrossRef]
- Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009;4(3):114-123 [FREE Full text]
- Brooke J. SUS-a quick and dirty usability scale. Usability Eval Ind 1996;189(194):4-7. [CrossRef]
- Sharma S, Kumari B, Ali A, Yadav R, Sharma A, Sharma K, et al. Mobile technology: a tool for healthcare and a boon in pandemic. J Family Med Prim Care 2022;11(1):37. [CrossRef]
- Agarwal S, Alhuwail D, John O. mHealth solutions in global health applications and implementation in low- and middle-income countries. In: Kickbusch I, Ganten D, Moeti M, editors. Handbook of Global Health. Berlin, Germany: Springer; 2021:1747-1774.
- Fortuin J, Salie F, Abdullahi LH, Douglas TS. The impact of mHealth interventions on health systems: a systematic review protocol. Syst Rev 2016 Nov 25;5(1):200 [FREE Full text] [CrossRef] [Medline]
- Chén OY, Roberts B. Personalized health care and public health in the digital age. Front Digit Health 2021 Mar 30;3:595704 [FREE Full text] [CrossRef] [Medline]
- Horvath KJ, Lammert S, MacLehose RF, Danh T, Baker JV, Carrico AW. A pilot study of a mobile app to support HIV antiretroviral therapy adherence among men who have sex with men who use stimulants. AIDS Behav 2019 Nov 15;23(11):3184-3198 [FREE Full text] [CrossRef] [Medline]
- Daniels J, Lane T, Struthers H, Maleke K, Moges W, McIntyre J, et al. Assessing the feasibility of smartphone apps for HIV-care research with MSM and transgender individuals in Mpumalanga, South Africa. J Int Assoc Provid AIDS Care 2017 Aug 09;16(5):433-439 [FREE Full text] [CrossRef] [Medline]
- Zhang A, Reynolds NR, Farley JE, Wang X, Tan S, Yan J. Preferences for an HIV prevention mobile phone app: a qualitative study among men who have sex with men in China. BMC Public Health 2019 Mar 12;19(1):297 [FREE Full text] [CrossRef] [Medline]
- Malaysian Communications and Multimedia Commission. Internet Users Survey: 2017. URL: https://www.mcmc.gov.my/skmmgovmy/media/general/pdf/mcmc-internet-users-survey-2017.pdf [accessed 2023-01-30]
- Amagai S, Pila S, Kaat AJ, Nowinski CJ, Gershon RC. Challenges in participant engagement and retention using mobile health apps: literature review. J Med Internet Res 2022 Apr 26;24(4):e35120 [FREE Full text] [CrossRef] [Medline]
- Bremer W, Sarker A. Recruitment and retention in mobile application-based intervention studies: a critical synopsis of challenges and opportunities. Inform Health Soc Care 2022 Jun 03:1-14. [CrossRef] [Medline]
- Mustafa AS, Ali N, Dhillon JS, Alkawsi G, Baashar Y. User engagement and abandonment of mHealth: a cross-sectional survey. Healthcare (Basel) 2022 Jan 24;10(2):221 [FREE Full text] [CrossRef] [Medline]
- Xu L, Shi H, Shen M, Ni Y, Zhang X, Pang Y, et al. The effects of mHealth-based gamification interventions on participation in physical activity: systematic review. JMIR Mhealth Uhealth 2022 Mar 03;10(2):e27794 [FREE Full text] [CrossRef] [Medline]
- Khati A, Wickersham JA, Rosen AO, Luces JRB, Copenhaver N, Jeri-Wahrhaftig A, et al. Ethical issues in the use of smartphone apps for HIV prevention in Malaysia: focus group study with men who have sex with men. JMIR Form Res 2022 Dec 23;6(12):e42939 [FREE Full text] [CrossRef] [Medline]
|CERiA: Centre of Excellence for Research in AIDS|
|DAI: discreet app icon|
|EHR: electronic health record|
|HIVST: HIV self-testing|
|JPP: JomPrEP points|
|LMIC: low- and middle-income country|
|mHealth: mobile health|
|MSM: men who have sex with men|
|PEP: postexposure prophylaxis|
|PrEP: pre-exposure prophylaxis|
|STI: sexually transmitted infection|
|SUS: System Usability Scale|
Edited by L Buis; submitted 20.11.22; peer-reviewed by JH Huang, M Torres; comments to author 09.12.22; revised version received 15.12.22; accepted 25.01.23; published 16.02.23Copyright
©Roman Shrestha, Frederick L Altice, Antoine Khati, Iskandar Azwa, Kamal Gautam, Sana Gupta, Patrick Sean Sullivan, Zhao Ni, Adeeba Kamarulzaman, Panyaphon Phiphatkunarnon, Jeffrey A Wickersham. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 16.02.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.