Evaluation of Chinese HIV Mobile Apps by Researchers and Patients With HIV: Quality Evaluation Study

Background: Against the backdrop of globalization, China remains one of the most heavily burdened countries in Asia with regard to AIDS. However, many high-risk groups and patients affected by AIDS may be less likely to actively seek care from medical institutions because of fear of experiencing shame or discrimination. Mobile apps provide a promising avenue for supporting the prevention, diagnosis, and treatment of AIDS. However, a comprehensive systematic evaluation of these mobile apps’ functionality and quality has not been conducted yet. Objective: This study aims to identify the available mobile apps for AIDS in China, assess and discuss the functional features and quality of these Chinese AIDS mobile apps, and offer decision support for patients and clinical practitioners in accessing high-quality AIDS mobile apps. Furthermore, based on the evaluation results, recommendations for improvement will be provided. Methods: A systematic search was conducted on the Qimai app data platform, the Aladdin WeChat applet data platform, and WeChat to identify mobile apps related to AIDS. A snowball sampling method was used to supplement the potentially overlooked apps. The selected mobile apps underwent a rigorous screening process based on unified criteria. Subsequently, assessments were independently undertaken by 3 separate researchers and 2 patients with HIV, using both the Mobile App Rating Scale (MARS) and the User Mobile App Rating Scale (uMARS). Quantitative interpretations of the data were facilitated by the MedCalc statistical software (version 20.217, MedCalc Software). Results: A total of 2901 AIDS mobile apps were included in the study, with 2897 identified through information retrieval and an additional 4 added via snowball sampling. After a rigorous selection process, 21 apps were determined to be usable. Among them, the Hong Feng Wan app achieved the highest combined average score, calculated based on the MARS (3.96, SD 0.33) and uMARS (4.47, SD 0.26). Overall, there was no significant correlation between MARS and uMARS ( r app quality total score =0.41; P =.07; r subjective quality =0.39; P =.08). A notable issue was the widespread lack of user privacy protection, with only 24% (5/21) of the apps offering this feature. Conclusions: The number of available Chinese AIDS mobile apps is limited, with WeChat applets dominating the market. Nonetheless, the performance of WeChat mini-apps is generally inferior to that of independent apps, and there may be significant discrepancies between assessments conducted by researchers and those provided by genuine end users, emphasizing the necessity of involving real users in the development and evaluation of HIV mobile apps. In addition, developers of these Chinese HIV mobile apps need to devote attention to improving privacy protection mechanisms, in addition to considering the evaluations of researchers and real users. This will help attract more users and increase user loyalty.


Background
AIDS is a chronic infectious disease caused by HIV infection, resulting in high mortality rates.Since the first reported case of AIDS, it has rapidly spread worldwide, becoming a major public health concern.China, influenced by globalization, is no exception.According to the 2021 National Notifiable Infectious Diseases Summary released by the Chinese Center for Disease Control and Prevention, AIDS has the highest mortality rate and death toll, with 19,623 reported deaths.Available data indicate that as of 2022, the number of confirmed AIDS cases in China has exceeded 1.05 million and is projected to surpass 1.6 million infections by 2023.AIDS remains a significant infectious disease affecting public health and socioeconomic development in China [1].
For individuals who are at a high risk of HIV infection but have not yet been infected, pre-exposure prophylaxis (PrEP) is a highly effective method of preventing HIV through daily medication.However, the current use of and adherence to PrEP remain low.However, mobile apps have shown effectiveness in promoting PrEP use and adherence among high-risk populations [2].Muessig et al [3] highlighted the advantages of internet and mobile-based interventions, which facilitate wider dissemination of PrEP at a lower cost compared with conventional methods.The MyChoices app developed by Biello KB demonstrated feasibility and potential in improving HIV PrEP use rates among gay and bisexual men in the United States [4].The prevalence of HIV infections remains disproportionately high among gay, bisexual, and other men who have sex with men (GBMSM) [5].With the widespread use and convenience of smartphones, sexual networking apps have replaced traditional dating websites as the primary online social platform for GBMSM.As such, apps have become crucial venues for sexual health research [6].Targeted video and text-based sexually transmitted infection and HIV information provided through mobile apps has proven effective in reducing new infection rates among GBMSM [5].
For people living with HIV, the highly active antiretroviral therapy developed by scientist David Ho represents the most effective treatment method.Highly active antiretroviral therapy can suppress HIV replication and prevent the emergence of drug-resistant viruses.However, strict adherence is required, necessitating patients to adhere to prescribed medication schedules [7].Despite simplified treatment regimens, adherence remains challenging for some people living with HIV [8].Evidence suggests that HIV case management can improve treatment adherence and quality of life and reduce risky sexual behavior [9,10].However, case managers face significant workloads and limited policy and funding support [11,12].Mobile health (mHealth) apps have demonstrated potential in assisting people living with HIV with effective self-management and delivering personalized interventions.Schnall et al [13,14] identified ideal features for an HIV app, including reminders, health information delivery, medication logs, communication, settings, and search functions.However, comprehensive apps meeting these criteria are scarce in the current market, and there is a lack of rigorously evaluated mHealth apps specifically designed for people living with HIV [13,14].Yang et al [15] found that most Chinese GBMSM apps, selected based on relaxed inclusion criteria, primarily focused on dating and lacked HIV prevention and health information.
For both high-risk populations and individuals infected with HIV, HIV testing plays a vital role in combating HIV.Early HIV testing allows individuals to learn about their infection status promptly, facilitating timely access to antiretroviral therapy and significantly reducing mortality rates.However, fear of shame or discrimination may discourage many high-risk populations and patients from actively seeking care at health care facilities [16][17][18][19].With the increasing prevalence of mobile phones, various mHealth interventions have been developed to diversify HIV self-testing (HIVST) approaches, including telephone hotlines, SMS text messaging-based interventions, and internet-based interventions.These interventions have shown potential in improving testing rates, particularly among hard-to-reach populations [20][21][22][23][24][25][26].Although these platforms have achieved varying levels of success, the use of mobile apps has emerged as a highly popular trend because of their flexibility and scalability.For instance, the mLab app serves to enhance users' understanding of their HIV test results while facilitating their access to pertinent HIV information and services [27].Another noteworthy app is Aspect HIVST, which offers an acceptable means of uploading mobile HIVST results and demographic information to a centralized database [28].In addition, ApiDé serves as a multilingual electronic tool (app) that assists health care providers in offering and explaining HIV screening to immigrants facing language barriers [29].
In conclusion, the use of mobile apps is becoming increasingly prevalent in HIV/AIDS prevention and control.These apps provide a convenient and accessible means for high-risk individuals and patients to access information, consultations, and support related to HIV/AIDS.They serve as effective adjunct measures in improving antiretroviral therapy adherence and reducing AIDS incidence rates, thereby mitigating the currently imbalanced resource allocation between patients and health care providers involved in antiviral treatment.However, research on Chinese AIDS mobile medical apps is relatively scarce and late, and empirical studies evaluating the effectiveness of HIV mobile medical apps remain sparse.Therefore, the effectiveness of Chinese HIV mobile apps in meeting user needs requires further assessment.To evaluate the efficacy of AIDS mobile apps in China, the Mobile App Rating Scale (MARS) and the User Mobile App Rating Scale (uMARS) scales, which are widely accepted and applied for uses such as assessing chronic disease management [30], COVID-19 tracking [31], psychiatric interventions [32], physical exercise among older people [33], and menstrual monitoring [34], were incorporated into this study as standardized assessment tools for the quality of mobile apps.

Objectives
This study aims to achieve the following objectives: 1.To identify a comprehensive list of available Chinese mobile apps for HIV/AIDS.

Data Collection of Apps
The Qimai platform, a well-known domestic mobile app data analytics platform, was used to retrieve AIDS-related apps.
Qimai provides comprehensive data on iOS and Android app markets (including Huawei, Baidu, Xiaomi, Vivo, etc), along with app store optimization and app store search marketing optimization service tools and SearchAds data reports.It also offers professional data analysis and optimization strategies.The platform features an intelligent keyword expansion and association tool, which proved valuable for obtaining the required data for this study.In this research, the Qimai Keyword Expansion Assistant was used to search for AIDS-related terms.
We selected keywords with a relevance score exceeding 50% and a search index surpassing 4605 (typically indicating a higher search frequency) to identify relevant apps that met the criteria.The search was conducted on February 18, 2023.

Supplementary Method
In collaboration with the HIV prevention and control team at the local Centers for Disease Control and Prevention (CDC), a snowball sampling method was used to distribute electronic questionnaires to individuals living with AIDS during their follow-up visits conducted by the CDC staff.The questionnaires (without personal information and providing adequate privacy protection) aimed to inquire about the HIV/AIDS apps they had used.Participants were encouraged to provide feedback using the Wenjuanxing platform and to share the electronic questionnaire with other individuals living with AIDS they knew.This supplementary method aimed to identify AIDS-related apps that were widely used within the AIDS community but were not captured by the aforementioned search strategies.The feedback collection period was extended from February 20 to May 19.
The exclusion criteria for apps were as follows: (1) nonsimplified Chinese language, (2) games, (3) apps with excessively limited functionality, (4) mobile apps that did not primarily focus on HIV or AIDS-related content, (5) >50% of the content is inaccessible, (6) apps that had terminated their services, (7) apps restricted to internal use by specific personnel (apps had to be accessible and functional for evaluation purposes), and (8) apps predominantly intended for advertising and product sales.

Evaluation Tool
In this study, we selected the MARS and uMARS scales as the evaluation tools for assessing the quality of the apps.These scales were chosen for 2 main reasons.First, these scales have been extensively validated and have demonstrated good reliability and validity across different contexts.Second, in our study, the included apps were a mix of independent apps and WeChat applets, with the majority being WeChat applets rather than independent apps.However, the MARS and uMARS scales offer broader applicability in this regard.
Developed by Zelmer et al [35], the MARS scale provides a standardized set of criteria for evaluating mobile apps in terms of engagement, functionality, esthetics, and information quality.The MARS scale has been widely used in user research, mobile app evaluations, and related fields.Previous studies have indicated that the MARS scale demonstrates good validity and reliability, making it a reliable measurement tool [30][31][32][33][34].The uMARS scale, developed by Manning et al [36], serves as an extension of the original MARS scale, specifically designed to prioritize the assessments of average users toward mobile apps.

Evaluation Process
The evaluation was conducted by 3 researchers.Rater 1 (PL) holds a master's degree in medical informatics and has extensive experience in medical information analysis.Rater 2 (Bin Li) holds a master's degree in computer science and has abundant software development experience.Rater 3 (FW), an esteemed scholar in medical informatics in China, possesses a master's degree in computer science and a PhD in social medicine.Being an expert in app quality evaluation, rater 3 boasts rich experience and high authority in research on assessing mobile app quality.Both rater 1 and rater 2 were recruited as real patients with HIV.

RenderX
The evaluations by the researchers and patients with HIV were conducted separately to avoid any interference.Each group followed a 3-step evaluation process.First, basic information for 21 apps, including their names, developers, platforms, and core functionalities, was collected from the SevenMa platform, Aladdin WeChat platform, and WeChat.Second, 2 WeChat applets and 2 independent apps were randomly selected for pilot evaluation.Before conducting the pilot evaluation, all 3 raters watched training videos on the MARS scale to better understand the purpose and significance of each item on the scale.Third, the 21 collected apps underwent a formal evaluation.To ensure consistent evaluation results, rater 1 and rater 2 independently assessed the same samples, completing the evaluation of the nth subdimension for all samples before moving on to evaluate the n+1th dimension.In cases where rater 1 and rater 2 had conflicting evaluations, we applied the Delphi method, which involved multiple rounds of consultation and feedback with experts, to achieve agreement regarding the discrepancies.Rater 3, an expert with relevant qualifications, reassessed the dimension that exhibited disagreement.The evaluation results from rater 3 were then shared with rater 1 and rater 2, along with appropriate explanations.This gave rater 1 and rater 2 the opportunity to revise their ratings and evaluations.If substantial differences persisted even after re-evaluation, the entire process was repeated until their evaluations converged or reached a satisfactory level of agreement.

Statistical Analysis
All analyses were performed using MedCalc software (version 20.217, MedCalc Software).Descriptive scores were derived from the MARS and uMARS scales.To evaluate the reliability of the raters' assessments, Bland-Altman analysis was used to assess both interrater agreement and the range of variability in their scores.Visual representations were also used to visualize the differences in ratings between the 2 evaluators, providing a more concrete quantification of the discrepancies.

Ethical Considerations
This study focused solely on mobile apps and did not involve the collection of any personal information or data.Therefore, there were no ethical concerns or disputes associated with this research.

Overview
A total of 21 apps that met the criteria were obtained through screening, with 18 being WeChat applets, dominating in terms of quantity, whereas only 3 were independent apps (Figure 1).Representative AIDS mobile apps were also displayed, as detailed in Figure 2. The characteristics of these apps are shown in Tables 1 and 2.

Functionality of the App
Most mobile apps (18/21, 86%) provided HIV/AIDS knowledge dissemination, and 71% (15/21) of the apps offered appointment booking for testing.Nearly half of the apps (10/21, 48%) provided counseling services, whereas 38% (8/21) of the apps offered features related to PrEP or PEP.Only 10% (2/21) of the apps had functionalities for live streaming and web-based community.
In addition to these functions related to function evaluation, we also included additional statistics on privacy protection settings.Among the 21 apps included in the study, only 6 (29%) provided privacy protection settings and 2 (10%) designed privacy passwords.

Overview of App Composite Scores
The composite scores for the app quality total score in MARS and uMARS were obtained by averaging the scores for each app.The overall composite score for the 21 included apps was 3.43 (mean app quality total score of MARS =3.47, SD 0.37; mean app quality total score of uMARS =3.38, SD 0.53).Hong Feng Wan achieved the highest composite score, with a score of 4.22 (mean app quality total score of MARS =3.96; mean app quality total score of uMARS =4.47), followed by Wu Ai Fang Hua with a composite score of 3.9 (mean app quality total score of MARS =4.06; mean app quality total score of uMARS =3.74), and Chabei secured the third rank with a composite score of 3.76 (mean app quality total score of MARS =3.66; mean app quality total score of uMARS =3.86).Suzhou Red Ribbon obtained the lowest composite score, with a score of 2.68 (mean app quality total score of MARS =3.67; mean app quality total score of uMARS =1.68).

Comparative Analysis of MARS Score and uMARS Score
Figure 3 presents a correlation analysis between the scores of MARS and uMARS.No significant relationship was observed between the app total quality scores of MARS and uMARS (0.41; P=.07) or between their subjective quality scores (0.39; P=.08).The app most reflective of the disparity between MARS and uMARS is Suzhou Red Ribbon, which had the greatest discrepancy between its app total quality scores on MARS (3.67) and uMARS (1.68).Its score on uMARS (1.68) was the lowest in the ranking, with respective dimensional scores of engagement (1.8), functionality (1.5), esthetics (1.67), information (1.75), and subjective quality (1).Conversely, for MARS, it ranked seventh for the app quality total score, with respective dimensional scores of engagement (2.9), functionality (4.38), esthetics (3.5), information (3.92), and subjective quality (2.5).There was considerable variation disclosed in the rankings of the most highly rated apps between MARS and uMARS.The top 5 app total quality scores on MARS were Wu Ai Fang Hua Interpreting composite data from Figures 4 and 5 and Table 3 reveal that within the 4 MARS scale dimensions of the evaluated 21 apps, the functionality dimension achieved the highest score (4.28, SD 0.39).

Comparative Analysis of Quality Between WeChat Mini-Apps and Independent Apps
As shown in Table 4, insignificant disparities are discernible between WeChat mini-apps and stand-alone apps regarding 2 metric domains: app quality (P MARS =.70; P uMARS =.54) and subjective app quality (P MARS =.48; P uMARS =.80).

Internal Consistency and Reliability Testing for MARS and uMARS Scores
Similarly, we formed 4 Bland-Altman plots using the disparities and mean values between the scorings by reviewers on app quality (both MARS and uMARS) and app subjective quality (MARS and uMARS).The relatively limited range of the 95% limits of agreements suggest that the evaluators' judgment outputs contain minor dissimilarities.A significant proportion of dots in all 4 Bland-Altman plots lie within the concordance interval (Figures 6-9), with their arithmetic means impressionably approaching 0 (Table 5).This provides evidence of the high degree of internal uniformity and dependability in both MARS and uMARS scores.

Principal Findings
This study provides a comprehensive statistical summary of the functionalities and characteristics of 21 Chinese HIV-related mobile apps, along with their quality assessment using the MARS scale.The primary objectives behind the development of these mobile apps were the dissemination of HIV prevention knowledge (18/21, 86%), appointment booking for testing (15/21, 71%), and counseling services (10/21, 48%).These mobile apps aim to assist users in HIV prevention and treatment while improving the quality of communication and interaction between patients and health care providers.

Overview of App Functions
In recent years, the Chinese government has implemented a series of regulations and policies for HIV/AIDS prevention and has established an effective mechanism for HIV/AIDS control.These efforts have achieved certain progress in HIV/AIDS prevention.However, the burden of AIDS disease continues to increase, and the prevention and control situation remains severe.With the widespread adoption of smartphones and the rapid development of mobile internet, mobile apps have played an increasingly important role in disseminating HIV/AIDS knowledge and raising public awareness of prevention [1,37].Among the 21 apps included in this study, 18 (86%) had HIV/AIDS knowledge dissemination functionality.This indicates that HIV/AIDS knowledge dissemination is an almost indispensable feature of Chinese HIV/AIDS mobile apps, which have become important tools for promoting HIV/AIDS knowledge in China.
In this study, 17 (81%) of the 21 mobile apps were identified, including 14 (67%) WeChat applets (such as Chabei, Danlan Happy Test, Baiyin HIV test, Suzhou Red Ribbon, Linqu County CDC, E Ai Jian, Ai Yi Jian, Rong Ai Jian, Liaocheng Dongchangfu District Anti-AIDS Service Platform, Zhecheng County AIDS consulting and test, Qing Ai Health Service, Ai Cheng Wang Shi, Beijing AIDS Association, and Dou Ai Jian) and 3 (14%) independent apps (Life4me+, Hong Feng Wan, and Xiao Ai).These apps provided various forms of health services, including appointment booking, counseling, risk assessment, and PrEP or PEP.A total of 2 mobile apps (Wu Ai Fang Hua and Hong Feng Wan) offered web-based community functionalities and scored excellently in terms of app quality and subjective quality, ranking among the top 2 positions.This finding is consistent with that of a previous study by Nour et al [38].Therefore, we recommend considering the incorporation of web-based community functions when developing and XSL • FO RenderX designing mobile apps related to AIDS.This can encourage users to share their experiences, ask questions, provide suggestions, and offer mutual assistance and support.Organizing regular web-based or offline activities can also promote face-to-face interactions among users, thereby enhancing their sense of belonging and engagement.
Privacy protection is another crucial topic that should be given sufficient attention in AIDS-related mobile apps.As social stigmatization and discrimination are still associated with AIDS, many users are concerned about their personal information being disclosed or misused, posing a threat to their privacy and security [39].Therefore, unlike general mobile apps, ensuring user privacy protection in AIDS mobile apps is of utmost importance, aligning with previous research findings [40].However, in this study, of the 21 apps, only 5 (24%) mobile apps, including 2 (10%) WeChat applets (Chabei and Su Zhou Red Ribbon) and 3 (14%) apps (My Life+, Hong Feng Wan, and Xiao Ai), provided privacy policies or agreements.The average subjective quality score (mean 2.85, SD 0.77) was significantly higher than that of the mobile apps without privacy protection features (mean 2.34, SD 0.58).This observed phenomenon may be attributed to the fact that inadequate privacy protection design may lead to user wariness and lower intentions and frequencies of use regarding AIDS mobile apps [41].Therefore, we strongly suggest that all AIDS-related mobile apps incorporate privacy policies or agreements to alleviate user concerns about privacy protection, enhance user trust, and promote willingness to use these apps.In addition to transparent privacy policies and user agreements, user education is also critical.Developers of AIDS mobile apps should provide users with relevant information and educational resources on privacy protection.This can be achieved through in-app prompts, tutorials, and frequently asked questions.Users should receive clear instructions on configuring their privacy settings, handling their personal data, and the steps to take in reporting privacy concerns or data breaches.These instructions should be easily accessible, comprehensible, and provide specific examples to guide users through each process.

Rating of App Quality
According to the results from MARS scores, among the 4 dimensions of app quality, both researchers and patients with HIV rated 21 mobile apps lowest on engagement (mean MARS 2.4, SD 0.53; mean uMARS 2.56, SD 0.63), especially the WeChat mini-apps, which had the lowest average scores (mean MARS 2.33, SD 0.43; mean uMARS 2.47, SD 0.42).The average engagement scores for the 7 mobile apps developed by the Chinese government were even lower (mean MARS 2.27, SD 0.33; mean uMARS 2.34, SD 0.30).This underlined the concern that the level of engagement would remain low, even for officially developed apps.An analysis of the respective scores for each item within the MARS and uMARS engagement portions revealed a predominant lack of interest and amusement within these mobile apps, thus leading to decreased user engagement.Many AIDS-related mobile apps merely provide basic information and functions without interactive and stimulating designs to attract user participation, lacking a strong appeal to users.In addition, the absence of personalized and customized features limits user engagement.A study has shown a significant correlation between user engagement and an increase in the adoption rate of mobile apps [42].Therefore, it is recommended that developers not only provide high-quality HIV/AIDS prevention and treatment information but also focus on meeting users' needs in terms of multidimensionality, functionality, and depth.In addition, attention should be paid to design in terms of amusement, entertainment, customization, interactivity, and other participatory aspects.This will help attract more user participation and enhance user stickiness.This investigation revealed a lack of correlation between the MARS and uMARS scores.Specifically, (1) the app with the lowest overall uMARS scores across all dimensions surprisingly ranked seventh in terms of MARS scores.(2) A notable discrepancy was found in the functionality dimension ratings among patients with HIV and researchers, exposing the highest demographic variance in this attribute.(3) Moreover, the apps predominantly preferred by patients with HIV exhibited robust performance in functionality and information dimensions, with the latter appearing particularly predominant.However, the apps gravitating toward researchers demonstrated high competence in functionality, esthetics, and information, with functionality being the most superior.These data imply possible significant divergences distinct between the evaluations of researchers and genuine users of the apps.To maintain superior app quality and consumer satisfaction, rigorous surveillance of app quality should be sustained from both research and real-user perspectives.This bifocal assessment permits the accurate identification of genuine user requirements and researcher appraisal, providing valuable insights for the pinpoint and scientific enhancement of both app quality and user experience.
In this study, we observed that most Chinese AIDS-related mobile apps are WeChat applets.A reason for this is that the WeChat applet has advantages such as not requiring downloading or installation, having minimal resource consumption, and high user retention rates, which independent apps do not possess [43].Another reason is that the HIV/AIDS population is relatively niche [44], which means that the market for AIDS mobile apps represents a low-frequency demand niche market.Independent apps often overlook this niche market owing to their high development costs, whereas WeChat applets, with their low development costs and the ability to cater to low-frequency demand niches through segmented scene construction, can effectively meet the needs of this market.Although WeChat mini-apps dominate the landscape of mobile apps for HIV/AIDS in China, their comprehensive performance lags behind that of stand-alone apps.The app quality score (mean MARS 3.45, SD 0.37 vs 3.58, SD 0.47; mean uMARS 3.33, SD 0.50 vs 3.66, SD 0.76) and subjective quality score (mean MARS 2.39, SD 0.57 vs 2.92, SD 1.05; mean uMARS 2.72, SD 0.58 vs 2.88, SD 0.90) for WeChat mini-apps are both lower compared with stand-alone apps.A granular analysis of the scores in different dimensions reveals the greatest discrepancy in the engagement dimension, with WeChat mini-apps scoring markedly lower (mean MARS 2.33, SD 0.43 vs 2.83, SD 0.93; mean uMARS 2.47, SD 0.42 vs 3.13, SD 1.40).This can be attributed to the constraints and limitations imposed by the WeChat platform, which prevent WeChat applets from providing the same user experience as independent apps in terms of interface design, interaction methods, and customization settings.Improvement in these aspects should be a key focus for the future development of AIDS-related WeChat applets.

Limitations
The mobile apps evaluated in this study represent a snapshot of the current status of Chinese HIV-related mobile apps during the research period.Over time, mobile apps may be removed or updated, so the list of included mobile apps meeting the criteria may change in the future.In addition, some mobile apps intended solely for internal organizational use or no longer available in the market were excluded.

Conclusions
This study presents a systematic introduction to the functionality and quality of the currently available Chinese mobile apps for AIDS, providing valuable decision-making support for AIDS-related groups and health care professionals in accessing high-quality AIDS mobile apps.Through our systematic search and evaluation of existing Chinese mobile apps for AIDS, it has been observed that because of the lower demand frequency of AIDS mobile apps in China, less costly WeChat mini-apps have become the primary mode of app, and the overall quality attains a merely average level.A significant implication of our research is identifying the potentially significant discrepancy between the assessments made by researchers and the authentic users of the apps.Consequently, the inclusion of genuine users during the assessment and refinement stages of HIV apps is crucial.The main purpose of developing these mobile apps is to spread HIV prevention knowledge and facilitate booking appointments for testing and counseling services.However, most of these apps lack privacy protection features.Unlike general mobile apps, privacy protection is especially crucial in AIDS-related mobile apps because it directly affects users' willingness to use them [40].Therefore, the introduction of legal and ethical frameworks for privacy protection as well as privacy protection technologies is essential.In addition, enhancing user education on privacy protection and ensuring informed consent is of utmost importance.Research related to privacy protection in Chinese AIDS mobile apps may be a vital and urgent topic to address in the future.It is our intent that the findings of our research may function as a road map and reference for the future development of HIV apps in China.Furthermore, we aim to provide crucial decision-making support for individuals living with HIV in their quest for superior HIV apps.

Figure 1 .
Figure 1.PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart of the selection process for inclusion of the apps.

Figure 2 .
Figure 2. Example of a typical Chinese AIDS mobile app.
a WC: WeChat applet.b PrEP: pre-exposure prophylaxis.c PEP: postexposure prophylaxis.dCDC: Centers for Disease Control and Prevention.

Figure 3 .
Figure 3. Results of correlation analysis between the Mobile App Rating Scale (MARS) and the User Mobile App Rating Scale (uMARS) scores.

Figure 4 .
Figure 4. Box plot of the Mobile App Rating Scale (MARS) score.

Figure 5 .
Figure 5. Box plot of the User Mobile App Rating Scale (uMARS) score.

Figure 6 .
Figure 6.Bland-Altman plot of the app quality (Mobile App Rating Scale).

Figure 7 .
Figure 7. Bland-Altman plot of the app subjective quality (Mobile App Rating Scale).

Figure 8 .
Figure 8. Bland-Altman plot of the app quality (User Mobile App Rating Scale).

Figure 9 .
Figure 9. Bland-Altman plot of the app subjective quality (User Mobile App Rating Scale).

Table 1 .
General characteristics of the AIDS mobile apps (N=21).

Table 2 .
Basic information about the AIDS mobile apps.

Table 3 .
The MARS a and uMARS b scales' scores for apps.The top 5 apps with app quality total score for each dimension are italicized.The top 5 apps in terms of app quality total score in MARS.The top 5 apps in terms of app quality total score in uMARS.
b uMARS: User Mobile App Rating Scale.c d

Table 4 .
Independent samples t test for unequal variances.
c Not available.