Characteristics and Quality of Mobile Apps Containing Prenatal Genetic Testing Information: Systematic App Store Search and Assessment

Introduction

Prenatal genetic testing, a set of genetic tests used to detect potential fetal disease risk, is an essential part of routine prenatal care [1,2]. Information provided by these tests allows pregnant women to make informed decisions about their pregnancy, including preparation for affected births, early management of infants with genetic disorders, and termination of affected pregnancies [1,3-5]. Although prenatal genetic testing is important for pregnant women, obstetricians often have insufficient time to provide comprehensive education about prenatal genetic testing in prenatal care [6,7]. Studies have found that pregnant women lack prenatal genetic testing knowledge, which may hinder informed decision-making during their pregnancies [4,8,9].

Mobile apps have been used to assist patients in accessing health information, facilitate engagement with their physicians, and strengthen patient-provider communication and relationships [10-12]. Lay people also perceive health information provided by apps to be accurate and trustworthy [13]. Among apps with medical information, pregnancy-related apps are the most common [14,15]. Nevertheless, the characteristics, quality, and number of available apps containing prenatal genetic testing information are unknown.

To fill this knowledge gap, we used a systematic approach to identify existing apps containing prenatal genetic testing information and summarize their characteristics. We then adapted existing app evaluation tools [16-18] to evaluate the quality of these apps. We believe our findings will have the potential to help obstetricians become familiar with prenatal genetic testing apps and make recommendations to their patients. Our results can assist researchers and app developers to recognize the strengths and limitations of existing apps that contain information pertaining to prenatal genetic testing, as well as to design apps that better meet the needs of pregnant women.

Methods

Overview

Because this study did not involve risk to human subjects, it was exempted from institutional review board oversight at Texas A&M University. A prenatal genetic counselor at a large medical center and a certified health education specialist who is an expert in prenatal genetic education guided, oversaw, and reviewed the research process and findings.

App Search

Figure 1 summarizes the search, screening, and selection procedures for the eligible prenatal genetic testing apps. Specifically, this process included four steps. First, 3 members of the research team (KW, RA, and JG) conducted a search on both the Apple App Store (iOS) and Google Play (Android) for apps with prenatal genetic testing information, between January 28, 2020, and March 23, 2020. The following search terms were used: “pregnancy,” “prenatal,” “pregnant,” “genetic testing,” “genetic screening,” “pregnancy screening,” “prenatal screening,” “prenatal screening test,” “prenatal diagnostic test,” “carrier testing,” “expanded carrier screening,” “amniocentesis,” “chorionic villus sampling,” “non-invasive prenatal testing,” “non-invasive prenatal screening,” “cell free DNA screening,” “cell free DNA testing,” “MaterniT21,” “Harmony,” “Panorama,” “QNatal,” “percutaneous umbilical blood sampling,” “cordocentesis,” “nuchal translucency scan,” “nuchal translucency ultrasound,” “quad screen,” “quadruple screen,” “quadruple test,” “first-trimester screening,” “second-trimester screening,” “modified sequential screening,” “sequential screening,” “alpha-fetoprotein screening,” “maternal serum screening,” “prenatal genetic screening test,” “prenatal genetic screening,” and “prenatal genetic testing”.

A total of 3795 apps were identified. In the second step, we used very few exclusion criteria for the initial screening in order to maximize the number of potential apps selected. A total of 2902 apps were excluded using the following exclusion criteria: (1) apps that were not relevant to prenatal genetic testing, pregnancy, and/or genetics, and (2) apps that were not in English. In the third step, we excluded apps that were duplicated within the Apple App Store (iOS) and in Google Play (Android). In the fourth step, a total of 414 apps were downloaded and thoroughly assessed. Among these, 28 apps were excluded because of workability issues, such as apps that could not be successfully downloaded or opened, and/or those that needed a specific user account from a hospital or clinic for access. Moreover, 284 apps were excluded due to the lack of prenatal genetic testing information, and 38 apps were excluded because of their duplication in both the Apple App Store and Google Play. Thus, our final sample consisted of 64 apps.

‎View this figure

Data Extraction

The entire research team assessed, extracted, and summarized detailed information from the included 64 apps (Multimedia Appendix 1). Extracted information included app name, operating system (ie, iOS or Android), app description, types of prenatal genetic testing mentioned in those apps, prenatal genetic test procedures mentioned therein, information on the timing of prenatal genetic tests, reliability or accuracy of prenatal genetic tests included, interpretation of the results of those prenatal genetic tests, specific disorders mentioned, prenatal genetic testing information citations, customer ratings, and cost. Moreover, according to the Mobile Application Rating Scale (MARS) app classification [18], we also extracted the app developer information (ie, details of the commercial company, hospital, or nongovernmental organization). Finally, based on the content of the apps, we utilized the Flesch-Kincaid grade level assessment to determine the readability level of the apps selected for further analysis.

App Quality Assessment

To evaluate the quality of the selected apps, we developed the App Quality Assessment Scoring System (AQASS) by adapting two common app evaluation tools—MARS [18] and the “APPLICATIONS” evaluation criteria [16,17]. The AQASS was used to assess 16 items in each app, including the number of prenatal genetic tests mentioned, comprehensiveness, information quality, quality of information sources, evidence base, readability, visual prenatal genetic testing information, customization, interactivity, text search field, ease of navigation, app layout, visual appeal, interdevice compatibility, connectivity, and price. Table 1 lists the detailed criteria for each item in AQASS. Possible AQASS scores ranged from 0 to 27, with a higher score representing higher app quality. Following the reliability checking procedure outlined in previous research [19,20], 2 raters on the team (RA and HH) used the AQASS criteria to independently score each app, after which interrater reliability was assessed using the Gwet AC1 calculation [21]. Findings suggest a strong interrater reliability between the two raters with a coefficient of 0.86 [22].

Results

Among 64 evaluated apps, only 2 (3.1%) apps were specifically developed for prenatal genetic testing. NIPT Insights, developed by Five Minutes Ltd., introduced noninvasive prenatal testing (NIPT) and compared it with first-trimester screening, triple/quad screening, anatomy ultrasound, chorionic villus sampling (CVS), and amniocentesis. Similarly, cfDNA Predictive Value Calculator (Perinatal Quality Foundation) offered links to a number of scientific articles regarding NIPT, and it provided users with the ability to calculate both positive and negative predictive values of NIPT. All other apps that provided prenatal genetic testing information were either pregnancy-related (61/64, 95%) or genetics (1/64, 2%) apps.

Table 2 summarizes the key characteristics of those 64 apps. Specifically, all 64 apps could be downloaded at no cost. Of these, 5 (8%) offered an optional upgrade, which were downloaded and assessed. However, upgrading did not provide any further prenatal genetic testing information. The majority of the apps (49/64, 77%) were created by commercial companies, and over half of the apps (38/64, 59%) were compatible with both the Android and iOS operating systems. Only 52 (81%) apps had a customer rating. With a theoretical range of 1 to 5 stars, the mean rating was 4.5 (SD 0.6; range 2.0-5.0).

A total of 10 different types of prenatal genetic tests were mentioned across all apps, including carrier screening, expanded carrier screening, first-trimester screening (blood test), first-trimester screening (nuchal translucency screening), cell-free DNA testing or NIPT, CVS, amniocentesis, triple/quad screening, anatomy ultrasound, and cordocentesis/percutaneous umbilical blood sampling. Amniocentesis and CVS were most common types of prenatal genetic tests referred to in apps (49/64, 77% for both), whereas carrier screening (6/64, 9%) was the least mentioned.

All apps specifically referred to 17 genetic disorders tested for by prenatal genetic testing; these included testing for the following disorders: trisomy 21 (53/64, 83%), trisomy 18 (33/64, 52%), cystic fibrosis (29/64, 45%), trisomy 13 (18/64, 28%), spina bifida (15/64, 23%), sickle cell anemia (14/64, 22%), Tay-Sachs disease (11/64, 17%), spinal muscular atrophy (8/64, 13%), anencephaly (6/64, 9%), neural tube defect as a general term (3/64, 5%), thalassemia (3/64, 5%), Fragile X syndrome (2/64, 3%), Huntington disease (2/64, 3%), muscular dystrophy (2/64, 3%), Klinefelter syndrome (1/64, 2%), Prader-Willi syndrome (1/64, 2%), trisomy 16 (1/64, 2%), and Turner syndrome (1/64, 2%). Nevertheless, 14.1% (9/64) of the apps did not mention any genetic disorders that could be identified by prenatal genetic testing. Information that was missing in the reviewed apps included context about the interpretation of test results (36/64, 56%), the reliability or the accuracy of prenatal genetic tests (33/64, 52%), testing procedures (17/64, 27%), and testing time (5/64, 8%). In terms of references and citations found in these apps, only 14 of 64 apps (22%) included reliable sources (eg, the Mayo Clinic, the American Pregnancy Association, and the American College of Obstetricians and Gynecologists [ACOG]) to support their prenatal genetic testing information.

Table 1 summarizes the AQASS ratings for the quality of the 64 apps we evaluated. With a possible score range of 0 to 27, the mean score for all apps was 13.5 (SD 2.89; range 5-18). In particular, a majority of the apps were found to be visually appealing (63/64, 98%), have appropriate and clear layouts (62/64, 97%), easy to navigate (55/64, 86%), contain interactivity functions (44/64, 69%), and require no internet to access once downloaded (37/64, 58%). Nevertheless, none of the apps included all types of prenatal genetic tests. Most apps (43/64, 67%) reported only 1 to 5 types of prenatal genetic tests. Only 23 (36%) of all the evaluated apps addressed procedures, timing, reliability or accuracy, and interpretation of the results of the prenatal genetic tests. Although 45 (70%) of the apps presented correct and up-to-date information, merely 8 apps (13%) provided reliable information sources from hospitals, specific obstetricians-gynecologists, and/or peer reviewed articles, or referenced authoritative organizations such as the ACOG (6/64, 9%). Furthermore, a large percentage of the apps had a reading level higher than the sixth grade (61/64, 95%), did not include any visual information for prenatal genetic testing (eg, photos, images, tables, figures, and videos) (59/64, 92%), and lacked a text search field (39/64, 61%). None of the apps had a customization function to allow for the tailoring of the prenatal genetic information to the patients’ circumstances (eg, inputting gestational weeks as well as prenatal genetic testing dates and results in settings or preferences). Nearly all apps (63/64, 98%) had not been tested by research studies to determine their quality and efficacy.

Discussion

Principal Findings

It is important for pregnant women to understand prenatal genetic testing in order to make informed decisions about whether or not to utilize these tests during their pregnancy [4]. Apps are a potential educational tool to help pregnant women understand such complex information [24]. To our knowledge, our study is the first systematic review of apps containing prenatal genetic testing information. We found 64 apps currently available that provide information about prenatal genetic testing. Nevertheless, none of those apps presented comprehensive information about all prenatal genetic tests.

Our results showed that commercial companies had developed the majority of the 64 apps with prenatal genetic testing information we found. We also noticed that approximately one-third of all evaluated apps contained incorrect and/or out-of-date information. It is recommended that developers in commercial companies collaborate with a team of health care professionals, including obstetricians, geneticists, and genetic counselors to design apps with accurate and up-to-date information. Moreover, due to the rapid advancement of genomic technologies, information and guidelines regarding prenatal genetic testing may evolve periodically [25]. Thus, developers should review their apps constantly and update information as needed.

The average AQASS score of the 64 apps containing prenatal genetic testing information for our final sample was 13.5, which was equal to the mean score of the possible theoretical AQASS score (range 0-27). This low score was in line with a previous review that suggested that the quality of genetics and genomics apps that need improvement [19]. The low scores of the apps we evaluated was mainly due to the limited number of prenatal genetic tests mentioned, incomprehensiveness of testing information, unreliable and missing information sources; absence of developmental and clinical testing with users; high readability levels; and the lack of visual information presented, customization options, and a text search field.

Unfortunately, the regulation of health apps has not been well established in the United States. Although the Food and Drug Administration (FDA) has published a policy regulating medical health apps, only mobile apps classified as medical devices are currently under federal oversight and require the FDA approval before they are released to the public [26,27]. Because the FDA does not regulate health apps focused on providing health information [26,27], the often-low quality we found in these apps that include information of prenatal genetic testing can cause pregnant women to be exposed to incomplete, unreliable information regarding prenatal genetic testing. As such, comprehensive regulatory standards and guidelines for these apps are urgently required.

Clinical and Research Implications

Our study has important clinical and research implications. First, due to the lack of high-quality and comprehensive apps that provide prenatal genetic testing information, we caution obstetricians to be prepared to address the questions raised by patients who may have previously obtained incorrect and incomplete information from those apps. Second, obstetricians should also be aware of the limitations of the apps with prenatal genetic testing information when communicating with their patients. Moreover, establishing policies to regulate apps that include prenatal genetic testing information is warranted. Finally, from both research and clinical standpoints, the creation of a high-quality app specifically designed for prenatal genetic testing that has been evaluated using a randomized control trial to test its efficacy in improving pregnant women’s prenatal genetic testing decision-making is recommended.

Strengths and Limitations

There are several limitations to this study. As we only reviewed apps with information on prenatal genetic testing in English, there may be apps developed in languages other than English that were not included in this review. Similarly, our app search was conducted in the United States. Future researchers may consider changing platform and search settings to include apps available in countries outside the United States to allow for a broader search. Furthermore, as a team, we carefully conducted the app search together and asked 3 additional researchers to independently check our search results. Some apps, however, may have been overlooked. As the app search was concluded on March 23, 2020, new apps with prenatal genetic testing information may have become available after that date.

Despite these limitations, this study has a number of strengths. It is a unique study and the first of its kind attempting to identify apps containing prenatal genetic testing information. After screening a total of 3795 apps on the Apple App Store and Google Play platforms, we identified 64 apps. In addition, we also carefully examined each app to present a thorough summary of the information they contained. Beyond reporting the characteristics of the identified apps, we developed the AQASS to capture a detailed overview of the quality of the apps we reviewed, as most apps were not created with the primary purpose of conveying prenatal genetic testing to pregnant women. Our AQASS was based on commonly used and validated app scoring systems (ie, MARS and APPLICATIONS) [16-18]. We adopted both these systems because several items of MARS (eg, “Quality of information: Is app content correct, well written, and relevant to the goal/topic of the app?”) and APPLICATIONS (eg, “advertisements” and “subjective presentation”) were not applicable to the evaluation of the apps that provide prenatal genetic testing information in our study. Conversely, some items from either MARS (eg, “Evidence base: Has the app been trialed/tested?”) or APPLICATIONS (eg, “connectivity” and “navigation ease”) were found to be suitable for the evaluation of apps that include information on prenatal genetic testing. Beyond the use of AQASS for our study, it can also be adopted to assess other apps that were not originally designed to primarily address the specific topic under evaluation. It should be noted that the reliability and validity of AQASS also need to be further examined in the future.

Conclusions

This study is the first to extensively search apps containing information on prenatal genetic testing in both the Apple App Store and Google Play, summarize their characteristics, and assess their quality. We identified 64 available apps containing information about prenatal genetic testing and found that the quality of those apps needs to be improved. Additionally, none of the apps we evaluated were specifically designed to introduce all prenatal genetic tests. As such, pregnant women should be cautious when using these apps for prenatal genetic testing information. Obstetricians should carefully examine apps before any recommendation is made for their use as an educational tool. Improving the quality of existing apps and developing new, evidence-based, high-quality apps with a targeted focus on prenatal genetic testing is strongly recommended in the future.