Background: COVID-19 digital contact-tracing apps were created to assist public health authorities in curbing the pandemic. These apps require users’ permission to access specific functions on their mobile phones, such as geolocation, Bluetooth or Wi-Fi connections, or personal data, to work correctly. As these functions have privacy repercussions, it is essential to establish how contact-tracing apps respect users’ privacy.
Objective: This study aimed to systematically map existing contact-tracing apps and evaluate the permissions required and their privacy policies. Specifically, we evaluated the type of permissions, the privacy policies’ readability, and the information included in them.
Methods: We used custom Google searches and existing lists of contact-tracing apps to identify potentially eligible apps between May 2020 and November 2021. We included contact-tracing or exposure notification apps with a Google Play webpage from which we extracted app characteristics (eg, sponsor, number of installs, and ratings). We used Exodus Privacy to systematically extract the number of permissions and classify them as dangerous or normal. We computed a Permission Accumulated Risk Score representing the threat level to the user’s privacy. We assessed the privacy policies’ readability and evaluated their content using a 13-item checklist, which generated a Privacy Transparency Index. We explored the relationships between app characteristics, Permission Accumulated Risk Score, and Privacy Transparency Index using correlations, chi-square tests, or ANOVAs.
Strategies to Contain COVID-19
Since the beginning of the COVID-19 pandemic in early 2020, in the absence of vaccines or pharmacological treatments for the SARS-CoV-2 virus, some researchers have urged governments and the global public health community to speed up the response to contain SARS-CoV-2, pushing for the implementation of integrated nonpharmaceutical or nonpharmacological interventions (NPIs) . Traditionally, NPIs adopted to curb an epidemic or pandemic such as influenza include mandating personal protective measures among health care professionals and citizens (eg, wearing masks), environmental measures such as isolating or quarantining positive cases [ ], physical distancing, lockdowns, and travel restrictions [ ]. However, more than two years into the pandemic, even after several vaccines were developed and rolled out worldwide, many countries have struggled to effectively and efficiently implement NPIs. In the absence of aggressive testing, contact tracing, and quarantining, an early study suggested that the only way to control COVID-19 would have included intermittent lockdowns until herd immunity was built up, resulting in unnecessary deaths [ ]. Unfortunately, this seems to have been the case in many countries of the world, which observed alarming surges in cases of the SARS-CoV-2 virus. As of April 2021, the virus had infected >130 million individuals and claimed the lives of >2.8 million worldwide [ ]. NPIs require significant investments in human resources and equipment and a level of coordination that might not be feasible in all contexts. For example, many low- and middle-income countries do not have the resources to enforce containment and testing policies [ ] or purchase large amounts of vaccines. In addition, quarantining and physical distancing may not work among underprivileged and vulnerable segments of the population relying on daily wages for survival [ ].
Contact tracing is one of the most cost-efficient NPIs available to break the chain of viral transmission . According to the interim guidance of the World Health Organization, contact tracing consists of systematically identifying individuals exposed to confirmed positive cases, quarantining them, following up with them to ensure rapid isolation, and, finally, testing and treating them in case they develop symptoms [ ]. This approach effectively controlled COVID-19 as long as quick and efficient processes were followed [ ]. A way to guarantee such efficiency was to include digital technologies, particularly mobile phone–based tools, which are widely available worldwide [ ]. In the last year, a few systematic literature reviews of COVID-19 apps mentioned contact-tracing apps as an essential type of app used in the context of the pandemic to curb virus transmission [ - ]. On the basis of the experience with the Ebola [ ] and H1N1 [ , ] viruses, digital technologies have been increasingly used to support governments in carrying out manual contact-tracing activities. Several conceptual papers and overviews exist on mobile apps for COVID-19 contact tracing [ - ]. There are also a few systematic reviews on the topic [ - ], including a Cochrane review [ ] and a literature review [ ], focusing on digital contact tracing. The Cochrane review analyzed technologies used in epidemics and was updated in May 2020 to include new COVID-19–related studies. This review showed that such technologies are most effective when used to complement rather than substitute manual contact-tracing activities [ ]. A literature review by Jalabneh et al [ ] identified 17 apps that could be used for contact tracing and mentioned the use of these apps to help governments contain the pandemic.
Digital Contact Tracing
According to the Centers for Disease Control and Prevention (CDC), 2 main types of digital contact-tracing tools are used for case management and proximity-tracing or exposure notification apps . Case management tools involve apps and devices that health workers involved in contact-tracing activities can use to capture data and manage contact databases of people tested for the virus. When a person tests positive, contact tracers interview them to recall where, when, and with whom they have been. The contacts are then triaged for assignment to case managers who call and notify contacts, providing options for testing, self-isolation, and referral to a health care provider if necessary. This activity can be done manually and on paper-and-pencil forms, so the technology allows for the streamlining of the process of data entry and management.
Proximity-tracing or exposure notification apps are designed for citizens who voluntarily download and activate such apps to assist in contact-tracing efforts. These apps rely on Bluetooth technology or location-based information stored on the phone to estimate the distance and duration of an encounter between users . The phones exchange alphanumeric strings or keys via Bluetooth that contain such information. This information can be stored on the phone only (decentralized framework) or on a central server (centralized framework) and retained for a limited amount of time [ , ]. Depending on the type of framework, once a positive case is identified, the user or the central server flag their profile as positive, triggering the network and urging them to take action and get tested, self-isolate, or seek the help of health care professionals. This way, exposure notification apps can expand the reach of traditional manual contact tracers, who may fail to identify cases. The apps can reduce the burden on public health staff by allowing for the electronic self-reporting by cases and contacts or by using location data or other features to identify community contacts unknown to the case to look at possible exposure to the virus. This study focuses on proximity-tracing or exposure notification apps as these are designed for citizens.
Use and Application of Contact-Tracing Apps
Many governments have developed digital contact-tracing apps following international guidelines (eg, the World Health Organization , CDC [ ], or European CDC [ ]) and Google and Apple exposure notification frameworks [ ]. For example, as of May 28, 2020, when we started working on this project, we identified 36 apps by searching in the Google Play and Apple App Stores. On the same date, the page entitled COVID-19 apps on Wikipedia [ ]—which was created on April 1, 2020—included information on 37 contact-tracing apps. As of June 15, 2020, there were already 68 apps and, by December 2, 2020, the page included approximately 100 apps. A recent review of COVID-19 apps in the Google Play and Apple App Stores identified 51 contact-tracing apps available until May 2, 2020 [ ]. In the same period, the Technology Review of the Massachusetts Institute of Technology (MIT) launched the Covid Tracing Tracker project [ ] with the purpose of monitoring and evaluating existing contact-tracing apps. A recent literature review published in July 2020 identified 17 apps in 15 countries [ ], whereas Wen et al [ ] analyzed 51 apps.
When can contact-tracing apps be considered effective? According to a seminal conceptual paper by Ferretti et al , contact-tracing apps can be deemed effective when at least 60% of the population uses them. More than a year after the COVID-19 pandemic was declared, some reviews asked the following question: Are these apps used by individuals [ ]? Although many calls for evaluations of contact-tracing apps have been made [ ], the evidence about contact-tracing app adoption and effectiveness is scant. A scoping review by Thorneloe et al [ ] reported only a couple of examples of apps used by 10% to 20% of the population using data reported in news media outlets. Similar numbers were presented in an overview of contact-tracing apps [ ] that provided descriptive information on 14 apps based on publicly available information. The authors focused on technical characteristics (eg, centralized or decentralized frameworks, tracing technology, and technical flaws) and the proportion of the country’s population that used the apps, showing wide ranges (between 0.1% for BlueZone Vietnam and 60% for the Chinese Health Code used on Alipay and WeChat) [ ]. In another conceptual paper, Seto et al [ ] argued that the concept of privacy is context-specific and that there is a trade-off between privacy and public health value. To the best of our knowledge, the only comprehensive evaluation of contact-tracing apps is a longitudinal study involving the German Corona-Warn-App [ ], one of the most downloaded contact-tracing apps in Europe totaling 26.5 million downloads as of March 25, 2021 [ ].
How can this low global uptake be explained? The study on the German Corona-Warn-App by Munzert et al  reported a differential app uptake depending on the users’ self-reported sociodemographic and behavioral profiles. For example, app use was positively associated with older age (≥50 years), education, socioeconomic status, health preconditions, and other preventive behaviors (eg, hand hygiene and mask wearing). App uptake was also higher among those who reported positive cases in their social network or who lived in areas of known outbreaks [ ]; it was also higher among users who trusted the national government, the health care system, and science in general, and among those with a strong digital literacy who were less concerned about privacy [ ].
Privacy and Transparency in Data Protection
Furthermore, in April 2020, our research group embarked on a project that a few months later resulted in the creation of a nationwide contact-tracing app (Ma3an)  in collaboration with the local Ministry of Public Health. Parallel to this project, we searched app databases to identify benchmark apps and used them as a reference for privacy-preserving contact-tracing apps. This was one of the main drivers urging us to undertake a comprehensive systematic review of contact-tracing apps and focus on data protection and privacy aspects.
Searches and Sources of Information
We used two main strategies to identify contact-tracing apps: (1) searching for keywords in the Google Play and on the Apple App Store using Google and (2) scanning the list of apps included in 5 websites identified via Google search.
For the first search strategy, we applied the following two search queries: (1) “allintext:COVID-19|covid|covid19|coronavirus AND tracing|exposure site:play.google.com” and (2) “allintext:COVID-19|covid|covid19|coronavirus AND tracing|exposure site:play.google.com.” We conducted the initial searches on May 7, 2020, and updated them almost monthly, on June 1, 2, and 24, 2020; August 18, 2020; November 27, 2020; April 8, 2021; August 7, 2021; and October 31, 2021.
The second search strategy consisted of scanning 5 webpages containing lists of COVID-19 apps, such as the Wikipedia page on COVID-19 apps (first published on April 1, 2020, and last edited on October 20, 2021) ; the MIT Covid Tracing Tracker project (first published on May 7, 2020, and last updated on January 25, 2021) [ ]; the database of contact-tracing apps of the Council of Europe (last updated on June 10, 2020, and then discontinued) [ ]; an article on COVID tracing app roundup on Android Police (published on September 1, 2020, and last updated on November 21, 2020) [ ], with 26 US states using Google Exposure Notification System (ENS), 37 international apps using the same ENS system, and 30 apps not using the ENS framework; and the List of countries using Google and Apple’s COVID-19 Contact Tracing API on the XDA Developers website (published on June 24, 2020, and updated on February 25, 2021) [ ]. All of these sources were last checked on October 31, 2021.
App Selection Process
We followed a multistage selection process. MB exported the Google search results by looking at the Apple App Store and Google Play in Microsoft Excel. MB then screened the links for relevance, and MAD confirmed the selection. Next, we resolved all disagreements through discussions. Finally, we entered the Google Play links in the Exodus Privacy database , which is the auditing platform for Android apps. The Exodus platform looks for embedded trackers (a software meant to collect user data) and permissions requested by each app. An app was excluded if the link to the Exodus database was not working.
In total, 4 authors (MB, MAD, NM, and RAS), in pairs and independently, extracted the information on permissions using a standardized web-based extraction form based on Exodus reports . All permission items were entered as binary values (1=yes; 0=no). Overall, the raters achieved excellent interrater agreement (percentage of agreement=98.3%; Cohen κ=0.954; Krippendorff α=0.953). All disagreements were resolved through discussion. The Exodus reports [ ] label permissions according to 2 levels of risk, as described on the Android developers' page [ ]: Normal or Dangerous, including Signature and SignatureOrSystem. As described in the book Android Application Security Essentials [ ], normal permissions are those that “cannot do much harm to the user. They generally do not cost users money, but they might cause users some annoyance...These permissions are automatically granted to the app.” Dangerous permissions are always shown to the user as “they can cause user privacy or financial loss.” Signature permissions allow 2 apps authored by the same developer to access each other’s components. This type of permission is automatically granted to the app if it has the same certificate as the app that declared the permission. Signature or system permissions are “granted to applications with the same certificate as the app that defined the permission. In addition, this protection level includes an app with the same certificate as the Android system image. This permission level is mainly used for applications built by handset manufacturers, carriers, and system apps. These permissions are not allowed for third-party apps. These permissions let apps perform some very powerful functions” [ ].
For this data extraction task, we conducted first a calibration exercise with a sample of 15 randomly selected apps to ensure sufficient reliability and adjust the instrument before applying it to the remaining set of apps. The aforementioned 4 authors individually and independently completed the same checklist. The exercise yielded a sufficient level of agreement (84.8%) as well as reliability indexes (Cohen κ=0.823; Krippendorff α=0.696). Disagreements were resolved through discussion, which allowed for the clarification of a few interpretation issues. After we resolved the disagreements, the 4 raters independently completed the checklist for other apps. The interrater reliability notably improved (percentage of agreement=87.5%; Cohen κ=0.749; Krippendorff α=0.749). Finally, the reviewers completed the data extraction for the remaining apps in pairs. As in the previous task, we resolved disagreements through discussion until we reached a consensus.
|Domain and items||Score|
|Privacy (25 points)|
|Does the app collect personally identifiable information?||Yes=0; partiala=5; no=10|
|Data management (50 points)|
|Legal framework (25 points)|
aIn this context, partial information is related to the use of location services only.
bNot applicable options for apps that do not collect personal or identifying information.
cN/A: not applicable.
dGDPR: General Data Protection Regulation.
On the basis of the information reported on the Google Play page or on the other sources we used, we categorized the apps by country and continent according to the NationsOnline classification . We also categorized the apps by the type of coverage (country or state, county, or city, depending on their geographical coverage) and type of sponsor (government; nonprofit organization; profit organization; and multistakeholder, involving a combination of the previous categories). Finally, we grouped the apps with the associated Google Play information according to the number of installs (ranging from ≥50 to ≥100 million) as a relative measure of popularity.
We counted the number of trackers and permissions identified through the Exodus platform . Next, we assigned numeric values to each protection level: normal permissions=1; dangerous, signature, or system permissions=2; and trackers=3 as they constitute a higher level of danger to users’ privacy. We then multiplied the number of permissions and trackers by the protection level to calculate a Permission Accumulated Risk Score. The higher this score, the higher the risk.
We also assessed the readability needed to understand the policy using a combined estimate of readability indexes provided by the Automatic Readability Checker, a web-based readability calculator . This tool outputs an estimate based on 7 popular readability formulas: the Flesch Reading Ease Formula, Flesch-Kincaid Grade Level, Gunning fog index, Simple Measure of Gobbledygook Index, Coleman-Liau Index, Automated Readability Index, and Linsear Write Formula. Naturally, the lower the grade, the easier it is to understand. For example, the Privacy Trust Framework recommends a reading grade level of ≤12 for policy documents [ ].
The selection process is illustrated in. We applied the search queries to Google on May 7, 2020; June 1, 2, and 24, 2020; August 18, 2020; November 27, 2020; April 8, 2021; August 7, 2021; and October 31, 2021. We exported 1055 records from Google Play and 1027 records from the Apple App Store in Microsoft Excel. After removing duplicate links, we screened 15.64% (165/1055) of unique app links available from Google Play and 16.85% (173/1027) of links available from the Apple App Store. In this first screening stage, we excluded 11.5% (19/165) and 25.4% (44/173) of apps from Google Play and the Apple App Store, respectively, that were deemed irrelevant as they were not related to COVID-19. The remaining 88.5% (146/165) and 74.6% (129/173) of apps from Google Play and the Apple App Store, respectively, were assessed for eligibility together with 152 apps from the Wikipedia page [ ], 81 from the MIT Covid Tracing Tracker project [ ], 52 from the Council of Europe database of contact-tracing apps [ ], 93 from the Android Police page [ ], and 65 from the XDA Developers page [ ]. Finally, we excluded apps that were not designed for contact tracing (52/146, 35.6% from Google Play; 42/129, 32.6% from the App Store; and 59/152, 38.8% from Wikipedia).
The identified 180 apps covered 152 geographical units (countries or regions, states or provinces, counties, territories, or cities) in 90 different countries spanning all 5 continents. Most apps came from the Americas (53/180, 29.4%), Asia (53/180, 29.4%), and Europe (46/180, 25.6%). The African continent had 6.1% (11/180) of the apps, Oceania had 5% (9/180), and 4.4% (8/180) of the apps covered multiple continents or were developed to cover different countries. The world map inrepresents the global distribution of the COVID-19 contact-tracing apps. The larger the bubble, the higher the number of apps for each country.
The United States had the highest absolute number of apps as 20% (36/180) were developed to cover different states. This number does not include 1.1% (2/180) of the apps, which came from the US unincorporated territories of Guam (Guam Covid Alert) and Puerto Rico (Rastrea el Virus). A total of 1.1% (2/180) of the apps (Care19 Alert and Care19 Diary) covered the states of North Dakota, South Dakota, and Wyoming. The country with the second-highest number of apps was Australia (6/180, 3.3%). Germany, Great Britain, India, and the Philippines had 2.8% (5/180) of the apps each; Brazil and Italy had 2.2% (4/180) of the apps each; France, Malaysia, Mexico, Nepal, Russia, Singapore, South Africa, and Spain had 1.7% (3/180) of the apps each; and Canada, Iran, the Netherlands, Oman, Switzerland, and the United Arab Emirates had 1.1% (2/180) of the apps each.
Most contact-tracing apps were sponsored by governments (132/180, 73.3%), followed by private organizations (28/180, 15.6%) and nonprofit organizations (14/180, 7.8%). A small number of apps involved multiple stakeholders, including consortia of private, nonprofit, and governmental organizations (6/180, 3.3%).
The list of 154 apps with permission data is provided in(tab 1). In , we report the basic descriptive information for the sample of apps grouped according to the number of installs. As of October 8, 2021, based on Google Play install categories, the 154 apps totaled 264.5 million installs (1.7 million on average), ranging from 10 to 100 million. The most installed app was AarogyaSetu, developed by the Indian National Informatics Centre eGov Mobile Apps department. The least installed app was Aggie-COVID-19, which was designed for New Mexico University. Most apps were installed between 100,000 and 1 million times (106/154, 68.8%), with approximately one-third being installed <100,000 times (48/154, 31.2%).
Most apps were developed by governments (116/154, 75.3%), private organizations (17/154, 11%), nonprofit organizations (11/154, 7.1%), and multistakeholder consortia (10/154, 6.5%). No significant association between the type of sponsor and number of install categories was detected. The average rating was 3.5 (SD 0.7) on Google Play and 3.6 (SD 0.9) on the Apple App Store based on a subsample of 120 apps with a valid App Store page. The average number of reviews was 26,412 (SD 143,803) on Google Play and 5120 (SD 28,826) on the App Store.
The average number of reviews on Google Play was significantly associated with the number of installs (F3=5.04; P<.001; η2=0.07), with apps installed ≥1 million times receiving more reviews than those installed between 100,000 and 500,000 times and more reviews than those installed <100,000 times. We detected a similar difference in the number of ratings on the Apple App Store (F3=3.59; P=.02; η2=0.05); in this case, the average number of ratings was significantly higher in apps installed <100,000 times and those installed between 500,000 and 1 million times.
|Variable||Total||Number of installs||P valuea|
|≥1 million (n=49)||500,000 to 1 million (n=12)||100,000 to 500,000 (n=45)||<100,000 (n=48)|
|Type of sponsor, n (%)||.05|
|Government||116 (75.3)||42 (85.7)||10 (83.3)||36 (80)||28 (58.3)|
|Private||17 (11)||2 (4.1)||1 (8.3)||3 (6.7)||11 (22.9)|
|Nonprofit||11 (7.1)||4 (8.2)||1 (8.3)||3 (6.7)||3 (6.3)|
|Multistakeholder||10 (6.5)||1 (2)||0 (0)||3 (6.7)||6 (12.5)|
|Average app ratings, mean (SD; range)|
|Google Play||3.49 (0.69; 1.00-5.00)||3.53 (0.72; 1.00-4.70)||3.16 (0.65; 1.80-4.10)||3.41 (0.62; 1.30-4.40)||3.61 (0.71; 1.70-5.00)||.17 (.13)|
|Apple App Storeb||3.59 (0.94; 1.00-5.00)||3.46 (0.97; 1.00-4.90)||3.42 (1.15; 1.20-5.00)||3.69 (0.91; 1.60-5.00)||3.74 (0.86; 1.50-5.00)||.52 (.64)|
|Average number of ratings, median (IQR; range)|
|Google Play||972 (5094; 1-1,600,000)||16,373 (32,382; 960-1,600,000)||2362 (1200; 1553-4094)||750 (569; 155-3275)||114 (137; 1-1135)||<.001 (<.001)|
|Apple App Storeb||103 (591; 1-287,200)||1047 (3553; 1-287,200)||464 (763; 4-1400)||112 (226; 1-2200)||25 (47; 1-595)||.02 (<.001)|
|Permission data, median (IQR; range)|
|Average number of permissions||9 (10; 2-44)||10 (9; 6-40)||7 (6; 6-44)||8 (7; 2-42)||12 (12; 4-41)||.27 (.19)|
|Average percentage of dangerous permissions||13 (21; 0-63)||13 (21; 0-36)||0 (20; 0-63)||11 (19; 0-50)||15 (21; 0-44)||.87 (.50)|
|Average number of trackers||1 (2; 0-11)||2 (2; 0-7)||0 (1.3; 0-4)||0 (2; 0-5)||1 (1.25; 0-11)||.38 (.23)|
|Permission Accumulated Risk Score||16 (26; 4-74)||14 (26; 6-63)||10 (13; 6-74)||14 (22; 4-70)||23.5 (26; 4-65)||.34 (.11)|
|Grade level, median (IQR; range)||12 (3; 7-23)||12 (4; 8-23)||11 (2; 7-16)||12 (3; 7-19)||12 (2; 8-18)||.07 (.13)|
|Readability level, n (%)b||.50|
|Very difficult to read||14 (9.1)||7 (15.9)||1 (10)||4 (10.8)||2 (4.9)|
|Difficult to read||67 (43.5)||24 (54.6)||4 (40)||18 (48.7)||21 (51.2)|
|Fairly difficult to read||47 (30.5)||11 (25)||4 (40)||14 (37.8)||18 (43.9)|
|Standard or average||4 (2.6)||2 (4.6)||1 (10)||1 (2.7)||0 (0)|
|Policy—transparency index, median (IQR; range)b||55 (30; 5-95)||60 (31.3; 25-95)||60 (36.3; 20-90)||60 (35; 5-85)||50 (20; 5-90)||.65 (.68)|
aP value for independent sample t tests (2-tailed), chi-square tests, or F tests comparing the number of install categories and the other variables. The P value for the Kruskal-Wallis test, the nonparametric equivalent of an ANOVA, is indicated in parentheses.
The typology of permissions, identified through the Exodus platform automatic permission extraction, is presented in(tab 2). Across the 154 apps with valid permission data, there were 94 different types of permissions, of which 17 (18%) were dangerous or special.
Among the normal permissions, the one used in all apps was Internet, have full network access (154/154, 100%). The permissions used by more than half of the apps were view network connections (150/154, 97.4%); wake lock, prevent phone from sleeping (142/154, 92.2%); run in foreground (137/154, 89%); run at startup (131/154, 85.1%); and the permissions related to Bluetooth as pair with Bluetooth devices (118/154, 76.6%). The most frequently used dangerous permission was access precise location (GPS and network-based), which was used by approximately half of the apps (73/154, 47.4%). Other dangerous permissions used by approximately one-third of the sample included access approximate location (network-based) (57/154, 37%), take pictures and videos (51/154, 33.1%), and modify or delete the contents of your SD card (44/154, 28.6%). On average, each app collected 9 permissions (IQR 10, range 2-44). Only 0.6% (1/154) of the apps collected 2 permissions (TRACE Taguig, the Philippines), and only 0.6% (1/154) collected 44 permissions (Shlonik, Kuwait); 46.1% (71/154) of the apps required fewer permissions.
The average proportion of dangerous permissions was 13% (IQR 21%, range 0%-63%). A total of 39% (60/154) of the apps did not use any dangerous permissions, and 0.6% (1/154) reported using the most dangerous permissions (Corona Watch, Karnataka province, India).
In addition, the Exodus platform extracted approximately 30 different trackers (, tab 3). Google Firebase Analytics was the most frequently used tracker (80/154, 51.9%), followed by Google CrashLytics for crash reporting (48/154, 31.2%). Although some apps had analytics and app statistic information trackers, others had trackers used to profile users (eg, Facebook log-in, Segment, AltBeacon, and DOV-E) or for advertising (Google AdMob; 6/154, 3.9%). On average, each app used 1 tracker (IQR 2, range 0-11). Although 41.6% (64/154) of the apps did not use any trackers, 0.6% (1/154) used the most trackers (Citizen SafePass).
On the basis of the number and type of permissions and trackers, the average Permission Accumulated Risk Score was 16 (IQR 26, range 4-74). Of the 154 apps, 2 (1.3%) scored the lowest—TRACE Taguig (the Philippines) and Beat COVID Gibraltar—and 1 (0.6%) scored the highest—Shlonik (Kuwait). Approximately one-fifth of the sample (40/154, 26%) obtained the second- and third-lowest Permission Accumulated Risk Score (score of 6: 23/154, 14.9%; score of 7: 17/154, 11%).
In terms of the legal framework used, most policy documents mentioned that they abided by the GDPR or other national-level legislative data protection frameworks (82/132, 62.1%). Another notable strength of the right to be forgotten is that most of the policy documents stated that the users had the right to delete the app or their profile (90/132, 68.2%). Nevertheless, a few policies mentioned the right to rectify or edit the profile (50/132, 37.9%).
|Domain, item, and score||Apps, n (%)|
|Does the app collect personally identifiable information?|
|Yes or N/Ab=5||48 (36.4)|
|Yes or N/A=5||116 (87.9)|
|Yes or N/A=5||78 (59.1)|
aPartial score when the app used location services only.
bN/A: not applicable.
cGDPR: General Data Protection Regulation.
The correlations among continuous variables representing app characteristics, Permission Accumulated Risk Score, readability, and Privacy Transparency Index are shown in. There was a small significant correlation between the average app ratings in the 2 app stores (r=0.21; P=.02; 116/154, 75.3%). Similarly, there was a larger, highly significant correlation between the number of ratings reported in the Google Play and Apple App Stores (r=0.87; P<.001; 116/154, 75.3%), which, in turn, was significantly correlated with the number of installs (Google Play ratings: r=0.96; P<.001; 150/154, 97.4%; Apple App Store ratings: r=0.90; P<.001; 120/154, 77.9%). This finding is consistent with the ANOVA reported at the end of the App Characteristics section. The Permission Accumulated Risk Score had a small negative correlation with the average rating on the Apple App Store (r=−0.20; P=.03; 120/154, 77.9%), suggesting that, the lower the rating, the higher the risk to the users’ privacy. The Privacy Transparency Index was negatively associated with the Permission Accumulated Risk Score (r=−0.25; P<.001; 132/154, 85.7%), suggesting that, the higher the risk to one’s data, the lower the transparency index of the related policy document.
is a screenshot of a map representing the relationship between the Permission Accumulated Risk Score and Privacy Transparency Index. The map is publicly available on Tableau Public [ ].
|1. Average rating (Google Play)||—a||—||—||—||—||—||—|
|2. Average rating (Apple App Store)||0.21b||—||—||—||—||—||—|
|3. Number of ratings (Google Play)||0.04||0.03||—||—||—||—||—|
|4. Number of ratings (Apple App Store)||0.11||0.13||0.87c||—||—||—||—|
|5. Number of installs||0.02||0.04||0.96c||0.90c||—||—||—|
|7. Grade level (readability)||0.04||0.10||0.05||0.11||0.08||0.02||—|
dPARS: Permission Accumulated Risk Score.
ePTI: Privacy Transparency Index.
Permission Data and Privacy Risk
To achieve the second objective, we analyzed publicly available information from Google Play webpages and extracted it using the Exodus platform scanner . This objective assessment and data extraction allowed us to systematically identify and classify the types of permissions and their relative risk to users’ privacy. We developed a Permission Accumulated Risk Score to qualify the level of risk, accounting for some dangerous permissions and the presence of invasive trackers. The wide variability in the number and type of permissions and trackers identified across the sample of 154 apps included in this study suggests that there is no single approach to privacy-preserving app development. Consistent with the conclusions of Azad et al [ ], many apps seem to collect more permissions than needed, some of which have the potential to breach users’ privacy. Although the number and type of permissions varied across the apps, it seems that some governments are particularly interested in collecting more data than others. On the one hand, most of the apps requested nondangerous permissions such as allowing for full network access, preventing the devices from sleeping, and asking to pair Bluetooth devices. The use of Bluetooth technology for contact tracing seems to be almost ubiquitous [ , ] and has been deemed a privacy-preserving approach [ , , ]; nevertheless, some apps included very invasive permissions or required constant internet connectivity, which might not be available at all times, making real-time exposure notification difficult or impractical [ ].
Moreover, some apps require read-and-write privileges to access the phone storage and camera to use QR codes, an approach that seems appropriate for some types of offline self-check actions for digital contact tracing . Other apps require access to the microphone, GPS location, and phone identity to allow for government operations of contact tracing and network exposure notification. Although it can be efficient from a public health perspective, this approach might generate some general privacy concerns. Our findings show a negative correlation between the Permission Accumulated Risk Score and the average rating of the selected apps on the Apple App Store, which might indicate that users did not like the design or usability or did not trust these apps, expressing a lower rating [ ].
Readability and Transparency of Privacy Policies
Most apps (81/132, 61.4%) included privacy policies that were very difficult to read, suggesting that only educated users could interpret the information presented. This finding is consistent with some studies evaluating the readability of contact-tracing app privacy policies  and with other apps for other health domains such as mental health [ , ], health and fitness [ ], and general health for young generations [ , ].
Strengths and Limitations
This is the first systematic review and evaluation of COVID-19 contact-tracing apps that combines an assessment of the privacy risk and the privacy policies’ transparency and readability. An essential strength of this study is the methodological approach following a specific protocol for selection, data extraction, and analysis. Another strength is the availability of the data collected across 154 apps developed worldwide. The limitations of this study include the use of bespoke measures to quantify the level of risk (the Permission Accumulated Risk Score) and the level of transparency (the Privacy Transparency Index). Although these instruments require formal validation, we tried to minimize the potential subjectivity and errors by completing a series of trainings and assessing interrater agreement and reliability indexes to establish a good level of agreement in evaluating the apps. Another limitation was the use of data generated from Google Play as some apps were developed only for iOS and were not included in the study. Unfortunately, the App Store for iOS does not include information about the permissions that the apps require; this is due to the different software architecture between iOS and Android. Another limitation is related to the extreme volatility of the mobile app market and its characteristics. We provided a global snapshot of all available contact-tracing apps as of October 31, 2021, after having monitored the market for approximately a year. Considering that the pandemic is still ongoing, existing contact-tracing apps might disappear, new ones could be developed, or different technological solutions could be adopted to provide exposure notifications (eg, merging databases or aligning data exchange protocols between European or US states). This would imply that the existing apps might have different software permissions and privacy policies. Our database provides a historical classification of contact-tracing apps that were developed over more than a year, and we made such a list of apps available from the Tableau link.
The authors would like to acknowledge the help of Ms Dalia Sarieldine, a graduate research assistant in the Department of Health Promotion and Community Health, Faculty of Health Sciences, the American University of Beirut, who helped verify the inclusion of selected apps and update the data on the selected apps.
MB conceived and designed the review with intellectual input from IHE, NM, and RAS. MB developed the search strategy and performed the searches. MB coordinated the selection process with the help of NM, RAS, and MAD. MB, NM, RAS, and MAD independently extracted the data. MB performed the data analyses, and NM, RAS, IHE, and MAD contributed to data interpretation. MB drafted the manuscript, which all authors then edited. All authors reviewed and approved the final version of the manuscript.
Conflicts of Interest
List of contact-tracing apps identified (N=180).XLSX File (Microsoft Excel File), 28 KB
Characteristics of selected contact-tracing apps with permission data available (n=154).XLSX File (Microsoft Excel File), 46 KB
Permission data for selected contact-tracing apps (n=154).XLSX File (Microsoft Excel File), 46 KB
- Fisher D, Wilder-Smith A. The global community needs to swiftly ramp up the response to contain COVID-19. Lancet 2020 Apr 04;395(10230):1109-1110 [FREE Full text] [CrossRef] [Medline]
- Heymann DL, Shindo N. COVID-19: what is next for public health? Lancet 2020 Feb 22;395(10224):542-545 [FREE Full text] [CrossRef] [Medline]
- Non-pharmaceutical public health measures for mitigating the risk and impact of epidemic and pandemic influenza: annex: report of systematic literature reviews. World Health Organization. 2019. URL: https://apps.who.int/iris/handle/10665/329439 [accessed 2021-04-03]
- Kissler SM, Tedijanto C, Goldstein E, Grad YH, Lipsitch M. Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period. Science 2020 May 22;368(6493):860-868 [FREE Full text] [CrossRef] [Medline]
- Coronavirus Update (Live) - Worldometer. Worldometer. 2022. URL: https://www.worldometers.info/coronavirus/ [accessed 2021-04-03]
- Chowdhury R, Luhar S, Khan N, Choudhury SR, Matin I, Franco OH. Long-term strategies to control COVID-19 in low and middle-income countries: an options overview of community-based, non-pharmacological interventions. Eur J Epidemiol 2020 Aug;35(8):743-748 [FREE Full text] [CrossRef] [Medline]
- Smith JA, Judd J. COVID-19: vulnerability and the power of privilege in a pandemic. Health Promot J Austr 2020 Apr;31(2):158-160 [FREE Full text] [CrossRef] [Medline]
- Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, et al. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Global Health 2020 Apr;8(4):e488-e496 [FREE Full text] [CrossRef] [Medline]
- Contact tracing in the context of COVID-19. World Health Organization. URL: https://www.who.int/publications-detail-redirect/contact-tracing-in-the-context-of-covid-19 [accessed 2021-04-04]
- Juneau C, Briand A, Pueyo T, Collazzo P, Potvin L. Effective contact tracing for COVID-19: a systematic review. BMJ 2020 Jul 25 (forthcoming). [CrossRef]
- Verhagen LM, de Groot R, Lawrence C, Taljaard J, Cotton M, Rabie H. COVID-19 response in low- and middle-income countries: don't overlook the role of mobile phone communication. Int J Infect Dis 2020 Oct;99:334-337 [FREE Full text] [CrossRef] [Medline]
- Ahmed N, Michelin RA, Xue W, Ruj S, Malaney R, Kanhere SS, et al. A survey of COVID-19 contact tracing apps. IEEE Access 2020;8:134577-134601. [CrossRef]
- Collado-Borrell R, Escudero-Vilaplana V, Villanueva-Bueno C, Herranz-Alonso A, Sanjurjo-Saez M. Features and functionalities of smartphone apps related to COVID-19: systematic search in app stores and content analysis. J Med Internet Res 2020 Aug 25;22(8):e20334 [FREE Full text] [CrossRef] [Medline]
- Davalbhakta S, Advani S, Kumar S, Agarwal V, Bhoyar S, Fedirko E, et al. A systematic review of smartphone applications available for corona virus disease 2019 (COVID19) and the assessment of their quality using the mobile application rating scale (MARS). J Med Syst 2020 Aug 10;44(9):164 [FREE Full text] [CrossRef] [Medline]
- Kondylakis H, Katehakis DG, Kouroubali A, Logothetidis F, Triantafyllidis A, Kalamaras I, et al. COVID-19 mobile apps: a systematic review of the literature. J Med Internet Res 2020 Dec 09;22(12):e23170 [FREE Full text] [CrossRef] [Medline]
- Li J, Guo X. COVID-19 contact-tracing apps: a survey on the global deployment and challenges. arXiv 2020 May 07. [CrossRef]
- Ming LC, Untong N, Aliudin NA, Osili N, Kifli N, Tan CS, et al. Mobile health apps on COVID-19 launched in the early days of the pandemic: content analysis and review. JMIR Mhealth Uhealth 2020 Sep 16;8(9):e19796 [FREE Full text] [CrossRef] [Medline]
- Danquah LO, Hasham N, MacFarlane M, Conteh FE, Momoh F, Tedesco AA, et al. Use of a mobile application for Ebola contact tracing and monitoring in northern Sierra Leone: a proof-of-concept study. BMC Infect Dis 2019 Sep 18;19(1):810 [FREE Full text] [CrossRef] [Medline]
- Hashemian M, Stanley K, Osgood N. Leveraging H1N1 infection transmission modeling with proximity sensor microdata. BMC Med Inform Decis Mak 2012 May 02;12:35 [FREE Full text] [CrossRef] [Medline]
- Hashemian M, Stanley K, Osgood N. Flunet: automated tracking of contacts during flu season. In: Proceedings of the 8th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks. 2010 Presented at: 8th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks; May 31 - Jun 04, 2010; Avignon, France.
- Alwashmi MF. The use of digital health in the detection and management of COVID-19. Int J Environ Res Public Health 2020 Apr 23;17(8):2906 [FREE Full text] [CrossRef] [Medline]
- Support in the UK for app-based contact tracing of COVID-19. OSF Home. URL: https://osf.io/huqtr [accessed 2022-07-04]
- McKendry RA, Rees G, Cox IJ, Johnson A, Edelstein M, Eland A, et al. Share mobile and social-media data to curb COVID-19. Nature 2020 Apr;580(7801):29. [CrossRef] [Medline]
- Owusu PN. Digital technology applications for contact tracing: the new promise for COVID-19 and beyond? Glob Health Res Policy 2020 Aug 3;5(1):36 [FREE Full text] [CrossRef] [Medline]
- Ferretti L, Wymant C, Kendall M, Zhao L, Nurtay A, Abeler-Dörner L, et al. Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 2020 May 08;368(6491):eabb6936 [FREE Full text] [CrossRef] [Medline]
- The Lancet Digital Health. Contact tracing: digital health on the frontline. Lancet Digit Health 2020 Nov;2(11):e561 [FREE Full text] [CrossRef] [Medline]
- Fagherazzi G, Goetzinger C, Rashid MA, Aguayo GA, Huiart L. Digital health strategies to fight COVID-19 worldwide: challenges, recommendations, and a call for papers. J Med Internet Res 2020 Jun 16;22(6):e19284 [FREE Full text] [CrossRef] [Medline]
- Braithwaite I, Callender T, Bullock M, Aldridge R. Automated and partially-automated contact tracing: a rapid systematic review to inform the control of COVID-19. medRxiv 2020 [FREE Full text] [CrossRef]
- Jalabneh R, Zehra Syed H, Pillai S, Hoque Apu E, Hussein MR, Kabir R, et al. Use of mobile phone apps for contact tracing to control the COVID-19 pandemic: a literature review. SSRN J 2020. [CrossRef]
- Anglemyer A, Moore T, Parker L, Chambers T, Grady A, Chiu K, et al. Digital contact tracing technologies in epidemics: a rapid review. Cochrane Database Syst Rev 2020 Aug 18;8:CD013699 [FREE Full text] [CrossRef] [Medline]
- Contact tracing: using digital tools. Centers for Disease Control and Prevention. URL: https://www.cdc.gov/coronavirus/2019-ncov/php/contact-tracing/index.html [accessed 2022-06-25]
- Martin T, Karopoulos G, Hernández-Ramos J, Kambourakis G, Fovino I. Demystifying COVID-19 digital contact tracing: a survey on frameworks and mobile apps. arXivcs 2020 [FREE Full text] [CrossRef]
- Digital tools for COVID-19 contact tracing. World Health Organization. 2020. URL: https://www.who.int/publications-detail-redirect/WHO-2019-nCoV-Contact_Tracing-Tools_Annex-2020.1 [accessed 2020-11-04]
- Mobile applications in support of contact tracing for COVID-19: a guidance for EU/EEA Member States. European Centre for Disease Prevention and Control. URL: https://www.ecdc.europa.eu/sites/default/files/documents/covid-19-mobile-applications-contact-tracing.pdf [accessed 2021-04-03]
- Michael K, Abbas R. Behind COVID-19 contact trace apps: the Google–Apple partnership. IEEE Consumer Electron Mag 2020 Sep 1;9(5):71-76. [CrossRef]
- COVID apps. Wikipedia. URL: https://en.wikipedia.org/wiki/COVID-19_apps [accessed 2021-10-31]
- A flood of coronavirus apps are tracking us. Now it’s time to keep track of them. MIT Technology Review. URL: https://www.technologyreview.com/2020/05/07/1000961/launching-mittr-covid-tracing-tracker/ [accessed 2020-06-15]
- Wen H, Zhao Q, Lin Z, Xuan D, Shroff N. A study of the privacy of COVID-19 contact tracing apps. In: Security and Privacy in Communication Networks. Cham: Springer International Publishing; 2020.
- Chiusi F. Digital contact tracing apps: do they actually work? A review of early evidence. AlgorithmWatch. URL: https://algorithmwatch.org/en/analysis-digital-contact-tracing-apps-2021/ [accessed 2021-10-31]
- Colizza V, Grill E, Mikolajczyk R, Cattuto C, Kucharski A, Riley S, et al. Time to evaluate COVID-19 contact-tracing apps. Nat Med 2021 Mar 15;27(3):361-362. [CrossRef] [Medline]
- Thorneloe R, Epton T, Fynn W, Daly M, Stanulewicz N, Kassianos A, et al. Scoping review of mobile phone app uptake and engagement to inform digital contract tracing tools for COVID-19. PsyArXiv 2020. [CrossRef]
- Seto E, Challa P, Ware P. Adoption of COVID-19 contact tracing apps: a balance between privacy and effectiveness. J Med Internet Res 2021 Mar 04;23(3):e25726 [FREE Full text] [CrossRef] [Medline]
- Munzert S, Selb P, Gohdes A, Stoetzer LF, Lowe W. Tracking and promoting the usage of a COVID-19 contact tracing app. Nat Hum Behav 2021 Feb;5(2):247-255. [CrossRef] [Medline]
- Corona-Warn-App Open Source Project homepage. Corona-Warn-App Open Source Project. URL: https://www.coronawarn.app/en/ [accessed 2021-04-04]
- Raskar R, Schunemann I, Barbar R, Vilcans K, Gray J, Vepakomma P, et al. Apps gone rogue: maintaining personal privacy in an epidemic. arXivcs 2020 [FREE Full text]
- Sharma T, Bashir M. Use of apps in the COVID-19 response and the loss of privacy protection. Nat Med 2020 Aug 26;26(8):1165-1167. [CrossRef] [Medline]
- Cho H, Ippolito D, Yu Y. Contact tracing mobile apps for COVID-19: privacy considerations and related trade-offs. arXiv 2020 [FREE Full text] [CrossRef]
- Berke A, Bakker M, Vepakomma P, Raskar R, Larson K, Pentland AS. Assessing disease exposure risk with location data: a proposal for cryptographic preservation of privacy. arXiv 2020 [FREE Full text] [CrossRef]
- Yasaka TM, Lehrich BM, Sahyouni R. Peer-to-peer contact tracing: development of a privacy-preserving smartphone app. JMIR Mhealth Uhealth 2020 Apr 07;8(4):e18936 [FREE Full text] [CrossRef] [Medline]
- Scantamburlo T, Cortés A, Dewitte P, Van Der Eycken, D, Billa V, Duysburgh P, et al. Covid-19 and contact tracing apps: a review under the European legal framework. arXiv 2020 [FREE Full text] [CrossRef]
- Hernández-Orallo E, Calafate CT, Cano J, Manzoni P. Evaluating the effectiveness of COVID-19 Bluetooth-based smartphone contact tracing applications. Applied Sci 2020 Oct 13;10(20):7113. [CrossRef]
- Peel D. Privacy trust framework. SSRN J 2013:13 [FREE Full text] [CrossRef]
- Federal Trade Commission. Mobile Privacy Disclosures Building Trust Through Transparency. Collingdale, PA: DIANE Publishing Company; 2013.
- Mulder T. Health apps, their privacy policies and the GDPR. Eur J Law Technol 2019;10(1):3 [FREE Full text]
- Das G, Cheung C, Nebeker C, Bietz M, Bloss C. Privacy policies for apps targeted toward youth: descriptive analysis of readability. JMIR Mhealth Uhealth 2018 Jan 04;6(1):e3 [FREE Full text] [CrossRef] [Medline]
- Oldeweme A, Märtins J, Westmattelmann D, Schewe G. The role of transparency, trust, and social influence on uncertainty reduction in times of pandemics: empirical study on the adoption of COVID-19 tracing apps. J Med Internet Res 2021 Feb 08;23(2):e25893 [FREE Full text] [CrossRef] [Medline]
- Elkhodr M, Mubin O, Iftikhar Z, Masood M, Alsinglawi B, Shahid S, et al. Technology, privacy, and user opinions of COVID-19 mobile apps for contact tracing: systematic search and content analysis. J Med Internet Res 2021 Feb 09;23(2):e23467 [FREE Full text] [CrossRef] [Medline]
- Azad MA, Arshad J, Akmal SM, Riaz F, Abdullah S, Imran M, et al. A first look at privacy analysis of COVID-19 contact-tracing mobile applications. IEEE Internet Things J 2021 Nov 1;8(21):15796-15806. [CrossRef]
- Blacklow S, Lisker S, Ng M, Sarkar U, Lyles C. Usability, inclusivity, and content evaluation of COVID-19 contact tracing apps in the United States. J Am Med Inform Assoc 2021 Aug 13;28(9):1982-1989 [FREE Full text] [CrossRef] [Medline]
- Zhang M, Chow A, Smith H. COVID-19 contact-tracing apps: analysis of the readability of privacy policies. J Med Internet Res 2020 Dec 03;22(12):e21572 [FREE Full text] [CrossRef] [Medline]
- Elhajj I, Abou ML, Germani A, Bardus M, Dohna H, Ghattas H, et al. Ma3an-together against corona, the official Lebanese contact tracing app: experience and lessons learned. In: Emerging science, frontier technologies, and the SDGs - Perspectives from the UN system and science and technology communities. New York: United Nations Interagency Task Team on Science, Technology and Innovation for the Sustainable Development Goals; 2021.
- Bardus M, van Beurden SB, Smith JR, Abraham C. A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. Int J Behav Nutr Phys Act 2016 Mar 10;13:35 [FREE Full text] [CrossRef] [Medline]
- Sunyaev A, Dehling T, Taylor PL, Mandl KD. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc 2015 Apr;22(e1):e28-e33. [CrossRef] [Medline]
- Contact tracing apps. Council of Europe. 2022. URL: https://www.coe.int/en/web/data-protection/contact-tracing-apps [accessed 2020-07-13]
- Sholtz M. COVID tracing app roundup: all of the countries and US states that currently offer exposure notification apps. Android Police. URL: https://www.androidpolice.com/2020/11/21/covid-tracing-apps-ens-android/ [accessed 2020-11-27]
- Rahman M. List of countries using Google and Apple's COVID-19 Contact Tracing API. XDA developers. 2021. URL: https://xda-developers.com/google-apple-covid-19-contact-tracing-exposure-notifications-api-app-list-countries/ [accessed 2021-08-06]
- Exodus privacy homepage. Exodus Privacy. 2022. URL: https://exodus-privacy.eu.org/en/ [accessed 2021-08-07]
- Manifest permission. Android Developer. URL: https://developer.android.com/reference/android/Manifest.permission [accessed 2020-07-13]
- Rai P. Android Application Security Essentials. Birmingham, UK: Packt Publishing; 2013.
- O'Loughlin K, Neary M, Adkins EC, Schueller SM. Reviewing the data security and privacy policies of mobile apps for depression. Internet Interv 2019 Mar;15:110-115 [FREE Full text] [CrossRef] [Medline]
- Continents of the World. Nations Online. URL: https://www.nationsonline.org/oneworld/continents.htm [accessed 2020-12-06]
- Automatic readability checker. Readability Formula. URL: https://readabilityformulas.com/free-readability-formula-tests.php [accessed 2020-12-02]
- Bardus M. Systematic review of C19-Contact tracing apps: App List - Marco Bardus. Tableau Public. 2022. URL: https://public.tableau.com/views/SR-C19-ContactTracingApps/AppList?:language=en-US&publish=yes&:display_count=n&:origin=viz_share_link [accessed 2021-09-02]
- Phillips N. The coronavirus is here to stay - here's what that means. Nature 2021 Feb 16;590(7846):382-384. [CrossRef] [Medline]
- Al Jutail M, Al-Akhras M, Albesher A. Associated risks in mobile applications permissions. J Inf Security 2019;10(02):69-90. [CrossRef]
- Min-Allah N, Alahmed BA, Albreek EM, Alghamdi LS, Alawad DA, Alharbi AS, et al. A survey of COVID-19 contact-tracing apps. Comput Biol Med 2021 Oct;137:104787 [FREE Full text] [CrossRef] [Medline]
- Wang H, Wang L, Wang H. Market-level analysis of government-backed COVID-19 contact tracing apps. In: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering Workshops. 2020 Presented at: ASE '20: 35th IEEE/ACM International Conference on Automated Software Engineering; Sep 21 - 25, 2020; Virtual Event Australia. [CrossRef]
- Robillard JM, Feng TL, Sporn AB, Lai J, Lo C, Ta M, et al. Availability, readability, and content of privacy policies and terms of agreements of mental health apps. Internet Interv 2019 Sep;17:100243 [FREE Full text] [CrossRef] [Medline]
- Huckvale K, Prieto JT, Tilney M, Benghozi P, Car J. Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment. BMC Med 2015 Sep 25;13(1):214 [FREE Full text] [CrossRef] [Medline]
|CDC: Centers for Disease Control and Prevention|
|ENS: Exposure Notification System|
|GDPR: General Data Protection Regulation|
|MIT: Massachusetts Institute of Technology|
|NPI: nonpharmacological intervention|
Edited by L Buis, A Mavragani; submitted 24.11.21; peer-reviewed by A Thongprasert, F Folkvord; comments to author 23.12.21; revised version received 04.01.22; accepted 17.02.22; published 12.07.22Copyright
©Marco Bardus, Melodie Al Daccache, Noel Maalouf, Rayan Al Sarih, Imad H Elhajj. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 12.07.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.