This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Due to the growing availability of consumer information, the protection of personal data is of increasing concern.
We assessed readability metrics of privacy policies for apps that are either available to or targeted toward youth to inform strategies to educate and protect youth from unintentional sharing of personal data.
We reviewed the 1200 highest ranked apps from the Apple and Google Play Stores and systematically selected apps geared toward youth. After applying exclusion criteria, 99 highly ranked apps geared toward minors remained, 64 of which had a privacy policy. We obtained and analyzed these privacy policies using reading grade level (RGL) as a metric. Policies were further compared as a function of app category (free vs paid; entertainment vs social networking vs utility).
Analysis of privacy policies for these 64 apps revealed an average RGL of 12.78, which is well above the average reading level (8.0) of adults in the United States. There was also a small but statistically significant difference in word count as a function of app category (entertainment: 2546 words, social networking: 3493 words, and utility: 1038 words;
Although users must agree to privacy policies to access digital tools and products, readability analyses suggest that these agreements are not comprehensible to most adults, let alone youth. We propose that stakeholders, including pediatricians and other health care professionals, play a role in educating youth and their guardians about the use of Web-based services and potential privacy risks, including the unintentional sharing of personal data.
Both Apple and Android have recently surpassed 1.5 million apps available on their respective markets [
Privacy policies should inform users of the risks of the product they are about to use. Whereas most users may not read the privacy policy, if they have concerns about their privacy while using an app, they should be able to refer back to the policy to understand how their information is being collected or used. Although the Federal Trade Commission (FTC) recommends that mobile apps make privacy statements available to app users [
Two existing regulations have attempted to address these issues: the FTC’s Children’s Online Privacy Protection Act (COPPA) and the California Online Privacy Protection Act (CalOPPA). The COPPA took effect in 2000 and created stipulations for the collection, usage, and sharing of information from children under 13 years by Web-based services. In 2013, COPPA rules were updated to address the privacy threats associated with “big data” and the ability for mobile apps and websites to collect highly granular information from consumers such as geolocation, relationships with friends, and different behaviors and preferences. The new COPPA guidelines also addressed parental concerns about websites collecting information about location, friends and contacts, and tracking software associated with mobile apps [
The unnoticed involvement of third parties is of particular concern when considering apps targeted toward minors. Although the COPPA legally restricts the ways in which information from minors younger than 13 years can be collected and used, language in the COPPA excludes teenagers from 13 to 18 years of age from these same protections. Although the responsibility of monitoring a child’s Web safety has traditionally fallen on the child’s parents [
Internet safety has become a public health issue that concerns health care providers. The American Academy of Pediatrics (AAP) encourages parents to open a dialogue with their children about Web safety [
App selection process flowchart (completed March 2016).
We made efforts to focus our study on apps actually used by youth, and this was done by further narrowing down the selection from the initial 1200 apps identified. Apps were characterized as available to and targeted toward minors if they generally did not require the use of money and did not facilitate interaction with unknown people. Specific exclusion criteria included apps that (1) encourage the use of money outside in-app purchases (eg, shopping, travel, or real-estate apps), (2) facilitate interaction with unknown people (eg, dating or ride-service apps), (3) are focused on tracking pregnancies or newborn development, or (4) serve as licensing keys that unlock premium features of other apps (only in the Google Play Store). Shopping apps included apps related to specific stores or corporations (eg, Kohl’s, Walmart, or Amazon), buy and sell apps (eg, letgo or eBay), and coupon or discount apps (eg, Groupon). Shopping apps did not include subscription streaming services such as HBO Now or Netflix. Dating and ride-service apps, including Tinder and Uber, were omitted because interaction with strangers is discouraged for youth.
Pregnancy and newborn development tracking apps were omitted because having and raising children is less common among teenagers and youth. A total of 96 apps were omitted. All other apps were included.
To determine the reliability of the exclusion criteria, a second rater who had not seen the original list of 1200 apps applied the exclusion criteria to a random sample of 120 apps (30 per app type—Apple Free, Apple Paid, Google Play Free, and Google Play Paid). Out of the 120 apps, there was disagreement on only one app, yielding a kappa statistic of .94 (
For the analysis of the apps, in each of the four app types, the highest ranked 30 apps, representing 10.00% (120/1200) of the apps, were reviewed for availability of a privacy policy. A total of 120 apps were considered a feasible number of privacy policies to analyze using a readability calculator. Of these 120 apps, 21 were available in both the Apple and Google stores and were analyzed only once. Out of the final 99 apps, 24 apps did not have privacy policies, and 11 apps had identical privacy policies because of those apps being products of the same developer. This left a total of 64 unique documents for our final readability analysis. Privacy policies of apps were found either via direct link to the privacy policy from the respective app store or from a link to the website of the app developer.
Comprehensibility was measured as “readability,” or the ease of understanding the given text. Readability was used as a measure of comprehensibility, as it provides an unbiased numerical value reflective of comprehensibility. Readability statistics of privacy policies for apps from the Apple and Google Play app stores were calculated using a Web-based readability calculator and analyzed. The average RGL was then compared with the average RGL of adults in the United States. Notably, there are no standards or guidelines for the readability of mobile app privacy policies, so the readability statistics were also compared with the PPR TF. The PPR TF is a set of criteria that measure how technology affects patient privacy. These criteria were developed by the Coalition for Patient Privacy, in collaboration with others, to offer suggested standards on how patient privacy can be protected.
The 64 privacy policies were entered into a Web-based readability calculator, the Readability Test Tool (WebpageFX, Inc, Harrisburg, PA) [
Statistics collected from the readability calculator were word count, Flesch reading ease, Flesch-Kincaid RGL, Gunning-Fog RGL, simplified measure of Gobbledygook (SMOG) RGL, sentence count, and number of complex words. Flesch reading ease computes a score on a scale from 0 to 100 with higher numbers representing greater reading ease. Flesch-Kincaid, Gunning-Fog RGL, and SMOG RGL are calculated by taking into account the sentence length and average word length. Gunning-Fog uses the average word length to determine the percentage of complex words or words with greater than three syllables. SMOG RGL typically overestimates the RGL of the text, and Flesh-Kincaid typically underestimates RGL. For a more accurate metric, RGL was calculated as the average of Flesch-Kincaid RGL, Gunning-Fog RGL, and SMOG RGL (
Mean RGL of the 64 apps was compared with the average adult reading level in the United States and to the PPR TF recommended RGL of 12.0. The Flesch reading ease score was compared with the PPR TF recommended reading ease score of 45.0. Apps were also divided into three broad app categories (entertainment, social networking, and utility) based on app store classifications. Entertainment apps included games, music, and video apps (eg, Angry Birds, Spotify, and Netflix). Social networking apps were categorized as such by the app stores and included messaging services associated with social networking (eg, Snapchat, Facebook Messenger, and Instagram). Utility apps encompassed all apps for general use and apps that did not fit into the other two categories (eg, flashlight, word processing, or email apps). RGL of the three categories were compared using a one-way analysis of variance (ANOVA). All reported
Flesch-Kincaid, Gunning-Fog, simplified measure of Gobbledygook (SMOG), and average reading grade levels (RGLs) for all apps included in the analysis. The average reading level column is the average of Flesch-Kincaid, Gunning-Fog, and SMOG RGLs.
App name | Average reading level | Flesch-Kincaid reading level | Gunning-Fog | SMOGa |
Disney Build It: Frozen | 17.1 | 16.8 | 19.4 | 15 |
Subway surfers | 15.9 | 16.1 | 18.2 | 13.6 |
Nova Launcher Prime | 15.6 | 15.8 | 16.9 | 14.1 |
Monument Valley | 15.5 | 15.9 | 16.6 | 14 |
15.5 | 16.2 | 17.4 | 13 | |
Du Battery Saver and phone charger | 15.2 | 14.5 | 17.9 | 13.3 |
Netflix | 14.9 | 14.6 | 17.2 | 13 |
Grand Theft Auto: San Andreas | 14.7 | 14.5 | 16.4 | 13.1 |
Mobile Strike | 14.6 | 14.6 | 16.6 | 12.5 |
Pages | 14.2 | 13.8 | 16.2 | 12.6 |
Terraria | 14.2 | 13.7 | 16.3 | 12.5 |
Faily brakes | 14.1 | 13.8 | 16.1 | 12.3 |
Pandora | 14.1 | 13.9 | 16 | 12.5 |
Rolling Sky | 14 | 13.4 | 16.4 | 12.3 |
Stick Texting: The Emoji Killer | 13.9 | 13.8 | 15.7 | 12.4 |
Gmail | 13.8 | 13.5 | 15.9 | 11.9 |
Assassin’s Creed Identity | 13.7 | 13.8 | 14.8 | 12.5 |
Minecraft: Story Mode | 13.7 | 13.5 | 15.6 | 12 |
Angry Birds | 13.6 | 13.3 | 15.2 | 12.4 |
NBA 2K16 | 13.5 | 13.3 | 15 | 12.2 |
Candy Crush Jelly Saga | 13.4 | 13 | 15 | 12.2 |
FaceSwap Live Lite | 13.4 | 13.3 | 14.5 | 12.3 |
Ultimate Guitar Tabs and Chords | 13.3 | 13 | 14.8 | 12 |
13.2 | 13.2 | 15.2 | 11.2 | |
Agar.io | 12.9 | 12.3 | 15.3 | 11.3 |
Hitman: Sniper | 12.9 | 12.7 | 14.2 | 11.7 |
Kimoji | 12.9 | 12.6 | 14.7 | 11.5 |
Spotify Music | 12.8 | 12.5 | 14.3 | 11.7 |
VivaVideo Pro | 12.8 | 12.3 | 14.8 | 11.2 |
Facetune | 12.7 | 12.7 | 13.9 | 11.6 |
Heads Up | 12.7 | 12.3 | 14.5 | 11.3 |
Swype keyboard | 12.7 | 12.5 | 13.9 | 11.6 |
Fishdom: Deep Dive | 12.6 | 12.4 | 14.4 | 11.1 |
Game of Life Classic Edition | 12.6 | 12.4 | 14.3 | 11.2 |
Geometry Dash | 12.6 | 12.2 | 14.4 | 11.3 |
Power Clean: Optimize cleaner | 12.6 | 12.4 | 13.9 | 11.5 |
Snapchat | 12.5 | 12.2 | 14.6 | 10.8 |
Super Bright LED Flashlight | 12.5 | 12.3 | 13.5 | 11.8 |
Clash Royale | 12.4 | 11.8 | 14.4 | 11.1 |
Plague Inc | 12.4 | 12.6 | 13.1 | 11.6 |
Sleep Cycle Alarm Clock | 12.4 | 12.4 | 13.2 | 11.6 |
Bloon TD 5 | 12.3 | 12.3 | 14.1 | 10.6 |
12.3 | 11.8 | 14.3 | 10.8 | |
12.3 | 12.1 | 13.9 | 11 | |
Akinator the Genie | 12.2 | 12.3 | 12.8 | 11.4 |
YouTube | 12.2 | 11.7 | 14.4 | 10.6 |
Please Don’t Touch Anything | 11.9 | 11.6 | 12.8 | 11.2 |
Musical.ly | 11.8 | 11.5 | 12.7 | 11.2 |
ZEDGE | 11.8 | 11.4 | 13.4 | 10.7 |
Kik | 11.7 | 11.3 | 13.3 | 10.6 |
PianoTiles 2 | 11.6 | 11.4 | 12.6 | 10.9 |
Dragon Land | 11.4 | 10.7 | 12.8 | 10.6 |
Kika Emoji Keyboard | 11.3 | 11 | 12.9 | 10 |
NeoMonsters | 11.3 | 10.8 | 12.8 | 10.3 |
11.3 | 10.8 | 12.9 | 10.2 | |
Toca Lab | 11.2 | 11 | 12.6 | 10 |
Afterlight | 10.8 | 10.4 | 11.9 | 10.2 |
Minecraft pocket edition | 10.7 | 10.2 | 11.9 | 10.1 |
Badland 2 | 10.5 | 9.7 | 12.3 | 9.4 |
True Skate | 10.2 | 10 | 11.4 | 9.1 |
Pocket Casts | 10 | 9.2 | 11.5 | 9.3 |
SuperPhoto Full | 10 | 9.4 | 11.7 | 9 |
The Room Three | 10 | 9.3 | 11.7 | 9.1 |
Papa’s Freezeria To Go | 8.5 | 8.6 | 8.8 | 8.2 |
aSMOG: simplified measure of Gobbledygook.
The privacy policies reviewed in our analysis had a mean length of 2425 words (standard deviation [SD] 1965) and ranged from 140 to 8290 words (
Importantly, none of the discovered privacy policies had an RGL below the average adult RGL in the United States of 8.0 (
The readability of policies from 30 free apps and 34 paid apps were compared. Free apps had an average RGL of 13.09 (SD 1.304), and paid apps had an average RGL of 12.51 (SD 1.815). Data are shown in
Mean readability statistics. Free versus paid: comparison of mean reading grade level (RGL), mean word count, and mean reading ease between free and paid apps from both the Android and Apple markets. The
Statistic | All appsa | Free apps | Paid apps | |
N | 64 | 30 | 34 | -- |
Mean RGLb | 12.78 | 13.09 | 12.51 | .15 |
Mean word count | 2425 | 2355 | 2487 | .79 |
Mean Flesch reading ease | 42.73 | 42.3 | 43.1 | .65 |
aColumn summarizes results for all apps included in the analysis; it was not included in the significance test for the
bRGL: reading grade level.
Mean readability statistics. Entertainment versus social networking versus utility: comparison of mean reading grade level (RGL), mean word count, and mean reading ease between entertainment, social networking, and utility apps. The
Statistic | All appsa | Entertainment | Social networking | Utility | |
N | 64 | 44 | 7 | 13 | -- |
Mean RGLb | 12.78 | 12.84 | 12.7 | 12.62 | .93 |
Mean word count | 2425 | 2546 | 3493 | 1038 | .02 |
Mean Flesch reading ease | 42.73 | 42 | 46.46 | 43.37 | .31 |
aColumn summarizes results for all apps included in the analysis; it was not included in the significance test for the
bRGL: reading grade level.
Privacy policy word count (N=64 apps). The average word count of the privacy policies was 2425 words. The Game of Life Classic Edition had the highest word count at 8290 words, and Plague Inc had the lowest word count at 140 words.
Privacy policy reading grade level (RGL; N=64). The RGL was an average of the Flesch-Kincaid, Gunning Fog, and simplified measure of Gobbledygook (SMOG) RGLs. The mean RGL of all the apps was 12.78, which is equivalent to a freshman in college. This average level is also higher than the Patient Privacy Rights (PPR) recommended RGL of 12.00 and higher than the US average adult RGL of 8.00. In terms of the individual apps, the highest RGL was for Disney Build It: Frozen at 17.07, which is equivalent to a graduate student reading level. The lowest RGL was for Papa’s Freezeria To Go at 8.53.
Analysis of privacy policies for 64 popular apps targeted toward youth revealed an average reading level of 12.78 or the equivalent RGL of a first year college student. Although this RGL is similar to the reading level recommended by the PPR TF, it is well above the average reading level of adults in the United States. These findings are similar to those from a 2015 study (Sunyaev et al), which noted that app developers and companies are not transparent about their privacy practices through their privacy policies [
Most parents are concerned about their child’s safety on the Internet. Whereas many have taken steps to protect their child’s safety while using the Web, such as through discussions with their children, it is often difficult for parents to know how their child’s privacy is protected on the Internet [
Results from a 2013 study conducted by the Pew Research Center show that 70% of teen Internet users do seek out advice about their Internet safety. Many teenagers turn to friends, peers, or their parents for advice about privacy settings on Web applications. The results of the Pew study also show that teenagers of all racial and socioeconomic backgrounds seek advice about Internet safety, but white teenagers are more likely than black or Hispanic teenagers to talk to their parents about Web privacy. Youth should have a trusted adult they can consult when considering privacy expectations with their Web presence. By having privacy policies written so that youth can understand them, children and teenagers are afforded a sense of autonomy over their Internet practices. They will be able to make informed decisions about what kind of privacy settings they desire on their Web-based accounts, and they can discuss these privacy settings and their safety with a trusted adult [
Much of the inaccessible language in privacy policies stems from legal terminology used by corporations to protect themselves from potential liability. We identified excerpts from privacy policies in our study with the highest RGL (
We noted that even the PPR TF criteria that was used as a base of comparison for readability in this study has recommended standards that are too difficult for the average adult in the United States to comprehend, as they recommend a RGL of 12.0. We recommend that a new set of guidelines for privacy policies target the average adult in the United States, with an average RGL of 8.0 or lower, a Flesch reading ease score of 70 or higher, and a word count of less than 500 words. These standards would also be understood by most high school students, allowing teenagers to read and comprehend privacy policies for the apps they download and potentially gain a better understanding of how their personal data are collected, used, and potentially sold to third parties.
The complexity and thus incomprehensibility of privacy policies poses a serious Internet safety concern for the youth in particular. A recent study on digital monitoring activity among teenagers shows that most parents do talk to their teenage children about appropriate Web behavior and what they should share on the Internet; however, most parents do not have these talks as frequently as they speak to their children about offline behavior [
Introducing educational curricula in schools about Web-based safety and increasing exposure to safe Internet practices may be an avenue to explore empirically. These curricula could provide children and adolescents with the tools they need to understand privacy risks and make choices about how their personal data are stored and shared over the Internet. Such resources are particularly important for older teenagers, who are less likely than younger children to involve their parents or ask for advice about Web privacy [
Sample text from privacy policies with highest reading grade level (top 5).
App name | Word count | Reading grade level | Sample text from privacy policy |
Disney Build It: Frozen | 2880 | 17.07 | “We collect...Usage, viewing and technical data, including your device identifier or IP address, when you visit our sites...or open emails we send.” |
“We acquire information from other trusted sources...” | |||
Subway Surfers | 1272 | 15.97 | “We log information about your use of the App...” |
“...if you log into the App using a third-party site or platform such as Facebook, we may access information about you from that site or platform...” | |||
“We may allow third parties to serve contextual advertisements and provide analytics services in connection with the App. These entities may use various identifiers to collect information...” | |||
Nova Launcher Prime | 1487 | 15.60 | “Information collected automatically from this Application (or third party services employed in this Application), which can include: the IP addresses or domain names of the computers utilized by the Users who use this Application...the country of origin...” |
2701 | 15.53 | “WhatsApp will periodically access your address book or contact list on your mobile phone...” | |
“WhatsApp uses both session cookies and persistent cookies. A persistent cookie remains after you close your browser...” | |||
Monument Valley | 984 | 15.50 | “For operation and maintenance purposes, this Application and any third party services may collect files that record interaction with this Application (System Logs) or use for this purpose other Personal Data (such as IP Address).” |
“This Application does not support “Do Not Track” requests.” |
Given the ubiquitous nature of Web-based applications and the increasing frequency of use among children and adolescents, combined with the potential for harm if these are used inappropriately, health care providers may need to consider how to address these harms in the context of their overall care of underage patients. Using clinicians as a vehicle for counseling patients on privacy and app safety practices would be analogous to the ways in which health professionals play an important role in informing patients about practices to promote a healthy lifestyle (eg, physical activity and nutrition). For example, health care providers who interact with youth (eg, orthodontists, dentists, or pediatricians) can leverage their access to youth to share information about safety practices to enhance protection of youth in Web-based settings. However, to do that, a systematic approach to document the need for and, subsequently, appropriate guidelines directed to the clinician, would be needed.
Overall, Internet safety has increasingly become a public health issue. Whereas parents may have the primary responsibility for Internet safety education [
Finally, institutional resources should be developed to help health professionals fulfill this role. An example of this is the AAP policy statement “Media Use in School-Aged Children and Adolescents” [
American Academy of Pediatrics
analysis of variance
Children’s Online Privacy Protection Act
California Online Privacy Protection Act
Federal Trade Commission
Patient Privacy Rights’ Trust Framework
reading grade level
standard deviation
simplified measure of Gobbledygook
The authors thank Joshua Quiroz for his assistance with aspects of this project. The authors also thank Kathryn Montgomery, Mark Hochhauser, and Kevin Patrick for their comments on study design and interpretation. This work was supported by a grant from the National Human Genome Research Institute “Impact of Privacy Environments for Personal Health Data on Patients” (PI: CB, R01 HG008753, 2015-2018).
None declared.