Background: Due to the growing availability of consumer information, the protection of personal data is of increasing concern.
Objective: We assessed readability metrics of privacy policies for apps that are either available to or targeted toward youth to inform strategies to educate and protect youth from unintentional sharing of personal data.
Results: Analysis of privacy policies for these 64 apps revealed an average RGL of 12.78, which is well above the average reading level (8.0) of adults in the United States. There was also a small but statistically significant difference in word count as a function of app category (entertainment: 2546 words, social networking: 3493 words, and utility: 1038 words; P=.02).
Conclusions: Although users must agree to privacy policies to access digital tools and products, readability analyses suggest that these agreements are not comprehensible to most adults, let alone youth. We propose that stakeholders, including pediatricians and other health care professionals, play a role in educating youth and their guardians about the use of Web-based services and potential privacy risks, including the unintentional sharing of personal data.
Both Apple and Android have recently surpassed 1.5 million apps available on their respective markets . Most of these apps collect user statistics and are able to make use of the built-in sensors on one’s mobile phone to track movement, location, and other personal behavior and activity [ ]. Although the use of built-in sensors may simplify the user interface and improve user experience, it can also allow app developers and third parties to gather potentially sensitive information about the consumer [ ]. Due to the growing availability of consumer information, protection of personal data is of increasing concern.
Two existing regulations have attempted to address these issues: the FTC’s Children’s Online Privacy Protection Act (COPPA) and the California Online Privacy Protection Act (CalOPPA). The COPPA took effect in 2000 and created stipulations for the collection, usage, and sharing of information from children under 13 years by Web-based services. In 2013, COPPA rules were updated to address the privacy threats associated with “big data” and the ability for mobile apps and websites to collect highly granular information from consumers such as geolocation, relationships with friends, and different behaviors and preferences. The new COPPA guidelines also addressed parental concerns about websites collecting information about location, friends and contacts, and tracking software associated with mobile apps . Similarly, the CalOPPA imposed regulations on apps available to California residents, requiring them to have a privacy statement informing consumers how their information is collected and shared [ ]. CalOPPA also requires privacy statements to include a list of personally identifiable information being collected and a list of third parties with whom information is shared [ ]. Unfortunately, it is still often unclear how third parties are collecting information that is entered into the app [ ]. This calls into question the effectiveness of such a policy if users are not aware that apps are collecting their information.
The unnoticed involvement of third parties is of particular concern when considering apps targeted toward minors. Although the COPPA legally restricts the ways in which information from minors younger than 13 years can be collected and used, language in the COPPA excludes teenagers from 13 to 18 years of age from these same protections. Although the responsibility of monitoring a child’s Web safety has traditionally fallen on the child’s parents , the teenage years are a time when parents tend to have less direct oversight of Web-based activities. Teens who use mobile apps and websites are less likely to involve their parents when interfacing with and providing information to Web-based services [ ] and may not be fully aware of how their information is collected and used. An open question, then, is the extent to which parents are able to adequately understand and advise on the privacy implications of their children’s Web-based activities.
App Selection Process
outlines the app selection process used. The Apple App Store and the Google Play Store have a combined total of over 3 million apps available for download on mobile devices [ , ]. Each store ranks their apps according to their respective ranking formulas, which take into account app ratings, reviews, and number of downloads. We identified and analyzed the highest-ranked 300 free and 300 paid apps in the Apple App Store and the highest-ranked 300 free and 300 paid apps in the Google Play Store, for a total of 1200 apps, which were reviewed manually.
Focus on Youth
We made efforts to focus our study on apps actually used by youth, and this was done by further narrowing down the selection from the initial 1200 apps identified. Apps were characterized as available to and targeted toward minors if they generally did not require the use of money and did not facilitate interaction with unknown people. Specific exclusion criteria included apps that (1) encourage the use of money outside in-app purchases (eg, shopping, travel, or real-estate apps), (2) facilitate interaction with unknown people (eg, dating or ride-service apps), (3) are focused on tracking pregnancies or newborn development, or (4) serve as licensing keys that unlock premium features of other apps (only in the Google Play Store). Shopping apps included apps related to specific stores or corporations (eg, Kohl’s, Walmart, or Amazon), buy and sell apps (eg, letgo or eBay), and coupon or discount apps (eg, Groupon). Shopping apps did not include subscription streaming services such as HBO Now or Netflix. Dating and ride-service apps, including Tinder and Uber, were omitted because interaction with strangers is discouraged for youth.
Pregnancy and newborn development tracking apps were omitted because having and raising children is less common among teenagers and youth. A total of 96 apps were omitted. All other apps were included.
To determine the reliability of the exclusion criteria, a second rater who had not seen the original list of 1200 apps applied the exclusion criteria to a random sample of 120 apps (30 per app type—Apple Free, Apple Paid, Google Play Free, and Google Play Paid). Out of the 120 apps, there was disagreement on only one app, yielding a kappa statistic of .94 (P<.001), which demonstrates high interrater agreement . After discussion, the 2 raters came to consensus on the one app of disagreement and included it in the sample as “available to youth.”
Comprehensibility was measured as “readability,” or the ease of understanding the given text. Readability was used as a measure of comprehensibility, as it provides an unbiased numerical value reflective of comprehensibility. Readability statistics of privacy policies for apps from the Apple and Google Play app stores were calculated using a Web-based readability calculator and analyzed. The average RGL was then compared with the average RGL of adults in the United States. Notably, there are no standards or guidelines for the readability of mobile app privacy policies, so the readability statistics were also compared with the PPR TF. The PPR TF is a set of criteria that measure how technology affects patient privacy. These criteria were developed by the Coalition for Patient Privacy, in collaboration with others, to offer suggested standards on how patient privacy can be protected.
The 64 privacy policies were entered into a Web-based readability calculator, the Readability Test Tool (WebpageFX, Inc, Harrisburg, PA) , which is one of multiple free resources that calculate readability. Before selecting this tool, privacy policies were entered into multiple Web-based calculators. As most tools were found to produce fairly consistent results, the Readability Test Tool was used because of its simple user interface.
Statistics collected from the readability calculator were word count, Flesch reading ease, Flesch-Kincaid RGL, Gunning-Fog RGL, simplified measure of Gobbledygook (SMOG) RGL, sentence count, and number of complex words. Flesch reading ease computes a score on a scale from 0 to 100 with higher numbers representing greater reading ease. Flesch-Kincaid, Gunning-Fog RGL, and SMOG RGL are calculated by taking into account the sentence length and average word length. Gunning-Fog uses the average word length to determine the percentage of complex words or words with greater than three syllables. SMOG RGL typically overestimates the RGL of the text, and Flesh-Kincaid typically underestimates RGL. For a more accurate metric, RGL was calculated as the average of Flesch-Kincaid RGL, Gunning-Fog RGL, and SMOG RGL ().
Mean RGL of the 64 apps was compared with the average adult reading level in the United States and to the PPR TF recommended RGL of 12.0. The Flesch reading ease score was compared with the PPR TF recommended reading ease score of 45.0. Apps were also divided into three broad app categories (entertainment, social networking, and utility) based on app store classifications. Entertainment apps included games, music, and video apps (eg, Angry Birds, Spotify, and Netflix). Social networking apps were categorized as such by the app stores and included messaging services associated with social networking (eg, Snapchat, Facebook Messenger, and Instagram). Utility apps encompassed all apps for general use and apps that did not fit into the other two categories (eg, flashlight, word processing, or email apps). RGL of the three categories were compared using a one-way analysis of variance (ANOVA). All reported P values are uncorrected.
|App name||Average reading level||Flesch-Kincaid reading level||Gunning-Fog||SMOGa|
|Disney Build It: Frozen||17.1||16.8||19.4||15|
|Nova Launcher Prime||15.6||15.8||16.9||14.1|
|Du Battery Saver and phone charger||15.2||14.5||17.9||13.3|
|Grand Theft Auto: San Andreas||14.7||14.5||16.4||13.1|
|Stick Texting: The Emoji Killer||13.9||13.8||15.7||12.4|
|Assassin’s Creed Identity||13.7||13.8||14.8||12.5|
|Minecraft: Story Mode||13.7||13.5||15.6||12|
|Candy Crush Jelly Saga||13.4||13||15||12.2|
|FaceSwap Live Lite||13.4||13.3||14.5||12.3|
|Ultimate Guitar Tabs and Chords||13.3||13||14.8||12|
|Fishdom: Deep Dive||12.6||12.4||14.4||11.1|
|Game of Life Classic Edition||12.6||12.4||14.3||11.2|
|Power Clean: Optimize cleaner||12.6||12.4||13.9||11.5|
|Super Bright LED Flashlight||12.5||12.3||13.5||11.8|
|Sleep Cycle Alarm Clock||12.4||12.4||13.2||11.6|
|Bloon TD 5||12.3||12.3||14.1||10.6|
|Akinator the Genie||12.2||12.3||12.8||11.4|
|Please Don’t Touch Anything||11.9||11.6||12.8||11.2|
|Kika Emoji Keyboard||11.3||11||12.9||10|
|Minecraft pocket edition||10.7||10.2||11.9||10.1|
|The Room Three||10||9.3||11.7||9.1|
|Papa’s Freezeria To Go||8.5||8.6||8.8||8.2|
aSMOG: simplified measure of Gobbledygook.
Policy Readability Versus Recommended Standards
Importantly, none of the discovered privacy policies had an RGL below the average adult RGL in the United States of 8.0 (). Privacy policies also had an average Flesch reading ease of 42.73 (SD 6.991), which is lower (ie, less readable) than the 45.0 recommended reading ease by the PPR (P<.05; ). The average RGL of 12.78 is similar to the PPR TF recommended RGL of 12.0.
App Category Comparisons
The readability of policies from 30 free apps and 34 paid apps were compared. Free apps had an average RGL of 13.09 (SD 1.304), and paid apps had an average RGL of 12.51 (SD 1.815). Data are shown inand illustrate no significant differences between free and paid apps on any of the metrics examined (P>.05). Apps were also divided into three broad categories (entertainment, social networking, and utility), as previously described. When privacy policies from these apps were compared as a function of category, we observed a significant difference in word count between the categories ( ), with social networking having the highest word count and utility the lowest. There were, however, no significant differences in average RGL.
|Statistic||All appsa||Free apps||Paid apps||P value|
|Mean word count||2425||2355||2487||.79|
|Mean Flesch reading ease||42.73||42.3||43.1||.65|
aColumn summarizes results for all apps included in the analysis; it was not included in the significance test for the P value in the last column.
bRGL: reading grade level.
|Statistic||All appsa||Entertainment||Social networking||Utility||P value|
|Mean word count||2425||2546||3493||1038||.02|
|Mean Flesch reading ease||42.73||42||46.46||43.37||.31|
aColumn summarizes results for all apps included in the analysis; it was not included in the significance test for the P value in the last column.
bRGL: reading grade level.
Most parents are concerned about their child’s safety on the Internet. Whereas many have taken steps to protect their child’s safety while using the Web, such as through discussions with their children, it is often difficult for parents to know how their child’s privacy is protected on the Internet . About 40% of parents of Internet users have read the privacy policies of the apps that their children are using. Previous studies that have assessed privacy policies of mobile apps have concluded that college-level literacy is required to comprehend the text of privacy statements [ ]. Likewise, our study reached similar conclusions even though the apps selected for analysis were specifically directed toward children and teenagers. Apps that are available to teenagers should have privacy statements that teenagers can understand, and apps that are available to children should have privacy statements that are accessible by their parents or guardians. To be COPPA compliant, apps and websites should post a policy regarding their privacy practices so that parents are aware of how information is collected and used, and these policies must be readable and comprehensible [ ].
Results from a 2013 study conducted by the Pew Research Center show that 70% of teen Internet users do seek out advice about their Internet safety. Many teenagers turn to friends, peers, or their parents for advice about privacy settings on Web applications. The results of the Pew study also show that teenagers of all racial and socioeconomic backgrounds seek advice about Internet safety, but white teenagers are more likely than black or Hispanic teenagers to talk to their parents about Web privacy. Youth should have a trusted adult they can consult when considering privacy expectations with their Web presence. By having privacy policies written so that youth can understand them, children and teenagers are afforded a sense of autonomy over their Internet practices. They will be able to make informed decisions about what kind of privacy settings they desire on their Web-based accounts, and they can discuss these privacy settings and their safety with a trusted adult .
We noted that even the PPR TF criteria that was used as a base of comparison for readability in this study has recommended standards that are too difficult for the average adult in the United States to comprehend, as they recommend a RGL of 12.0. We recommend that a new set of guidelines for privacy policies target the average adult in the United States, with an average RGL of 8.0 or lower, a Flesch reading ease score of 70 or higher, and a word count of less than 500 words. These standards would also be understood by most high school students, allowing teenagers to read and comprehend privacy policies for the apps they download and potentially gain a better understanding of how their personal data are collected, used, and potentially sold to third parties.
The complexity and thus incomprehensibility of privacy policies poses a serious Internet safety concern for the youth in particular. A recent study on digital monitoring activity among teenagers shows that most parents do talk to their teenage children about appropriate Web behavior and what they should share on the Internet; however, most parents do not have these talks as frequently as they speak to their children about offline behavior . With the increasing use of Web-based applications in entertainment, education, and social networking, young people are making more and more information available over the Web, potentially leading to harmful consequences.
Introducing educational curricula in schools about Web-based safety and increasing exposure to safe Internet practices may be an avenue to explore empirically. These curricula could provide children and adolescents with the tools they need to understand privacy risks and make choices about how their personal data are stored and shared over the Internet. Such resources are particularly important for older teenagers, who are less likely than younger children to involve their parents or ask for advice about Web privacy . Indeed, teenagers are often already in the position of making their own choices about their behavior and practices in Web-based and digital environments. Web-based safety programs, such as the one developed by Common Sense Education, allow teachers to tailor their curricula to specific grade levels to make Internet safety relevant to minors of different ages [ ].
|Disney Build It: Frozen||2880||17.07||“We collect...Usage, viewing and technical data, including your device identifier or IP address, when you visit our sites...or open emails we send.”|
|“We acquire information from other trusted sources...”|
|Subway Surfers||1272||15.97||“We log information about your use of the App...”|
|“...if you log into the App using a third-party site or platform such as Facebook, we may access information about you from that site or platform...”|
|“We may allow third parties to serve contextual advertisements and provide analytics services in connection with the App. These entities may use various identifiers to collect information...”|
|Nova Launcher Prime||1487||15.60||“Information collected automatically from this Application (or third party services employed in this Application), which can include: the IP addresses or domain names of the computers utilized by the Users who use this Application...the country of origin...”|
|2701||15.53||“WhatsApp will periodically access your address book or contact list on your mobile phone...”|
|“WhatsApp uses both session cookies and persistent cookies. A persistent cookie remains after you close your browser...”|
|Monument Valley||984||15.50||“For operation and maintenance purposes, this Application and any third party services may collect files that record interaction with this Application (System Logs) or use for this purpose other Personal Data (such as IP Address).”|
|“This Application does not support “Do Not Track” requests.”|
Given the ubiquitous nature of Web-based applications and the increasing frequency of use among children and adolescents, combined with the potential for harm if these are used inappropriately, health care providers may need to consider how to address these harms in the context of their overall care of underage patients. Using clinicians as a vehicle for counseling patients on privacy and app safety practices would be analogous to the ways in which health professionals play an important role in informing patients about practices to promote a healthy lifestyle (eg, physical activity and nutrition). For example, health care providers who interact with youth (eg, orthodontists, dentists, or pediatricians) can leverage their access to youth to share information about safety practices to enhance protection of youth in Web-based settings. However, to do that, a systematic approach to document the need for and, subsequently, appropriate guidelines directed to the clinician, would be needed.
Overall, Internet safety has increasingly become a public health issue. Whereas parents may have the primary responsibility for Internet safety education , the literature documents research findings that underscore the expertise required to understand privacy policies. The AAP has posted a guide on their website to assist parents in opening a dialogue to talk to their kids about Internet safety and social media [ ]. Social networking features have become increasingly prevalent in apps—even apps that are not directly associated with social media are often linked to Facebook accounts or have the option to share on social networking. This expansive network increases opportunities for exposure to cyberbullying or material that is unsuitable for minors, which can lead to mental health and safety issues in the pediatric population [ ]. Until there are clear standards for pediatricians and other health care providers specific to privacy and app safety education, they can assist by sharing information about available tools and educational resources.
Finally, institutional resources should be developed to help health professionals fulfill this role. An example of this is the AAP policy statement “Media Use in School-Aged Children and Adolescents”  that specifically highlights the privacy risks of social media and other Web-based activities and recognizes pediatricians’ role in helping parents set rules for Web-based activities and mentor their children about Web safety. Although the AAP tools are a good beginning, there is a need for further tools and training to help health care workers understand, navigate, and educate others about Web-based privacy and Internet safety. Overall, there is evidence that youth are concerned about maintaining their privacy, so training pediatricians and other health care providers to address privacy concerns with their patients will provide an additional safe place to ask questions and open a dialogue about Internet safety.
The authors thank Joshua Quiroz for his assistance with aspects of this project. The authors also thank Kathryn Montgomery, Mark Hochhauser, and Kevin Patrick for their comments on study design and interpretation. This work was supported by a grant from the National Human Genome Research Institute “Impact of Privacy Environments for Personal Health Data on Patients” (PI: CB, R01 HG008753, 2015-2018).
Conflicts of Interest
- Statista. Number of apps available in leading app stores as of July 2015 URL: https://www.statista.com/statistics/276623/number-of-apps-available-in-leading-app-stores/ [accessed 2017-03-01] [WebCite Cache]
- Lane N, Miluzzo E, Lu H, Peebles D, Choudhury T, Campbell A. A survey of mobile phone sensing. IEEE Commun Mag 2010 Sep 02;48(9):140-150. [CrossRef]
- FTC. 2013 Feb. Mobile privacy disclosures: building trust through transparency URL: https://www.ftc.gov/reports/mobile-privacy-disclosures-building-trust-through-transparency-federal-trade-commission [accessed 2017-09-18] [WebCite Cache]
- Sunyaev A, Dehling T, Taylor PL, Mandl KD. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc 2015 Apr;22(e1):e28-e33. [CrossRef] [Medline]
- Democratic Media. 2016 Jun 26. The new children's online privacy rules: what parents need to know URL: https://www.democraticmedia.org/sites/default/files/CDDCOPPAParentguideJune2013.pdf [accessed 2017-09-18] [WebCite Cache]
- Zang J, Dummit K, Graves J, Lisker P, Sweeney L. Who knows what about me? a survey of behind the scenes personal data sharing to third parties by mobile apps. Technology Science 2015 Oct 30:1-53 [FREE Full text]
- Moreno MA, Egan KG, Bare K, Young HN, Cox ED. Internet safety education for youth: stakeholder perspectives. BMC Public Health 2013 Jun 5;13(1):543 [FREE Full text] [CrossRef] [Medline]
- Lenhart A, Madden M, Cortesi S, Gasser U, Smith A. Pew Research Center. 2013 Aug 15. Where teens seek online privacy advice URL: http://www.pewinternet.org/2013/08/15/where-teens-seek-online-privacy-advice/ [accessed 2017-09-18] [WebCite Cache]
- American Academy of Pediatrics. 2013 May 31. Talking to kids and teens about social media and sexting URL: https://www.aap.org/en-us/about-the-aap/aap-press-room/news-features-and-safety-tips/Pages/Talking-to-kids-and-Teens-about-social-media-and-sexting.aspx [accessed 2017-07-27] [WebCite Cache]
- Diomidous M, Chardalias K, Magita A, Koutonias P, Panagiotopoulou P, Mantas J. Social and psychological effects of the internet use. Acta Inform Med 2016 Feb;24(1):66-68 [FREE Full text] [CrossRef] [Medline]
- Livingstone S, Carr J, Byrne J. Cigionline. 2015 Nov 02. One in three: Internet governance and children’s rights URL: https://www.cigionline.org/publications/one-three-internet-governance-and-childrens-rights [accessed 2017-09-18] [WebCite Cache]
- Chia PH, Yamamoto Y, Asokan N. Is this app safe? a large scale study on application permissions and risk signals. New York, NY: International World Wide Web Conference Committee (IW3C2); 2012 Apr Presented at: Proceedings of the 21st International conference on World Wide Web; April 16-20, 2012; Lyon, France p. 1-10.
- Peel D. Privacy Trust Framework. Patient Privacy Rights 2013 Feb 27:1-13 [FREE Full text] [CrossRef]
- Apple Inc. URL: https://itunes.apple.com/us/genre/ios/id36?mt=8 [accessed 2017-03-01] [WebCite Cache]
- Google Inc. URL: https://play.google.com/store/apps?hl=en [accessed 2017-09-18] [WebCite Cache]
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977 Mar;33(1):159-174. [CrossRef] [Medline]
- WebpageFX Inc. Readability test tool URL: http://www.webpagefx.com/tools/read-able/ [accessed 2017-03-02] [WebCite Cache]
- Children's Commissioner. London; 2017 Jan. Growing up digital: a report of the growing up digital taskforce URL: https://www.childrenscommissioner.gov.uk/wp-content/uploads/2017/06/Growing-Up-Digital-Taskforce-Report-January-2017_0.pdf [WebCite Cache]
- McDonald AM, Reeder RW, Kelley PG, Cranor LF. A comparative study of online privacy policies and formats. In: Privacy Enhancing Technologies. Berlin, Heidelberg: Springer; 2009:37-55.
- FTC. 2015. Complying with COPPA: frequently asked questions URL: https://www.ftc.gov/tips-advice/business-center/guidance/complying-coppa-frequently-asked-questions [accessed 2017-03-02] [WebCite Cache]
- Anderson M. Pew Research Center. 2016 Jan 07. Parents, teens, and digital monitoring URL: http://www.pewinternet.org/2016/01/07/parents-teens-and-digital-monitoring/ [accessed 2017-09-18] [WebCite Cache]
- Common Sense. Digital citizenship URL: https://www.commonsense.org/education/digital-citizenship [accessed 2017-03-02] [WebCite Cache]
- Council on Communications and Media. Media use in school-aged children and adolescents. Pediatrics 2016 Nov;138(5):1-8 [FREE Full text] [CrossRef]
|AAP: American Academy of Pediatrics|
|ANOVA: analysis of variance|
|COPPA: Children’s Online Privacy Protection Act|
|CalOPPA: California Online Privacy Protection Act|
|FTC: Federal Trade Commission|
|PPR TF: Patient Privacy Rights’ Trust Framework|
|RGL: reading grade level|
|SD: standard deviation|
|SMOG: simplified measure of Gobbledygook|
Edited by G Eysenbach; submitted 02.03.17; peer-reviewed by J Bender, J Li, P Boisrond, A Cyr; comments to author 05.06.17; revised version received 01.08.17; accepted 23.08.17; published 04.01.18Copyright
©Gitanjali Das, Cynthia Cheung, Camille Nebeker, Matthew Bietz, Cinnamon Bloss. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 04.01.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.