Published on in Vol 8, No 2 (2020): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/15364, first published .
Medication Management Apps for Diabetes: Systematic Assessment of the Transparency and Reliability of Health Information Dissemination

Medication Management Apps for Diabetes: Systematic Assessment of the Transparency and Reliability of Health Information Dissemination

Medication Management Apps for Diabetes: Systematic Assessment of the Transparency and Reliability of Health Information Dissemination

Original Paper

1Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

2NTU Institute for Health Technologies, Interdisciplinary Graduate School, Nanyang Technological University, Singapore, Singapore

3Institute of Health and Biomedical Innovation, Queensland University of Technology, Brisbane, Australia

4School of Clinical Sciences, Faculty of Health, Queensland University of Technology, Brisbane, Australia

Corresponding Author:

Zhilian Huang, MPH

Centre for Population Health Sciences

Lee Kong Chian School of Medicine

Nanyang Technological University

11 Mandalay Road

Clinical Sciences Building, Level 18

Singapore, 308322

Singapore

Phone: 65 65923945

Email: ZHUANG014@e.ntu.edu.sg


Background: Smartphone apps are increasingly used for diabetes self-management because of their ubiquity and ability to help users to personalize health care management. The number of diabetes apps has proliferated in recent years, but only a small subset of apps that pose a higher risk are regulated by governmental agencies. The transparency and reliability of information sources are unclear for apps that provide health care advice and are not regulated by governmental agencies.

Objective: This study aimed to assess the transparency and reliability of information disseminated via diabetes apps against 8 criteria adapted from the Health On the Net code of conduct (HONcode) principles.

Methods: English-language diabetes-related terms were searched on a market explorer (42matters) on June 12, 2018. Apps with medication and blood glucose management features were downloaded and evaluated against the App-HONcode criteria adapted from the 8 HONcode principles: authoritative, complementarity, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy. Apps were profiled by operating platforms (ie, Android and iOS) and the number of downloads (ie, Android only: ≥100,000 downloads and <100,000 downloads).

Results: A total of 143 apps (81 Android and 62 iOS) were downloaded and assessed against the adapted App-HONcode criteria. Most of the apps on the Android and iOS platforms fulfilled between 2 and 6 criteria, but few (20/143, 14.0%) apps mentioned the qualifications of individuals who contributed to app development. Less than half (59/143, 39.2%) of the apps disclaimed that the information provided or app functions do not replace the advice of the health care provider. A higher proportion of iOS apps fulfilled 5 or more App-HONcode criteria compared with Android apps. However, Android apps were more likely to have the developer’s email listed on the app store (Android: 75/81, 98%; and iOS: 52/62, 84%; P=.005) compared with iOS apps. Of the Android apps assessed, a significantly higher proportion of highly downloaded apps had a privacy and confidentiality clause (high downloads: 15/17, 88%; and low downloads: 33/64, 52%; P=.006) and were more likely to discuss their financial sources (high downloads: 14/17, 82%; and low downloads: 32/64, 50%; P=.03) compared with apps with a low number of downloads.

Conclusions: Gaps in the disclosure of the developer’s qualification, funding source, and the complementary role of the app in disease management were identified. App stores, developers, and medical providers should collaborate to close these gaps and provide more transparency and reliability to app users. Future work can further examine the consent-seeking process for data collection, data management policies, the appropriateness of advertising content, and clarity of privacy clause of these apps.

JMIR Mhealth Uhealth 2020;8(2):e15364

doi:10.2196/15364

Keywords



Background

Smartphone apps, software designed to run on mobile devices, have integrated into many aspects of daily lives and are increasingly used for disease management. For example, there are more than 2000 consumer apps to choose from to self-manage diabetes [1]. In contrast to Web-based information accessed from the computer, apps can help the user to conveniently access information with their portability and improved technological capabilities, such as increased human interaction, interoperability with other devices, and easy data collection. Health apps can have a multitude of uses, including offering advice on healthy living, communication with health care providers, and providing decision support through granular biometric data collection (eg, blood glucose) [2-4]. The adoption of health apps for chronic conditions such as diabetes is expected to grow [5]. There are currently 2.5 billion smartphone users in the world [6] and more than 300,000 health apps available for consumer download [5].

One main reason for the proliferation of health apps is the low barrier of entry for app developers to publish apps [3]. Although the 2 major app stores (ie, Apple and Google Play) review apps before publication (Multimedia Appendix 1), many apps that do not conform to the prereview checklist fall through the cracks and are published. The regulation of app stores also falls outside the purview of governmental agencies, such as purview of the Food and Drug Administration (FDA) [7]. Only a small subset of apps that can pose a higher risk and meet the regulatory definition of device are regulated by the FDA [8-10]. Therefore, apps with inaccurate content or advertisements may still be published and be available to consumers [11-13]. The lack of transparency regarding an app’s source of content may compromise the reliability of the information it disseminates [9,14,15] and can potentially mislead or cause harm to patients who have low health literacy [16].

Concerns over the credibility and reliability of Web-based health information sources began to surface in the early days of internet usage [17,18]. The Health On the Net code of conduct (HONcode), which covers 8 principles (ie, authoritative, complementarity, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy) for website certification, was developed to help guide and standardize the reliability of health and medical information published on the internet [19]. Websites that were certified by the HONcode were assessed to be of higher quality and may reduce consumers’ burden of searching for good-quality websites [20,21].

Objectives

As part of a larger study investigating the medication management features of diabetes apps [22], this study aimed to assess the transparency and reliability of information disseminated via these apps against 8 criteria adapted from the HONcode principles.


Development of App Assessment Criteria

We adapted the 8 HONcode principles (ie, authoritative, complementarity, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy) and termed it the App-HONcode criteria to suit the context of health apps and apps assessment. The initial versions of our criteria were piloted with highly downloaded (≥100,000 downloads) diabetes apps to refine and improve the clarity of the assessment criteria. Unclear statements were discussed among the app assessment team members until a consensus was reached. The not applicable (N/A) option was included for the assessment of attribution and justifiability to account for apps that did not provide health information within the app. The final assessment criteria are shown in (Table 1).

Table 1. Adapted Health On the Net code of conduct criteria for health app assessment.
AttributeHONcodeaApp-HONcode criteriaOptions
AuthoritativeThe qualifications of the authors are indicated.Does the app indicate the qualifications of specific individuals who developed or contributed to the development of the app?Yes or no
ComplementarityInformation should support, not replace, the doctor-patient relationship.Is there a disclaimer stating or which implies that the information provided and/or app functions do not replace the health care provider’s advice?Yes or no
PrivacyRespect the privacy and confidentiality of personal data submitted to the site by the visitor.Is there a privacy and confidentiality clause in the app?Yes or no
AttributionCite the source(s) and date of published information on medical and health pages.Are information sources in the app cited?Yes, no, or N/Ab
JustifiabilityJustifiability: site must back up claims relating to benefits and performance.Are the claims relating to benefits and performance in the app description backed up by evidence? (Answer N/A if there are no claims)Yes, no, or N/A
TransparencyAccessible presentation and accurate email contact.Are the developers contactable by email?Yes or no
Financial disclosureIdentify funding sources.Does the app indicate any funding sources? (Yes if the app is managed by a registered commercial company)Yes or no
Advertising policyClearly distinguish advertising from editorial content.Are advertorials distinguishable from content of the app?Yes, no or no advertising

aHONcode: Health On the Net code of conduct.

bNot applicable.

App Selection and Assessment

As the app assessments were part of a larger study investigating medicines management functionalities of diabetes apps, a more detailed description of app selection and assessment can be found in another paper [22].

App Search Strategy

Diabetes terms in the English language were searched in the Google Play and Apple app stores via an app market explorer, 42matters [23], on June 12, 2018, to identify apps that were marketed for diabetes self-management. The search terms (Diabetes OR Diabetic OR Diabetics) OR (glucose OR glycaemic OR glycemic OR blood sugar OR HbA1c OR A1c) OR insulin were used to search app descriptions, and a list of app titles with descriptions was produced for screening.

App Selection

A random sample of 100 apps was first screened by 2 researchers (ZH and MLT) to ensure consistency in interpretation of the inclusion and exclusion criteria listed below. Differences in interpretations were resolved via consensus discussion with 2 other team members (EL and GJ). Random samples of apps were rescreened until an interrater agreement of above 80.0% was achieved.

Inclusion Criteria

The inclusion criteria were as follows: Apps with medication self-management features (ie, medication scheduling, reminders, tracking, information provision, and adherence review), apps with any blood glucose logging features, apps in the English language, free apps, and apps requiring payment.

Exclusion criteria

The exclusion criteria were as follows: Patient health portals linking to patients’ electronic health records, apps that were not updated after January 1, 2017, intended only for health care professionals, insulin calculators/bolus correctors only, apps with exclusive blood glucose monitoring device tie-in requirement (ie, not permitting manual entry of blood glucose values), apps duplicated on the same platform, apps with regional restrictions, and technical problems (eg, crashes, screen hangs, unable to login, and unable to download).

App Assessments

Included apps were evaluated against the adapted App-HONcode criteria. In total, 5 team members (ZH, EL, GJ, CT, and MLT) underwent a calibration exercise to ensure consistency in criteria interpretation before the app assessments. Apps available on the iOS and Android platforms were treated as unique apps and assessed on both platforms because of possible differences in versions across platforms. The number of yes responses was summed up for each app to determine the number of App-HONcode criteria met. N/A responses for attribution and justifiability were treated as not meeting the criteria for a more conservative approach, whereas the no advertising response for advertising policy was treated as a yes (conforming to this principle).

Statistical Analyses

Apps were grouped by operating platform (ie, Android and iOS) and profiled according to each App-HONcode criterion using descriptive statistics. Only Android apps were further classified by the number of downloads (ie, ≥100,000 downloads and <100,000 downloads), as information on the number of downloads was not available for iOS apps. Pearson chi-square test was used for comparisons between groups, and a 2-tailed Fisher exact test was used where the expected count was less than 5 in a group. Statistical significance was set at a P value of less than .05. All analyses were performed using SPSS (version 22; IBM Corp).


App Screening

We identified and downloaded 351 apps (191 Android and 160 iOS) after app title and description screening, of which 143 apps (81 Android and 62 iOS) met the inclusion criteria and were assessed against the app assessment criteria. The detailed app search results can be found in our study [22].

Characteristics of Included Apps

The number and proportion of the assessed apps meeting the App-HONcode criteria are shown in Figure 1. Most of the apps on the Android and iOS platforms fulfilled between 2 and 5 criteria; 1 Android app met all 8 criteria, whereas another did not meet any criteria. A higher proportion of apps published on the iOS platform met more App-HONcode criteria compared with apps on the Android platform. For example, 43.6% of iOS apps met at least five App-HONcode criteria compared with 28.3% of Android apps.

Figure 1. The number and proportion of diabetes apps meeting the App-HONcode app criteria. The number of Yes responses were summed up for each app to determine the number of app-HONcode criteria met. N/A responses for attribution and justifiability were treated as no, whereas the no advertising response for advertising policy was treated as a yes. HONcode: Health On the Net code of conduct.
View this figure

The profile of app attributes grouped by platform is shown in Table 2. Few (20/143, 14.0%) apps mentioned the qualifications of individuals who contributed to app development. Less than half (56/143, 39.2%) of the apps had a disclaimer stating that the information provided/app functions do not replace a health care provider’s advice, and approximately two-thirds (93/143, 65.0%) of apps had a privacy and confidentiality clause. Of the apps providing health or medical information or made claims on its efficacy, only one-third cited their information sources (15/42, 36%) and/or backed up the claims relating to benefits and performance in the app by evidence (7/23, 30%).

Table 2. Profile of app attribute grouped by platform.
HONcodea principleApp-HONcode criteriaAll apps, n (%)Android, n (%)iOS, n (%)P value
AuthoritativeDoes the app indicate the qualifications of specific individuals who developed or contributed to the development of the app?20 (14.0)b9 (11)c11 (18)d.33
ComplementarityIs there a disclaimer stating or which implies that the information provided and/or app functions do not replace the health care provider’s advice?56 (39.2)b27 (33)c29 (47)d.12
PrivacyIs there a privacy and confidentiality clause in the app?93 (65.0)b48 (59)c45 (73)d.11
AttributionAre information sources in the app cited?15 (34)e,f8 (36)e,g7 (32)e,h>.99i
JustifiabilityAre the claims relating to benefits and performance in the app description backed up by evidence? (Answer not applicable if there are no claims)7 (30)e,j4 (29)e,k3 (33)e,l>.99i
TransparencyAre the developers contactable by email?131 (91.6)b79 (98)c52 (84)d.005m
Financial disclosureDoes the app indicate any funding sources? (Yes if the app is managed by a registered commercial company)88 (61.5)b46 (57)c42 (68)d.23
Advertising policyThere are no advertisements in the app119 (83.2)b64 (79)c55 (89)d.18
Are advertorials distinguishable from content of the app?18 (75)n,o12 (71)n,p6 (86)n,q.63

aHONcode: Health On the Net code of conduct.

bN=143.

cN=81.

dN=62.

eNot applicable removed before statistical analysis and percentage computation.

fN=44.

gN=22.

hN=22.

iTwo-tailed P value calculated using Fisher exact test, as the expected count is less than 5 in at least one group.

jN=23.

kN=14.

lN=9.

mStatistical significance P<.05 in the comparison between Android and iOS app features.

nPercentage is computed by dividing the number of apps with distinguishable advertisements with the total number of apps with advertisements.

oN=24.

pN=17.

qN=7.

There were no significant differences between the Android and iOS platforms in the proportion of apps fulfilling each criterion except for the principle of transparency. Android apps had a significantly higher proportion of apps with the developer’s email listed on the Google play store compared with apps listed on the Apple store (Android: 75/81, 98%; and iOS: 52/62, 84%; P=.005). More than half (88/143, 61.5%) of the apps disclosed funding sources. Finally, most 119/143, 83.2%) of the assessed apps did not have advertisements; of apps with advertisements, three-fourths (18/24, 75%) were distinguishable from the content of the app.

Android Apps by Downloads

Table 3 shows the profile of app attribute grouped by a low (<100,000 downloads) and high number of downloads (≥100,000 downloads) for Android apps. There were no significant differences between apps with a low and high number of downloads in terms of the authoritative, complementarity, attribution, justifiability, transparency, and advertising policy. A significantly higher proportion of highly downloaded apps had a privacy and confidentiality clause (high downloads: 15/17, 88%; and low downloads: 33/64, 52%; P=.006) and were more likely to discuss their funding sources (high downloads: 15/17, 82%; and low downloads: 32/64, 50%;

Table 3. Profile of Android app attributes grouped by the number of downloads.
HONcodea principleApp-HONcode criteriaAll Android apps, n (%)<100,000 downloads, n (%)≥100,000 downloads, n (%)P value
AuthoritativeDoes the app indicate the qualifications of specific individuals who developed or contributed to the development of the app?9 (11)b8 (13)c1 (6)d.68e
ComplementarityIs there a disclaimer stating or which implies that the information provided and/or app functions do not replace the health care provider’s advice?27 (33)b18 (28)c9 (53)d.08
PrivacyIs there a privacy and confidentiality clause in the app?48 (59)b33 (52)c15 (88)d.006f
AttributionAre information sources in the app cited?8 (36)g,h7 (37)g,i1 (33.3)g,j>.99e
JustifiabilityAre the claims relating to benefits and performance in the app description backed up by evidence? (Answer not applicable if there are no claims)4 (29)g,k3 (30)g,l1 (25)g,m>.99e
TransparencyAre the developers contactable by email?79 (98)b63 (98)c16 (94)d.38
Financial disclosureDoes the app indicate any funding sources? (Yes if the app is managed by a registered commercial company)46 (57)b32 (50)c14 (82)d.026f
Advertising policyThere are no advertisements in the app64 (79)b50 (78)c14 (82)d>.99e
Are advertorials distinguishable from content of the app?12 (71)n,o11 (79)n,p1 (33)n,q.19n

aHONcode: Health On the Net code of conduct.

bN=81.

cN=64.

dN=17.

eTwo-tailed P value calculated using Fisher exact test, as the expected count is less than 5 in at least one group.

fStatistical significance P<.05 in the comparison between Android and iOS app features.

gNot applicable removed before statistical analysis and percentage computation.

hN=22.

iN=19

jN=3.

kN=14.

lN=10.

mN=4.

nPercentage is computed by dividing the number of apps with distinguishable advertisements with the total number of apps with advertisements.

oN=17.

pN=14.

qN=3.


Principal Finding

We evaluated 143 apps against 8 App-HONcode criteria to assess the transparency and reliability of information disseminated via diabetes apps. Most apps fulfilled between 2 and 5 assessment criteria, and only 1 Android app fulfilled all 8 criteria. Overall, a higher proportion of iOS apps fulfilled more App-HONcode criteria compared with Android apps, although the differences were not significant.

Many apps were not transparent in indicating the content source of the app. More than half of the assessed apps did not fulfill important criteria, such as indicating the qualifications of individuals involved in the app development and disclaiming that the app does not replace health care provider’s advice. This concurs with a study assessing eczema apps, where only 15% of the app developers indicated their qualifications, and 46% disclaimed that the app does not replace the advice of the health care provider [24]. Although it may be challenging to indicate the qualifications of all individuals involved in the development of a complex app, the qualifications of the main content contributors and a representation of their collaborators should minimally be quoted to account for the content source of the app.

Approximately three-fourths of the assessed apps did not provide any health information. This was not surprising, as disease management apps tend to emphasize management aspects rather than educating the patient [1], which, in our view, presents a missed opportunity for patient education, which can be incorporated into apps. Of the apps that provided health information, only one-third cited the source of information. Few of the assessed apps had claims relating to the benefits and performance of the app. However, only one-third of these apps backed the claims with evidence. Apps or any consumer products with unsubstantiated claims have the potential to mislead and cause harm to the undiscerning consumer [16,25]. Therefore, it is imperative for app stores to check the veracity of claims used in the app description before its publication on the app stores.

Most apps had an email of the developer displayed, but the email address does not guarantee access to the app developer. We contacted the developers of apps that had access restrictions, and only 10% responded to our emails (2 emails sent a week apart). This percentage should be higher for apps that are accessible, but there is a possibility of inoperative email addresses being displayed on the app store. App stores should ensure the inclusion of a valid email address for all health apps for consumer inquiries.

Privacy breaches can erode consumers’ trust in the app. Two-thirds of the apps assessed had a privacy and confidentiality clause. This was an improvement from a study published 6 years ago assessing the availability, scope, and transparency of mobile health (mHealth) app privacy policies on 600 commonly used mHealth iOS and Android apps [15]. One explanation for this improvement could be the changes made to the app store policies to improve the quality of apps over the years [26,27]. However, stricter scrutiny is required on the part of the app stores, given the absence of privacy policies in many of the assessed apps. Although there is an improvement in the presence of privacy policy of the English-language apps we assessed, those that are published on other platforms and in other languages may yield different assessment results.

Approximately 40% of the assessed apps did not have their funding sources indicated. The funding source of an app will affect its development, quality, and the services provided. This represents a gap in which app stores can play a role to improve the quality assurance of health apps. Advertisements were not present in 80% of the assessed apps. The proportion of apps with advertisements in our study may be lower, as we assessed the best version of apps requiring in-app payments for feature upgrades. Previous studies have shown that paid apps were not of higher quality compared with free apps [28,29]. Approximately one-third of the in-app advertisements were judged as being nondistinguishable from the content of the app. We did not scrutinize the appropriateness of the advertising content, which may present an additional gap in the quality and reliability of information disseminated via these apps.

The originators of the HONcode recently published an extension for apps—mHONcode—to cover the certification of health information disseminated on apps [30]. This was only available after we completed our app assessments, but there are minimal conceptual differences between our App-HONcode and the mHONcode. Our criteria were worded to minimize the subjectivity of assessment by different researchers.

Limitations

There were limitations to the study, despite attempts to minimize bias. First, the scope was limited to diabetes management apps because of being part of a larger study investigating the medication management features of diabetes apps. However, our findings are generalizable to other diabetes apps, as the apps were identified using a systematic search and selection strategy. Second, our assessment criteria may have underrated apps that do not provide health information (eg, medication logging apps). Even so, many apps were not transparent in data privacy and in clearly distinguishing the complementarity of the app (ie, not replacing the health care provider’s advice). Third, the assessment may not reflect the current state of the apps because of constant app updates. However, we believe that our findings remain unchanged, as app updates were mainly for bug fixes and feature upgrades. Fourth, we neither assessed issues surrounding data management nor the content of privacy and confidentiality clauses, which may not accurately disclose the sharing of some personal information [31]. Finally, app assessments were subjective to researcher interpretation, although we attempted to reduce researcher bias by piloting the assessment and using a standardized patient profile when interacting with apps.

Implications and Future Research

The fulfillment of the 8 App-HONcode criteria are actionable, but not many developers may be aware of the need to indicate background information or to check the content of advertisements, as their main aim is to get the app published. App developers and consumers would benefit from the availability of a standardized checklist to assess the information quality of health apps. It would be challenging for governments to regulate all health apps because of its ubiquity, pace of development, and ambiguity in definition. Therefore, app stores should play a larger role in making the transparency and reliability of information dissemination a basic requirement for app publication. As observed from our app assessments, owing to the higher barriers of entry currently set by the Apple app store (see Multimedia Appendix 1), it had a larger proportion of apps that fulfilled more App-HONcode criteria.

As health apps collect a lot more user data than the internet, the consent-seeking process for data collection and data management policies of these apps should be evaluated in the future. The appropriateness of advertising and clarity of privacy clauses should also be checked, for diabetes management and other chronic disease management apps in other languages and on other platforms, to provide a more complete landscape of the transparency and reliability of information disseminated and collected through these apps.

Conclusions

Our systematic app assessments of the transparency and reliability of health information disseminated in diabetes apps discovered gaps in the disclosure of the developer’s qualification, funding source, and the complementary role of the app in disease management. App stores should play a larger role in scrutinizing app publication, as higher barriers of app entry will lead to the publication of apps with better disclosure of the app’s content source. As the development of the App-HONcode criteria is preliminary, future work can further examine the consent-seeking process for data collection, data management policies, the appropriateness of advertising content, and the clarity of privacy clauses.

Acknowledgments

This study was funded by the Nanyang Technological University, Singapore, through (1) the Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore; (2) the Ageing Research Institute for Society and Education, Nanyang Technological University, Singapore; and (3) the Institute for Health Technologies, Nanyang Technological University, Singapore. The funder did not have direct involvement in the research including manuscript preparation, apart from employing the authors. The research presented in this paper is solely the responsibility of the authors and does not reflect the views of the funder. The authors wish to acknowledge Mr Geronimo Jimenez for his input in piloting and refining the app assessment criteria and Mr Geronimo Jimenez, Ms Christina Tan, and Ms My Linh Thai for their assistance in app assessment. ZH would like to acknowledge funding from the Nanyang Technological University Research Scholarship.

Authors' Contributions

ZH conceptualized and contributed to the development and refinement of the adapted HONcode app assessment criteria, screened apps, assessed apps, cleaned and analyzed data, interpreted data, and drafted and revised the manuscript. EL coconceptualized and provided critical input into the developed app assessment criteria, assessed apps, interpreted data, and revised the manuscript. JC conceptualized the study, obtained the funding, supervised the team, and provided critical input into all stages of the study and critical review of the draft manuscript. All authors reviewed and approved the final version of the manuscript before submission.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Selected pre-review app publication checklist of Apple and Google Play app stores.

DOCX File , 38 KB

  1. Huang Z, Soljak M, Boehm BO, Car J. Clinical relevance of smartphone apps for diabetes management: a global overview. Diabetes Metab Res Rev 2018 May;34(4):e2990. [CrossRef] [Medline]
  2. Dayer L, Heldenbrand S, Anderson P, Gubbins PO, Martin BC. Smartphone medication adherence apps: potential benefits to patients and providers. J Am Pharm Assoc (2003) 2013;53(2):172-181 [FREE Full text] [CrossRef] [Medline]
  3. West JH, Hall PC, Hanson CL, Barnes MD, Giraud-Carrier C, Barrett J. There's an app for that: content analysis of paid health and fitness apps. J Med Internet Res 2012 May 14;14(3):e72 [FREE Full text] [CrossRef] [Medline]
  4. Tran J, Tran R, White JR. Smartphone-based glucose monitors and applications in the management of diabetes: an overview of 10 salient 'apps' and a novel smartphone-connected blood glucose monitor. Clin Diabetes 2012;30(4):173-178. [CrossRef]
  5. IQVIA. 2017 Nov 7. The Growing Value of Digital Health: Evidence and Impact on Human Health and the Healthcare System   URL: https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health [accessed 2019-06-03] [WebCite Cache]
  6. Statista Research Department. Wayback Machine - Internet Archive. 2016. Number of Smartphone Users Worldwide From 2014 to 2020 (in Billions)   URL: https:/​/web.​archive.org/​web/​20190704081902/​https:/​www.statista.com/​statistics/​330695/​number-of-smartphone-users-worldwide/​ [accessed 2019-07-04]
  7. Food and Drug Administration. Wayback Machine - Internet Archive. 2015. Mobile Medical Applications: Guidance for Industry and Food and Drug Administration Staff   URL: https://web.archive.org/web/20190704082425/https://www.fda.gov/media/80958/download [accessed 2019-07-04]
  8. Cortez NG, Cohen IG, Kesselheim AS. FDA regulation of mobile health technologies. N Engl J Med 2014 Jul 24;371(4):372-379. [CrossRef] [Medline]
  9. Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry 2017 May 1;74(5):437-438. [CrossRef] [Medline]
  10. Bates DW, Landman A, Levine DM. Health apps and health policy: what is needed? J Am Med Assoc 2018 Nov 20;320(19):1975-1976. [CrossRef] [Medline]
  11. Huckvale K, Adomaviciute S, Prieto JT, Leow MK, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med 2015 May 6;13:106 [FREE Full text] [CrossRef] [Medline]
  12. Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med 2012 Nov 22;10:144 [FREE Full text] [CrossRef] [Medline]
  13. Chen J, Cade JE, Allman-Farinelli M. The most popular smartphone apps for weight loss: a quality assessment. JMIR Mhealth Uhealth 2015 Dec 16;3(4):e104 [FREE Full text] [CrossRef] [Medline]
  14. Wicks P, Chiauzzi E. 'Trust but verify'--five approaches to ensure safe medical apps. BMC Med 2015 Sep 25;13:205 [FREE Full text] [CrossRef] [Medline]
  15. Sunyaev A, Dehling T, Taylor PL, Mandl KD. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc 2015 Apr;22(e1):e28-e33. [CrossRef] [Medline]
  16. Ghaddar SF, Valerio MA, Garcia CM, Hansen L. Adolescent health literacy: the importance of credible sources for online health information. J Sch Health 2012 Jan;82(1):28-36. [CrossRef] [Medline]
  17. Zhang Y, Sun Y, Xie B. Quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results. J Assn Inf Sci Technol 2015;66(10):2071-2084. [CrossRef]
  18. Lee K, Hoti K, Hughes JD, Emmerton LM. Interventions to assist health consumers to find reliable online health information: a comprehensive review. PLoS One 2014;9(4):e94186 [FREE Full text] [CrossRef] [Medline]
  19. Health On the Net. 1995. HONcode Certification   URL: https://www.hon.ch/en/certification.html#principles [accessed 2019-07-04]
  20. Taki S, Campbell KJ, Russell CG, Elliott R, Laws R, Denney-Wilson E. Infant feeding websites and apps: a systematic assessment of quality and content. Interact J Med Res 2015 Sep 29;4(3):e18 [FREE Full text] [CrossRef] [Medline]
  21. Loy JS, Ali EE, Yap KY. Quality assessment of medical apps that target medication-related problems. J Manag Care Spec Pharm 2016 Oct;22(10):1124-1140 [FREE Full text] [CrossRef] [Medline]
  22. Huang Z, Lum E, Jimenez G, Semwal M, Sloot P, Car J. Medication management support in diabetes: a systematic assessment of diabetes self-management apps. BMC Med 2019 Jul 17;17(1):127 [FREE Full text] [CrossRef] [Medline]
  23. 42matters: Mobile App Intelligence.   URL: https://42matters.com/ [accessed 2019-07-04]
  24. van Galen LS, Xu X, Koh M, Thng S, Car J. Eczema apps conformance with clinical guidelines: a systematic assessment of functions, tools and content. Br J Dermatol 2019 Jun 9. [CrossRef] [Medline]
  25. Vladeck DC. Truth and Consequences: The Perils of Half-Truths and Unsubstantiated Health Claims for Dietary Supplements. J Public Policy Mark 2000;19(1):132-138. [CrossRef]
  26. Apple Developer. 2019. App Store Review Guidelines   URL: https://developer.apple.com/app-store/review/guidelines/ [accessed 2019-10-27]
  27. Salim S. Digital Information World. 2019 Apr 18. Google Tweaks Its Policy With Aim to Improve 'App Quality' in Play Store   URL: https:/​/www.​digitalinformationworld.com/​2019/​04/​google-android-policy-improve-app-quality-in-play-store.​html [accessed 2019-10-27]
  28. Middelweerd A, Mollee JS, van der Wal CN, Brug J, te Velde SJ. Apps to promote physical activity among adults: a review and content analysis. Int J Behav Nutr Phys Act 2014 Jul 25;11:97 [FREE Full text] [CrossRef] [Medline]
  29. Bondaronek P, Alkhaldi G, Slee A, Hamilton FL, Murray E. Quality of publicly available physical activity apps: review and content analysis. JMIR Mhealth Uhealth 2018 Mar 21;6(3):e53 [FREE Full text] [CrossRef] [Medline]
  30. Health On the Net. mHONcode: The New Certification of Health Mobile Applications   URL: https://www.hon.ch/en/certification/app-certification-en.html [accessed 2019-07-04]
  31. Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open 2019 Apr 5;2(4):e192542 [FREE Full text] [CrossRef] [Medline]


FDA: Food and Drug Administration
HONcode: Health On the Net code of conduct
mHealth: mobile health


Edited by G Eysenbach; submitted 04.07.19; peer-reviewed by E Sezgin, S Baptista, T Hussain, F Fatehi; comments to author 21.10.19; revised version received 29.10.19; accepted 17.12.19; published 19.02.20

Copyright

©Zhilian Huang, Elaine Lum, Josip Car. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 19.02.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.