Background: The widespread adoption of smartphones provides researchers with expanded opportunities for developing, testing and implementing interventions. National Institutes of Health (NIH) funds competitive, investigator-initiated grant applications. Funded grants represent the state of the science and therefore are expected to anticipate the progression of research in the near future.
Objective: The objective of this paper is to provide an analysis of the kinds of smartphone-based intervention apps funded in NIH research grants during the five-year period between 2014 and 2018.
Methods: We queried NIH Reporter to identify candidate funded grants that addressed mHealth and the use of smartphones. From 1524 potential grants, we identified 397 that met the requisites of including an intervention app. Each grant’s abstract was analyzed to understand the focus of intervention. The year of funding, type of activity (eg, R01, R34, and so on) and funding were noted.
Results: We identified 13 categories of strategies employed in funded smartphone intervention apps. Most grants included either one (35.0%) or two (39.0%) intervention approaches. These included artificial intelligence (57 apps), bionic adaptation (33 apps), cognitive and behavioral therapies (68 apps), contingency management (24 apps), education and information (85 apps), enhanced motivation (50 apps), facilitating, reminding and referring (60 apps), gaming and gamification (52 apps), mindfulness training (18 apps), monitoring and feedback (192 apps), norm setting (7 apps), skills training (85 apps) and social support and social networking (59 apps). The most frequently observed grant types included Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) grants (40.8%) and Research Project Grants (R01s) (26.2%). The number of grants funded increased through the five-year period from 60 in 2014 to 112 in 2018.
Conclusions: Smartphone intervention apps are increasingly competitive for NIH funding. They reflect a wide diversity of approaches that have significant potential for use in applied settings.
Public Health Interventions
Through its various federal agencies, the US government has evolved a broad public health mandate to prevent disease and improve health. Part and parcel of this mandate is to find creative and innovative ways to enhance the public’s health and reduce the burdens of disease and disability. In recent years, the National Institutes of Health (NIH) and its 27 Institutes and Centers have searched for promising ways to capitalize on the public’s use of mobile phone technology to service its mission. Otherwise known as mobile health (mHealth), the use of cellular and smartphone technology has provided a platform to extend the reach of health promotion and disease prevention activities. We differentiate mHealth technology, which is defined as “mobile communications for health information and services” from electronic health (eHealth), the latter which primarily involves the use of the Internet for delivery of behavioral and health-related interventions [- ]. Mobile communication can include, but not be limited to, smartphones, tablets, patient monitoring devices, personal digital assistants, and bodily or household sensors that rely on wireless communication. Given the widespread presence of mobile technology, potential users are no longer bound by periodically showing up for clinic visits and the inconvenience of waiting rooms and stalled scheduling. The monitoring and reporting of behavior, tracking events that precipitate disease conditions, or monitoring adherence can all be conducted in real-time, and a wide assortment of pertinent information can be transported electronically to medical records or even stored in comprehensive research databases.
Several systematic reviews and meta-analyses have reinforced the utility of mobile phones and smartphones (including wireless hand-held tablet devices and cellular phones) in a variety of health domains. These include application of mHealth technology in care delivery and chronic disease management [, ] with specific applications monitoring hypertension [ , ], medication adherence support [ ], obesity and weight loss [ ], diabetes mellitus [ ], substance use prevention [ , ], mental health [ - ] and physical activity [ , ], to name a few. Apps have also been developed for asthma self-care management [ ], smoking cessation [ ], maternal and neonatal care [ ], and stress reduction using mindfulness strategies [ ], all of which include provision of biofeedback [ ]. Additional reviews have focused more exclusively on mobile apps targeting diseases identified by the WHO as major global health priorities [ ]. For the most part, these reviews support promising results with potential healthcare benefits for a wide selection of evidence-based mobile and smartphone app interventions.
There is a considerable upside to the plethora of mobile phone apps now available. According to the trade organization GSMA Intelligence, which tracks trends in mobile phone usage worldwide, cell phone use worldwide has topped five billion, with roughly 67% market penetration and 3.2% annual projected growth . In the United States alone, there are over 262 million mobile phone subscribers [ ]. Increases in mobile phone usage are slated for the traditionally developing or low-income countries where expansion of mobile phone infrastructure (ie, 4G networks, cell transmission towers, and LTE broadband transmission) is taking place [ ]. Using digital distribution platforms like Apple’s AppStore for iOS products or Google’s Google Play for Android products, approximately 1500 apps are downloaded daily with 178 billion downloaded in 2017 and 205 billion in 2018 [ ]. A mobile app industry tracking company reports there are 318,000 health-related mobile phone apps with approximately 200 new ones added daily [ ]. Based on Apple Store statistics, health and fitness apps represented 3.0% of the total downloads, lifestyle represented 8.3% of the downloaded apps, medical was 1.8% and the leading app download category was games at 24.9%.
More importantly, availability of free and low-cost health-related applications has dramatically increased as well. FitBit, a digital healthcare company, is the leading fitness tracking app among those labeled as wireless-enabled wearable sensor technology and that are used specifically for monitoring health and vitals. The free, downloadable app monitors health and vitals like heart rate, pulse, sleep quality, and calories burned, and can both accurately measure steps using a pedometer, as well as monitor activity through an accelerometer-based motion sensor. A new sector is growing to handle digital biomarkers that can instantly regulate a patient’s insulin intake or monitor their breathing capacity (ie, smart inhalers) and report this to clinical providers. The availability of apps in general as well as the increased market for health-related apps all speaks volumes regarding demand. One barometer of this demand is revenue from global mobile app sales. In 2014, revenues from mobile app sales hovered around 35 billion US dollars. In 2018, these sales skyrocketed to 92 billion US dollars  with the duo of Samsung and Apple sharing the lion’s share of revenues at 50%. This number is projected to reach 188.9 billion US dollars in 2020.
The use of mobile phone apps for healthcare piggybacks onto several emerging trends. Smartphone sales have skyrocketed in the past decade, and with their advanced features they now practically function like a handheld computer. This is due to improved technological capabilities which include wireless connectivity allowing Internet access, faster chip processors and download speeds, longer battery life, finer digital pixilation, and multimedia graphic imaging capability, all at a lower cost to subscribers. Freeware apps like Skype, Fring, and iCall utilize Voice of Internet Protocol (VoIP) to achieve real-time synchronous remote teleconferencing (ie, video chat). The global positioning system (GPS) capability enables remote location monitoring, making it possible to monitor disease vectors as part of public health surveillance [, ], to track physical activity in cardiac rehabilitation [ ] and to find dementia patients who wander outside of care facilities [ ].
Smartphone camera technology has improved dramatically, and with higher resolution digital imaging it can now enable bio-optical sensing and real-time synchronous sharing of pictures of patients’ skin with their dermatologist or sharing of medical conditions with healthcare staff for diagnosis, treatment and remediation. Connectivity to cloud computing allows physicians, for example, to exchange laboratory or radiographic images for medical consultations using servers that are compliant with data privacy regulations . More and more, even in resource limited countries, mobile technology provides a bridge to healthcare connectivity, making it possible for patients (or their families) to receive educational materials, monitor their health, track symptoms or conduct self-assessments [ ].
Smartphone apps allow users to connect with their healthcare provider and share patient experiences as well as receive individualized instruction by transmitting data in real-time. This may help individuals overcome resource limitations that may arise from the large distances between their residence and their provider (access to care) and structural barriers that arise from cultural or behavioral factors that limit care provision. There are now numerous projects underway supporting public health surveillance in Sub-Saharan Africa that are leveraging mobile health-related apps [, ] to increase treatment adherence, improve delivery of care [ ], and monitor communicable diseases (ie, malaria) and other health problems (eg, child malnutrition). Even physician providers themselves are using mobile apps to learn about care delivery in remote areas of Africa [ ], and this is part of an ongoing, worldwide trend for health care professionals who require greater access to point of care tools for decision making [ ]. The burgeoning use of mobile phones for provision of healthcare is coupled with what is termed lab-on-a-chip technology that allows biochemical and diagnostic assays (eg, blood glucose monitoring or cholesterol testing) to be performed with sweat, saliva, urine or blood using the smartphone in vivo [ ]. In all these examples, the portability, ease of use, flexibility, customization, convenience, and privacy of information was highly attractive to users.
Missing Pieces of the Mobile Phone Healthcare Puzzle
What is missing from this picture, however, is a deeper understanding of the organization of mobile apps, in particular whether there are common themes underlying mobile app intervention strategies (ie, treatment adherence, behavior modification, appointment reminders, and data collection using self-reporting or remote bio-sensors for symptom tracking), service delivery strategies (ie, platforms used to deliver interventions), the medical and behavioral conditions that apps emphasize, their intervention goals, integration with other service platforms (ie, web or in person), and characterization of the target populations, all of which can help clarify the event horizon. To our knowledge, this type of summarization has not been previously conducted, with the few research-based systematic reviews of mHealth applications highly topic dependent (ie, focusing on at most one disease or health focus with a specific target population). It is particularly relevant to understand the US government’s scientific research investment portfolio, which through various grant mechanisms supports app development that specifically addresses the nation’s stated public health agenda . Therefore, in the current study, we undertake the synthesizing of NIH-funded mobile phone apps between 2014 and 2018 based on a set of organized criteria that will inform the future of public health initiatives using mHealth strategies.
We searched NIH Reporter  using the following key terms in the text search field: (1) mHealth or smartphone or mobile or phone or Android or iOS or game or gaming or gamification and (2) intervention or treatment or randomized or RCT or program. We limited the search period to the five fiscal years of 2014 through 2018. At times, insufficient information was available on the NIH Reporter that made it hard to detect whether an app was being used in the research. This prompted us to widen the search engines (Google) and to include Grantome.com (an independent resource for searching for grant awards) as a means of locating additional information. In some cases, we used Google Scholar to track down publications, when available, that could clarify the research focus and utilization of a true app.
We completed this search for each of the following agencies:
- National Institute on Alcohol Abuse & Alcoholism (NIAAA)
- National Institute on Aging (NIA)
- National Institute of Allergy and Infectious Diseases (NIAID)
- National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS)
- National Institute on Drug Abuse (NIDA)
- National Institute on Deafness and other Communication Disorders (NIDCD)
- National Institute of Dental and Craniofacial Research (NIDCR)
- National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)
- National Institute of Biomedical Imaging and Bioengineering (NIBIB)
- National Institute of Environmental and Health Sciences (NIEHS)
- National Eye Institute (NEI)
- National Institute of General Medicine Sciences (NIGMS)
- National Institute of Child Health and Human Development (NICHD)
- National Human Genome Research Institute (NHGRI)
- National Heart, Lung, and Blood Institute (NHLBI)
- National Library of Medicine (NLM)
- National Institute on Minority Health and Health Disparities (NIMHD)
- National Institute of Mental Health (NIMH)
- National Institute of Nursing Research (NINR)
- National Institute of Neurological Disorders and Stroke (NINDS)
For each funded grant, we retrieved the abstract, title, start date, activity code, and first year funding amount. We subsequently retained only the following grant activity codes: R01, R03, R15, R21, R34, R35, R41, R42, R43, R44, R56 and R61. We excluded all D, F, K, O, P, S, U and Z awards as well as R13, R25, R36, R37 and R90 awards. This left us with the bulk of peer reviewed research grant mechanisms that included technology-related research .
For each grant we reviewed the abstract, the only publicly available description of the project that might elucidate the content and methods. We first eliminated grants that did not have a planned intervention. We also eliminated grants that only used standard features of smartphones, such as SMS or text messaging, although if there were added features that were novel in any way those grants were retained.
We read through each abstract and provided brief summaries about: (1) the topic of the app (eg, the specific disease or behavior addressed); (2) the specific strategy employed by the app (ie, how the app was to be used for intervention); (3) the intended outcome (ie, goals of the intervention); (4) whether the app was applied to a standalone intervention or used in an ancillary way in conjunction with other forms of care; (5) the age group targeted by the app; (6) the population on which it would be tested or used; and (7) the name of the app.
Review proceeded agency by agency. Following an initial review of specific strategies employed in four behavior focused (NIAAA, NICHD, NIDA and NIMH) and four disease focused agencies (NCI, NIDDK, NHLBI and NIA), we developed a tentative list of categories that summarize the kinds of interventions that we noted in the abstract review process. When a subsequently reviewed grant had an approach that could not be easily accommodated within the existing list of categories, we added a new category. For each grant, the proposed categories were noted as present or not present. Single grants could include multiple categories. All grants included at least one category.
In the context of smartphone apps, artificial intelligence implies the use of embedded computational algorithms that give the user guidance for making ideal or maximally satisfying decisions . The user provides input either by defining situations and selecting options provided by the app or through data provided from wearable devices. The intent is to guide the user in taking next steps toward enhancing prevention or engaging in treatment [ ]. Included in this category are apps that feature tailoring to meet an individual’s personalized needs.
This category of apps included grants that intend to develop smartphone assisted or adaptive technologies and prosthetic devices to compensate for a variety of biological limitations or disease conditions. Conceivably, bionic adaptations can be used to compensate for challenges with vision [, ], vestibular balance [ , ], speech and hearing [ , ], and the use of artificial limbs [ , ]. The goal is often to replace a missing component of normal human functioning or enhance a component that has been damaged due to aging, illness or injury. Included in this category are wearable and injected devices that communicate wirelessly with smartphones.
Cognitive and Behavioral Therapies
This category includes a wide variety of interventions. Among these, cognitive behavior therapy (CBT) involves treatments that seek to change behavior by modifying how people process self-relevant information . The underlying assumption is that people have dysfunctional thoughts and that these thoughts lead them to behave in maladaptive ways that can be corrected with supportive counseling. The CBT approach seeks to challenge a person’s assumptions and how they interpret events around them. The goal of CBT is to replace negative or self-defeating beliefs, thoughts, and feelings with positive and self-enhancing cognitions. Therapeutic treatments involving CBT typically train individuals to self-monitor and identify harmful thoughts as well as eliminate distorted cognitions (eg, unnecessary worry) that lead to dysfunctional behaviors. Core features include developing problem solving and coping strategies, anxiety reducing techniques, progressive relaxation, cognitive restructuring, activity scheduling, and other skills training approaches (eg, assertiveness) that reduce the influence of unhelpful thinking patterns. An added approach, behavioral activation (BA) [ ] specifically addresses scheduling positive activities. We categorized grants as having cognitive and behavioral therapy components even if CBT or BA were not mentioned, but cognitive interventions falling under the CBT rubric were addressed.
This generally refers to interventions that provide overt consequences, typically in the form of rewards, when behavior standards are met [, ]. Apps that address contingency management often provide some form of monetary reward. Using rewards is most common in cases where app users are required to forego behavior, such as in treating different forms of addiction [ ], or users want to promote a behavior, such as physical exercise among sedentary individuals [ ] or individuals recovering from surgery [ ]. The provision of financial incentives (ie, the carrot and stick approach in behavioral economic theory), suggests that human beings are rational agents that will deliberate consciously when faced with a decision, and if offered a valued incentive, change their behavior [ - ].
Education and Information
Apps in this category provide information and serve to educate patients about their disease, condition, or disorder. One of the primary aspects that characterize apps in this category is providing access to research on the disorder in a manner that is acceptable and understandable to a lay audience . There may be a variety of reasons for providing this information, including increasing understanding of and countering misperceptions about the course of a disease [ - ], increasing beliefs about negative consequences of inaction or dysfunctional action [ , ] and increasing beliefs that taking proper action will be beneficial [ , ].
This merits an umbrella classification because it involves interventions that attempt to increase a user’s inherent desire to behave in a manner that improves health behaviors. Included in this rubric is motivational interviewing (MI) [- ] as well as a variety of other strategies. The focus of motivational enhancement is to increase a person’s desire to align their behaviors with their primary values and reduce their ambivalence through a focus on the pursuit of goal directed activities. This approach assumes that people can articulate their thinking about goals and aspirations. Aligning their behavior with goals, priorities and values helps individuals reduce cognitive dissonance [ ]. Thus, instead of focusing on changing or developing new cognitions, motivational approaches capitalize on and strengthen existing cognitions. Apps that focus on motivation often focus on helping users remember their overarching life goals and engage in goal setting activities.
Facilitating, Reminding, and Referring
These smartphone apps provide a means for users to connect with resource providers, primarily clinicians and testing facilities, and they address improving access to healthcare [, ]. This can involve finding resources as well as providing reminders and scheduling alerts to assist users in remembering appointments, when to take medication [ ] and finding relevant resources they need [ , ].
Gaming and Gamification
This category of apps involves users in some form of imaginative (ie, virtual simulation) and competitive play [- ]. The goal is to provide a basis for active learning and foster skill development. This can include stealth learning, where the user is not entirely aware of the game logic but acquires new cognitive skills while playing the game [ , ]. This is typically done irrespective of directly providing instruction. Rather, users are expected to learn through the experiences provided by the game’s design. Games include a variety of formats including timed challenges, simulations, and exercises in which users earn points (ie, rewards) through correct responses.
An alternative cognitive approach that encourages individuals to be introspective and pay attention to moment by moment experiences and heed one’s inner voice [- ]. The goal of mindfulness training in smartphone apps is to heighten users’ awareness, encourage them to live in the moment and be aware of their surroundings in a more relaxed and contemplative state of mind [ ]. Meditation is a core feature of mindfulness training, using deep relaxation and breathing exercises to accompany skills and strategies that target defeating negative thoughts and reducing temptations to overreact to nonessential stimuli.
Monitoring and Feedback
This a broad catchall for apps that collect biological data (from sources such as saliva, blood or urine), activity data (from telemetry or accelerometer devices that detect movement or location) or detect vital signs (involving sensors), as well as self-report data (from ecological momentary assessments). Apps that are linked with wireless wearable devices or subcutaneous embedded sensors [- ] are also included. In part, the value of monitoring is that, without adequate information, errors in diagnosis and treatment may occur [ ]. Thus, there is a clear benefit to the clinician who intends to understand how the patient responds or acts when not present in the clinic.
A second value of monitoring is that it creates opportunities for clinic staff to keep patients and other non-clinical parties informed and provide them feedback. Many apps address monitoring as well as provide the app user feedback about their performance, such as adherence to medication and prescribed health promotion routines based on the monitoring just received. Indeed, without monitoring, feedback would not be possible, so it became a corollary activity of these apps. Research has demonstrated the potential that positive feedback can have on health outcomes [, ].
These apps draw from the literature in which these norm setting approaches have been tried with adolescents  and college-age populations [ , ]. The hypothesis underlying these approaches is that many people overestimate the prevalence and acceptability of negative behaviors, such as alcohol and drug use. Interventions reduce risk by providing accurate information about the prevalence and unacceptability of behaviors among referent peer groups. Correcting misperceptions regarding social acceptability and prevalence also serves to modify expectancies about perceived benefits that underlie engaging in risky behavior.
Apps in this category tend to focus on stress and emotional self-regulation. These apps provide cues on how to respond to challenging situations. Apps often focus on pain, anxiety, stress and emotion management [, ]. Skills taught may also include how to be assertive and refuse unwanted offers to participate in risky behaviors [ ]. These apps walk users through different strategies they can apply when handling challenging situations. Examples of skills frequently taught include general life skills such as decision making, goal setting and communication skills.
Social Support and Social Networking
This category of apps provides ways to link a patient or app user to family and friends . These apps may provide access to a network of individuals similar to the user who share similar health conditions and who can provide support and information. The implied effect by increasing support and including a user’s social network is that there will be both an empathetic response and the potential for those in the network to provide some type of instrumental support or assistance. Apps that fall under this broad rubric are in part driven by the literature linking social support with better health outcomes [ , ].
Based on the search criteria, a total of 1524 grants met the activity code requirements. After reading the abstract for each grant to verify that a smartphone intervention app was indeed proposed, and that enough detail was provided to categorize the intervention, the resulting qualifying pool included 397 grants in the final sample.
The most common activity codes represented were Research Project Grants (R01) (103 grants; 25.9%) and R43 Phase I Small Business Innovation Research (SBIR) grants (also 104 grants; 26.2%). Together SBIR (R43 and R44) and Small Business Technology Transfer (STTR) (R41 and R42) grants accounted for 162 (40.8%) of the grants. Two categories of exploratory grants, R21s (70 grants; 17.6%) and R34s (37 grants; 9.3%), were also prominent. R03 (10 grants), R15 (5 grants), R56 (5 grants), R61 (3 grants), R35 (1 grant) and R37 (1 grant) together accounted for 6.2% of the grants.
summarizes the number of grants that met the selection criteria and shows the total first grant year dollars awarded for these grants for the years 2014 through 2018 (dollars are not adjusted for inflation.) Total funding for the first year of projects increased steadily and the percentage these grants represent doubled throughout the five-year period. While this represents a sizable investment in absolute dollars, funding for smartphone intervention apps was a miniscule part of overall NIH funding for the agencies that sponsored this research.
|Year||Number of Grants||Total Dollars ($)||Average Grant ($)||Agency Funding, %|
All NIH agencies reviewed, except for NHGRI, contributed projects to our sample.presents the number of smartphone intervention app grants awarded by each NIH agency. NIA awarded a large number of grants (76). Several other agencies were also prolific in awarding grants for developing and testing smartphone intervention apps: NCI (47), NIMH (41), NIDDK (37), NIDA (31) and NHLBI (31). During any given year, the size of an agency’s budget was not significantly associated with how many smartphone intervention app grants were funded (r=.018).
The next step involved coding each grant’s abstract based on the 13 intervention categories. All grants could be coded based on single or multiple categories, should such evidence exist. In the 397 grant applications, the number of categories of intervention ranged from 1 to 5 with a mean of 1.99. Most grants proposed either one (139 grants, 35.0%) or two (155 grants, 39.0%) intervention approaches.
|Agency||Grants||Grants Awarded, %||Total Funding ($)||Average Grant ($)|
|Content Category||Number of Apps||Total Apps, %||Rank|
|Cognitive and Behavioral Therapies||68||17.1||4|
|Education and Information||85||21.4||3|
|Facilitating, Reminding and Referring||60||15.1||5|
|Gaming and Gamification||52||13.1||8|
|Monitoring and Feedback||192||48.4||1|
|Social Support and Social Networking||59||14.9||6|
summarizes the relative emphasis placed on the types of intervention approaches that were coded. By far, the most commonly included intervention approach provided monitoring and feedback to targeted individuals (48.4%). In 15.7% of cases, monitoring and feedback apps included some form of a wearable device that provided data that was transmitted through the smartphone. Also included were apps that used geo-sensors that could relay information about a user’s location as well as self-report functions. Contingency management that would involve some form of monetary reward or incentive was much less frequently observed but was always tied to monitoring. Providing education and information (21.4%) and separately providing skills training (21.4%) were each represented in about one in five proposed apps. Apps that intended to provide cognitive and behavior therapies (17.1%), facilitating, reminding and referring (15.1%), social support and opportunities for joining social networks (14.9%), enhancing a user’s motivation to improve their behaviors or comply with prescribed regimens (12.6%) and apps that proposed to use some form of game (13.1%) were also observed with some frequency. On the rare side were apps that focused on norm setting (1.8%) and mindfulness training (4.5%). Bionic adaptation apps that proposed linking smartphones to prosthetic devices were also relatively rare (8.3%).
It is worth emphasizing that all abstracts represented proposed projects that had undergone rigorous peer review and been funded. NIH funds anywhere between 10% and 20% of submitted applications on an annual basis. Successfully competing for NIH funding therefore provides some assurance that the abstracts reviewed represent the state of the science.
Smartphone intervention apps are increasing in popularity among researchers being funded by NIH. The increase in numbers of grants funded between 2014 and 2018 attests to the belief in the potential for smartphone technology to be useful in health promotion and disease prevention. It is conceivable that this increase is due primarily to the increased technology capability that smartphones offer, their broad reach, ease of use by potential users, flexibility, and increasing ease of app programming.
The current review found that each NIH agency’s research agenda was matched with possible smartphone intervention apps that addressed their programmatic goals. The behaviorally focused agencies (eg, NIAAA, NICHD, NIDA and NIMH) funded grants that emphasized psychosocial interventions. However, disease focused agencies also included funding for smartphone intervention app projects that addressed behavioral concerns, notably focusing on monitoring and feedback. The diversity of app categories and intervention strategies was interesting given that we identified 13 discrete intervention categories in comparison to the four categories suggested by Abraham and Michie . Their categories included: (1) adherence and remote monitoring; (2) remote dissemination of information; (3) data collection and disease outbreak surveillance; and (4) diagnostic treatment and support. Their classification scheme was specific to physical activity and dietary interventions and not extended to a broad variety of smartphone apps.
There are yet many approaches to intervention that are currently less robustly represented in the current taxonomy. For example, smartphones have significant potential to deliver interventions that can augment participant engagement through gamification. In the grants reviewed, gamification (13.1%) was most often included with grants that also addressed skills training (34.6%), monitoring and feedback (26.9%), and education and information (25.0%). Less frequently it was coupled with cognitive and behavioral therapies (17.3%), contingency management (11.5%), social support and social networking (11.5%), artificial intelligence (7.7%), bionic adaptation (5.8%), enhanced motivation (7.7%), mindfulness training (1.9%) and facilitating, reminding and referring (1.9%). It was never coupled with norm setting. This coupling of strategies may indicate the desire to utilize fun gaming logic with other active program ingredients. Conceivably, strategies could be more frequently coupled to more robustly utilize the gamification potential of smartphones.
Grant review has been noted as fostering conservatism in its outcomes , with relative ease in providing fundable scores to traditional topics and difficulty in assigning equally meritorious scores to novel approaches. When examined from this point of view, norm setting and mindfulness training were the least utilized approaches for interventions. Perhaps these may represent newer approaches for which researchers have not yet successfully competed for grant funding.
The creation of a categorization scheme is intended to provide a simplified structure by which topics covered can be grouped for analysis. NIH funded grants are truly multifarious, covering a wide array of diseases and conditions that are addressed through an equally diverse set of interventions. The categorization scheme presented reflects both our understanding of the underlying theoretical and practical assumptions investigators used when crafting their grant applications. The diversity of wording used in abstracts, even within a topic area, presents a challenge to anyone who would attempt to simplify an entire field. We note especially that there were many cases where abstracts were somewhat vague about the approach being proposed. Nonetheless, the resulting classification system provides an initial way in which intervention types may be characterized and meets the needs of our analysis.
NIH grant summaries and abstracts presented in NIH Reporter are the only freely available descriptions of grant applications that can be accessed without making a freedom of information request. They are abbreviated descriptions that, under current guidelines, are limited to 30 lines of text. As a result, they often lack sufficient detail that would otherwise be found in a grant’s research plan section. Because of this, it is likely that details about interventions in an abstract may be missing or underspecified. Nonetheless, abstracts provided enough information for the purpose of the current paper to construct a general picture of the state of the science in funded grants.
Our review was limited to examining funded grants and not the outcomes of these grants. Future research will need to examine the strategies proposed to determine if researchers’ planned interventions were successful at achieving intended results.
Our classification results are publicly available for download .
Our review of NIH-funded abstracts over a five-year timeframe suggests that there is growing interest in using smartphone apps as either standalone or auxiliary components of health promotion and disease, as well as injury prevention and treatment. While smartphone intervention apps being developed with NIH grants are no longer novel, there is still great opportunity for innovation and rigorous science to provide a body of evidence-based strategies.
Conflicts of Interest
- Nacinovich M. Defining mHealth. Journal of Communication in Healthcare 2013 Jul 18;4(1):1-3 [FREE Full text] [CrossRef]
- Eysenbach G. What is e-health? J Med Internet Res 2001;3(2):E20 [FREE Full text] [CrossRef] [Medline]
- Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. J Med Internet Res 2005 Feb 24;7(1):e1 [FREE Full text] [CrossRef] [Medline]
- Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth). Annu Rev Clin Psychol 2007;3:53-76 [FREE Full text] [CrossRef] [Medline]
- Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [CrossRef] [Medline]
- Beratarrechea A, Lee A, Willner J, Jahangir E, Ciapponi A, Rubinstein A. The impact of mobile health interventions on chronic disease outcomes in developing countries: a systematic review. Telemed J E Health 2014 Jan;20(1):75-82 [FREE Full text] [CrossRef] [Medline]
- Andre N, Wibawanti R, Siswanto B. Mobile Phone-Based Intervention in Hypertension Management. Int J Hypertens 2019;2019:9021017 [FREE Full text] [CrossRef] [Medline]
- Morawski K, Ghazinouri R, Krumme A, Lauffenburger J, Lu Z, Durfee E, et al. Association of a Smartphone Application With Medication Adherence and Blood Pressure Control: The MedISAFE-BP Randomized Clinical Trial. JAMA Intern Med 2018 Dec 01;178(6):802-809 [FREE Full text] [CrossRef] [Medline]
- Park L, Howie-Esquivel J, Dracup K. A quantitative systematic review of the efficacy of mobile phone interventions to improve medication adherence. J Adv Nurs 2014 Sep;70(9):1932-1953 [FREE Full text] [CrossRef] [Medline]
- Bacigalupo R, Cudd P, Littlewood C, Bissell P, Hawley M, Buckley Woods H. Interventions employing mobile technology for overweight and obesity: an early systematic review of randomized controlled trials. Obes Rev 2013 Apr;14(4):279-291 [FREE Full text] [CrossRef] [Medline]
- Bonoto B, de Araújo VE, Godói IP, de Lemos LLP, Godman B, Bennie M, et al. Efficacy of Mobile Apps to Support the Care of Patients With Diabetes Mellitus: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. JMIR Mhealth Uhealth 2017 Mar 01;5(3):e4 [FREE Full text] [CrossRef] [Medline]
- Kapitány-Fövény M, Vagdalt E, Ruttkay Z, Urbán R, Richman M, Demetrovics Z. Potential of an Interactive Drug Prevention Mobile Phone App (Once Upon a High): Questionnaire Study Among Students. JMIR Serious Games 2018 Dec 04;6(4):e19 [FREE Full text] [CrossRef] [Medline]
- Tofighi B, Chemi C, Ruiz-Valcarcel J, Hein P, Hu L. Smartphone apps targeting alcohol and illicit substance use: systematic search in commercial app stores and critical content analysis. JMIR mHealth and uHealth 2019 Jul:e11831 [FREE Full text] [CrossRef]
- Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013 Nov 15;15(11):e247 [FREE Full text] [CrossRef] [Medline]
- Luxton D, Kayl R, Mishkind M. mHealth data security: the need for HIPAA-compliant standardization. Telemed J E Health 2012 May;18(4):284-288 [FREE Full text] [CrossRef] [Medline]
- Luxton D, McCann R, Bush N, Mishkind M, Reger G. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Professional Psychology: Research and Practice 2011 Dec;42(6):505-512 [FREE Full text] [CrossRef]
- Bort-Roig J, Gilson N, Puig-Ribera A, Contreras R, Trost S. Measuring and influencing physical activity with smartphone technology: a systematic review. Sports Med 2014 May;44(5):671-686 [FREE Full text] [CrossRef] [Medline]
- Direito A, Dale L, Shields E, Dobson R, Whittaker R, Maddison R. Do physical activity and dietary smartphone applications incorporate evidence-based behaviour change techniques? BMC Public Health 2014 Jun 25;14:646 [FREE Full text] [CrossRef] [Medline]
- Tinschert P, Jakob R, Barata F, Kramer J, Kowatsch T. The Potential of Mobile Apps for Improving Asthma Self-Management: A Review of Publicly Available and Well-Adopted Asthma Apps. JMIR Mhealth Uhealth 2017 Aug 02;5(8):e113 [FREE Full text] [CrossRef] [Medline]
- Whittaker R, McRobbie H, Bullen C, Rodgers A, Gu Y. Mobile phone-based interventions for smoking cessation. Cochrane Database Syst Rev 2016 Apr 10;4:CD006611 [FREE Full text] [CrossRef] [Medline]
- Sondaal S, Browne J, Amoakoh-Coleman M, Borgstein A, Miltenburg A, Verwijs M, et al. Assessing the Effect of mHealth Interventions in Improving Maternal and Neonatal Care in Low- and Middle-Income Countries: A Systematic Review. PLoS One 2016;11(5):e0154664 [FREE Full text] [CrossRef] [Medline]
- Mani M, Kavanagh D, Hides L, Stoyanov S. Review and Evaluation of Mindfulness-Based iPhone Apps. JMIR Mhealth Uhealth 2015 Aug 19;3(3):e82 [FREE Full text] [CrossRef] [Medline]
- Dillon A, Kelly M, Robertson I, Robertson D. Smartphone applications utilizing biofeedback can aid stress reduction. Frontiers in psychology 2016 Jul:832 [FREE Full text] [CrossRef]
- Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. Mobile health applications for the most prevalent conditions by the World Health Organization: review and analysis. J Med Internet Res 2013 Jun 14;15(6):e120 [FREE Full text] [CrossRef] [Medline]
- GSMA Intelligence. 2017. Global Mobile Trends 2017 URL: https://www.gsmaintelligence.com/research/2017/09/global-mobile-trends-2017/639/2019 [accessed 2019-04-28]
- Siedner MJ, Lankowski A, Musinga D, Jackson J, Muzoora C, Hunt PW, et al. Optimizing network connectivity for mobile health technologies in sub-Saharan Africa. PLoS One 2012;7(9):e45643 [FREE Full text] [CrossRef] [Medline]
- Statista. 2019. Number of mobile app downloads worldwide in 2017, 2018 and 2022 (in billions) URL: https://www.statista.com/statistics/271644/worldwide-free-and-paid-mobile-app-store-downloads/2019 [accessed 2019-05-06]
- IQVIA. 2017 Nov. The Growing Value of Digital Health: Evidence and Impact on Human Health and the Healthcare System URL: https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health [accessed 2019-05-06]
- Newzoo. Newzoo. 2018 Sep. Newzoo’s 2018 Global Mobile Market Report: Insights into the World’s 3 Billion Smartphone Users URL: https://newzoo.com/insights/articles/newzoos-2018-global-mobile-market-report-insights-into-the-worlds-3-billion-smartphone-users/2018 [accessed 2019-05-06]
- Brinkel J, Krämer A, Krumkamp R, May J, Fobil J. Mobile phone-based mHealth approaches for public health surveillance in sub-Saharan Africa: a systematic review. Int J Environ Res Public Health 2014 Nov 12;11(11):11559-11582 [FREE Full text] [CrossRef] [Medline]
- Rajput Z, Mbugua S, Amadi D, Chepngeno V, Saleem J, Anokwa Y, et al. Evaluation of an Android-based mHealth system for population surveillance in developing countries. J Am Med Inform Assoc 2012;19(4):655-659 [FREE Full text] [CrossRef] [Medline]
- Worringham C, Rojek A, Stewart I. Development and feasibility of a smartphone, ECG and GPS based system for remotely monitoring exercise in cardiac rehabilitation. PLoS One 2011 Feb 09;6(2):e14669 [FREE Full text] [CrossRef] [Medline]
- Miskelly F. Electronic tracking of patients with dementia and wandering using mobile phone technology. Age Ageing 2005 Sep;34(5):497-499 [FREE Full text] [CrossRef] [Medline]
- Kratzke C, Cox C. Smartphone technology and apps: rapidly changing health promotion. Global Journal of Health Education and Promotion 2012;15(1).
- Ha YP, Littman-Quinn R, Antwi C, Seropola G, Green RS, Tesfalul MA, et al. A mobile health approach to tuberculosis contact tracing in resource-limited settings. Stud Health Technol Inform 2013;192:1188. [Medline]
- Ha Y, Tesfalul M, Littman-Quinn R, Antwi C, Green R, Mapila T, et al. Evaluation of a Mobile Health Approach to Tuberculosis Contact Tracing in Botswana. J Health Commun 2016 Dec;21(10):1115-1121 [FREE Full text] [CrossRef] [Medline]
- Chang A, Ghose S, Littman-Quinn R, Anolik R, Kyer A, Mazhani L, et al. Use of mobile learning by resident physicians in Botswana. Telemed J E Health 2012;18(1):11-13 [FREE Full text] [CrossRef] [Medline]
- Ventola C. Mobile devices and apps for health care professionals: uses and benefits. Pharmacy and Therapeutics 2014;39(5):356. [Medline]
- Oncescu V, Mancuso M, Erickson D. Cholesterol testing on a smartphone. Lab Chip 2014;14(4):759-763 [FREE Full text] [CrossRef]
- HealthyPeople. 2018. 2020 Topics and Objectives URL: https://www.healthypeople.gov/2020/topics-objectives [accessed 2019-05-06]
- NIH. National Institutes of Health. 2019. NIH Research Portfolio Online Reporting Tools URL: https://projectreporter.nih.gov/reporter.cfm [accessed 2019-05-06]
- NIH. National Institutes of Health. 2019. Activity Codes Search Results URL: https://grants.nih.gov/grants/funding/ac_search_results.htm [accessed 2019-05-06]
- Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol 2017 Dec;2(4):230-243 [FREE Full text] [CrossRef] [Medline]
- Arnrich B, Mayora O, Bardram J. Pervasive or Ubiquitous Healthcare? Methods Inf Med 2018 Jan 17;49(01):65-66 [FREE Full text] [CrossRef]
- Suaning G, Lovell N, Schindhelm K, Coroneo M. The bionic eye (electronic visual prosthesis): A review. Australian and New Zealand journal of ophthalmology 1998:A [FREE Full text] [CrossRef]
- Guenther T, Lovell N, Suaning G. Bionic vision: system architectures--a review. Expert Rev Med Devices 2012 Jan;9(1):33-48 [FREE Full text] [CrossRef] [Medline]
- Alsalaheen B, Mucha A, Morris L, Whitney S, Furman J, Camiolo-Reddy C, et al. Vestibular rehabilitation for dizziness and balance disorders after concussion. J Neurol Phys Ther 2010 Jun;34(2):87-93 [FREE Full text] [CrossRef] [Medline]
- Reyes A, Qin P, Brown C. A standardized review of smartphone applications to promote balance for older adults. Disabil Rehabil 2018 Dec;40(6):690-696 [FREE Full text] [CrossRef] [Medline]
- Peters J, Ramakers G, Smit A, Grolman W. Cochlear implantation in children with unilateral hearing loss: A systematic review. Laryngoscope 2016 Mar;126(3):713-721 [FREE Full text] [CrossRef] [Medline]
- Vlastarakos P, Proikas K, Papacharalampous G, Exadaktylou I, Mochloulis G, Nikolopoulos T. Cochlear implantation under the first year of age--the outcomes. A critical systematic review and meta-analysis. Int J Pediatr Otorhinolaryngol 2010 Feb;74(2):119-126 [FREE Full text] [CrossRef] [Medline]
- Clement R, Bugler K, Oliver C. Bionic prosthetic hands: A review of present technology and future aspirations. Surgeon 2011 Dec;9(6):336-340 [FREE Full text] [CrossRef] [Medline]
- Ortiz-Catalan M, Brånemark R, Håkansson B, Delbeke J. On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion. Biomed Eng Online 2012 Jun 20;11:33 [FREE Full text] [CrossRef] [Medline]
- Butler A, Chapman J, Forman E, Beck A. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev 2006 Jan;26(1):17-31 [FREE Full text] [CrossRef] [Medline]
- Mazzucchelli T, Kane R, Rees C. Behavioral activation treatments for depression in adults: a meta-analysis and review. Clinical Psychologycience and Practice 2009;16(4):383-411 [FREE Full text] [CrossRef]
- Prendergast M, Podus D, Finney J, Greenwell L, Roll J. Contingency management for treatment of substance use disorders: a meta-analysis. Addiction 2006 Nov;101(11):1546-1560 [FREE Full text] [CrossRef] [Medline]
- Petry N. Contingency management treatments: controversies and challenges. Addiction 2010 Sep;105(9):1507-1509 [FREE Full text] [CrossRef] [Medline]
- Carroll K, Easton C, Nich C, Hunkele K, Neavins T, Sinha R, et al. The use of contingency management and motivational/skills-building therapy to treat young adults with marijuana dependence. J Consult Clin Psychol 2006 Oct;74(5):955-966 [FREE Full text] [CrossRef] [Medline]
- Finkelstein E, Brown D, Brown D, Buchner D. A randomized study of financial incentives to increase physical activity among sedentary older adults. Prev Med 2008 Aug;47(2):182-187 [FREE Full text] [CrossRef] [Medline]
- Pozzi F, Snyder-Mackler L, Zeni J. Physical exercise after knee arthroplasty: a systematic review of controlled trials. European journal of physical and rehabilitation medicine 2013;49(6):877-892.
- Gilad B, Kaish S, Loeb P. From economic behavior to behavioral economics: The behavioral uprising in economics. Journal of Behavioral Economics 1984 Jun;13(1):1-22 [FREE Full text] [CrossRef]
- Katona G. Consumer behavior: Theory and findings on expectations and aspirations. The American Economic Review 1968 May;58(2):19-30.
- Katona G. On the function of behavioral theory and behavioral research in economics. The American Economic Review 1968 Mar;58(1):146-150.
- Kahneman D. Maps of Bounded Rationality: Psychology for Behavioral Economics. American Economic Review 2003 Nov;93(5):1449-1475 [FREE Full text] [CrossRef]
- Tversky A, Kahneman D. Judgment under Uncertainty: Heuristics and Biases. Science 1974 Sep 27;185(4157):1124-1131 [FREE Full text] [CrossRef] [Medline]
- Kaplan B, Cassel J, Gore S. Social Support and Health. Medical Care 1977;15(Supplement):47-58 [FREE Full text] [CrossRef]
- Reijula E, Halkoaho A, Pietilä AM, Selander T, Martikainen K, Kälviäinen R, et al. Comparable indicators of therapeutic misconception between epilepsy or Parkinson's disease patients between those with clinical trial experience and trial non-participants. Seizure 2018 Aug;60:61-67 [FREE Full text] [CrossRef] [Medline]
- Lowe D, Balsis S, Hughes M, Shine H, Carpenter B. Misconceptions of Alzheimer’s Disease. Clinical Gerontologist 2014 Dec 31;38(2):149-156 [FREE Full text] [CrossRef]
- Kukkadapu T, Filo A, Jia L, Keshavamurthy J, Dillard T. High Price of Getting High: Marijuana-Induced Bullous Lung Disease. In: B51. Cigarettes, E-cigarettes, and Hookahs.: American Journal of Respiratory and Critical Care Medicine; 2018 Presented at: American Thoracic Society 2018 International Conference; May 21, 2018; San Diego p. A3581.
- Sandberg T, Hutter R, Richetin J, Conner M. Testing the role of action and inaction anticipated regret on intentions and behaviour. Br J Soc Psychol 2016 Sep;55(3):407-425 [FREE Full text] [CrossRef] [Medline]
- Sheeran P, Maki A, Montanaro E, Avishai-Yitshak A, Bryan A, Klein W, et al. The impact of changing attitudes, norms, and self-efficacy on health-related intentions and behavior: A meta-analysis. Health Psychol 2016 Nov;35(11):1178-1188 [FREE Full text] [CrossRef] [Medline]
- Wu M, Wu S, Lee M, Peng L, Tsao L, Lee W. Health-promotion interventions enhance and maintain self-efficacy for adults at cardiometabolic risk: A randomized controlled trial. Arch Gerontol Geriatr 2019;82:61-66 [FREE Full text] [CrossRef] [Medline]
- Rubak S, Sandbaek A, Lauritzen T, Christensen B. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract 2005 Apr;55(513):305-312 [FREE Full text] [Medline]
- Barnett E, Sussman S, Smith C, Rohrbach L, Spruijt-Metz D. Motivational Interviewing for adolescent substance use: a review of the literature. Addict Behav 2012 Dec;37(12):1325-1334 [FREE Full text] [CrossRef] [Medline]
- Rollnick S, Miller W, Butler C, Aloia M. Motivational Interviewing in Health Care: Helping Patients Change Behavior. COPD: Journal of Chronic Obstructive Pulmonary Disease 2009 Jul 02;5(3):203-203 [FREE Full text] [CrossRef]
- Elliot A, Devine P. On the motivational nature of cognitive dissonance: Dissonance as psychological discomfort. Journal of Personality and Social Psychology 1994;67(3):382-394 [FREE Full text] [CrossRef]
- Aday L, Andersen R. A framework for the study of access to medical care. Health services research 1974;9(3):208.
- Andersen R, Davidson PL. Improving access to care in America. In: Andersen R, Rice TH, Kominski GF, editors. Changing The US Health Care System: Key Issues In Health Services Policy And Management. San Francisco, CA: Jossey-bass; 2007:3-31.
- Kannisto K, Koivunen M, Välimäki MA. Use of mobile phone text message reminders in health care services: a narrative literature review. J Med Internet Res 2014 Oct 17;16(10):e222 [FREE Full text] [CrossRef] [Medline]
- Horner S, Ambrogne J, Coleman M, Hanson C, Hodnicki D, Lopez S, et al. Traveling for Care: Factors Influencing Health Care Access for Rural Dwellers. Public Health Nurs 1994 Jun;11(3):145-149 [FREE Full text] [CrossRef]
- Nicholas D, Williams P, Huntington P, Jamali H. Digital Health Information. Farnham: Ashgate Publishing, Limited; 2007.
- Blakely G, Skirton H, Cooper S, Allum P, Nelmes P. Educational gaming in the health sciences: systematic review. J Adv Nurs 2009 Feb;65(2):259-269 [FREE Full text] [CrossRef] [Medline]
- Ricciardi F, De Paolis L. A Comprehensive Review of Serious Games in Health Professions. International Journal of Computer Games Technology 2014;2014:1-11 [FREE Full text] [CrossRef]
- Adams SA. Use of "serious health games" in health care: a review. Stud Health Technol Inform 2010;157:160-166. [Medline]
- Gee J. What Video Games Have to Teach Us about Learning and Literacy. Second Edition: Revised and Updated Edition. New York: Palgrave Macmillan; 2007.
- Prensky M. Digital game-based learning. Comput. Entertain 2003 Oct 01;1(1):21 [FREE Full text] [CrossRef]
- Bishop S, Lau M, Shapiro S, Carlson L, Anderson N, Carmody J. Mindfulness: A proposed operational definition. Clinical psychology: science and practice 2004;11(3):230-241 [FREE Full text] [CrossRef]
- Shapiro S, Carlson L, Astin J, Freedman B. Mechanisms of mindfulness. J Clin Psychol 2006 Mar;62(3):373-386 [FREE Full text] [CrossRef] [Medline]
- Grossman P, Niemann L, Schmidt S, Walach H. Mindfulness-based stress reduction and health benefits. Journal of Psychosomatic Research 2004 Jul;57(1):35-43 [FREE Full text] [CrossRef]
- Brown K, Ryan R. The benefits of being present: Mindfulness and its role in psychological well-being. Journal of Personality and Social Psychology 2003;84(4):822-848 [FREE Full text] [CrossRef]
- Chiesa A, Serretti A. Mindfulness-based stress reduction for stress management in healthy people: a review and meta-analysis. J Altern Complement Med 2009 May;15(5):593-600 [FREE Full text] [CrossRef] [Medline]
- Mukhopadhyay S. Wearable Sensors for Human Activity Monitoring: A Review. IEEE Sensors J 2015 Mar;15(3):1321-1330 [FREE Full text] [CrossRef]
- Kumari P, Mathew L, Syal P. Increasing trend of wearables and multimodal interface for human activity monitoring: A review. Biosens Bioelectron 2017 Apr 15;90:298-307 [FREE Full text] [CrossRef] [Medline]
- Piwek L, Ellis D, Andrews S, Joinson A. The Rise of Consumer Health Wearables: Promises and Barriers. PLoS Med 2016 Feb;13(2):e1001953 [FREE Full text] [CrossRef] [Medline]
- Varshney U. Pervasive Healthcare and Wireless Health Monitoring. Mobile Netw Appl 2007 Jul 12;12(2-3):113-127 [FREE Full text] [CrossRef]
- Etkind S, Daveson B, Kwok W, Witt J, Bausewein C, Higginson I, et al. Capture, transfer, and feedback of patient-centered outcomes data in palliative care populations: does it make a difference? A systematic review. J Pain Symptom Manage 2015 Mar;49(3):611-624 [FREE Full text] [CrossRef] [Medline]
- Kluger A, DeNisi A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin 1996;119(2):254-284 [FREE Full text] [CrossRef]
- Hansen W, Graham J. Preventing alcohol, marijuana, and cigarette use among adolescents: Peer pressure resistance training versus establishing conservative norms. Preventive Medicine 1991 May;20(3):414-430 [FREE Full text] [CrossRef]
- Granfield R. Alcohol use in college: Limitations on the transformation of social norms. Addiction Research & Theory 2009 Jul 11;13(3):281-292 [FREE Full text] [CrossRef]
- Doumas D, Hannah E. Preventing high-risk drinking in youth in the workplace: a web-based normative feedback program. J Subst Abuse Treat 2008 Apr;34(3):263-271 [FREE Full text] [CrossRef] [Medline]
- Compas B, Jaser S, Bettis A, Watson K, Gruhn M, Dunbar J, et al. Coping, emotion regulation, and psychopathology in childhood and adolescence: A meta-analysis and narrative review. Psychol Bull 2017 Sep;143(9):939-991 [FREE Full text] [CrossRef] [Medline]
- Mueller C, Wesenberg S, Nestmann F, Stubbs B, Bebbington P, Raymont V. Interventions to enhance coping after traumatic brain injury: A systematic review. International Journal of Therapy and Rehabilitation 2018 Mar 02;25(3):107-119 [FREE Full text] [CrossRef]
- Müller R, Peter C, Cieza A, Geyh S. The role of social support and social skills in people with spinal cord injury--a systematic review of the literature. Spinal Cord 2012 Feb;50(2):94-106 [FREE Full text] [CrossRef] [Medline]
- Dennis C. Peer support within a health care context: a concept analysis. International Journal of Nursing Studies 2003 Mar;40(3):321-332 [FREE Full text] [CrossRef]
- Gurung R, Sarason B, Sarason I. Predicting Relationship Quality and Emotional Reactions to Stress from Significant-Other-Concept Clarity. Pers Soc Psychol Bull 2016 Jul 02;27(10):1267-1276 [FREE Full text] [CrossRef]
- Thoits P. Mechanisms linking social ties and support to physical and mental health. J Health Soc Behav 2011 Jun;52(2):145-161 [FREE Full text] [CrossRef] [Medline]
- Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008 May;27(3):379-387 [FREE Full text] [CrossRef] [Medline]
- Stanford P. Unconceived alternatives and conservatism in science: the impact of professionalization, peer-review, and Big Science. Synthese 2015 Aug 30:1-18 [FREE Full text] [CrossRef]
- Hansen WB, Scheier LM. Zenodo.org. 2019. NIH-funded smartphone intervention apps, SPSS dataset URL: http://doi.org/10.5281/zenodo.3252153 [accessed 2019-05-06]
|BA: behavioral activation|
|CBT: cognitive behavior therapy|
|eHealth: electronic health|
|GPS: global positioning system|
|mHealth: mobile health|
|MI: motivational interviewing|
|NEI: National Eye Institute|
|NHGRI: National Human Genome Research Institute|
|NHLBI: National Heart, Lung, and Blood Institute|
|NIA: National Institute on Aging|
|NIAAA: National Institute on Alcohol Abuse & Alcoholism|
|NIAID: National Institute of Allergy and Infectious Diseases|
|NIAMS: National Institute of Arthritis and Musculoskeletal and Skin Diseases|
|NIBIB: National Institute of Biomedical Imaging and Bioengineering|
|NICHD: National Institute of Child Health and Human Development|
|NIDA: National Institute on Drug Abuse|
|NIDCD: National Institute on Deafness and other Communication Disorders|
|NIDCR: National Institute of Dental and Craniofacial Research|
|NIDDK: National Institute of Diabetes and Digestive and Kidney Diseases|
|NIEHS: National Institute of Environmental and Health Sciences|
|NIGMS: National Institute of General Medicine Sciences|
|NIH: National Institutes of Health|
|NIMH: National Institute of Mental Health|
|NIMHD: National Institute on Minority Health and Health Disparities|
|NINDS: National Institute of Neurological Disorders and Stroke|
|NINR: National Institute of Nursing Research|
|NLM: National Library of Medicine|
|VoIP: Voice over Internet Protocol|
Edited by G Eysenbach; submitted 08.05.19; peer-reviewed by J Derzon, L Ranandeh; comments to author 18.06.19; revised version received 25.06.19; accepted 27.06.19; published 29.07.19Copyright
©William B Hansen, Lawrence M Scheier. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 29.07.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.