This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
The now ubiquitous catchphrase, “There’s an app for that,” rings true owing to the growing number of mobile phone apps. In excess of 97,000 eHealth apps are available in major app stores. Yet the effectiveness of these apps varies greatly. While a minority of apps are developed grounded in theory and in conjunction with health care experts, the vast majority are not. This is concerning given the Hippocratic notion of “do no harm.” There is currently no unified formal theory for developing interactive eHealth apps, and development is especially difficult when complex messaging is required, such as in health promotion and prevention.
This paper aims to provide insight into the creation of interactive eHealth apps for complex messaging, by leveraging the Safe-D case study, which involved complex messaging required to guide safe but sufficient UV exposure for vitamin D synthesis in users. We aim to create recommendations for developing interactive eHealth apps for complex messages based on the lessons learned during Safe-D app development.
For this case study we developed an Apple and Android app, both named Safe-D, to safely improve vitamin D status in young women through encouraging safe ultraviolet radiation exposure. The app was developed through participatory action research involving medical and human computer interaction researchers, subject matter expert clinicians, external developers, and target users. The recommendations for development were created from analysis of the development process.
By working with clinicians and implementing disparate design examples from the literature, we developed the Safe-D app. From this development process, recommendations for developing interactive eHealth apps for complex messaging were created: (1) involve a multidisciplinary team in the development process, (2) manage complex messages to engage users, and (3) design for interactivity (tailor recommendations, remove barriers to use, design for simplicity).
This research has provided principles for developing interactive eHealth apps for complex messaging as guidelines by aggregating existing design concepts and expanding these concepts and new learnings from our development process. A set of guidelines to develop interactive eHealth apps generally, and specifically those for complex messaging, was previously missing from the literature; this research has contributed these principles. Safe-D delivers complex messaging simply, to aid education, and explicitly, considering user safety.
In 2013, there were 97,000 eHealth apps in major app stores [
The effectiveness of these apps can vary if development does not draw on change theories [
More recently, clinician involvement has occurred [
High-level exposure to ultraviolet (UV) radiation from the sun is a major risk factor for skin cancer, yet UV exposure is needed for vitamin D synthesis in the skin. In Australia, skin cancer awareness has risen considerably in recent years. At the same time, research has demonstrated low vitamin D levels in various population samples despite abundant natural UV radiation. On the whole, Australians have adopted the Cancer Council’s “slip-slop-slap” campaign message [
In addition, only a few foods such as oily fish, sun-exposed mushrooms, and eggs contain appreciable amounts of vitamin D [
To address these diverse variables, we selected an app as the best medium to provide the complex messaging required to guide safe but sufficient UV exposure, as it affords the two-way information exchange required for interactive, tailored recommendations. Utilizing an interactive eHealth app enables portability to record sun exposure duration and frequency as it occurs. Hence, more reliable exposure measurements [
The Safe-D study is limited to English-reading females aged 16-25 years residing in the State of Victoria, Australia. Therefore, the Safe-D app is in English only and designed for this target audience. Safe-D, and thus this research, does not include social media integration during its evaluation. There is a need for this limitation to restrict the opportunity for participants to share information with non-app intervention participants during its evaluation, insuring RCT integrity. Further to these limitations, the app is available only on Apple and Android devices, using existing technologies of Global Positioning System (GPS) and readily available UV data from the Australian Radiation Protection and Nuclear Safety Agency (ARPANSA) and Bureau of Meteorology (BOM). The limitation to Apple and Android devices only was based on Australian Interactive Media Industry Association (AIMIA) results that 49% of Australians own a smartphone, predominantly Apple and Android devices [
Safe-D development conformed to eHealth, persuasion, and interactivity best practices. A literature review was conducted to align development with current literature, presented thematically in this section. The aim of Safe-D was to persuasively encourage UV exposure. Persuasive technologies are those to prompt behavior change and reinforce learning [
Eysenbach [
Dennison et al [
Rewarding positive behaviors, rather than punishing non-compliance, leads to continued engagement [
Gamification is the use of game elements in non-game contexts to provoke competition [
It is well published that tailored messages are most effective at motivation [
UX encompasses “reactions and feelings, which arise from interactions with a system” [
This research had dual goals of developing Safe-D while utilizing the process to understand development principles for interactive eHealth apps for complex messaging. These dual goals contribute to the development of a tangible solution in the form of the Safe-D app, while corresponding with intended outcomes of participatory action research [
As is commonplace in qualitative research, development began relatively theory-free [
A preliminary survey was conducted to further understand what young women wanted in an app, to inform design decisions. The survey was conducted online and advertised via a Melbourne Health media release, the “Netball Scoop” forum, VGen Core group, the Youth Affairs Council of Victoria monthly e-newsletter, and an email to past participants in the Young Female Health Initiative (a collaborative comprehensive health study of 16-25 year-old women in Victoria, Australia). A total of 71 respondents commenced the survey. Of these, 57 passed all eligibility criteria to participate, that is, being English-speaking females aged 18-25.
Developers were hired, and development followed stages from the software development lifecycle, selected due to its applicability to both business and health care scenarios [
Development phases.
Development phase | Process and data generated to help develop the Safe-D App |
Requirements analysis | Requirements were written and a literature review and focus groups were conducted. |
Design | Wireframes were created to elicit non-functional and evolving requirements. |
Design validation | Designs were validated in informal focus groups, an advisory private Facebook group, and informal “guerilla testing,” asking young women approached in public places for opinions regarding designs. Outcomes were used to (1) determine preferences, (2) validate designs as flexible, non-offensive, and culturally sensitive, (3) validate messages conveyed the intended message, and (4) test the app concept. |
Development | The contracted developers created Safe-D for Apple and Android platforms iteratively and collaboratively with the other researchers. |
Beta testing | Core team members who used with Apple and Android devices were given access to Safe-D. Additional beta testers were seconded from the larger team to ensure a range of platform and operating system configurations, making a total of 24 beta testers, consisting of 6 from the target demographic while others acted as expert reviewers. |
Retrospective | This is the final phase, adopted from agile methodologies. Retrospectives are specific meetings reflecting on the development and identifying improvements. Retrospective data enabled triangulation so findings did not rely solely on observations of the authors. |
Throughout each stage, available recommendations from the literature (as previously outlined) were used to help create the best experience in the app.
Researcher reflections and observational notes were used to highlight recommendations for future developments. This analysis was triangulated through data collected during retrospectives. Each member (ie, information systems and medical researchers, medical professionals, beta testers and app developers) shared what they thought went well, what did not, and what they would do differently next time based on this experience. Post-development, retrospective data were organized thematically through affinity mapping to check against initial researcher observations. This process condensed the data in a manageable format to identify any themes and relationships between themes, using an inductive approach, as per Neuman [
As this research had the dual goals of first developing the Safe-D app and then using this process to develop recommendations for future app development, the results will be presented separately: the Safe-D app and development recommendations.
Safe-D aims to safely improve vitamin D status, through individualized UV exposure recommendations, messages, and learning. Safe-D will be presented by the major actions that users take within the app, which are initial setup, tracking UV exposure, monitoring progress, recording missed exposure, and viewing messages.
Initial app setup is required when the user accesses the app for the first time. Safe-D prompts for (1) push notification and GPS permissions needed to provide interactive personalized recommendations based on real-time locational UV, (2) answers to the Fitzpatrick skin-type questionnaire [
Tunneling, the practice of guiding users through 1 question per screen, was implemented to reduce cognitive load. All data entry is simple to encourage usage, and logon details can be remembered for automatic sign-in to remove barriers to future use. As Safe-D is intended for initial evaluation in an RCT, access must be restricted to only participants through password protection.
To record UV exposure, the avatar must first be dressed. This is a seamless and fun interaction used to record exposed skin and sunscreen protection. Sunscreen and clothing are required for exposure calculations. Therefore, they are explicitly prompted. Exposed skin entry is simple: the avatar enables easy, visual, touch entry of clothing to calculate exposed skin. For usability, the avatar defaults to clothing from the last exposure, removing a step if exposed skin remains stable exposure-to-exposure.
These inputs, with locational UV levels and skin type, are used to calculate and display safe exposure recommendations. As exposure has a safe limit, the timer cannot simply count down. Exposure counts down to zero, then up in red to indicate, alert to, and dissuade overexposure. Gamification promotes usage, inspired by Fish n’ Steps metaphor [
There are two ways to monitor progress on the app: viewing today’s exposure or overall exposure. Today’s exposure is visible on the homepage by assessing the sunflower’s health state. The sunflower health state can also be assessed while the timer is running. Overall progress is monitored via statistics, displaying the current consecutive days-of-use in the form of a streak, and longest consecutive use. Streak here is used in the sporting sense to refer to a running total of consecutive days when the app was accessed. To encourage reaching exposure, without exceeding the defined safe limit, Safe-D enables easy entry and viewing of exposure, making it apparent when the safe limit is exceeded. The progress display of how much exposure the user has had each day allows them to adjust behavior accordingly. This visual display takes into account that exposure can be too high, too low, or just right for the most number of users. Inspiration was garnered from discussions with other app designers [
Missed exposure can occur for two reasons: a user may be unable to get sun or may forget to record it. When unable to get sun, the “I Can’t Get Sun Today” feature records the reason. If users forget to start or stop the timer, they can retrospectively enter or delete using “My Sun Exposure.” Users may be unable to get sun for a number of reasons, and this function handles this sensitively without punishing a user by forcing them to lose their in-app streak.
Safe-D aims to safely improve long-term vitamin D status through UV exposure. Thus, to prevent indefinite app reliance, information is required for learning. The complex messaging occurs on a per scenario basis (see
Safe-D messages and medium per scenario.
Scenario | In app | Push | |
Timer stopped without reaching exposure | Yes |
|
|
Exposure reached | Yes | Yes |
|
Exposure exceeded | Yes | Yes |
|
Exposure exceeded and timer not stopped | Yes | Yes |
|
Exposure exceeded and “Get Vitamin D” accessed when UV is still above low | Yes |
|
|
Exposure exceeded and “Get Vitamin D” accessed when UV is now low | Yes |
|
|
Too much sun without reaching exposure, via the “Too Much Sun” function | Yes | Yes |
|
Safe-D not used |
|
Yes |
|
UV in current location changes during exposure | Yes | Yes |
|
Forecast UV is extreme or very low |
|
Yes | Yes |
From our preliminary survey, 64% (7/11) of respondents provided an answer for the acceptable number of notifications per day, reporting 1-2 notifications per day as acceptable. However, beta testing revealed that 1 per day when the user was not actively using Safe-D was annoying and could potentially inhibit use. Push notifications send to a user’s phone only under the following circumstances: the timer is running and/or behavior needs to occur, Safe-D was not used the previous day, the user recorded “Too Much Sun” yesterday, and an educational message sent once a week.
Messages can be viewed either as push notifications or through in-app mail. Messages can be sent to groups to improve effectiveness rather than generic messages sent to all. Other messages are interactive, based on user inputs. Mail was designed in a way that enables tailored messages to be sent to defined groups. Hardcoded events are recorded in the Safe-D dashboard that researchers can access and identify individuals who may need special messages. An example of how this could be implemented is to look at users who continually flag that they cannot get sun today because they are “too busy.” A sensitively tailored message could then be sent to such a participant to encourage them to make time. In more extreme cases if a participant continuously exceeds app-defined safe exposure limits, it would be possible for the non-blinded researcher to identify and contact them directly for their safety.
Educational notifications are delivered weekly. These are designed to be fun, non-clinical, and short. Messages cannot build on previously received messages as participants can start Safe-D at different times. Clinical experts in bone health and dermatology first reviewed all written copy, and expert bodies, such as the Cancer Council, are referenced where appropriate. Messages are framed positively to avoid reprimanding users [
As previously indicated, the literature [
As recommended in the literature [
It is important to note that despite all the preparation in our study, it was not always possible to ensure that all roles and messages were fully understood. For example, developers may want a decision quickly, not appreciating that clinician review and approval was required. Similarly, clinicians may have unrealistic technical expectations. It is important to work through these issues and also ensure that there is adequate time set aside for the complexities of working in the multidisciplinary team. This recommendation reflects Gold et al’s findings [
Conveying the complex and potentially confounding sun exposure factors to users was challenging. The recommendations from the study for managing the complex messaging were three-fold: (1) minimize user input requirements into the app to keep it relatively simple, (2) deliver safe personalized key messages to users when appropriate, and (3) manage the push messages from a central server.
First, the principle of minimizing input while making the complexity as invisible to the user was important. Safe-D achieved this by managing external factors in the background and only asking users for information that the app could not detect. The external UV level data sources (ARPANSA and BOM) were input into an algorithm, which was essential to calculate safe, personalized recommendations. Use of an algorithm enabled clear output based on individualized inputs, as is required for complex messaging. The mechanics of this algorithm, however, should not be apparent to users. The algorithm helped to determine which messages are most relevant and appropriate to individuals. If no algorithm exists, it is recommended that teams take the time to create one that provides the required output by taking into consideration all inputs and develop automated test coverage to run algorithms for complex messaging through all possible input combinations, especially when there is no existing algorithm available.
Second, it is the messages themselves that convey the complex information simply, with the use of gamification to further simplify understanding. These messages can be delivered in one or multiple modalities: in-app messages to deliver point-of-interaction information, push notifications when action is required, garnering attention to trigger the desired behavior as per Fogg [
This research has built on work by Fogg [
Third, the health messages should be simplified and controlled from a central server. This is so that messages can be updated when needed and pushed to users when relevant, without requiring them to update the app. It is useful to enable the delivery of customized messages to user subsets from the central server. In addition, this allows for later retrieval of education messages to reinforce education. It is then also easier to track which user has received which messages to help avoid repetitiveness.
By designing for interactivity, we applied concepts from the literature [
While the literature posits that tailored messages have greater persuasion [
Dennison et al [
While the literature states that data entry must be simple to encourage use [
These strategies simplified the interface to a few simple clicks, rather than extensive data entry. The push notifications reduced the need for users to remember recommendations and exposure time. As a result, the Safe-D app is easy to use, requiring minimal time to learn, and adopts an intuitive interface.
In summary, development followed the concepts from the literature of iterative and flexible development, continuous evaluation and refinement, participatory development, a multidisciplinary team and involvement of end-users, and active participation to fully appreciate and understand the issue up front [
The principles we have presented in this paper were created through drawing inferences from Safe-D development. Safe-D takes multiple inputs and provides individualized and varying messaging to aid learning for users to optimize ongoing cutaneous vitamin D synthesis; continued exposure is required indefinitely to maintain vitamin D status. These principles can be leveraged for development of eHealth apps with similar individualized and complex messaging requirements where a one-size-fits-all message would be harmful. Examples of some potential eHealth apps that would require complex messages are described in
Potential future eHealth apps with complex messaging needs.
eHealth app topic | Complex messaging |
Allergy | Allergy management requires ongoing monitoring and actions. An app would require personalized messages to each individual as their allergies and combination of allergies would vary. |
Diabetes, hyperglycemia, hypoglycemia, and insulin resistance | Disordered blood glucose requires complex messaging based on real-time individual readings to provide correct advice for the individual’s requirement at the time, be it to take insulin, glucose, or maintain current levels. |
Management of chronic illness | Many chronic conditions require individualized care plans based on the severity of the condition and any comorbidities. Care plan apps need to be able to accept multiple metrics that the user needs to track and deliver the right messages based on how they are tracking against each metric. |
Musculoskeletal disorders | The recommendations such an app would need to provide on the amount and type of physical activity would need to take in to account confounding factors of disorder (eg, osteoporosis), age, flexibility, and any temporary injuries. |
Potentially most of these conditions will require individualized and complex messaging with the move towards personalized medicine [
This paper has not presented a detailed description of the Safe-D app to maintain the integrity of the RCT. This information will be included in future reports after the RCT is completed. While Safe-D has been extensively beta tested, resulting in improvements, formal evaluation was not performed as part of this research. The RCT will evaluate whether Safe-D succeeds in its purpose to safely improve vitamin D status in young women and, therefore, the applicability of these principles to complex messaging. The proposed principles may not be applicable to all complex messaging and are not currently ranked in priority. Further, as the findings are based on a reflective analysis from participatory action research, it is possible that the guidelines may have inherent bias. This was mitigated somewhat through the triangulation of data, including using beta testers who were not involved directly in the app development. These limitations can be addressed through future research directions.
Future research, as part of the Safe-D study, should include an evaluation of the usability and utility of the app. The greater study will confirm whether or not the app is effective in safely improving vitamin D status. For example, it is possible to reach exposure limits for the day, to stop sun damage, without synthesizing enough vitamin D when limited skin is exposed. The algorithm may be updated during the study, as well as other changes based on participant interviews during site-visits regarding experiences, to improve effectiveness. A future improvement to Safe-D may include indicating to the user whether they will likely get adequate vitamin D based on what they are wearing, enabling them to change clothing or alter sunscreen use to improve synthesis.
As this research provides recommended principles for developing interactive eHealth apps for complex messaging, the results are largely theoretical at this stage and require validation before their suitability can be entirely understood. The lack of evaluation is a limitation to these principles. While behavior change was experienced anecdotally among the core researchers and beta testers, the Safe-D study will validate the app and therefore the principles applied in its development. Furthermore, future work in the eHealth app area should leverage these principles to validate them.
This research has provided insights into how to develop interactive eHealth apps for complex messaging. This outcome was achieved by drawing on examples from the literature. A set of guidelines to develop interactive eHealth apps generally, and specifically those for complex messaging, was previously missing from the literature. This research has contributed these principles. Safe-D delivers complex messaging to simplify education, explicitly considering user safety. Therefore, it showed a set of recommendations that might help develop a complex eHealth app that is simple for target users to engage with.
Australian Interactive Media Industry Association
Australian Radiation Protection and Nuclear Safety Agency
Bureau of Meteorology
general practitioner
Global Positioning System
randomized controlled trial
ultraviolet radiation
user experience
This paper is on behalf of the Safe-D Study group: Professor John Wark, Associate Professor Shanton Chang, Dr Nicola Reavley, Associate Professor Marie Pirotta, Professor George Varigos, Professor Kim Bennell, Professor Suzanne Garland, Ms Alexandra Gorelik, Professor Anthony Jorm, Dr Tharshan Vaithianathan, Ms Emma Callegari, Ms Skye Maclean, Ms Kayla Heffernan, Ms. Audrey Grech, Ms. Anna Scobie, Ms Adele Rivers, Ms Marjan Tabesh, Ms Stefanie Hartley, Ms Karen Gillett and Ms Ashleigh Buckland.
Safe-D was developed by Boosted Human: Mr Ashemah Harrison and Mr Thomas Ruffie.
Safe-D was funded by a National Health and Medical Research Council (NHMRC) grant APP1049065.
Swisse Wellness provided in-kind support for the Safe-D RCT.