The model used to develop the app involved a process of research and analysis, development, implementation, and evaluation. In developing new technologies, it was important that the framework was systematic (clear steps following a logical order), systemic (all processes critical for success are incorporated), reliable (steps are clearly described so that they can be replicated by other designers in other projects), iterative (the cycle of analysis design development testing revision can be repeated a number of times), and empirical (data gathering is built into the process and decisions are made on the basis of data) [27]. The development process utilized a 3-step approach based on the intervention mapping protocol [28]. Similarly, processes have been used successfully for mHealth app-based interventions [29]. The predominate emphasis of this paper is that of the third step, as other steps have been reported elsewhere [30,31].

Employees in MDIs were specifically targeted. The process took place from January 2016 to April 2017. An interdisciplinary team of computer engineers, psychiatrists, psychologists, and design experts (user experience and graphic designers) collaborated in the design and development of the app.

Although the effectiveness of eHealth and mHealth technologies for treating moderate levels of mental ill-health in general and clinical populations has been established, less is known about workplace eHealth interventions and eHealth prevention. The problem led the team to conduct a series of systematic reviews and meta-analyses to determine the effectiveness of workplace interventions for common mental disorders (CMD) [32], workplace depression prevention [33], the use of eHealth for prevention of CMD in general populations [34], and the use of eHealth tools for CMD in the workplace [23].

A recent meta-analysis of work-based depression prevention programs found such programs to be encouraging, with a number of different types of work-based interventions, particularly those based on cognitive behavioral models, demonstrating an ability to reduce depressive symptoms on unselected working populations [9].

To develop a relevant and engaging program, it is important to involve end users in participatory design and user experience research [35]. This stage comprised several components, including 6 focus groups (N=60) with industry partners and an in-depth survey of 1 specific industry partner (N=105). The findings of these are reported separately [30,31], and following feedback, we aimed to design and prototype an app that is engaging for men in the target workplaces.

Building on the outcomes of the initial 2 steps, the app content and design were finalized. The pilot testing of the app involved a 2-stage approach to test both initial utility and acceptability (using an alpha version [acceptance testing] of the app) and feasibility (engagement and perceived usefulness to users) and preliminary efficacy (using a beta version [operational testing] of the app). There were several reasons for this approach. Primarily, the costs involved in the creation of such technology are considerable. During the participatory engagement step, no prototype was used to generate unbiased input. However, it was necessary to be able to make modifications based on this testing and the usability of the app. Subsequently, modifications were made to the app between the 2 (alpha or beta) stages of the design and pilot testing step to refine usability elements and to test preliminary efficacy.

HeadGear is a smartphone app–based intervention centered on behavioral activation and mindfulness therapy. The main therapeutic component of the HeadGear app takes the form of a 30-day challenge in which users’ complete 1 challenge daily (approximately 5-10 min; Figure 1). These challenges include psychoeducational videos on coping skills or resiliency, mindfulness, and behavioral activation; mindfulness exercises; value-driven activity planning, goal-setting, and review; and coping skill development (problem solving, sleep, grounding, alcohol use, assertiveness, and training in adaptive forms of coping). The inclusion of these specific components was driven by the findings of stages 1 and 2 (specifically, [9,23,30,31,34]).

The first daily challenge involves the completion of a risk calculator, which assesses and provides participants with personalized feedback regarding their risk for future mental health issues. The risk calculator consists of 20 items developed from the Household, Income and Labour Dynamics in Australia Survey (HILDA) and has been validated in the Australian adult population [36]. The risk factor items are based on participant self-report. The HILDA risk items include age, gender, Aboriginal and Torres Strait Islander status, active career status, freedom to decide work, satisfaction with hours worked, satisfaction with employment opportunities, physical activity, alcohol use, episodes of distress in the previous 2 years, satisfaction with health, satisfaction with the neighborhood, satisfaction with partner, satisfaction with the way tasks are divided with partner, having someone to confide in, the feeling of being pushed around, and English as a second language. The HILDA questions and response items were replicated from the original items included in the HILDA survey, apart from age, which is measured here as a continuous measure. Users received personalized risk feedback immediately after completing the risk calculator. The personalized risk feedback involves an interactive icon array, which displays the calculated numerical risk estimate of developing anxiety and depression within the next year, along with a text description (Figure 2). Although much of the app is not specific to a workplace (or even MDI) setting and is likely to have utility to a general population, it was within these populations that development occurred. The outcomes of early development work [30,31] led to the inclusion of certain elements, determined to be the most relevant among these groups. Importantly, the risk algorithm was built from a working population sample and was fundamental to its working population delivery.

Other components of the app include a mood monitoring widget, a toolbox of skills (which is built from the challenges as they are completed), and support service helplines. Users had access to the app indefinitely. The app monitors use time and frequency and mission completion rates.

In total, 21 participants downloaded the app, 12 of whom responded to the follow-up survey. However, 6 participants consented but did not download the app and were subsequently removed from the study. The average age of the participants was 37.86 years (SD=10.98); approximately half of the participants were female (n=12). The majority of the sample worked in freight and postage (n=11), followed by mining (n=6) and agriculture (n=2); however, 3 participants declined to provide their industry. Approximately half of the participants were working in a manager role (n=9) and the majority were based in an urban center (n=15).

The sample was entirely male, with almost half working in an MDI (Table 3). Participants had a mean age of 38 years (SD=9.23). On average, participants spent just under an hour in the app (mean=58.24 min; SD=62.98) and completed a third (mean=9.11; SD=10.25) of the challenge days. Over half (n=48) of the participants reported a prior episode of mental ill-health and were considered high-risk on the HILDA-derived risk algorithm [36].

This study provides both a framework for the development and testing of a new smartphone app intervention, HeadGear, and investigates the use and acceptability, along with the feasibility and preliminary effectiveness, of the app in a working population, specifically MDIs. The core features and functionality of the app were developed through a participatory design process, and the content of the app was based on current best available evidence-based theory. The research team encompassed computer engineers, psychiatrists, psychologists, and design (user experience and graphic design) experts allowing for a multidisciplinary approach to development. The pilot testing of the app incorporated a 2-stage process that utilized different samples and different outcomes measures to reflect the progression of the app from alpha to beta testing. Overall, the app was well received in both stages of the pilot testing, and preliminary testing indicated significant improvements in measures of psychopathology and workplace productivity.

The results from this feasibility and efficacy pilot trial suggest that an mHealth app can be an engaging, useful, and acceptable intervention. Across the 2 stages of the study, the majority of the participants acknowledged the utility, helpfulness, and overall ease and acceptability of use of the HeadGear app. With regards to preliminary efficacy of the intervention, the results are in line with previous findings that have shown mindfulness and behavioral activation to be effective in the treatment of mood and anxiety disorders [58,59], even in mobile app forms [24]. The dose-effect response seen between level of usage of the HeadGear app and improvements in both depression and anxiety symptoms was also encouraging; however, due to high attrition, findings need to be interpreted with caution.

Although improvements in well-being and resilience were found, these findings were not significant. As the sample included both well and unwell individuals, it is likely to have been underpowered to detect such changes; this underscores the need for a full-scale efficacy trial. Results also indicated that there were significant reductions in absenteeism and increases in worker productivity. This is especially encouraging given medical interventions in isolation have not shown as positive an effect on work-related outcomes when compared with workplace interventions [60]. This finding suggests the utility in incorporating evidence-based interventions in the workplace.

Despite the positive reviews of stage 1, there was a low level of challenge days completed. This may reflect a number of functionality issues resolved for stage 2 and that the sample’s characteristics were not representative of MDIs (from which this sample was taken and for which the app was designed) as 50% of participants were women. When contrasted with a technically improved iteration and a more representative population (stage 2), there was significantly more engagement with the app. Nevertheless, engendering motivation to complete the program was a concern raised in this review process. Although reasons for disengagement are complex and rarely only due to dissatisfaction [61] and somewhat unsurprising given the unguided nature of the trial [62,63], it does raise some feasibility concerns. O’Brien and Toms [64] suggest that engagement is not static but a process operating over a continuum; therefore, understanding this process more specifically across each of the challenge days might assist in improving adherence, which may be garnered through a larger trial. It was determined in earlier development steps [30,31] that end users were familiar with month-long health endeavors (eg, FebFast, Steptember, Dry July), and this played a role in the selection of the 30-day challenge period. Mobile apps, in general, suffer from poor rates of retention. Overall, 43% of global mobile users were still using apps (at least once) 1 month after download [65]. However, 23% will use an app only once, and only 1 in 3 will use an app at least 11 times [66]. Ultimately, this presents new obstacles in regards to engagement with a mental health and well-being app that need to be considered over the full intervention [67]. Encouragingly, results indicated the more time spent in the app was associated with more positive outcomes on the primary outcome and that participants used the app irrespective of their current symptom level, suggesting it has wider appeal than simply those with heightened symptomatology. Nevertheless, further research is required to better understand ways in which to enhance engagement.

A substantial strength of the study was the development process, which allowed for detailed and systematic analysis of a product in multiple stages of testing. Additionally, the mobile-based delivery of the program holds a number of advantages over traditional methods particularly in MDIs [68]. Despite some limitations to generalizability, the study indicates that the intervention may have value in engaging this difficult-to-reach and at-risk group [69]. Indeed, tailoring the treatment to the feedback received during participatory user research meant that goal-directed and skill-based activities were the predominate focus of the intervention, which is in line with other recommendations for this group [70].

In addition to modest rates of intervention completion, the follow-up rate was also a limitation, and as mentioned, this has implications for the findings. Despite email (stage 1) and SMS text message (stage 2) reminders and incentives for assessment completion, follow-up rates were low compared with the literature [71]. Some reasons postulated for this include the source of recruitment (social media in stage 2), limited exclusion criteria (ie, those who were well may have had less motivation to engage), and a lack of personalized follow-up. However, a key factor which is unique to this trial is onboarding, whereby participants downloaded the app, consented, and subsequently completed baseline within the app. Therefore, users may have had little desire to participate in the trial but simply wanted access to the app. In an attempt to streamline the user experience (avoiding filtering participants through an arduous onboarding, which may lose all but the most conscientious participants), the study may, in fact, have recruited a less research-engaged (though perhaps more real-world) sample. Clearly, alternate and intensive strategies are required, as these low levels of retention raise feasibility concerns for a larger randomized controlled trial (RCT) trial. Additionally, low levels of mental health sick days were reported in the sample. This is unsurprising considering the small size and short follow-up; however, it limits what can be derived from this outcome. Sample size limited further investigation of change in outcomes based on baseline risk category or industry; again, larger RCT studies are required to explore these relationships. An additional limitation is despite targeting MDIs, there was significant interest from non-MDIs, and consequently, the conclusions that can be reached pertaining solely to MDIs are limited; conversely, the app may have wider utility. Finally, and perhaps most importantly, the lack of a control group limits any conclusions that can be made regarding the beneficial impact of this app; an RCT would help to ameliorate the biases inherent in uncontrolled trials.

The results from this pilot trial suggest that the HeadGear app can be an engaging, acceptable, and potentially effective intervention. Although preliminary results were encouraging, noted limitations in the pilot design highlight the need for a full-scale efficacy trial to better understand the utility of smartphone apps in the prevention and treatment of depression symptoms.