This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Around the world, depression is both under- and overtreated. The
The aim of the study was to maximize patient engagement in an app designed to improve treatment allocation for depression.
An iterative, user-centered design process was employed. Qualitative data were collected via 2 focus groups with a community sample (n=17) and 7 semistructured interviews with people with depressive symptoms. The results of the focus groups and interviews were used by the computer engineering team to modify subsequent protoypes of the app.
Iterative development resulted in 3 prototypes and a final app. The areas requiring the most substantial changes following end-user input were related to the iconography used and the way that feedback was provided. In particular, communicating risk of future depressive symptoms proved difficult; these messages were consistently misinterpreted and negatively viewed and were ultimately removed. All participants felt positively about seeing their results summarized after completion of the clinical prediction tool, but there was a need for a personalized treatment recommendation made in conjunction with a consultation with a health professional.
User-centered design led to valuable improvements in the content and design of an app designed to improve allocation of and engagement in depression treatment. Iterative design allowed us to develop a tool that allows users to feel hope, engage in self-reflection, and motivate them to treatment. The tool is currently being evaluated in a randomized controlled trial.
Depression affects at least 350 million people worldwide [
To improve treatment allocation for depression, we developed the
Despite the wide availability of apps, they have yet to revolutionize health care, due in part to lack of uptake. User attrition from or nonadherence to electronic health (eHealth) technologies is well documented, both for patients and clinicians [
Explicit user-centered design, a process in which end users influence how a design takes shape [
In this study, we describe the user-centered design process of an app to assess individual risk of persistent depressive symptoms and recommend individually tailored treatment based on current knowledge about best-evidence treatment for depression. Our aim was to focus on users’ emotional and cognitive experience to design an acceptable tool for clinical decision support. Users were involved to determine
How the tool should look (to ensure it was credible, easy to use, and visually attractive)
What feedback was most likely to promote engagement with treatment recommendations
How the feedback should be presented.
User-centered design is an umbrella term that encompasses a range of models and approaches that software developers can employ to produce a highly usable and accessible product [
To increase the likelihood that the
Emotion-driven goal modeling was used to identify requirements based on patient, clinician, and research team goals regarding the app. Emotion-driven goal modeling is based on the theory of agent-oriented modeling [
Two focus groups, each lasting approximately 2 hours, were conducted by CW and AC. The 10 participants in focus group 1 (1) were presented with icons from the app without any associated text and were requested to write down and verbally present the thoughts and feelings they associated with the image; (2) formed groups of two and took turns using the app prototype on an iPad provided by the research team, followed by general discussion based on semistructured questions; and (3) were presented with the options for risk communication, followed by semistructured interview questions (
First impressions
What are your first or general impressions of the app?
What were the best things about the app?
What was the biggest problem you had with the app?
Results or feedback
How did you feel about the results page?
Imagine you’re in your general practitioner (GP) clinic and you complete the app, how would you feel?
Risk communication
What are your first impressions of (the risk communication)?
How do you feel about the faces?
How do you feel about the stick figures and numbers?
How would you feel if both the stick figures and the faces were presented?
Iconography
In front of you is a workbook with a picture on each page. For each picture write down what you think each picture represents. We will then discuss as a group.
First impressions
What were your first impressions of the app?
What did you like or dislike?
How did the app make you feel?
Did the app make you think of any questions or other thoughts when completing it?
Results or feedback
How did you feel about the message at the end (treatment recommendation)?
Was the information clear?
The third prototype of the app was tested in individual interviews. Seven face-to-face semistructured interviews were conducted by CW and observed by AC. Participants were given prototype 3 on an iPad while the observer took notes. After completing the app, they were interviewed using a broad topic guide (
The focus groups were audiorecorded. All audio recordings and text produced by participants, moderator, and observers were included in the analysis. AC collated the written and spoken words associated with part 1 of the focus group. CW transcribed the audio recordings. We conducted thematic analysis of the data by iteratively coding individual words, concepts, and phrases and then organizing these codes into a structure of themes and subthemes using the constant comparison method [
All community-dwelling adults in Melbourne, Australia, who were in the age range of 18 to 65 years and able to respond to recruitment materials in English were eligible for the study. The only exclusion criteria were being outside this age range, and, for participation in an individual interview, the absence of any depressive symptoms (as assessed by the Patient Health Questionnaire-9, PHQ-9 [
This study was approved by the University of Melbourne Human Research Ethics Committee (1442318, 1442584).
In total, 17 individuals participated in two focus groups (10 in focus group 1 and 7 in focus group 2), and 7 participated in individual interviews. The demographic characteristics of each group of participants are presented in
Characteristics of focus group and interview participants.
Characteristics | Focus group 1 (n=10) | Focus group 2 (n=7) | Interviews (n=7) | |
Male | 6 | 4 | 3 | |
Female | 4 | 3 | 4 | |
Range | 26-60 | 25-57 | 25-45 | |
Mean (SD) | 39.33 (13.36) | 39.14 (13.54) | 33.14 (7.64) | |
White | 5 | 6 | 6 | |
Asian | 3 | 0 | 0 | |
Hispanic | 0 | 1 | 0 | |
Other | 2 | 0 | 1 | |
Technical and further education | 10 | 2 | 3 | |
Bachelor | 40 | 3 | 2 | |
Postgraduate | 50 | 2 | 2 |
End users of the tool were identified as primary care patients and primary care doctors. All primary care patients could use the tool; however, only those whose initial responses to two questions on depressive symptoms indicated that they had depressive symptoms would be taken through to the full assessment and treatment recommendation phases. Our review of the context in which the tool would be used indicated that it should be used by the patient in the waiting room before a consultation with a GP or during the consultation itself. It was believed that this approach was most likely to promote use of the tool, motivate patients to engage in decisions around their health care, and increase the efficiency of the mental health care consultation.
Emotion goal modeling identified that patients wanted the app to make them feel emotionally supported, and they wanted to feel confident that the information presented to them was relevant and important. Most importantly, users wanted to see the results of their assessment (ie, the risk of having depressive symptoms in 3 months’ time) in a way that was meaningful to them.
Our review of the clinical prediction tool literature identified that a risk communication component, using numerical, verbal, or graphical depictions of risk, is built into most clinical prediction tools. We also identified several challenges in communicating risk to patients: low numerical literacy even in educated populations and the attendant problem of interpretation, considerable margin of error in risk probability, the fact that risk identified by a clinical prediction tool represents a population probability rather than an individualized risk, and ethical issues surrounding the use of risk communication tools as a persuasive mechanism. Adding to these challenges was that, unlike some health problems, the risk probability around persistent depression is very wide, thus increasing the margin of error and the validity of the result. No one type of risk communication emerged as superior to another in communicating risk for persistent depression.
Clinicians wanted to have confidence that the app provided scientifically accurate information, that it looked professional, and that it was useful for improving depression care. Therefore, it was essential that the tool retain, without alteration, all the data items that make up the prognostic algorithm in the
Items forming the
Item number | Text |
1 | Do you identify more strongly as male or female? |
2 | In general, would you say your health is |
3 | Do you have any long-term illnesses, health problem, which limits your daily activities or the work you can do (including problems that are due to old age)? |
4 | Do you live alone? |
5 | How do you manage on your available income? |
Over the last 2 weeks, how often have you been bothered by... | |
6 | ...Little interest or pleasure in doing things? |
7 | ...Feeling down, depressed or hopeless? |
8 | ...Trouble falling or staying asleep, or sleeping too much? |
9 | ...Feeling tired or having little energy? |
10 | ...Poor appetite or overeating? |
11 | ...Feeling bad about yourself, or that you are a failure, or have let yourself or your family down? |
12 | ...Trouble concentrating on things such as reading the newspaper or watching television? |
13 | ...Moving or speaking so slowly that other people could have noticed. Or the opposite—being so fidgety or restless that you have been moving around a lot more than usual? |
14 | ...Thoughts that you would be better off dead, or of hurting yourself in some way? |
15 | Have you ever been bothered by feeling down, depressed, or hopeless for longer than 2 weeks? |
16 | Have you ever been bothered by little interest or pleasure in doing things for longer than 2 weeks? |
17 | Over the last 4 weeks, how often have you been bothered by feeling nervous, anxious, on edge or worrying a lot about different things? |
The initial prototype of the tool consisted of three content areas:
First impressions of the app were positive, with participants reporting it was easy to use, as illustrated in the following quote:
I think the app in general looks quite clean and clinical, for some people a good thing, if they feel they have a problem they want to be handled in a professional way.
The purpose of the app was seen to be to raise awareness of existing mental health problems for the individual, to give hope for improvement, and to motivate the individual to pursue the next step in getting help, as illustrated in the following quotes:
It’s about education, awareness, by answering these questions your becoming aware of some problems you may have, presenting hope, there are things people can do, that you can get better.
If you are this person, then you get this educational fact—these are the areas [you need help with], [this is] how it is impacting you, [it’s] a nudge to get to the next step.
Several participants noted that the “next” button, presented under each question, should be removed to streamline the app, which was agreed upon by the rest of the group.
There was a clear mismatch for participants between nine of the 12 the visual icons and their intended meaning. For example, participants interpreted an icon depicting a pair of hands as a representation of “charity” or “religion,” when the intended concept was “health.”
Risk communication presented to participants in prototype 1.
Participants liked having a summary of their results reflected to them. However, they felt that only reflecting the “difficult” areas might have negative consequences, as illustrated in the following quotes:
That’s validation, yeah I feel that way.
You see your problem areas, when you are depressed you just feel bad but this makes it clear.
It could be a negative thing, like look at everything that’s wrong with me...If it were only one or two, that could be identification, but getting a lot back, that could be quite detrimental to some people...It would help to see what was working well.
Participants wanted more meaningful feedback about their results. They wanted an explanation of the severity of the problem, advice on prioritizing areas for attention, and a personalized treatment recommendation based on their results, as illustrated in the following quotes:
Maybe if there is the option to emphasize some problems here it would be better, for example, I’d like to be able to emphasize my sleep problems.
I think there needs to be an answer, to show you have a problem, to contact this GP or call this number, someone to discuss the results.
The risk communication component was identified as the most problematic aspect of the tool. Participants were concerned that presenting risk might make already depressed people feel worse, as illustrated in the following quotes:
If I get help I still have a one in three chance of still feeling bad?
If it has gone 2 months and you are still sick, it’s like there’s only one month left
Participants were confused that the app reported risk at a population level rather than their own personal risk of suffering depression in the future, as illustrated in the following quote:
Impersonal, I’m going to be pigeonholed.
Some participants misinterpreted the message that “with help, you will feel better in three months’ time,” as illustrated in the following quote:
It could be shorter intervals...I mean, if you’re suicidal and you have to wait three months.
All participants misunderstood the risk communication in the form of stick figures, and they felt that it had a negative message, as illustrated in the following quote:
If you are depressed maybe you identify with the sad figure, feels hopeless, not sure this works
Most participants expressed their dislike of the portrayal of risk using emotional faces, as illustrated in the following quotes:
Pretty scary, this looks like a Halloween pumpkin! It’s a bit impersonal.
I feel condescended to [by the animated face].
On the basis of the results of focus group 1, a second prototype was developed. The nine most problematic icons were removed and replaced with new icons (see
Example of icons replaced and more easily interpreted in prototype 2.
The “next” button was removed allowing screens to automatically transition from question to question once a response had been entered. The summary screen was redesigned to reflect areas that “seem to be ok for you right now” in addition to the “difficult” areas presented in prototype 1. Although risk communication was identified as problematic in the first focus group, it was included in the second focus group so that user preferences regarding this component could be explored further. An additional component, treatment recommendation, was added to the end of the tool. This screen informed participants that they could access an online portal with information about mental health treatment options and that a named health care professional was available to talk them through this portal.
Similar to focus group 1, participants felt that the second prototype was simple and easy to use. It was seen to be a tool that could guide a conversation with a health provider. Additionally, the app gave an opportunity for the individual to reflect on his or her symptoms, give hope for improvement, and motivate help-seeking. Two participants stated the following:
It would be useful in telling you things you didn’t know, you didn’t think about, and then you’d go into the GP and say, this is right, I’m not sleeping well, it would be a prompt.
This would be a starting point for talking to the GP, for sure, you could focus on what the problems actually are.
In the second prototype, participants reported that nine of the 12 icons were congruent with the intended concepts for each icon, with the remaining three approaching congruency. Participants indicated that the icons were helpful when interpreting the question and that they should be more prominent.
Although participants generally felt positively toward the summary screen, they also expressed concern that if the summary did not accord with the user’s experience, there was potential for a loss of trust in the tool, as illustrated in the following quotes:
I find it interesting just to look at the two sides, what seems to be shaping up ok, and where the struggle points are. So I think there is a bit of personal reflection that can go on the results page with these emblematic little icons, I find that quite interesting.
This is the make or break point, I mean if there are things here that people don’t see in their own lives, if it’s not an accurate reflection of what they answered, they’ll lose trust.
I was expecting more personalized results at the end, not just what I’m doing well in, something more detailed.
Like their counterparts in focus group 1, many participants interpreted the information as a negative prognosis, and there was a sense that the message was impersonal and untrustworthy, as illustrated in the following quotes:
If I’m depressed, I can always find the figure down at the bottom that doesn’t get better, if I were to look at that through a dark cloud, I would see myself.
I mean, if somebody is depressed, you don’t want to tell them “you’re depressed, you’re a sad face.”
It was a bit like it wasn’t even paying attention to the answers I gave, just saying ok you’ll be ok in 3 months.
Participants responded positively to the treatment recommendation screens but expressed a desire for more information from the treatment recommendation regarding what they could do get better, as illustrated in the following quotes:
Yeah its very positive, isn’t it, that you can get help, that’s great.
So on the results page, maybe more like you should do this, take action. I mean I already know my sleep is not good but what should I do, how should I get better.
Given that participants in both focus groups expressed problems with the risk communication, and in the absence of a compelling alternative, we removed this element of the app entirely. Information on the treatment recommendation screen was rewritten to direct participants to specific evidence-based treatment options depending on their predicted depressive symptom severity. Treatment recommendations were based on the principles of stepped care, where the intensity of treatments increased in line with symptom severity. So, for example, patients predicted to have mild symptoms were recommended to access Internet-based self-help and psychoeducation via the myCompass program [
Participants did not report any negative aspects of using the app, and all participants explicitly said it was professional and easy to use.
Consistent with focus group results, interview participants indicated that the app could raise awareness of their problems, give hope, and potentially motivate them to treatment, as illustrated in the following quotes:
It made me reflect on how my feeling have been over the past weeks and months, which did make me think, it is a bit more frequent than I thought or hoped it was.
Summary of iterative development process (N/A: not applicable).
Themes and feedback from focus group 1 | Revisions | Feedback from focus group 2 | Revisions | Feedback from interviews | Revisions | |
Dislike having to press “next” to navigate to next page | Remove next button | N/A | N/A | N/A | Removed a nonfunctional “tap here” button | |
Mismatch in interpretation for 9/12 icons | Revise nine icons | Correct interpretation for 9/12 icons, with the remaining three approaching congruency | N/A | Icons viewed as helpful for people with English as a second language | N/A | |
Seeing only where things are difficult could be detrimental | Include feedback on both “difficult” areas and areas that “seem to be ok for you right now” | Desire for more personalized results | None; app should be administered in a health care setting where responses can be discussed | Seeing “difficult” and “OK” areas useful for patients with depression—counteracts overgeneralization that everything is difficult | N/A | |
Focus on negative message | None; reassess in focus group 2 | Focus on negative message | Removed entirely from app | N/A | N/A | |
Need action-oriented message | Add recommendation to review available resources through online portal | Need more tailored recommendation | Revise recommendation to direct to specific evidence-based treatment, matched to predicted depressive symptom severity | Like being provided treatment option | Minor changes to phrasing |
It made me think about how could I get more help.
It would be a motivation, or maybe an opportunity.
The iconography was seen to be a seamless part of the app that could enhance understanding of the clinical prediction tool items. No participant commented on a mismatch between icon and question concept, as illustrated in the following quotes:
Yes the pictures make sense.
Seems pretty clear, and for people with English as a second language the infographics would help if they can’t understand the more complex words.
Refinements to the feedback and treatment recommendation and removal of risk communication in response to the focus groups appeared to improve the match between the app and participant needs. Although focus group participants felt the combination of the clinical prediction tool results and risk communication left them focused on the negatives, interview participants felt positively about the feedback and recommendation, as they felt it provided solutions and a way of moving forward, as illustrated in the following quotes:
This would feel pretty good...the fact that it offers options.
I’d be looking forward to the [treatment recommendation], to see what options were available, of more information.
The individual interviews identified only minimal changes in phrasing of the treatment recommendations and in one technical aspect of the app (a nonfunctional tap here button). These corrections were made in the final app.
In this paper, we describe the process of user involvement in the iterative development of an app designed to estimate prognosis and guide treatment choice for patients with current depressive symptoms (using the
This study is the first to our knowledge to use explicit user-centered design principles in development of an app-based clinical prediction tool for mental health in primary care. The long-standing problem of engaging patients in mental health treatment has not to date been solved by the advancement of health technologies such as apps, due in part to limited engagement with the technologies themselves. User-centered development has been posited to address this issue by leading to more acceptable, usable, and effective mental health technologies. For example, a lifestyle and mental health screening tool developed using a user-centered approach was deemed acceptable and usable by end users [
As discussed above, Norman’s formative theory of user-centered design suggests that an individual’s interaction with a product can be conceived of as three levels of processing: visceral, behavioral, and reflective [
Although the majority of participants in this study did not suffer from depression, they consistently interpreted the communication of their risk for persistent depression through a negative lens. As far as we are aware, this is the first study to have examined how best to communicate risk for depression. Given the negative biases inherent in many psychiatric illnesses, it is possible that the requirements for effectively communicating risk for these conditions may be very different to those for chronic physical conditions (including, eg, genetic disorders and cancer), which have to date received most attention in the risk communication field.
High-quality communication is considered an important component of shared decision making [
Given that risk communication has been shown to be highly influential on patient decision making [
During our development process, it became apparent that participants were more interested in the app as an action-oriented rather than informational tool. They desired a tailored treatment recommendation based on their individual symptoms, rather than information on their risk of persistent depression, even when this provided generic advice on the benefits of help-seeking (ie, “7 out of 10 people with extra help from a GP or health professional feel better”). This finding is consistent with research showing that patients with mental illness desire personalized information about available treatment [
Importantly, participants in this trial were free to respond to the app however they chose, and the treatment recommendation we presented in prototype 3 was designed to identify the acceptability of this action-oriented message relative to the more passive information presented in prototypes 1 and 2 and not to provide specific treatment advice. Although our findings suggest this solution-focused approach was preferred, we did not set out to test how best to personalize treatment recommendations for depression. This is likely to be an important area of future investigation; although several models of personalized care have been proposed, particularly in the fields of cancer [
We employed an iterative development process that allowed us to improve the app on a step-by-step basis. This allowed us to track shortcomings at each stage and avoid any flow on effects by making appropriate changes to the app as we became aware of them. We were also able to add new functions to the app as suggested by participants in the focus groups.
Our use of qualitative methodologies is also a strength of this study. Focus groups are a valuable way to generate information about what a group of people think is important and how they understand a problem [
There is a risk for a biased sample in both our focus groups and our interviews. Participants responded to recruitment advertising in the community that made clear the focus on mental health and mobile apps, so it is likely we recruited primarily individuals with interest in or experience of one or both of these topics. Additionally, we recruited in an area with a highly-educated, urban population, and therefore, our recruited population may not reflect the demographics of all end users of the app.
The app developed in this study is being used in a randomized controlled trial to identify whether delivering the
In this study, we described how an iterative, user-centered design process led to an easy to use, engaging, and motivating app that assists in assessing prognosis and guiding treatment choice for patients with depressive symptoms. Future initiatives aimed at improving engagement with mental health assessment or treatment may consider digital apps as a platform of delivery.
electronic health
general practitioner
The data used to develop the
None declared.