This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.
The benefits of involving those with lived experience in the design and development of health technology are well recognized, and the reporting of co-design best practices has increased over the past decade. However, it is important to recognize that the methods and protocols behind patient and public involvement and co-design vary depending on the patient population accessed. This is especially important when considering individuals living with cognitive impairments, such as dementia, who are likely to have needs and experiences unique to their cognitive capabilities. We worked alongside individuals living with dementia and their care partners to co-design a mobile health app. This app aimed to address a gap in our knowledge of how cognition fluctuates over short, microlongitudinal timescales. The app requires users to interact with built-in memory tests multiple times per day, meaning that co-designing a platform that is easy to use, accessible, and appealing is particularly important. Here, we discuss our use of Agile methodology to enable those living with dementia and their care partners to be actively involved in the co-design of a mobile health app.
The aim of this study is to explore the benefits of co-design in the development of smartphone apps. Here, we share our co-design methodology and reflections on how this benefited the completed product.
Our app was developed using Agile methodology, which allowed for patient and care partner input to be incorporated iteratively throughout the design and development process. Our co-design approach comprised 3 core elements, aligned with the values of patient co-design and adapted to meaningfully involve those living with cognitive impairments: end-user representation at research and software development meetings via a patient proxy; equal decision-making power for all stakeholders based on their expertise; and continuous user consultation, user-testing, and feedback.
This co-design approach resulted in multiple patient and care partner–led software alterations, which, without consultation, would not have been anticipated by the research team. This included 13 software design alterations, renaming of the product, and removal of a cognitive test deemed to be too challenging for the target demographic.
We found patient and care partner input to be critical throughout the development process for early identification of design and usability issues and for identifying solutions not previously considered by our research team. As issues addressed in early co-design workshops did not reoccur subsequently, we believe this process made our product more user-friendly and acceptable, and we will formally test this assumption through future pilot-testing.
In January 2019, the National Health Service published its long-term plan, setting out key ambitions for the next 10 years. One of the most ambitious targets of this plan was in the field of digital technology, with a vision toward increasing care at home using remote monitoring and digital tools [
Within the context of software development, co-design can be defined as a process that draws on the shared creativity of software developers and people not trained in software working together [
In dementia research, patient and public involvement (PPI) and co-design is still a developing field [
An area in which digital technology can help with the care and management of dementia is through the monitoring of cognitive change and variability, which is an issue that is considered important for this population [
Recent developments in computerized cognitive testing have made it possible to measure microlongitudinal patterns of cognitive function [
Despite the diverse and divergent lived experiences of those living with dementia, software apps are rarely designed with this patient population in mind [
Agile software development focuses on collaboration with users and rapid software deployment [
In this paper, we describe how we modified co-design approaches to involve members of the public living with dementia and their care partners in the production of a smartphone app. Although we worked specifically with people with dementia, the principles could be applied to other patient groups who do not find it easy to engage with standard co-design approaches. We also explain the benefits of the Scrum development methodology as a way of integrating user feedback into the design and development process.
Public involvement in research is defined by the National Institute for Health Research’s INVOLVE as research being conducted
Working alongside individuals living with cognitive impairments necessitates a tailored approach to involvement and co-design. Therefore, it is necessary to balance facilitating meaningful involvement alongside being mindful of individuals’ capacity, capability, and preferences.
Therefore, we approached the challenge of co-design alongside individuals with cognitive impairments by adopting the following three methodological steps: (1) end-user representation at research and software development meetings via a patient proxy; (2) equal decision-making power for all stakeholders based on their expertise; and (3) continual user consultation, user-testing, and feedback.
On the basis of the combination of a short timescale for app development, limitations in the availability of clinical advisors, and a desire to reduce unnecessary burden on contributors living with dementia and their care partners, we chose to represent the patient or public voice at research group meetings via a proxy. Our proxy was a PPI officer who worked alongside our research group. They were responsible for developing and facilitating co-design workshops and representing end users at research group meetings. This ensured that the patient voice was represented in all important decisions and was given equal weight as the voice of other research team members.
Input from those with lived experience of cognitive impairments was integral to the development of this app. Therefore, those involved were encouraged to input into all the elements of the design process. To this end, input from those with lived experience led to 13 design alterations across the life of the project (listed in the following sections). Feedback from co-design workshops also led to the removal of 1 cognitive test, which was deemed too challenging for those living with dementia, and rebranding of the app.
Following the development of an initial prototype app, which was designed to act as a scaffolding example app for use in the first co-design workshop, all subsequent software development sprints were based on end-user feedback. This ensured that any emerging design or software features were reviewed and modified by end users before being added to the following sprint. The extent of the end-user modifications adopted in the creation of this app can be visualized by comparing
(A) Simple test of cognitive processing speed and (B) a more cognitively demanding tests of working memory developed by the software development team before patient and public involvement input. PPI: patient and public involvement.
(A) Redesigned shopping list task and (B) new shopping list+ task following first patient and public involvement workshop.
Final software alterations following second patient and public involvement workshop showing flow through the app.
A total of 4 co-design workshops were run collaboratively with community dementia support groups and were tailored to those living with cognitive impairments. Workshops were planned around familiar venues and, in some cases, to coincide with existing support group meetings. The materials used were dementia-friendly [
Participants of workshops 1 and 2 comprised a mix of individuals living with a dementia diagnosis and current and past care providers of people living with dementia (workshop 1: 5/7, 71% with dementia and 2/7, 29% current or past carers; workshop 2: 3/6, 50% with dementia and 3/6, 50% current or past carers). Participants were recruited from 2 local dementia support groups following informal visits and presentations from the members of the research team. Approximately 30% (3/10) of the participants (1/3, 33% living with dementia, and 2/3, 67% current or past carers) attended both workshops 1 and 2.
Workshops 1 and 2 adopted a similar format: each workshop lasted approximately 2 hours and included (1) lunch and informal ice-breaker conversations; (2) a short, accessible project discussion and feedback; (3) introduction and testing of a visual working prototype; and (4) the collection of informal one-to-one and group feedback on the prototype. Workshop 1 also included an activity in which participants were encouraged to discuss their views on and responses to candidate words and phrases for the app’s name. This was achieved via a discussion of flashcards containing keywords associated with the cognitive testing app (eg, cognition, test, training, research, brain,
These workshops were designed to involve patients in the co-design of the
Workshops 3 and 4 took place 2 months after workshop 2 and spanned a week-long period of user-testing. Participants were recruited by the research team from a local community dementia support group with a focus on technology; these were older individuals with current or past experience of supporting someone living with dementia (n=4 current or past carers). This group was targeted as we expected individuals attending a technology-focused group to be inclined to take part in our week-long user-testing phase.
Potential participants from this group were approached during one of the group’s regular meetings, and the project was introduced, and the app was demonstrated. From this meeting, 4 individuals consented to the 7-day testing period, whereas 4 declined, citing time commitments as a barrier to participation. We returned to the group the following week to distribute phones preloaded with the
Workshop 4 took place after the week-long user-testing period and comprised a short informal discussion regarding participants’ experiences of using the software. Participants were asked to comment not only on their own experience with the software but also on its suitability for someone living with a dementia diagnosis. Paper diaries were referred to during this discussion as a memory prompt; data from these diaries were not stored or analyzed further outside this workshop.
Feedback from each workshop was reviewed and discussed by the project team shortly after each workshop. This resulted in an agreed set of changes for the subsequent shippable products. As with any feedback of this nature, the project team prioritized changes based on both the effort required to implement and the likely impact on the end user.
The
Scrum uses sprints, which are timeboxed development efforts, usually 1 to 4 weeks in duration [
Visual working prototypes were used during the initial PPI workshop events. These prototypes allowed users to see interactive screens that portrayed key design elements and the flow through the app and were produced by a user experience designer embedded in the software team. This enabled early testing with the research team and target user group without requiring significant investment in software development.
The initial prototypes were based on a validated computerized cognitive task, in which participants were presented with a short list of grocery items (eg, carrots) and a quantity for each [
Before the first PPI workshop, research team members, who had experience in working with people with cognitive impairment, reviewed the prototypes. They suggested modifications to simplify these tasks, making them more visually appealing and quicker to navigate to maintain user engagement. These modifications included the following:
The addition of images to both tasks
Start screens containing instructions on how to complete the tasks
Simplification of the second, harder, N-back task by introducing images of meals and the use of the days of the week, as these were familiar items (
The team also generated the provisional name
Health-e-Mind
for the prototype app
These prototypes were then presented for review to the target user group during co-design workshops arranged by the team’s public involvement lead. On the basis of initial feedback on the visual working prototypes, the software team began the development of version 1 of the app. This process continued in an iterative manner for each of the key components of the app.
Throughout the project, the research and software development teams met on an approximately monthly basis to review the approach being taken. This enabled improvements to be made to the process within the project and resulted in some key learning points that could be applied to future projects.
The initial prototype comprised a shopping list task and an N-back task (
Much of the participant feedback collected during workshop 1 correlated poorly with the research team’s prior assumptions. Some of the key feedback from workshop 1 and the actions taken to address this feedback are presented in
Feedback and resulting software modifications from workshop 1.
Item and feedback | Software modification | ||
|
|||
|
Testers found pictures to be distracting. Specifically, it was noted that some pictures were confusing (ie, onion and apple looked similar) and that the images shifted focus away from reading written information, making it harder to follow instructions. |
From this feedback, the software development team chose to remove images from both tasks. | |
|
Participants reported that the color scheme (dark blue text on a light blue background) might be inappropriate for those with reading or perceptual difficulties. Black writing on a yellow background was suggested to be optimal for improving reading speed and for assisting people with reading difficulties. |
The display was altered to black text on a yellow background. | |
|
|||
|
Testers noted that detailed introductory text explaining the task was not necessary for the simple shopping list task. Indeed, several participants stated that they skipped reading the introductory message and were still able to perform the task. |
The development team removed the introduction text from this task, replacing it with a simple “Are you ready to start <yes>, <no>” structure. | |
|
It was noted that the screen flow used in the shopping list task left some participants confused. Specifically, several participants felt that displaying the shopping list followed by a probe question was less logical (harder to follow) than displaying the probe question first followed by the shopping list. |
Text flow was altered in line with workshop preferences in the next design iteration ( |
|
|
The shopping list task relied on measures of task completion time as a proxy for cognition. Therefore, instructions for this task asked participants to complete the task “As quickly as possible.” Workshop participants noted that although they read this instruction, they did not feel a sense of urgency while completing the task, suggesting that they had not remembered it. |
To encourage users to complete the shopping list task as quickly as possible, the development team added a circular bar countdown timer to the bottom of the task screen. | |
|
|||
|
Participants felt that the written explanation for the second, harder (N-back) task was insufficient, and, even after a verbal explanation and demonstration, many were still uncomfortable interacting with this task. |
It was decided that the N-back task was too complicated and not fit for purpose. Therefore, the development team removed this task and replaced it with a more memory-intensive variant of the shopping list task, subsequently referred to as shopping list+, in which the shopping list was removed from the screen before and during each probe question ( |
|
|
|||
|
Participants were asked whether they would appreciate feedback on their performance on these tasks. Opinions were mixed, with some participants wanting graphed data, or indications of low and high performance, whereas others felt that feedback on poor performance might reduce their motivation to complete future tasks. |
It was decided that a generic positive feedback message would be added to the tasks, that is, “Great job, well done.” | |
|
|||
|
Participants did not like the name the research team chose for the app—Health-e-Mind. Most were unaware that the e stood for electronic, and 1 individual mentioned that it made him think of drug use. Group feedback on flashcard word association included the following: Brain was seen to be too biological, whereas mind was preferred as this sounded more holistic and accessible. Although some participants were comfortable with the words test and memory, others suggested that these terms may be off-putting and could cause anxiety. It was suggested that the word test could be replaced by check as this sounded less daunting and clinical. Participants also liked the addition of the word my to the name, personalizing the app. |
From this feedback, the team chose to change the name to |
Overall, workshop participants seemed positively disposed to the purpose of the app and said that assuming certain alterations were made, they would be willing to interact with such a program on a subdaily basis.
Building on feedback from workshop 1, the software team undertook a second development sprint, updating the original prototype to incorporate feedback from workshop 1, including removal of N-back task and replacement with shopping list+ task (
Feedback from this workshop and actions taken are listed below in
Feedback and resulting software modifications from workshop 2.
Item and feedback | Software modification | |
|
||
|
For the new shopping list+ task, a number of participants noted that until they reached the screen containing the question and multiple-choice answers, they did not realize that they had to remember both the objects listed and the associated number of items. | This was addressed by altering the prompt used on the first screen of this task to read “Remember how many of each item.” |
|
||
|
For the shopping list+ task, several participants were unable to read the entire list of 4 items displayed on the first screen before it timed out and moved on to the probe question. | To address this, the team increased the display duration of the first screen to give users more time to read the instruction and object list. They also reduced the list length from 4 items to 3. |
|
This version included a countdown timer on both tasks, specifically, a circular bar countdown timer. Although most testers said that they did not notice this timer, they did note that they had been trying to respond quickly. However, 1 tester did say that she noticed the timer and felt stressed about completing the task in time. | The countdown timer remains in the app as a visual cue to complete in a timely manner. However, the timer was altered from a model which showed a finite time counting down to a timer that did not count down to a finite point. It was hoped that this maintained a sense of urgency but would mitigate stress caused by a finite countdown. |
|
||
|
This version of the app included a generic positive feedback message after each task that was not linked to performance, that, “Well done.” This was included to avoid user discouragement because of low scores. However, participants did not appreciate being given positive feedback when they were aware that they had performed badly. | Feedback was altered to maintain a positive tone while also remaining performance neutral: “Task complete! You have finished the task. See you at the next alarm.” |
In line with feedback from the first workshop, no major objections were raised to the usability and acceptability of the
Workshops 3 and 4 aimed to test the software alterations implemented as a result of workshop 2 and to trial new functionality, including prompts and alarms. Feedback from these workshops and the actions taken to address this feedback are listed as follows:
Although most participants complied with the assigned in-app tests, the most common reasons for noncompliance were
The alarm was not loud or long enough.
Fear of breaking the phone if they took it out of the house.
Fatigue at being asked to complete tasks 4 times a day.
To address these concerns, the software team implemented the following modifications:
Increased the alarm volume and duration
Decided to provide phone cases when using a study phone to reduce fear of dropping or damaging phones
Decided to implement further PPI regarding prompt number and frequency before further implementation or testing
One tester also noted in her diary that, for the first 2 days of testing, the phone did not register her responses, and therefore was timing out on the tests. Similar issues had surfaced with other testers to a lesser extent in some of the preceding workshops. On the basis of this feedback and previous observations, it was noted that some participants were holding the response buttons rather than tapping them, perhaps reflecting a level of unfamiliarity with mobile technology among this group. Therefore, to address this issue the software was modified to identify both on-press and on-hold events as valid answers.
Testers were confident using both tasks, although they noted that they found the shopping list+ task harder and that it required more concentration. Testers with experience caring for someone with dementia stated that they believed that these tasks could be completed by someone living with dementia, assuming they had support from a care partner.
We used a co-design approach to develop the
By adopting Scrum in this context, we were able to realize the benefits of an iterative co-design approach, with the software evolving throughout each of the 4 workshops. In addition, our use of prototype designs in the first 2 workshops provided the team with a low-cost opportunity to receive feedback and evaluate the idea before commencing software development.
Participants in the workshops gave positive feedback about the experience, showed strong engagement during the sessions, and provided constructive comments on the app. Notably, points raised in early workshops did not resurface in the week-long test undertaken by a different participant group at a later stage. This could be because users in the week-long test focused on different aspects of the app. However, it could also suggest that our approach was effective in addressing design issues at an early stage of development.
Although the co-design methodology enabled the team to iteratively develop the app, we still had to overcome several challenges. For instance, it was agreed early on that embedding of an end user (in this case, someone living with a dementia diagnosis into the Scrum team), as per co-design best practice [
There were some logistical challenges in running the Agile development using a co-design process. Specifically, recruiting participants for workshops required multiple interactions with community groups to garner interest in the project and plan suitable times and venues for workshops. Therefore, it was necessary to set the date for each workshop several weeks in advance. However, software development does not always run according to the plan, as it is not possible to estimate development tasks with a high degree of accuracy. Therefore, there is a risk that a date could be set for a workshop only for the software not to be ready in time. We mitigated this risk by setting a date for each workshop, which allowed a suitable leeway for any unexpected delays. We also worked with an experienced and established Scrum team, meaning that the estimates could usually be provided with a reasonable level of confidence.
Given the need for health research, particularly the development of health technology, to be approached in a patient-centered manner [
We also highlighted several instances where input provided by people with lived experience of dementia helped our team to identify and address usability issues early in the development process, speeding up delivery and reducing software development waste. Our experience evidences how co-design can benefit the software development process and be sustainably tailored to the needs of diverse patient populations [
The next step for the
patient and public involvement
This work was funded by the Medical Research Council Momentum Award: Institutional pump-priming award for dementia research (MC_PC_16033). The authors would like to thank the Alzheimer’s Society and their patient network for support in the setup and design of workshops and all the participants who contributed to this project, including special thanks to the Peter Quinn Friendship Group, Together Dementia Support, and the Humphrey Booth Resource center.
None declared.