Background: Human-centered design (HCD) approaches to health care strive to support the development of innovative, effective, and person-centered solutions for health care. Although their use is increasing, there is no integral overview describing the details of HCD methods in health innovations.
Objective: This review aims to explore the current practices of HCD approaches for the development of health innovations, with the aim of providing an overview of the applied methods for participatory and HCD processes and highlighting their shortcomings for further research.
Methods: A narrative review of health research was conducted based on systematic electronic searches in the PubMed, CINAHL, Embase, Cochrane Library, Web of Science, PsycINFO, and Sociological Abstracts (2000-2020) databases using keywords related to human-centered design, design thinking (DT), and user-centered design (UCD). Abstracts and full-text articles were screened by 2 reviewers independently based on predefined inclusion criteria. Data extraction focused on the methodology used throughout the research process, the choice of methods in different phases of the innovation cycle, and the level of engagement of end users.
Results: This review summarizes the application of HCD practices across various areas of health innovation. All approaches prioritized the user’s needs and the participatory and iterative nature of the design process. The design processes comprised several design cycles during which multiple qualitative and quantitative methods were used in combination with specific design methods. HCD- and DT-based research primarily targeted understanding the research context and defining the problem, whereas UCD-based work focused mainly on the direct generation of solutions. Although UCD approaches involved end users primarily as testers and informants, HCD and DT approaches involved end users most often as design partners.
Conclusions: We have provided an overview of the currently applied methodologies and HCD guidelines to assist health care professionals and design researchers in their methodological choices. HCD-based techniques are challenging to evaluate using traditional biomedical research methods. Previously proposed reporting guidelines are a step forward but would require a level of detail that is incompatible with the current publishing landscape. Hence, further development is needed in this area. Special focus should be placed on the congruence between the chosen methods, design strategy, and achievable outcomes. Furthermore, power dimensions, agency, and intersectionality need to be considered in co-design sessions with multiple stakeholders, especially when including vulnerable groups.
Health systems are experiencing a progressive imbalance between available resources and increasing needs. The world population is growing, and the incidence of chronic diseases is rising; however, the funds allocated to health care are limited [, ]. The need to provide optimized, individualized, and person-centered care is growing. Addressing these competing needs and complex problems requires novel and creative approaches for the development of health care solutions. Design approaches to health care promise to aid the development of innovative, effective, and person-centered solutions to health challenges, supporting the realization of a future for health care that is preventative, personalized, and participatory in nature [ , ]. Different medical disciplines are increasingly applying human-centered design (HCD) to a range of complex questions, from process optimization to product design and social innovation [ - ]. HCD is often described as an iterative, collaborative, and people-centered approach for designing products, services, and systems and is argued to be particularly well-suited for solving complex challenges [ ]. In recent years, a growing number of health care professionals have applied HCD to develop person-centered health care solutions in collaboration with patients [ ]. For example, the Department of Obstetrics and Gynecology at Mayo Clinic used HCD to develop a new prenatal care model designed to demedicalize a healthy pregnancy experience [ ]. By enabling women to meaningfully participate in the process through the use of self-measurement tools, their levels of engagement, sense of control, confidence, and reassurance significantly increased. Another example is the nurse-led quality improvement project at Kaiser Permanente Northern California. HCD principles were used for a patient-centered approach to improve inpatient pain management. The experiences of frontline nurses, patients, and managers were collected, evaluated, and applied to improve the care experience of patients and the work experience of care providers [ ].
However, the application of HCD beyond the design sector and its adoption in health research is still in its infancy [, ]. The number of HCD studies that describe a full project cycle is limited, and even fewer publications focus on the evaluation of research projects that use HCD [ ]. A recent scoping review on the application of HCD in global health provided a first overview of its application and health outcomes in public health. The review concluded that increased methodological rigor in the application and reporting of HCD is needed to allow for more acceptance and integration of design practices into research and development [ , ]. However, currently, there is no integral collection of HCD approaches and methods used in the development of health innovations. We performed this review to fill this gap.
HCD evolved from the collaborative design movement and covers a range of overlapping collaborative processes and techniques such as, and not limited to, participatory design, ethnography, cocreation, contextual design, co-design, and empathic design. These processes share several principles: the active involvement of users, an iterative design process, and the organization of multidisciplinary teamwork [- ]. The term HCD, as a collaborative multimethod approach, is often used interchangeably with terms such as design thinking (DT) or user-centered design (UCD) because of their similar design philosophies. DT is an approach that prioritizes developing empathy for users, working in collaborative multidisciplinary teams, and using an iterative process with rapid prototyping techniques for potential solutions [ ]. Similarly, UCD, although deeply rooted in human-computer interactions, is described as both a philosophy and a set of methods in which end users actively influence and are involved in the design process [ ]. As these principles are akin to those of HCD, this review includes both DT and UCD as variations that apply HCD principles to further explore their similarities and differences.
In this review, we systematically explore the following question: how is HCD, and the closely related approaches of DT and UCD, applied in the development of innovations for health research? We specifically focus on the applied research methodologies and design methods used throughout the study. We investigate the level of engagement of end users during the HCD design processes. As a result, we provide an overview of the current application practices of HCD in health research and a practice-oriented collection of the used design methods to aid future researchers in their choice of methodology.
A total of 2 librarians, 1 from medical sciences and 1 from social sciences, assisted with the development of a search strategy and the selection of the appropriate databases. Our research included health research related to biomedical, nursing, and allied health and public health sciences. We performed multiple test runs to optimize the search strategy before the first search in July 2019. A final search was performed in August 2020 to update the included publications. The protocol for this review can be found in.
We performed electronic searches in the following databases: PubMed, CINAHL, Embase, Cochrane Library, Web of Science, PsycINFO, and Sociological Abstracts. Gray literature searches were not included. We searched for studies in the English language that were published between 2000 and 2020. For medical databases, the following terms were used: Human-centered OR Human-centred OR User-centered OR User-centred AND Design OR approach OR Design thinking. For nonmedical databases, the following search terms were added: Health OR Medic OR Clinic. The exact search algorithms per database can be found in.
We included health research studies that applied HCD, UCD, or DT; focused on the development process of a health innovation; and provided a detailed description of the design process, which included the applied process steps or phases, the applied design methods per process step or phase, and a description of the involved design team and end users. We excluded studies if they did not focus on the design process and did not provide a detailed description of the design processes and the HCD, DT, or UCD methods used in the study. No specific criteria were formulated related to the end user population.
We conceptualized a health innovation as it is applied within the context of health research according to the World Health Organization concept of “Health innovation identifies new or improved health policies, systems, products and technologies, and services and delivery methods that improve people’s health and wellbeing.”
Screening and Data Extraction
We downloaded relevant papers on the Endnote bibliographic software (Clarivate Analytics) and removed duplicates. We then uploaded the Endnote database with the remaining papers on Rayyan, a web application that supports the initial screening of publication titles and abstracts . A total of 2 reviewers independently screened the titles and abstracts for inclusion eligibility and subsequently screened the full-text articles independently for inclusion. We resolved disagreements through discussions. To determine the level of agreement, both Cohen κ value and the percentage of agreement were calculated.
Data Retrieval and Analysis
We conducted a stepwise analysis of the included publications, focusing on (1) study characteristics, including design phases and methods, (2) level of end user involvement, and (3) quality assessment.
We extracted the following data from each article: year of publication, first author, title of the study, aim of the study, end user of the innovation, type of innovation, study design, design approach, design approach reference, design process phases, applied research and design methods, and the design-based problem-solving strategy.
For the classification of the applied qualitative and quantitative research and design methods, research methods were defined as “methods traditionally used within scientific research, oriented towards understanding” and design methods were defined as “methods not traditionally used with scientific research, oriented towards action or solution creation for defined problems” [, ]. These distinctions were made based on the discussions between the authors. To define the design-based problem-solving strategy, we used the categories of problem-focused strategy (PFS) versus solution-focused strategy (SFS). Studies that use a PFS aim to define or reframe the problem before formulating possible solutions. Studies that use an SFS approach focus on the development of a predefined solution, investing little time in defining or reframing the problem [ ].
Level of Involvement of the End User
To define the level of engagement of the end user, we adopted a modified framework proposed by Druin , which was originally used to categorize the participating role of children in a design process. The participating roles were users, testers, informants, or design partners, with increased levels of involvement for each role. Users help researchers and designers understand the problem context and user needs. The role of testers builds upon this role by including end users as part of the initial or functional prototype testing. In the role of informants, end users are involved during various stages of the design process, and they contribute to idea generation and provide feedback on the initial and functional prototypes. In the role of design partners, end users are considered equal partners of the design team and are involved at all stages of the design process and fully included during the decision-making processes.
We assessed the quality of reporting and analysis of the study designs using the Mixed Methods Appraisal Tool (MMAT), which allows for the appraisal of studies for literature reviews that include qualitative, quantitative, and mixed methods studies . As most HCD studies apply a multimethod approach, we considered this tool fit for purpose. The MMAT contains 2 general screening questions and 5 study design–specific criteria for assessing quantitative and qualitative studies. For mixed methods studies, we applied both sets of criteria, in addition to 5 specific mixed methods criteria. The scores per item could vary between yes (criterion is met), no (criterion is not met), and can’t tell (paper did not report appropriate information to rate this criterion).
One of the authors first performed the data retrieval and conducted the stepwise analysis described above. Subsequently, both authors reviewed and discussed the results.
In the following sections, we have provided an overview of the literature search results and the study characteristics of the included studies. Subsequently, several aspects of the studies have been highlighted, including the applied design theories, guidelines, strategies, and design process steps. Furthermore, we evaluated the applied research and design methods and the role in which end users were involved throughout the studies.
Our literature search identified 7560 records. Of the 7560 papers, after the removal of 4072 (53.86%) duplicates and exclusions on the basis of abstract for 3097 (40.97%) papers and full text for 309 (4.09%) papers, 82 (1.08%) articles were included in the final analysis (). Interrater agreement on the inclusion and exclusion of the studies was 96%, with Cohen κ=0.81.
Of the 82 included papers, 68 (83%) were published between 2015 and 2020. Most studies originated from the United States (34/82, 41%), the Netherlands (7/82, 9%), the United Kingdom (6/82, 7%), and Canada (6/82, 7%). Most studies focused on patients as the end users and developed health innovations with a focus on improving patient care. The most common type of study design was a mixed methods strategy (47/82, 57%), that is, a combination of qualitative, quantitative, and design methods. Of the 82 studies, 33 (40%) combined only qualitative methods with design methods. A detailed overview of the study characteristics is presented in.
|Author||Country||Title||End user population||Innovation type||Design approach||Study design|
|Bae et al ||Korea||Development of a user-centered health information service system for depressive symptom management||Patients who experience depression||Web-based system||User-centered system development||Mixed methods|
|Birnie et al ||Canada||ICanCope PostOp: user-centered design of a smartphone-based app for self-management of postoperative pain in children and adolescents||Children and adolescents who have recently undergone any type of day surgery||Pain self-management app||UCDa||Mixed methods|
|Brox et al ||Norway||User-centered design of serious games for older adults following 3 years of experience with exergames for seniors: a study design||Seniors||Serious game||UCD||Mixed methods|
|Cairns et al ||United Kingdom||Rethinking the foam cosmesis for people with lower limb absence||People with lower limb absence||Foam cosmesis for prosthetic limbs||User-centered product design||Mixed methods|
|Carey-Smith et al ||United Kingdom||A user-centered design process to develop technology to improve sleep quality in residential care homes||Older people with sleep or wake pattern disturbance||Sleep improvement technology||UCD||Qualitative|
|Caro et al ||Mexico||FroggyBobby: an exergame to support children with motor problems practicing motor coordination exercises during therapeutic interventions||Children with motor coordination problems||Exergames for children with motor problems||UCD||Qualitative|
|Catalani et al ||Kenya||A clinical decision support system for integrating tuberculosis and HIV care in Kenya: a human-centered design approach||HIV clinical care providers||Clinical shared decision support system||HCDb||Mixed methods|
|Cawood et al ||New Zealand||Creating the optimal workspace for hospital staff using human centered design||Hospital staff||Nonclinical workspaces||HCD||Qualitative|
|Civan-Hartzler et al ||United States||Bringing the field into focus: user-centered design of a patient expertise locator||Survivors of breast cancer||Patient expertise locator for web-based health communities||UCD||Qualitative|
|Connelly et al ||United States||Development of an ecological momentary assessment mobile app for a low-literacy, Mexican American population to collect disordered eating behaviors||Mexican-American women||Patient experiences assessment app||User-centered, iterative design||Mixed methods|
|Crespin et al ||Canada||Feasibility of adapting the fundamentals of laparoscopic surgery trainer box to endoscopic skills training tool||Surgeons and gastroenterologists||Laparoscopic surgery training box||UCD||Mixed methods|
|Curtis et al ||United Kingdom||Targeting parents for childhood weight management: development of a theory-driven and user-centered healthy eating app||Parents of children with weight management problems||Healthy eating app||UCD||Mixed methods|
|Dabbs de Vito et al ||United States||User-centered design and interactive health technologies for patients||Patients with a lung transplant||Personal health tracking app||UCD||Mixed methods|
|Das and Svanaes ||Norway||Human-centered methods in the design of an eHealth solution for patients undergoing weight loss treatment||Patients undergoing weight loss treatment||eHealth solution for weight loss treatment||HCD||Mixed methods|
|Davies et al ||United Kingdom||Recommendations for developing support tools with people suffering from chronic obstructive pulmonary disease: co-design and pilot testing of a mobile health prototype||People with COPDc||Mobile app for COPD self-management||User-centered, iterative design||Mixed methods|
|Dijkstra et al ||The Netherlands||Development of ehome, a mobile instrument for reporting, monitoring, and consulting drug-related problems in home care: human-centered design study||Home care nurses, general practitioners, and pharmacists||e-home solution for monitoring and consulting||HCD||Mixed methods|
|Eberhart et al ||United States||Using a human-centered design approach for collaborative decision-making in pediatric asthma care||Parents and children who are dealing with asthma management in a lower income environment||Physical decision-making aids||HCD||Qualitative|
|Erol Barkana and Açik ||Turkey||Improvement of design of a surgical interface using an eye tracking device||Surgeons who perform kidney tumor cryoablations||Eye-tracking device||UCD||Qualitative|
|Erwin et al ||United States||Development of a framework and tool to facilitate cost-of-care conversations with patients during prenatal care||Patients receiving prenatal care||Conversation framework||HCD||Qualitative|
|Ettinger et al ||South Africa||Building quality mHealthd for low resource settings||Community health care workers||mHealth app to inform clinical decision-making||HCD||Mixed methods|
|Fabri et al ||United Kingdom||Using design thinking to engage autistic students in participatory design of an online toolkit to help with transition into higher education||Students with autism||Web-based toolkit||DTe||Mixed methods|
|Farinango et al ||Colombia||Human-centered design of a personal health record system for metabolic syndrome management based on the ISO 9241-210:2010 standard||Individuals at risk for metabolic syndrome||Personal health record system||HCD||Mixed methods|
|Ferris and Shepley ||United States||The design of neonatal incubators: a systems-oriented, human-centered approach||Infants, medical practitioners, and family members||Neonatal incubators||HCD||Qualitative|
|Foley et al ||United States||Primary care women’s health screening: a case study of a community engaged human centered design approach to enhancing the screening process||Women receiving health screening in primary care||Health screening tool||HCD||Qualitative|
|Fortuna et al ||United States||Adapting a psychosocial intervention for smartphone delivery to middle-aged and older adults with serious mental illness||Middle-aged and older adults with serious mental illnesses||Mobile app for medical and psychiatric self-management||UCD||Qualitative|
|Furberg et al ||United States||A digital decision support tool to enhance decisional capacity for clinical trial consent: design and development||People diagnosed with fragile X syndrome and clinicians||Tablet-based decision support tool||UCD||Qualitative|
|Gačnik et al ||Slovenia||User-centered app design for speech sound disorders interventions with tablet computers||Children with speech-language pathology||App for speech sound disorder therapy||UCD||Mixed methods|
|Garvelink et al ||Canada||Development of a decision guide to support the elderly in decision making about location of care: an iterative, user-centered design||Older adults and their informal caregivers||Decision guide (physical)||UCD||Mixed methods|
|Garvelink et al ||Canada||Deciding how to stay independent at home in later years: development and acceptability testing of an informative web-based module||Seniors with loss of autonomy||Interactive website||UCD||Qualitative|
|Garvin et al ||United States||Descriptive usability study of CirrODS: clinical decision and workflow support tool for management of patients with cirrhosis||Clinicians caring for patients with cirrhosis||Clinical decision and workflow support tool (digital)||UCD||Mixed methods|
|Garzo et al ||France||Design and development of a gait training system for Parkinson’s disease||People with Parkinson disease||Gait training app||UCD||Mixed methods|
|Gaynor et al ||United States||A user-centered, learning asthma smartphone application for patients and providers||People with asthma||Mobile app for asthma self-management||UCD||Qualitative|
|Gill et al ||Canada||Feasibility and acceptability of a mobile technology intervention to support post abortion care (The FACTSf study phase II) after surgical abortion: user-centered design||Women who underwent an abortion||Web-based intervention for postabortion care support||UCD||Mixed methods|
|Giunti et al ||Spain||More stamina, a gamified mHealth solution for persons with multiple sclerosis: research through design||Young adults who have been diagnosed with multiple sclerosis||mHealth solution||UCD||Qualitative|
|Godinho et al ||Portugal||Improving accessibility of mobile devices with EasyWrite||Motor-disabled persons who experience text-entry difficulties when using mobile devices||Text-entry method for mobile devices||User-centered approach||Mixed methods|
|Gould et al ||United States||Development and refinement of educational materials to help older veterans use VAg mental health mobile apps||Older veterans||Educational material for mobile mental health apps||UCD||Mixed methods|
|Green et al ||United States||Tracking care in the emergency department||Emergency department physicians||Emergency department tracking board||UCD||Qualitative|
|Griffin et al ||United States||Creating an mHealth app for colorectal cancer screening: user-centered design approach||People at risk for colorectal cancer aged ≥50 years||mHealth screening solution||UCD||Mixed methods|
|Grossman et al ||United States||Leveraging patient-reported outcomes using data visualization||Patients with heart failure and health care providers for patients with heart failure||Data visualization||UCD||Mixed methods|
|Hafiz et al ||Denmark||The internet-based cognitive assessment tool: system design and feasibility study||Patients with unipolar and bipolar disorder||Web-based cognitive assessment tool||UCD||Mixed methods|
|Hardy et al ||United Kingdom||How inclusive, user-centered design research can improve psychological therapies for psychosis: development of SlowMo||People who fear harm from others||Digital solution for psychological therapy||UCD||Qualitative|
|Harte, R. ||Ireland||Human-centered design study: enhancing the usability of a mobile phone app in an integrated falls risk detection system for use by older adult users||Older adults with fall risk||Mobile app for fall risk detection||HCD||Mixed methods|
|Hartlzer et al ||United States||Design and feasibility of integrating personalized PROh dashboards into prostate cancer care||Patients following prostate cancer treatment||Patient dashboard||HCD||Mixed methods|
|Herschman et al ||Canada||Development of a smartphone app for adolescents with lupus: a collaborative meeting-based methodology inclusive of a wide range of stakeholders||Adolescents with lupus||Mobile app for adolescents||UCD||N/Ai|
|Horsky and Ramelson ||United States||Development of a cognitive framework of patient record summary review in the formative phase of user-centered design||Clinicians||Patient record summary review||UCD||Qualitative|
|Huberty et al ||United States||Development and design of an intervention to improve physical activity in pregnant women using Text4baby||Pregnant women||SMS text messaging||UCD||Mixed methods|
|Isenberg et al ||United States||An advance care plan decision support video before major surgery: a patient- and family-centered approach||Patients who are preparing for major surgery||Advance care planning decision support video||HCD||Mixed methods|
|Johnston et al ||United States||Designing and testing a web-based interface for self-monitoring of exercise and symptoms for older adults with COPD||Older adults with COPD||Web-based interface for self-monitoring of exercise||UCD||Mixed methods|
|Lan Hing Ting et al ||France||Examining usage to ensure utility: co-design of a tool for fall prevention||Older adults with fall risk||Balance assessment tool||HCD||Mixed methods|
|Luna et al ||Argentina||User-centered design improves the usability of drug-drug interaction alerts: experimental comparison of interfaces||Physicians||Drug–drug interaction alert system||UCD||Mixed methods|
|Ma, Wu and Chang ||Taiwan||A new design approach of user-centered design on a personal assistive bathing device for hemiplegia||Patients with stroke and hemiplegia||Personal assistive bathing device||UCD||Qualitative|
|Madrigal-Cadavid et al ||Colombia||Design and development of a mobile app of drug information for people with visual impairment||People with visual impairment||Mobile app for drug information||UCD||Qualitative|
|Marker and Monzon ||United States||Iterative development of a web-based intervention for families of young children with type 1 diabetes: DIPPer academy||Parents of children with type 1 diabetes||Web-based intervention||UCD||Mixed methods|
|Marko-Holguin et al ||United States||A two-way interactive text messaging application for low-income patients with chronic medical conditions: design-thinking development approach||Patients with low income and chronic medical conditions||Interactive SMS text messaging app||DT||Mixed methods|
|Martin et al ||Ireland||A qualitative study adopting a user-centered approach to design and validate a brain computer interface for cognitive rehabilitation for people with brain injury||People with brain injury||Brain-computer interface||UCD||Qualitative|
|McGinn et al ||Ireland||A human-oriented framework for developing assistive service robots||People with disabilities||Assistive service robot||HCD||Qualitative|
|McMullen et al ||United States||Designing for impact: identifying stakeholder-driven interventions to support recovery after major cancer surgery||Patients who recover from major cancer surgery||Web-based educational platform for patients||UCD||Qualitative|
|Melnick et al ||United States||Patient-centered decision support: formative usability evaluation of integrated clinical decision support with a patient decision aid for minor head injury in the emergency department||Emergency department physicians||Electronic clinical decision support||UCD||Mixed methods|
|Nunez-Nava et al ||Colombia||Human-centered development of an online social network for metabolic syndrome management||People with metabolic syndrome||Web-based social network||HCD||Mixed methods|
|Person et al ||Tanzania||Community co-designed schistosomiasis control interventions for school-aged children in Zanzibar||School-aged children||Intervention to reduce schistosomiasis transmission||HCD||Qualitative|
|Petersen, and Hempler ||Denmark||Development and testing of a mobile application to support diabetes self-management for people with newly diagnosed type 2 diabetes: a design thinking case study||People with newly diagnosed type 2 diabetes||Mobile app for newly diagnosed patients with type 2 diabetes||DT||Qualitative|
|Ragouzeos et al ||United States||Am I OK? using human centered design to empower rheumatoid arthritis patients through patient reported outcomes||Patient with rheumatoid arthritis||Dashboard to display PROs||HCD||Qualitative|
|Ray et al ||United States||Computerized clinical decision support system for emergency department–initiated buprenorphine for opioid use disorder: user-centered design||Emergency department physicians||Computerized clinical decision support system||UCD||Qualitative|
|Rothgangel et al ||The Netherlands||Design and development of a telerehabilitation platform for patients with phantom limb pain: a user-centered approach||Patients with phantom limb pain||Tele-rehabilitation platform||UCD||Mixed methods|
|Salmon et al ||Congo||Alternative ultrasound gel for a sustainable ultrasound program: application of human centered design||Local clinicians who use point of care ultrasound||Alternative ultrasound gel||HCD||Mixed methods|
|Schild et al ||Germany||A digital cognitive aid for anesthesia to support intraoperative crisis management: results of the user-centered design process||Anesthesiologists||Digital cognitive aid for intraoperative crisis management||UCD||Mixed methods|
|Sedlmayr et al ||Germany||User-centered design of a mobile medication management||People who use medication||Mobile interface for medication management||UCD||Mixed methods|
|Seeber et al ||Germany||A design thinking approach to effective vaccine safety communication||Parents and babies||Effective vaccine safety communication||DT||Qualitative|
|Sonney et al ||United States||Applying human-centered design to the development of an asthma essentials kit for school aged children and their parents||School-aged children and their parents who deal with asthma management||Asthma essential kit||HCD||Qualitative|
|Srinivas et al ||United States||Context-sensitive ecologic momentary assessment: application of user-centered design for improving user satisfaction and engagement during self-report||Middle-aged women with obesity||Patients’ experiences assessment app||UCD||Mixed methods|
|Stevens et al ||The Netherlands||The development of a patient-specific method for physiotherapy goal setting: a user-centered design||Physiotherapists and patients||A new method for goal setting||UCD||Qualitative|
|Taylor et al ||United States||User-centered development of a web-based preschool vision screening tool||Parents of preschool-aged children with amblyopia||Web-based vision screening tool||UCD||Mixed methods|
|Timmerman et al ||The Netherlands||Cocreation of an ICTj-supported cancer rehabilitation application for resected lung cancer survivors: design and evaluation||Health care professionals and survivors of lung cancer||ICT-supported cancer rehabilitation program||UCD||Mixed methods|
|Tucker Edmonds et al ||United States||Creation of a decision support tool for expectant parents facing threatened periviable delivery: application of a user-centered design approach||Prospective parents||Decision support tool||UCD||N/A|
|van der Weegen et al ||The Netherlands||The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: a user-centered design||People with chronic disease||Mobile monitoring and feedback tool||UCD||Qualitative|
|Vechakul et al ||United States||Human-centered design as an approach for place-based innovation in public health: a case study from Oakland, California||Citizens of Castlemont neighborhood||Novel programs to reduce inequities in infant mortality rates||HCD||Qualitative|
|Vermeulen et al ||The Netherlands||User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people||Older adults||Mobile interface for a monitoring system||User-centered development process||Mixed methods|
|Vilardaga et al ||United States||User-centered design of learn to quit, a smoking cessation smartphone app for people with serious mental illness||People with serious mental illnesses who smoke||Smoking cessation app||UCD||Mixed methods|
|Wachtler et al ||Australia||Development of a mobile clinical prediction tool to estimate future depression severity and guide treatment in primary care: user-centered design||People with depressive symptoms||App for improvement of treatment allocation for depression||UCD||Qualitative|
|Willard et al ||The Netherlands||Development and testing of an online community care platform for frail older adults in The Netherlands: a user-centered design||Frail older adults||Web-based community platform||UCD||Mixed methods|
|Woodard et al ||United States||The Pathways fertility preservation decision aid website for women with cancer: development and field testing||Women survivors of cancer||Decision aid website for young women with cancer||UCD||Mixed methods|
|Wysocki et al ||United States||A web-based coping intervention by and for parents of very young children with type 1 diabetes: user-centered design||Parents of young children with type 1 diabetes||Web-based coping resource||UCD||Qualitative|
aUCD: user-centered design.
bHCD: human-centered design.
cCOPD: chronic obstructive pulmonary disease.
dmHealth: mobile health.
eDT: design thinking.
fFACTS: factors affecting combination trial success.
gVA: veterans affairs.
hPRO: patient-reported outcome.
iN/A: not applicable.
jICT: information and communication technology.
Design Theories and Methodologies
This review explores the various applications of HCD approaches, including HCD, UCD, and DT. Of the 82 studies, HCD was used in 21 (26%) studies, whereas 4 (4%) studies applied a DT approach. Most (57/82, 70%) used a UCD approach. All approaches prioritized the users’ needs and the participatory and iterative nature of the design process. Some HCD definitions included a focus on a multiple stakeholder or system perspective, whereas some UCD definitions aimed at increasing usability or user friendliness of the solution. These design approaches are generally characterized by the use of different standards or models.
A total of 3 standards or models were frequently mentioned in the studies and used as references. These models overlap in their attempt to classify the distinct phases of the design process but operationalize the steps differently. The UCD ISO Standard 9241-210 for HCD of interactive systems encompasses a 5-phase design process including (1) understanding and specifying the context of use, (2) specifying user requirements, (3) producing design solutions, (4) evaluating design against requirements, and (5) delivering design solutions that meet user requirements. The HCD IDEO Field Guide to Human-Centered Design and the Hasso Plattner Institute (HPI) School of Design Thinking models are characterized by different versions of a similar 3-phase design process: (1) inspiration, (2) ideation, and (3) implementation. Studies that applied DT worked with a multiphase approach that included versions of the following phases: (1) empathizing with stakeholders, (2) defining the problem, (3) generating ideas for solutions, (4) prototyping the solutions, and (5) testing the solutions. In, we have
illustrated how the different approaches to the HCD process align.
Of the 82 articles identified, 57 (70%) applied a UCD approach, 21 (26%) used HCD, and 4 (5%) used DT. In 17% (14/82) of the studies, the concepts of HCD and UCD were referred to interchangeably; of these 14 studies, 9 (64%) studies referred to the use of the ISO 9241-210 standard. In the 5% (4/82) of studies that applied DT, the concept was used interchangeably with HCD in all cases. These studies referred to the IDEO Field Guide to Human-Centered Design or the HPI School of Design Thinking Guide as standards. For clarity, we have continued to report the results of the HCD and DT studies and UCD studies separately in this review.
Design Strategies and Methods
Of the 82 studies, 74 (90%) applied an SFS versus 8 (10%) applied a PFS to drive the design process. Thus, most design studies focused on directly generating solutions or developing a specific predefined solution. Only a minority used design-based methods to define the problem and selectively gather information before proceeding to solution development. Of the 74 studies that applied an SFS, 55 (74%) applied the UCD approach. Of the 8 studies that applied a PFS, 6 (75%) applied an HCD and DT approach. Overall, HCD and DT appears to be the preferred approach for problem-driven strategies, whereas UCD is generally applied for solution-driven strategies.
The design processes comprised several design cycles during which multiple qualitative and quantitative methods were used in combination with specific design methods. Of the 82 studies, 47 (57%) applied a mixed methods approach, and 33 (40%) applied qualitative methodology. A synthesis of the methods used in the different phases of the included studies is presented in(details about the described design methods can be found in ). The first design phase—understanding the context—was often characterized by the use of a limited range of design-based methods. During the second and third phases—problem specification and idea generation—a broader range of design methods was used in different studies. In the fourth phase—testing of solutions—the range of design methods was reduced again. Some design-based methods were applied in multiple phases of the process, for example, personas, intervention mapping, or the Wizard of Oz technique; however, most were uniquely used in a single phase.
Overall, qualitative methods or mixed methods were mostly used in the first and last phases of the design process to understand user needs or to evaluate user experiences. In the first phase of the process, qualitative methods such as interviews and observations as well as literature reviews were commonly used to understand the problem context. In later stages, the use of methods diverges based on the type of foreseen solution, for example, digital or nondigital solutions. Quantitative methods were used to either support qualitative findings during the first phase of the process or as an evaluation instrument in the later design phases.
|Design phase||Qualitative methods||Quantitative methods||Design methods|
|Understanding the context||Surveys (not specified)|
|Specify the problem or user need|
|Generate ideas and design solutions|
aMoSCoW: must have, should have, could have, won’t have.
bSWOT: strengths, weaknesses, opportunities, and threats.
cEMA: ecological momentary assessment.
End User Involvement
In 6 studies (UCD 5/6, 83%; HCD and DT 1/6, 17%), the end users were actively involved as users, that is, as information sources but not as active participants in the design process. In 27 (UCD 21/27, 78%; HCD and DT 6/27, 22%) studies, the end users participated as testers; that is, they were involved in the first and last phases of the design process as testers of the developed solutions. In 28 (UCD 22/28, 79%; HCT and DT 6/28, 21%) studies, the end users were involved as informants. Here, end users were involved in various phases of the process and asked for input on the design prototypes, such as sketches and low-fidelity prototypes. Participation as design partners, that is, as contributors to all phases and being involved in the decision-making process, was identified in 21 (UCD 9/21, 43%; HCD and DT 12/21, 57%) studies (). Although UCD approaches involved end users primarily in the role of tester (21/57, 37%) and informant (22/57, 39%), HCD and DT approaches involved end users as design partners in 48% (12/57) of the studies.
Quality Assessment of the Studies
Using the MMAT, 16% (13/82) of the included studies met ≥1 MMAT reporting criteria, based on the study type. The remaining studies had to be rated as unclear on all MMAT reporting criteria. An overview of the quality assessment results can be found in[ - ]. The biggest limitation to the quality assessment was the lack of uniformity in reporting and the broad extent of the design studies that needed to be captured in limited words for publication. In fact, most studies used multiple research and design cycles and generally offered limited details about the applied methodology.
In this review, we explored how different HCD approaches, including DT and UCD, were applied for the development of innovations in health research. Overall, the concepts of HCD and DT, and HCD and UCD, were used interchangeably in 22% (18/82) of the included studies. This applied to all studies that referred to HCD and DT; however, UCD was defined as a standalone entity in 84% (48/57) of the papers that used this approach. Most of the studies using HCD and UCD interchangeably referred to the ISO 9241-210 standard. This aligns with the theoretical framework pursued by the studies, that is, a problem-driven versus a solution-driven strategy. DT- and HCD-based studies commonly engaged in understanding the underlying problem and focused on a broad range of health, social, or medical topics. They often included a focus on human values and a multistakeholder or systems perspective. Instead, UCD-based approaches focused primarily on the direct identification of a solution and were mostly used in health technology innovation. They often focused on human factors to increase the usability or user friendliness of the solution. The limitations of this functional approach in promoting human interests have been previously described as a potential shortcoming of UCD .
It has been reported that designers who use a problem-driven design strategy produce solutions with the best balance between quality and creativity . However, in this review, 90% (74/82) of the included studies used a solution-driven strategy. Although the evaluation of solutions can be used to further define the design problem, this was not an objective of the included studies. Their solution-driven approach generally focused on generating a large number of ideas and solutions, potentially leaving the initial design problem ill-defined and ignoring the relationships between various stakeholders. However, health care innovation could significantly benefit from problem-driven design processes, especially from the perspective of resource efficiency. Innovation in health care is characterized by a development or implementation cost trade-off. Therefore, it is critical that the most impactful innovations be prioritized based on a critical understanding of the underlying problem [ ].
HCD in health research is often perceived as a single unitary method, as emphasized by the reference to a single practitioner guideline in the included studies. However, in this review, we found that the application of HCD entails a wide array of design methods and techniques that can be used selectively and that are dependent on the specific design case. Design methods diverge from the traditional methods of academic research as they are primarily oriented toward action or solution of defined problems rather than toward theory and hypotheses building. To date, little is known about their effectiveness according to evidence-based medical standards. The creation of a new product, system, or service to improve health might be considered an outcome from a design perspective but would not be considered a health outcome from a scientific perspective [, ]. In the literature, a scientific method is described as a strategy to understand the nature of a phenomenon, whereas a design method is a strategy to invent things of value. According to this distinction, science is analytical and design is constructive and it is therefore difficult to assess both methods according to the same standard [ ]. However, according to Frey and Dym [ ], many of the validation techniques found in medicine can be used for the validation of design methods. For example, where medicine uses animal models and clinical trials to test medical treatments, detailed simulations and controlled field experiments of design methods could be developed for the explicit purpose of evaluating design methodologies [ ].
This logical, empirical approach toward the evaluation of design methods fits well with, for example, the field of engineering design, which is based on mathematical modeling, as it is most appropriate for closed, objective problems that can lead to binary (yes or no) answers. However, HCD approaches often address open, complex problems that involve both objective and subjective elements without a single correct answer. For design methods addressing open, complex problems, a relativist validation approach that gradually builds confidence in the usefulness of the methods can be considered a more appropriate paradigm . A relativist approach to design claims no absolute objectivity for methods or models; however, it assumes that a valid method or model is only one of the many possible ways of measuring or describing a real situation. In a relativist approach to design methods, validity becomes a matter of practical use and contextual functionality rather than formal and universal accuracy. The validity of design methods becomes a contextual, semiformal, and conversational process, because establishing models of usefulness is a conversational matter [ ]. It is important to note that a relativist approach toward the evaluation of design methods does not antagonize the logical, empirical approach toward the evaluation of scientific research methods used in HCD processes.
There is an ongoing demand for the development of a design science with systematic and formalized design methods that adhere to the values of the empirical scientific method: objectivity, rationality, and universalism [, ]. Scientific design methods have been developed in engineering and computer science; however, there is limited evidence that the systematic use of design practices leads to measurable and reproducible results in health research [ ]. Design researchers themselves still debate whether design conforms to a scientific activity or represents an academic discipline with a rigorous culture of its own [ , ]. As a result, critical appraisal and best practice selections of design methods in health research remain challenging.
In this review, the diverse reporting formats challenged our ability to assess the quality of the studies from an evidence-based perspective. Although initial guidelines have been proposed to improve the reporting of design studies in health research, this is still an area that is in development . The guidelines by Bazzano et al [ ] represent the first detailed overview of reporting items for health research that includes design approaches. Although we acknowledge that this reporting guideline is an important first step toward improving transparency, evaluability, and wider dissemination of design approaches in health research, it is, however, debatable whether the application of these guidelines is feasible in the context of health research manuscripts. The level of detail that the Bazzano [ ] guidelines propose implies that the design research component should be reported as a standalone article, separate from the connected empirical studies. Most of the design studies included in this review offered limited details about their multimethod design cycles, possibly because of the word count limits that most scientific journals apply. It would be almost impossible to describe a multimethod design process in adequate detail and also effectively report on the research and design outcomes in a single manuscript. Applying the Bazzano [ ] guidelines with rigor is likely to result in the reporting of separate design cycles across multiple manuscripts, and essential findings for the design process might appear fragmented or be lost among reports that are published separately.
However, it could be argued that the separate publication of multiple waves of data collection in design research is preferable for both researchers and reviewers to support the validity, reliability, and reproducibility of design-based health research. Rather than aiming for complex integrated manuscripts, multiple publications would allow researchers to report in more detail on both their methods and findings and also allow for easier critical appraisal and quality assessment by reviewers. In addition to traditional research articles, innovative publication formats such as registered reports could be used to submit design research protocols and results that are judged on their methodological robustness rather than the potential novelty of the findings . We recommend registering the design research protocols in a research registry to address the issue of potential fragmented data publication. This would allow for systematic referencing to previous design activities, even when their results have not been published.
The active engagement of stakeholders is one of the key principles of the HCD approach. Stakeholders can be defined as “individuals, organizations or communities that have a direct interest in the process and outcomes of a project, research or policy endeavor” . In health care innovation, the engagement of diverse stakeholders is essential to the development of a shared agenda for responsible innovation and for the cocreation of social value [ ]. However, a multistakeholder innovation process brings about several challenges. HCD practitioners acknowledge the challenge of equitably including the experience and expertise of all participants in the design process. Although the importance of creative interdisciplinary collaborations between various disciplines in health care is increasing, it is still a relatively new and complex phenomenon [ ]. Each stakeholder brings their own motivations, attitudes, priorities, and incentives to the process, and such differences will influence the cocreative space and interpersonal interactions. HCD practitioners should critically reflect on the participatory methods that they intend to apply, considering the possible contribution of each participant in the design process to facilitate the effective use of their expertise and experiences [ ]. This is particularly important when working with vulnerable patient groups or health care professionals with limited time to participate in co-design sessions [ ].
An earlier study suggested that HCD processes can rely too much on anecdotal evidence of key stakeholders who might not fully understand what they want and need . However, a more strategic application of HCD aims to identify themes that describe people’s deeper needs and values rather than their wishes and desires and uses those themes to inform the creation of innovative strategies and solutions [ ]. Field studies with the use of qualitative methods, such as observations, to study key stakeholders and their activities in their own environments could offer a valid alternative [ ].
In addition, it is essential for HCD practitioners to take power dimensions and the agency of different stakeholders into account, especially during co-design sessions. To achieve inclusive design processes, intersectional aspects should be considered for stakeholder engagement and methodological choices, such as gender identity, class, sexuality, geography, age, and disability and ability [, ]. Reflective project planning aids and frameworks for involving patients and the public in research and design projects should be used to guarantee meaningful engagement of stakeholders and facilitate democratic design processes [ , ].
At present, MMAT is the most comprehensive tool available for appraising multimethod studies . Although the MMAT is a tool that allows for the critical appraisal of most common types of study designs, the tool seems less appropriate for HCD, DT, and UCD because of the inclusion of multiple research and design cycles and the often-limited word space to describe the applied methodologies and methodological choices in detail. To our knowledge, there is no appropriate tool available for the critical appraisal of design studies in health research.
In this review, we have only reviewed articles that described the complete development processes of a health innovation. This criterion might have limited the inclusion of studies that describe the complete process through multiple publications. For example, in a few studies, the authors referred to future studies in which they expressed the intention to test a designed solution in a randomized controlled trial. Those studies were not included in this review. Furthermore, no selection criteria for the end user populations were applied. This might have influenced the choice for the use of particular design methods, as design researchers need to take intersectional aspects into account, as mentioned in the Discussion. Although this was not the main objective of this review, future research could focus on the application of design methods and their suitability for specific stakeholder populations in health care.
In addition, our search strategy was limited to scientific databases related to biomedical, nursing, and allied health and public health sciences, and gray literature was not included. Disciplines that publish design research related to health systems outside this scope were not considered in our searches. Finally, the existence of different design methods and models with principles related to HCD and the interchangeable use of these terms in the literature made it challenging to scope and perform a fully systematic search.
A wide variety of design practices and methods such as HCD, DT, and UCD are increasingly being applied in health research. In our analysis, HCD- and DT-based projects tended to primarily follow integrated and problem-driven approaches, whereas UCD-based projects engaged in more functional and solution-driven approaches. Most of these design studies used mixed methods approaches, combined qualitative and quantitative research with design methods, and frequently referred to the following 3 design guides: the IDEO Field Guide to Human-Centered Design, the HPI School of Design Thinking Guide, and the ISO Standard 9241-210.
The increasing use of design-based approaches such as HCD and DT and UCD in health research subjects them to evaluation according to traditional biomedical standards. However, the analytic approach of the scientific method versus the constructive approach of the design method impedes the assessment of both methods according to the same standard. To address the validation of design methods, a relativist validation approach that gradually builds confidence in the usefulness of methods could be considered a more appropriate paradigm for design methods, particularly those that are concerned with subjective elements of the design process.
Specific standards for reporting HCD practices in health and biomedical research have been developed in recent years. However, these reporting standards remain challenging to apply for single design research papers because of the extensiveness of multimethod design processes in combination with customary word limits in biomedical publications. Separate publications detailing the multiple waves of data collection in design research might be preferable for both researchers and reviewers to support the validity, reliability, and reproducibility of design-based health research. In addition, innovative publication formats such as registered reports could be used to submit design research protocols and results that are judged on their methodological robustness rather than the potential novelty of the findings. Furthermore, future research on HCD approaches in health should focus on the development of an HCD practitioner guideline for stakeholder engagement that takes stakeholder roles, experiences, expertise, agency, and power dimensions into account.
The authors thank the librarians OY Chan (Radboud University Medical Center Library) and S van Putten (Radboud University Library) for their continued support in formulating and testing the search strategy for this review. The authors also thank M Wijnen (Radboud University Medical Center; Department of Primary and Community Care; Nijmegen, the Netherlands) for her contribution during literature screening.
Conflicts of Interest
Review protocol.DOC File , 52 KB
Search strategy.DOC File , 31 KB
Descriptions of the design method included in the review.DOC File , 53 KB
Mixed Methods Appraisal Tool checklist.DOC File , 274 KB
- Lehnert T, Heider D, Leicht H, Heinrich S, Corrieri S, Luppa M, et al. Review: health care utilization and costs of elderly persons with multiple chronic conditions. Med Care Res Rev 2011 Aug 03;68(4):387-420. [CrossRef] [Medline]
- McPhail S. Multimorbidity in chronic disease: impact on health care resources and costs. Risk Manag Healthc Policy 2016 Jul;Volume 9:143-156. [CrossRef]
- Matheson GO, Pacione C, Shultz RK, Klügl M. Leveraging human-centered design in chronic disease prevention. Am J Prev Med 2015 Apr;48(4):472-479. [CrossRef] [Medline]
- Chan K. A design thinking mindset beyond the public health model. World Med Health Policy 2018;10(1):111-119. [CrossRef]
- Vesely R. Applying 'design thinking' to health care organizations. Health Facil Manag 2017 Mar;30(3):10-11. [Medline]
- Valentine L, Kroll T, Bruce F, Lim C, Mountain R. Design thinking for social innovation in health care. Design J 2017 Oct 17;20(6):755-774. [CrossRef]
- Roberts JP, Fisher TR, Trowbridge MJ, Bent C. A design thinking framework for healthcare management and innovation. Healthc (Amst) 2016 Mar;4(1):11-14. [CrossRef] [Medline]
- Zuber CD, Moody L. Creativity and innovation in health care: tapping into organizational enablers through human-centered design. Nurs Adm Q 2018;42(1):62-75. [CrossRef] [Medline]
- Kalaichandran A. Design thinking for doctors and nurses. New York Times. 2017. URL: https://www.nytimes.com/2017/08/03/well/live/design-thinking-for-doctors-and-nurses.html [accessed 2021-11-20]
- de Mooij MJ, Hodny RL, O'Neil DA, Gardner MR, Beaver M, Brown AT, et al. OB nest: reimagining low-risk prenatal care. Mayo Clin Proc 2018 Apr;93(4):458-466. [CrossRef] [Medline]
- Trail-Mahan T, Heisler S, Katica M. Quality improvement project to improve patient satisfaction with pain management: using human-centered design. J Nurs Care Qual 2016;31(2):105-113. [CrossRef] [Medline]
- van der Bijl-Brouwer M. The challenges of human-centred design in a public sector innovation context. Design Research Society. 2016. URL: https://dl.designresearchsociety.org/drs-conference-papers/drs2016/researchpapers/125/ [accessed 2021-11-20]
- Bazzano AN, Martin J, Hicks E, Faughnan M, Murphy L. Human-centred design in global health: a scoping review of applications and contexts. PLoS One 2017 Nov;12(11):e0186744 [FREE Full text] [CrossRef] [Medline]
- Bazzano AN, Yan SD, Martin J, Mulhern E, Brown E, LaFond A, et al. Improving the reporting of health research involving design: a proposed guideline. BMJ Glob Health 2020 Feb 16;5(2):e002248 [FREE Full text] [CrossRef] [Medline]
- Leiying PU. Effect of co-design approach on working efficiency and satisfaction in user support design processes - Co-Design Approach: A Collective Design Method for Improving Working Efficiency and Satisfaction in User Support Design Processes. Master graduation thesis, University of Twente, The Netherlands. 2019. URL: https://essay.utwente.nl/79455/1/Pu_MA_BMS.pdf [accessed 2021-11-21]
- Steen M. Tensions in human-centred design. CoDesign 2011 Mar;7(1):45-60. [CrossRef]
- Scariot CA, Heemann A, Padovani S. Understanding the collaborative-participatory design. Work 2012;41 Suppl 1:2701-2705. [CrossRef] [Medline]
- Altman M, Huang TT, Breland JY. Design thinking in health care. Prev Chronic Dis 2018 Sep 27;15:E117 [FREE Full text] [CrossRef] [Medline]
- Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
- Cross N, Naughton J, Walker D. Design method and scientific method. Design Stud 1981 Oct;2(4):195-201. [CrossRef]
- Galle P, Kroes P. Science and design: identical twins? Design Stud 2014 May;35(3):201-231. [CrossRef]
- Kruger C, Cross N. Solution driven versus problem driven design: strategies and outcomes. Design Stud 2006 Sep;27(5):527-548. [CrossRef]
- Druin A. The role of children in the design of new technology. Behav Inform Technol 2002 Jan;21(1):1-25. [CrossRef]
- Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf 2018 Dec 18;34(4):285-291. [CrossRef]
- Bae J, Wolpin S, Kim E, Lee S, Yoon S, An K. Development of a user-centered health information service system for depressive symptom management. Nurs Health Sci 2009 Jun;11(2):185-193. [CrossRef] [Medline]
- Birnie KA, Campbell F, Nguyen C, Lalloo C, Tsimicalis A, Matava C, et al. iCanCope PostOp: user-centered design of a smartphone-based app for self-management of postoperative pain in children and adolescents. JMIR Form Res 2019 Apr 22;3(2):e12028 [FREE Full text] [CrossRef] [Medline]
- Brox E, Konstantinidis ST, Evertsen G. User-centered design of serious games for older adults following 3 years of experience with exergames for seniors: a study design. JMIR Serious Games 2017 Jan 11;5(1):e2 [FREE Full text] [CrossRef] [Medline]
- Cairns N, Corney J, Murray K, Moore-Millar K, Hatcher GD, Zahedi S, et al. Rethinking the foam cosmesis for people with lower limb absence. Prosthet Orthot Int 2018 Apr;42(2):223-227 [FREE Full text] [CrossRef] [Medline]
- Carey-Smith B, Evans N, Orpwood R. A user-centred design process to develop technology to improve sleep quality in residential care homes. Technol Disabil 2013 Mar 25;25(1):49-58. [CrossRef]
- Caro K, Tentori M, Martinez-Garcia AI, Zavala-Ibarra I. FroggyBobby: an exergame to support children with motor problems practicing motor coordination exercises during therapeutic interventions. Comput Hum Behav 2017 Jun;71:479-498. [CrossRef]
- Catalani C, Green E, Owiti P, Keny A, Diero L, Yeung A, et al. A clinical decision support system for integrating tuberculosis and HIV care in Kenya: a human-centered design approach. PLoS One 2014 Aug;9(8):e103205 [FREE Full text] [CrossRef] [Medline]
- Cawood T, Saunders E, Drennan C, Cross N, Nicholl D, Kenny A, et al. Creating the optimal workspace for hospital staff using human centred design. Intern Med J 2016 Jul 12;46(7):840-845. [CrossRef] [Medline]
- Civan-Hartzler A, McDonald DW, Powell C, Skeels MM, Mukai M, Pratt W. Bringing the field into focus: user-centered design of a patient expertise locator. Proc SIGCHI Conf Hum Factor Comput Syst 2010 Apr;2010:1675-1684 [FREE Full text] [CrossRef] [Medline]
- Connelly K, Stein KF, Chaudry B, Trabold N. Development of an ecological momentary assessment mobile app for a low-literacy, Mexican American population to collect disordered eating behaviors. JMIR Public Health Surveill 2016 Jul 14;2(2):e31 [FREE Full text] [CrossRef] [Medline]
- Crespin OM, Okrainec A, Kwong AV, Habaz I, Jimenez MC, Szasz P, et al. Feasibility of adapting the fundamentals of laparoscopic surgery trainer box to endoscopic skills training tool. Surg Endosc 2018 Jun 2;32(6):2968-2983. [CrossRef] [Medline]
- Curtis KE, Lahiri S, Brown KE. Targeting parents for childhood weight management: development of a theory-driven and user-centered healthy eating app. JMIR Mhealth Uhealth 2015 Jun 18;3(2):e69 [FREE Full text] [CrossRef] [Medline]
- De Vito Dabbs A, Myers BA, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs 2009;27(3):175-183 [FREE Full text] [CrossRef] [Medline]
- Das A, Svanæs D. Human-centred methods in the design of an e-health solution for patients undergoing weight loss treatment. Int J Med Inform 2013 Nov;82(11):1075-1091. [CrossRef] [Medline]
- Davies A, Mueller J, Hennings J, Caress A, Jay C. Recommendations for developing support tools with people suffering from chronic obstructive pulmonary disease: co-design and pilot testing of a mobile health prototype. JMIR Hum Factors 2020 May 15;7(2):e16289 [FREE Full text] [CrossRef] [Medline]
- Dijkstra NE, Sino CG, Heerdink ER, Schuurmans MJ. Development of eHOME, a mobile instrument for reporting, monitoring, and consulting drug-related problems in home care: human-centered design study. JMIR Hum Factors 2018 Mar 07;5(1):e10 [FREE Full text] [CrossRef] [Medline]
- Eberhart A, Slogeris B, Sadreameli S, Jassal M. Using a human-centered design approach for collaborative decision-making in pediatric asthma care. Public Health 2019 May;170:129-132. [CrossRef] [Medline]
- Barkana DE, Açık A, Duru DG, Duru AD. Improvement of design of a surgical interface using an eye tracking device. Theor Biol Med Model 2014 May 07;11 Suppl 1(Suppl 1):S4 [FREE Full text] [CrossRef] [Medline]
- Erwin K, Fitzpatrick V, Norell S, Gilliam M. Development of a framework and tool to facilitate cost-of-care conversations with patients during prenatal care. Ann Intern Med 2019 May 07;170(9_Supplement):S62. [CrossRef]
- Ettinger KM, Pharaoh H, Buckman RY, Conradie H, Karlen W. Building quality mHealth for low resource settings. J Med Eng Technol 2016 Aug 25;40(7-8):431-443. [CrossRef] [Medline]
- Fabri M, Andrews PC, Pukki HK. Using design thinking to engage autistic students in participatory design of an online toolkit to help with transition into higher education. J Assist Technol 2016;10(2):102-114. [CrossRef]
- Farinango C, Benavides J, Cerón J, López D, Álvarez R. Human-centered design of a personal health record system for metabolic syndrome management based on the ISO 9241-210:2010 standard. J Multidiscip Healthc 2018 Jan;Volume 11:21-37. [CrossRef]
- Ferris TK, Shepley MM. The design of neonatal incubators: a systems-oriented, human-centered approach. J Perinatol 2013 Apr 28;33 Suppl 1(S1):24-31. [CrossRef] [Medline]
- Foley KA, Shelton J, Richardson E, Smart N, Smart-McMillan C, Mustakem O, et al. Primary care women's health screening: a case study of a community engaged human centered design approach to enhancing the screening process. Matern Child Health J 2019 Nov 27;23(11):1446-1458. [CrossRef] [Medline]
- Fortuna KL, Lohman MC, Gill LE, Bruce ML, Bartels SJ. Adapting a psychosocial intervention for smartphone delivery to middle-aged and older adults with serious mental illness. Am J Geriatr Psychiatry 2017 Aug;25(8):819-828 [FREE Full text] [CrossRef] [Medline]
- Furberg RD, Ortiz AM, Moultrie RR, Raspa M, Wheeler AC, McCormack LA, et al. A digital decision support tool to enhance decisional capacity for clinical trial consent: design and development. JMIR Res Protoc 2018 Jun 06;7(6):e10525 [FREE Full text] [CrossRef] [Medline]
- Gačnik M, Starčič AI, Zaletelj J, Zajc M. User-centred app design for speech sound disorders interventions with tablet computers. Univ Access Inf Soc 2017 Jul 1;17(4):821-832. [CrossRef]
- Garvelink MM, Emond J, Menear M, Brière N, Freitas A, Boland L, et al. Development of a decision guide to support the elderly in decision making about location of care: an iterative, user-centered design. Res Involv Engagem 2016 Jul 19;2(1):26 [FREE Full text] [CrossRef] [Medline]
- Garvelink MM, Jones CA, Archambault PM, Roy N, Blair L, Légaré F. Deciding how to stay independent at home in later years: development and acceptability testing of an informative web-based module. JMIR Hum Factors 2017 Dec 14;4(4):e32 [FREE Full text] [CrossRef] [Medline]
- Garvin JH, Ducom J, Matheny M, Miller A, Westerman D, Reale C, et al. Descriptive usability study of CirrODS: clinical decision and workflow support tool for management of patients with cirrhosis. JMIR Med Inform 2019 Jul 03;7(3):e13627 [FREE Full text] [CrossRef] [Medline]
- Garzo A, Silva PA, Garay-Vitoria N, Hernandez E, Cullen S, De Cock VC, et al. Design and development of a gait training system for Parkinson's disease. PLoS One 2018 Nov 12;13(11):e0207136 [FREE Full text] [CrossRef] [Medline]
- Gaynor M, Schneider D, Seltzer M, Crannage E, Barron ML, Waterman J, et al. A user-centered, learning asthma smartphone application for patients and providers. Learn Health Syst 2020 Jul;4(3):e10217 [FREE Full text] [CrossRef] [Medline]
- Gill RK, Ogilvie G, Norman WV, Fitzsimmons B, Maher C, Renner R. Feasibility and acceptability of a mobile technology intervention to support postabortion care (The FACTS Study Phase II) after surgical abortion: user-centered design. JMIR Hum Factors 2019 Oct 10;6(4):e14558 [FREE Full text] [CrossRef] [Medline]
- Giunti G, Mylonopoulou V, Rivera Romero O. More stamina, a gamified mHealth solution for persons with multiple sclerosis: research through design. JMIR Mhealth Uhealth 2018 Mar 02;6(3):e51 [FREE Full text] [CrossRef] [Medline]
- Godinho R, Condado PA, Zacarias M, Lobo FG. Improving accessibility of mobile devices with EasyWrite. Behav Inform Technol 2014 Dec 03;34(2):135-150. [CrossRef]
- Gould CE, Loup J, Scales AN, Juang C, Carlson C, Ma F, et al. Development and refinement of educational materials to help older veterans use VA mental health mobile apps. Prof Psychol Res Pr 2020 Aug;51(4):414-423 [FREE Full text] [CrossRef] [Medline]
- Green R, Buckley A, Scott B. Tracking care in the emergency department. Ergon Des 2015 Nov 11;23(4):5-12. [CrossRef]
- Griffin L, Lee D, Jaisle A, Carek P, George T, Laber E, et al. Creating an mhealth app for colorectal cancer screening: user-centered design approach. JMIR Hum Factors 2019 May 08;6(2):e12700 [FREE Full text] [CrossRef] [Medline]
- Grossman L, Feiner S, Mitchell E, Creber R. Leveraging patient-reported outcomes using data visualization. Appl Clin Inform 2018 Jul 01;9(3):565-575 [FREE Full text] [CrossRef] [Medline]
- Hafiz P, Miskowiak KW, Kessing LV, Jespersen AE, Obenhausen K, Gulyas L, et al. The internet-based cognitive assessment tool: system design and feasibility study. JMIR Form Res 2019 Jul 26;3(3):e13898 [FREE Full text] [CrossRef] [Medline]
- Hardy A, Wojdecka A, West J, Matthews E, Golby C, Ward T, et al. How inclusive, user-centered design research can improve psychological therapies for psychosis: development of SlowMo. JMIR Ment Health 2018 Dec 05;5(4):e11222 [FREE Full text] [CrossRef] [Medline]
- Harte R, Quinlan LR, Glynn L, Rodríguez-Molinero A, Baker PM, Scharf T, et al. Human-centered design study: enhancing the usability of a mobile phone app in an integrated falls risk detection system for use by older adult users. JMIR Mhealth Uhealth 2017 May 30;5(5):e71 [FREE Full text] [CrossRef] [Medline]
- Hartzler AL, Izard JP, Dalkin BL, Mikles SP, Gore JL. Design and feasibility of integrating personalized PRO dashboards into prostate cancer care. J Am Med Inform Assoc 2016 Jan;23(1):38-47 [FREE Full text] [CrossRef] [Medline]
- Herschman J, Kasenberg T, Levy D, Ruth N, Taberner C, Kaufman M, et al. Development of a smartphone app for adolescents with lupus: a collaborative meeting-based methodology inclusive of a wide range of stakeholders. Rev Panam Salud Publica 2014;35(5-6):471-476. [Medline]
- Horsky J, Ramelson HZ. Development of a cognitive framework of patient record summary review in the formative phase of user-centered design. J Biomed Inform 2016 Dec;64:147-157 [FREE Full text] [CrossRef] [Medline]
- Huberty J, Rowedder L, Hekler E, Adams M, Hanigan E, McClain D, et al. Development and design of an intervention to improve physical activity in pregnant women using Text4baby. Transl Behav Med 2016 Jun 25;6(2):285-294 [FREE Full text] [CrossRef] [Medline]
- Isenberg SR, Crossnohere NL, Patel MI, Conca-Cheng A, Bridges JF, Swoboda SM, et al. An advance care plan decision support video before major surgery: a patient- and family-centred approach. BMJ Support Palliat Care 2018 Jun 18;8(2):229-236. [CrossRef] [Medline]
- Johnston SK, Nguyen HQ, Wolpin S. Designing and testing a web-based interface for self-monitoring of exercise and symptoms for older adults with chronic obstructive pulmonary disease. Comput Inform Nurs 2009;27(3):166-174 [FREE Full text] [CrossRef] [Medline]
- Ting K, Dessinger G, Voilmy D. Examining usage to ensure utility: co-design of a tool for fall prevention. IRBM 2020 Oct;41(5):286-293. [CrossRef]
- Luna DR, Lede DA, Otero CM, Risk MR, de Quirós F. User-centered design improves the usability of drug-drug interaction alerts: experimental comparison of interfaces. J Biomed Inform 2017 Feb;66:204-213 [FREE Full text] [CrossRef] [Medline]
- Ma M, Wu F, Chang R. A new design approach of user-centered design on a personal assistive bathing device for hemiplegia. Disabil Rehabil 2007 Jul 30;29(14):1077-1089. [CrossRef] [Medline]
- Madrigal-Cadavid J, Amariles P, Pino-Marín D, Granados J, Giraldo N. Design and development of a mobile app of drug information for people with visual impairment. Res Social Adm Pharm 2020 Jan;16(1):62-67. [CrossRef] [Medline]
- Marker AM, Monzon AD, Goggin KJ, Clements MA, Patton SR. Iterative development of a web-based intervention for families of young children with type 1 diabetes: DIPPer Academy. Clin Pract Pediatr Psychol 2019 Mar;7(1):20-30 [FREE Full text] [CrossRef] [Medline]
- Marko-Holguin M, Cordel SL, Van Voorhees BW, Fogel J, Sykes E, Fitzgibbon M, et al. A two-way interactive text messaging application for low-income patients with chronic medical conditions: design-thinking development approach. JMIR Mhealth Uhealth 2019 May 01;7(5):e11833 [FREE Full text] [CrossRef] [Medline]
- Martin S, Armstrong E, Thomson E, Vargiu E, Solà M, Dauwalder S, et al. A qualitative study adopting a user-centered approach to design and validate a brain computer interface for cognitive rehabilitation for people with brain injury. Assist Technol 2018 Jul 14;30(5):233-241. [CrossRef] [Medline]
- McGinn C, Cullinan MF, Culleton M, Kelly K. A human-oriented framework for developing assistive service robots. Disabil Rehabil Assist Technol 2018 Apr 24;13(3):293-304. [CrossRef] [Medline]
- McMullen C, Nielsen M, Firemark A, Price PM, Nakatani D, Tuthill J, et al. Designing for impact: identifying stakeholder-driven interventions to support recovery after major cancer surgery. Support Care Cancer 2018 Dec 6;26(12):4067-4076 [FREE Full text] [CrossRef] [Medline]
- Melnick ER, Hess EP, Guo G, Breslin M, Lopez K, Pavlo AJ, et al. Patient-centered decision support: formative usability evaluation of integrated clinical decision support with a patient decision aid for minor head injury in the emergency department. J Med Internet Res 2017 May 19;19(5):e174 [FREE Full text] [CrossRef] [Medline]
- Núñez-Nava J, Orozco-Sánchez PA, López DM, Ceron JD, Alvarez-Rosero RE. Human-centered development of an online social network for metabolic syndrome management. In: Studies in Health Technology and Informatics. Amsterdam: IOS Press; 2016:100-104.
- Person B, Knopp S, Ali SM, A’kadir FM, Khamis AN, Ali JN, et al. Community co-designed schistosomiasis control interventions for school-aged children in Zanzibar. J Biosoc Sci 2016 Jul 18;48(S1):56-73. [CrossRef]
- Petersen M, Hempler NF. Development and testing of a mobile application to support diabetes self-management for people with newly diagnosed type 2 diabetes: a design thinking case study. BMC Med Inform Decis Mak 2017 Jun 26;17(1):91 [FREE Full text] [CrossRef] [Medline]
- Ragouzeos D, Gandrup J, Berrean B, Li J, Murphy M, Trupin L, et al. "Am I OK?" using human centered design to empower rheumatoid arthritis patients through patient reported outcomes. Patient Educ Couns 2019 Mar;102(3):503-510 [FREE Full text] [CrossRef] [Medline]
- Ray JM, Ahmed OM, Solad Y, Maleska M, Martel S, Jeffery MM, et al. Computerized clinical decision support system for emergency department-initiated buprenorphine for opioid use disorder: user-centered design. JMIR Hum Factors 2019 Feb 27;6(1):e13121 [FREE Full text] [CrossRef] [Medline]
- Rothgangel A, Braun S, Smeets R, Beurskens A. Design and development of a telerehabilitation platform for patients with phantom limb pain: a user-centered approach. JMIR Rehabil Assist Technol 2017 Feb 15;4(1):e2 [FREE Full text] [CrossRef] [Medline]
- Salmon M, Salmon C, Bissinger A, Muller MM, Gebreyesus A, Geremew H, et al. Alternative ultrasound gel for a sustainable ultrasound program: application of human centered design. PLoS One 2015 Aug 7;10(8):e0134332 [FREE Full text] [CrossRef] [Medline]
- Schild S, Sedlmayr B, Schumacher A, Sedlmayr M, Prokosch H, St Pierre M, German Cognitive Aid Working Group. A digital cognitive aid for anesthesia to support intraoperative crisis management: results of the user-centered design process. JMIR Mhealth Uhealth 2019 Apr 29;7(4):e13226 [FREE Full text] [CrossRef] [Medline]
- Sedlmayr B, Schöffler J, Prokosch H, Sedlmayr M. User-centered design of a mobile medication management. Inform Health Soc Care 2019 Mar 05;44(2):152-163. [CrossRef] [Medline]
- Seeber L, Michl B, Rundblad G, Trusko B, Schnjakin M, Meinel C, et al. A design thinking approach to effective vaccine safety communication. Curr Drug Saf 2015 Apr 07;10(1):31-40. [CrossRef] [Medline]
- Sonney J, Duffy M, Hoogerheyde LX, Langhauser E, Teska D. Applying human-centered design to the development of an asthma essentials kit for school-aged children and their parents. J Pediatr Health Care 2019 Mar;33(2):169-177. [CrossRef] [Medline]
- Srinivas P, Bodke K, Ofner S, Keith NR, Tu W, Clark DO. Context-sensitive ecological momentary assessment: application of user-centered design for improving user satisfaction and engagement during self-report. JMIR Mhealth Uhealth 2019 Apr 03;7(4):e10894 [FREE Full text] [CrossRef] [Medline]
- Stevens A, Köke A, van der Weijden T, Beurskens A. The development of a patient-specific method for physiotherapy goal setting: a user-centered design. Disabil Rehabil 2018 Aug 13;40(17):2048-2055. [CrossRef] [Medline]
- Taylor DP, Bray BE, Staggers N, Olson RJ. User-centered development of a web-based preschool vision screening tool. AMIA Annu Symp Proc 2003:654-658 [FREE Full text] [Medline]
- Timmerman JG, Tönis TM, van Weering MG, Stuiver MM, Wouters MW, van Harten WH, et al. Co-creation of an ICT-supported cancer rehabilitation application for resected lung cancer survivors: design and evaluation. BMC Health Serv Res 2016 Apr 27;16:155 [FREE Full text] [CrossRef] [Medline]
- Edmonds BT, Hoffman SM, Lynch D, Jeffries E, Jenkins K, Wiehe S, et al. Creation of a decision support tool for expectant parents facing threatened periviable delivery: application of a user-centered design approach. Patient 2019 Jun 29;12(3):327-337 [FREE Full text] [CrossRef] [Medline]
- van der Weegen S, Verwey R, Spreeuwenberg M, Tange H, van der Weijden T, de Witte L. The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: a user-centered design. JMIR Mhealth Uhealth 2013 Jul 02;1(2):e8 [FREE Full text] [CrossRef] [Medline]
- Vechakul J, Shrimali BP, Sandhu JS. Human-centered design as an approach for place-based innovation in public health: a case study from Oakland, California. Matern Child Health J 2015 Dec 22;19(12):2552-2559. [CrossRef] [Medline]
- Vermeulen J, Neyens J, Spreeuwenberg M, van Rossum E, Sipers W, Habets H, et al. User-centered development and testing of a monitoring system that provides feedback regarding physical functioning to elderly people. Patient Prefer Adher 2013 Aug:843-854. [CrossRef]
- Vilardaga R, Rizo J, Zeng E, Kientz JA, Ries R, Otis C, et al. User-centered design of learn to quit, a smoking cessation smartphone app for people with serious mental illness. JMIR Serious Games 2018 Jan 16;6(1):e2 [FREE Full text] [CrossRef] [Medline]
- Wachtler C, Coe A, Davidson S, Fletcher S, Mendoza A, Sterling L, et al. Development of a mobile clinical prediction tool to estimate future depression severity and guide treatment in primary care: user-centered design. JMIR Mhealth Uhealth 2018 Apr 23;6(4):e95 [FREE Full text] [CrossRef] [Medline]
- Willard S, Cremers G, Man YP, van Rossum E, Spreeuwenberg M, de Witte L. Development and testing of an online community care platform for frail older adults in the Netherlands: a user-centred design. BMC Geriatr 2018 Apr 07;18(1):87 [FREE Full text] [CrossRef] [Medline]
- Woodard TL, Hoffman AS, Covarrubias LA, Holman D, Schover L, Bradford A, et al. The Pathways fertility preservation decision aid website for women with cancer: development and field testing. J Cancer Surviv 2018 Feb 15;12(1):101-114. [CrossRef] [Medline]
- Wysocki T, Pierce J, Caldwell C, Aroian K, Miller L, Farless R, et al. A web-based coping intervention by and for parents of very young children with type 1 diabetes: user-centered design. JMIR Diabetes 2018 Dec 17;3(4):e16 [FREE Full text] [CrossRef] [Medline]
- Gasson S. Human-centered vs. user-centered approaches to information system design. J Inf Theory Appl 2003;5(2):29-46 [FREE Full text]
- Soliman E, Mogefors D, Bergmann JH. Problem-driven innovation models for emerging technologies. Health Technol 2020 Jun 23;10(5):1195-1206. [CrossRef]
- Frey DD, Dym CL. Validation of design methods: lessons from medicine. Res Eng Design 2006 May 25;17(1):45-57. [CrossRef]
- Pedersen K, Emblemsvåg J, Bailey R, Allen JK, Mistree F. Validating design methods and research: the validation square. In: Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (ASME 2000). 2000 Presented at: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (ASME 2000); September 10–13, 2000; Baltimore, Maryland, USA p. 379-390. [CrossRef]
- Barlas Y, Carpenter S. Philosophical roots of model validation: two paradigms. Syst Dyn Rev 1990;6(2):148-166 [FREE Full text] [CrossRef]
- Cross N. Designerly ways of knowing: design discipline versus design science. Design Issues 2001 Jul;17(3):49-55. [CrossRef]
- You X, Hands D. A reflection upon Herbert Simon’s vision of design in "The Sciences of the Artificial". Design J 2019 May 31;22(sup1):1345-1356. [CrossRef]
- No authors listed. An innovative way to publish. Nature 2019 Jul 23;571(7766):447-447. [CrossRef] [Medline]
- Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res 2012 Mar;1(2):181-194 [FREE Full text] [CrossRef] [Medline]
- Abrishami P, Repping S. Nurturing societal values in and through health innovations comment on "What health system challenges should responsible innovation in health address?". Int J Health Policy Manag 2019 Oct 01;8(10):613-615 [FREE Full text] [CrossRef] [Medline]
- Moirano R, Sánchez MA, Štěpánek L. Creative interdisciplinary collaboration: a systematic literature review. Think Skills Creat 2020 Mar;35:100626 [FREE Full text] [CrossRef]
- Langley J, Bec R, Wheeler G, Partridge R. The need for distributed co-design in healthcare contexts. In: Proceedings of the 5th European International Conference on Design4Health. 2018 Presented at: 5th European International Conference on Design4Health; September 4 – 6, 2018; Sheffield Hallam University, Sheffield, UK p. In Press URL: http://shura.shu.ac.uk/23417/3/Langley%20The%20need%20for%20distributed%20co-design%20in%20healthcare%20contexts.pdf
- van der Bijl-Brouwer M, Dorst K. Advancing the strategic impact of human-centred design. Design Stud 2017 Nov;53:1-23 [FREE Full text] [CrossRef]
- Kujala S. User involvement: a review of the benefits and challenges. Behav Inform Technol 2003 Jan;22(1):1-16. [CrossRef]
- Bauer GR. Incorporating intersectionality theory into population health research methodology: challenges and the potential to advance health equity. Soc Sci Med 2014 Jun;110:10-17 [FREE Full text] [CrossRef] [Medline]
- Erete S, Israni A, Dillahunt T. An intersectional approach to designing in the margins. interactions 2018 Apr 23;25(3):66-69. [CrossRef]
- Pratt B. Sharing power in global health research: an ethical toolkit for designing priority-setting processes that meaningfully include communities. Int J Equity Health 2021 May 25;20(1):127 [FREE Full text] [CrossRef] [Medline]
- Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect 2019 Aug 22;22(4):785-801 [FREE Full text] [CrossRef] [Medline]
- Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. J Clin Epidemiol 2019 Jul;111:49-59 [FREE Full text] [CrossRef] [Medline]
|DT: design thinking|
|HCD: human-centered design|
|HPI: Hasso Plattner Institute|
|MMAT: Mixed Methods Appraisal Tool|
|PFS: problem-focused strategy|
|SFS: solution-focused strategy|
|UCD: user-centered design|
Edited by L Buis; submitted 21.02.21; peer-reviewed by A Bazzano, I Holeman, L Powell, S Ren, D Szinay; comments to author 10.05.21; revised version received 16.07.21; accepted 21.10.21; published 06.12.21Copyright
©Irene Göttgens, Sabine Oertelt-Prigione. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 06.12.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.