Effective health care strategies are required to ensure the right patient receives the right treatment at the right time. Advancements in mobile phones and tablets have led to the emergence of mHealth. The Global Observatory for eHealth of the World Health Organization defines mHealth as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants, and other wireless devices” [10]. Recent advances allow seamless integration between smartphones and medical devices. This integration enables smartphones to store and analyze objective measurements such as heart rate, lung volume, and medication adherence. Advancements in machine learning and artificial intelligence have the potential to use these measurements, in combination with data collected via smartphones, to improve our understanding of disease etiology [11,12].

The significance of mHealth is highlighted by its ability to deliver timely care over distance to manage diseases. It is particularly important for rural areas with limited access to health care [13,14]. Moreover, mHealth strategies can enhance treatment outcomes while mitigating health care costs [15,16]. Hayes et al [16] illustrated why mHealth could reduce physician visits, resource consumption, and emergency room visits. The literature continues to evolve on applications of mHealth. For example, several published studies indicate that mHealth interventions are beneficial for patients across various diseases and age groups [1-4]. However, research on the development and usability methodology of such interventions remains in a relatively early stage.

The International Organization for Standardization (ISO) 9241-210 standard defines human-centered design (HCD) as “an approach to systems design and development that aims to make interactive systems more usable by focusing on the use of the system and applying human factors/ergonomics and usability knowledge and techniques” [17]. The ISO uses the term HCD instead of user-centered design as it” addresses impacts on a number of stakeholders, not just those typically considered as users” [17]. However, in practice, these terms are often used synonymously.

HCD has 4 defined activity phases: (1) identify the user and specify the context of use, (2) specify the user requirements, (3) produce design solutions, and (4) evaluate design solutions against requirements. The process model of HCD as defined in ISO 9241-210 is illustrated in Figure 1.

Researchers advocate for involving patients during development who are going to use the mHealth intervention to meet the patient’s needs and facilitate successful uptake. Testing mHealth interventions with patients reveals preferences and concerns unique to the tested population [5,18,19]. Developing an mHealth intervention with insights from stakeholders will potentially improve the process and outcome of mHealth interventions. The main goal of HCD is to increase the usability of mHealth technology.

This study offers an in-depth account of the HCD’s fourth activity phase, evaluate design solutions against requirements. This clarifies that this framework intends to focus on usability testing as one component of the more extensive design process.

The ISO defines usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use [20]. Although this definition was published in 1998, it has been updated in 2018 without any changes to the core concepts. The definition is widespread and generalizable [21,22].

mHealth involves the interaction between multiple user groups through a system. As a result, the usability aspect is vital for the effective, efficient, and satisfactory use of mHealth interventions. Although patients express an interest in using mHealth to manage their health and chronic conditions, many mHealth interventions are not easy to use [5,23]. Difficulty in using an mHealth intervention may limit the user retention rate. A high dropout rate is one of the most significant barriers to mHealth adoption [24,25]. The majority of mHealth app publishers (83%) have less than 10,000 users who have used the app at least once a month [26]. These numbers are discouraging as according to a 2018 estimate, the average mHealth app costs $425,000 to develop [26]. By putting a more significant emphasis on usability, iterative improvements can reduce costs and enhance the long-term use and adoption of mHealth interventions [27-29].

Researchers recommend frequent and iterative usability testing to respond to users’ preferences, technical issues, and shortcomings [18,30,31]. It is also important to ensure that errors in understanding or using the intervention are addressed before testing the intervention in an efficacy trial [32]. A systematic review investigated the usability evaluation processes described in 22 studies related to mHealth applications [33]. The results suggest that the adoption of automated mechanisms could improve usability and stress the importance of adapting health applications to users’ need [33]. Including insights from key users of mHealth has the potential to improve the process and the outcome of the intervention [34].

Contemporary iterative development methods, such as prototyping, reduce the challenges that evolve during the development lifecycle [35]. Prototyping is defined as creating a simulation of the final mHealth technology that is used for testing before launch. Furthermore, researchers suggest including patients who are going to use the mHealth interventions to assist in the development of the intervention [5,18,19,33].

The ISO 9241-11 established usability standards [20]. These standards provide a measure of patients’ experienced usability. It focuses on effectiveness, efficiency, and satisfaction [20]. It is easier to quantify effectiveness and efficiency compared with satisfaction. Brooke developed the System Usability Scale (SUS) [36] and noted that “if there is an area in which it is possible to make more generalized assessments of usability, which could bear cross-system comparison, it is the area of subjective assessments of usability” [36]. Thus, the SUS was developed to quantify satisfaction (users’ subjective reactions to using the system) [36]. The SUS is an affordable and effective tool for assessing the usability of products [36]. It contains 10 statements that are answered on a 5-point Likert scale (Multimedia Appendix 1). Although this scale was developed in 1996, it is relevant and applicable to current research because it is short and easy to use. Many contemporary mHealth studies have been successful in combining both ISO and SUS to assess usability [37-39]. Table 1 describes the ISO 9241-11 usability constructs.

MMR is gaining popularity and acceptance across disciplines and the world [41]. It draws from multiple scientific traditions and disciplinary backgrounds. MMR is defined as “the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches for the broad purposes of breadth and depth of understanding and corroboration” [42]. MMR combines both closed-ended response data (quantitative) and open-ended personal data (qualitative) [41].

Although quantitative research historically has predominated in health sciences research, many contemporary phenomena in health care are difficult, if not impossible, to measure using quantitative methods alone [43]. The goal of qualitative research is to produce a deep understanding of a phenomenon. It can also be used to generate a hypothesis regarding a phenomenon, its precursors, and its consequences [44]. When the study phenomenon of interest is multifaceted and complex, a mixed methods approach is appropriate [43]. The National Institutes of Health best practices guideline and many mixed methods researchers advise distinguishing the quantitative purpose, the qualitative research questions, and the mixed methods questions [45]. Consequently, MMR can capitalize on the strengths of both methods, the depth of qualitative research and the breadth of quantitative research. The resulting mixed data can be integrated to balance the strengths and limitations of either method to provide a more comprehensive understanding under potentially complementary sources of evidence [43].

Understanding the principles and practices of integration is essential for leveraging the strengths of MMR. Fetters and Molina-Azorin [7] defined integration as the linking of qualitative and quantitative approaches and dimensions together to create a new whole or a more holistic understanding than achieved by either alone. Fetters et al examined vital integration principles and practices in MMR [46]. They provide approaches to integrating both research procedures and data in the design, methods, interpretation, and reporting dimensions of research [46]. Table 2 provides the relevant dimensions of MMR integration and illustrates how researchers can integrate those dimensions. These dimensions are relevant to mHealth, and additional information about MMR dimensions is explained elsewhere [7]. Through increasingly sophisticated approaches, MMR is viewed as an opportunity to address the highly complex, compelling, and even wicked research problems facing researchers in the health and social sciences [47]. Investigation of novel mHealth technologies is an important example of a highly complex research challenge that can benefit from a systematic mixed methods approach.

Usability is a complex phenomenon. It is challenging to investigate usability comprehensively using only quantitative methods or qualitative methods in isolation, so-called monomethod approaches [6]. The alternative to using a monomethod approach is using diverse methods to generate a complete picture and reveal hidden patterns and novel relationships between variables and concepts [43]. To identify and resolve usability issues, various researchers emphasize the importance of using multiple methods and sources of data [18,48]. Although many studies collect both quantitative and qualitative data to test usability [49-51], mHealth researchers could benefit from advances being made for integration in mixed methods studies [46,52].

Despite the recognized and intuitive value of using mixed methods for mHealth usability testing, mixed methodologists have yet to articulate specific designs that guide the development and testing of mHealth interventions. A core MMR study design that is attractive for usability testing is the convergent design [52]. Also called by some authors as a concurrent parallel study [53] or, historically, a concurrent triangulation design [54], the convergent mixed methods design features the collection and analysis of both types of data and then merging of the data for the final interpretation [41].

The mixed methods aim is to illustrate, explore, and measure how to improve the usability of an mHealth intervention. A mixed methods aim should imply both qualitative and quantitative data collection methods. For example, illustrate and explore imply qualitative data collection, whereas measure implies qualitative data collection [7].

Appropriate quantitative research aims include measuring effectiveness, efficiency, and satisfaction, as illustrated in Table 3. These constructs provide a measure of patients’ experienced usability. Effectiveness, efficiency, and satisfaction can be compared across iterations to identify the most usable mHealth technology.

As illustrated in Table 3, appropriate qualitative research questions include clarifying and characterizing our understanding of mHealth intervention usability. Qualitative inquiry is particularly valuable for understanding how and why a phenomenon occurs, a theory explaining a phenomenon, or the nature of someone’s experience [56]. In usability testing, specific applications can include how and why participants make certain choices when using a prototype or their overall assessment of the utility. Usability testing may require or suggest a theory for its utility. The quality of the user’s experience is critical for an mHealth developer who is creating a desirable user-friendly system.

A recent study by Beatty et al [50] illustrates the mixed methods process as they collected both quantitative and qualitative data to determine the usability of a mobile app for technology-facilitated home cardiac rehabilitation. Quantitative data included the SUS and task completion rate, whereas the qualitative data included questions about the functionality of the mobile app [50].

The matching integration strategy involves intentionally asking qualitative questions that address the scales or constructs of quantitative instruments such that both instruments will elucidate data about the same concepts or domains [7]. For example, the constructs of both the ISO standards and SUS used during quantitative data collection can be matched with similar or related qualitative questions to generate related quantitative and qualitative data, as illustrated in Table 3. A mixed methods data collection joint display includes the major constructs of inquiry in the first column. The latter 2 columns include the quantitative data, for example, scales or items, and the qualitative data, for example, qualitative questions or observation types. For example, Beatty et al [50,58] used matching by integrating the task completion time (quantitative) with asking “I noticed that the _____ feature took you longer than some of the others. Tell me more about that?” (qualitative). They also expanded on the SUS by asking open-ended questions regarding the user’s experience with the mHealth intervention.

The qualitative questions in this table include both general and specific questions. Depending on the development needs, more general questions may be used initially, whereas later, more specific questions may be asked. When data become available, the same table structure can be populated with the findings; see Step 4: Integration in the Data Interpretation Dimension. The ordering of the columns is flexible according to specific project procedures.

The diffracting integration strategy involves intentionally asking qualitative questions that will address different aspects of the quantitative data, in the case of mHealth, the usability measure [58]. The intent is to obtain different cuts of data that will reveal information about different aspects of the usability that will not be addressed with the quantitative scales or items that are being collected [58]. Hence, for the ISO measures of effectiveness, efficiency, and satisfaction, qualitative questions might explore other facets, for example, animations, color patterns, sounds, and font size.

Diffracting can be used to address external factors to the user; such as the ease of connecting to the internet or connecting medical devices via Bluetooth. It is also important to develop an mHealth intervention that is energy efficient. mHealth interventions that require frequent charging of the smartphone or medical device are not recommended. Finally, developers should ensure that adequate resources are available to address medical and technical difficulties related to the mHealth intervention.

The expanding integration strategy occurs when the findings from the 2 sources of data diverge and expand upon the phenomenon of interest by addressing both different aspects of a single phenomenon as well as a central phenomenon of interest [46]. Expansion involves intentionally asking qualitative questions that will be the same as the quantitative scales, while also measuring and asking qualitative questions that will address different aspects of usability. In essence, it reflects a hybrid strategy of using both matching, an area of overlap, and diffracting by looking at different aspects or facets of mHealth during the collection of data. Each of these integration strategies could be used effectively in usability testing.

The interactive analysis strategy [64], also called a crossover-tracks analysis [65], means that the researcher considers the qualitative and quantitative findings, in real time, as the data are collected and analyzed. That is, the data are openly, actively, and interactively considered in the context of each other. Metaphorically, the data are talking to each other.

The independent intramethod strategy, also called a parallel-tracks analysis [65], means that the researcher uses an intramethod (ie, within method) qualitative data analysis strategy separately or independently to the quantitative data analysis strategy. First, each type of data is examined using a strategy appropriate for the type of data, for example, statistical analysis of quantitative data and thematic analysis of the qualitative data. After the separate/independent analysis, the findings are then integrated to draw an overarching interpretation, so-called metainferences in MMR methodology [65].

For an iterative convergent design, the research can and likely will use both strategies depending on the stage of testing. The interactive analysis strategy is preferred, especially during early prototype testing when the number of users will invariably be smaller and there is an urgency for identifying major issues. As statistical analyses will not be feasible or necessary, this approach allows rapidly assessing user rankings of certain features, for example, using the SUS as well as their qualitative experiences with the system.

In later cycles of testing, the analysis may shift to a more independent intramethod analysis strategy. As a higher number of users engage and real-time automated digital user data emerge, the interactive analysis approach may become more challenging to conduct. Moreover, the independent intramethod analysis may be preferred when the scale of testing expands such that blinded quantitative data collection becomes more important. Doing so can enable the researcher to avoid validity threats to the data quality that could occur by changing the data collection approach or by sharing patterns with users in real time. For example, Kron et al [66] linked the qualitative findings from learners’ reflections on their experiences after completing the qualitative and quantitative analyses.

The third option can involve an iteration of both interactive and independent data analyses, that is, user survey and interview data conducted in real time can be looked at interactively, whereas automated data collection that accumulates as the number of users expands may be brought into the results of the interactive analysis after being examined independently. The exact approach may vary and evolve according to development needs.

Comparing both the qualitative and quantitative findings allows researchers to examine the similarities, differences, or contradictions. This comparison also allows researchers to obtain an expanded understanding of when the qualitative and quantitative findings from the analyses are merged for an interpretation. Similarities occur when there is convergence or confirmation between the qualitative and quantitative findings. Differences occur when the 2 types of data illustrate different, nonconflicting interpretations, so-called complementarity [67]. There is an expanded understanding, namely, expansion, when qualitative and quantitative finding provides a broader understanding of a central commonality [7,46]. Contradictions occur when there is discordance or divergence between the findings of the qualitative and quantitative data. To handle discordance, Fetters et al recommend gathering additional data, reanalyzing existing databases to resolve differences, seeking explanations from theory, or challenging the validity of the constructs [7,46].