As previously stated, the MAUQ, developed and validated by Zhou and colleagues [7], is designed to assess the usability of mHealth apps among patients and health care providers. There are two versions of MAUQ, one for interactive apps and one for stand-alone apps. Each also addresses two of its potential users: patient or health care professional. We will work on MAUQ for standalone apps intended for patients. Always consider that the term “patient” refers to a person who uses an mHealth app to manage their health either for maintenance or improvement.

The questionnaire is comprised of 3 subscales or dimensions: ease of use (items #1 to #5), interface and satisfaction (items #6 to #12), and usefulness (items #13 to #18). Participants are required to indicate their level of agreement with each item on a 7-point Likert scale, with 1 representing strong disagreement and 7 representing strong agreement. The overall usability of the app is determined by calculating the total score and the mean score of all statements. A higher overall mean score is indicative of superior app usability. Conversely, a mean score <4 indicates suboptimal usability.

The original MAUQ (standalone, for patient version) exhibited robust internal consistency, with an overall Cronbach α value of 0.914. This indicates a high degree of correlation between the questionnaire items, indicating good reliability. The study was conducted in 2020 [7].

The methodology used to translate and validate the original MAUQ (standalone, for patient version) into Spanish followed the approach proposed by Sousa and Rojjanasrirat [20], with the objective of ensuring equivalence between the original and translated versions. The process is comprised of several steps (illustrated in Figure 1), which can be grouped into the following 4 stages: translation and adaptation, content validation, face validation, and internal consistency reliability.

The preliminary phase of the translation process entailed the participation of two certified translators, each with a specialized understanding of both Spanish culture and linguistics. These professionals engaged in a forward translation from English to Spanish. One translator had a background in computer science, while the other had a background in health sciences. This resulted in the generation of two distinct documents designated T1 and T2.

Subsequently, the documents (T1 and T2) underwent a harmonization process. A third translator, a Spanish language teacher with more than 5 years of teaching experience, was tasked with identifying any ambiguities; discrepancies; and differences in words, sentences, grammar, and meaning. This was accomplished through a comparison of T1 and T2 with the original MAUQ. Any discrepancies and ambiguities were discussed and resolved through a process of consensus decision-making involving the third translator, the first two translators, and two members of the research team. The process culminated in the production of an initial translated version of the MAUQ document (T3).

In the third phase of the process, the translated version of the MAUQ (T3) was provided to 2 independent native Spanish speakers who had acquired English as a second language. One individual (a computer science lecturer) was a content expert in the instrument’s construct, while the other had no prior knowledge of the instrument (an English language teacher). The 2 participants then produced independent back-translated versions of the MAUQ (T4 and T5).

Last, a committee was established comprising 3 researchers, the translators involved in the initial and concluding stages, and the original developers of the MAUQ (when necessary). The committee conducted a comparative analysis of the documents with the original MAUQ, engaging in discussions via email and web-based meetings. Any ambiguities or inconsistencies in cultural meaning and expression of words and sentences and the format of the answers were addressed and resolved through consensus among the committee members. This process resulted in the development of the prefinal Spanish questionnaire, designated as S-MAUQ’.

Content validation is achieved by a panel of experts who assess the relevance and clarity of questionnaire items for measuring the usability of health-related mobile apps. Face validation is accomplished by a set of nonexperts or target users who specifically assess the clarity of the items.

No financial compensation was provided to participants at any stage of the study. Participation was voluntary and anonymous, except for members of the expert panel responsible for content validation, who were intentionally selected for their expertise. All participants (including the expert panel) were provided and signed an informed consent online, which included the possibility of opting out. The study adhered to the Declaration of Helsinki and was approved by the Bioethics Committee of the Medical University of Málaga (17‐2022-H).

Please refer to Table 1, which presents the calculated values (using Excel) of the I-CVI for relevancy and clarity, the (Pc), and the modified kappa agreement (κ) by the 10 experts for each item of the S-MAUQ. The S-CVI was calculated to be 0.778. However, as can be seen, there were 2 items (#3 and #6) with κ<0.4 (interpreted as poor agreement) and 2 others (#15 and #18) with 0.4<κ<0.6 (interpreted as fair). Considering these findings, we sought to further refine the instrument in accordance with the recommendations of the expert panel.

The results of the second round are presented in Table 2, which indicates an S-CVI of 0.850. However, 2 items (#14 and #15) demonstrated a markedly low level of relevance and clarity (I-CVI), accompanied by a similarly low κ (0.4≤κ≤0.6, interpreted as “fair”). Following a thorough review with our panel of experts, it was determined that both items should be removed from the preliminary Spanish questionnaire (S-MAUQ). In conclusion, the S-CVI was calculated to be 0.881 (without considering the 2 items that were subsequently eliminated).

Please refer to Table 3 for the calculated values of the I-FVI, the Pc, and the modified kappa agreement (κ) for each item. These were determined by the 12 target users. The S-FVI was calculated to be 0.927. A score >0.79 indicates that the items in the questionnaire are sufficiently clear and understandable for the nonexpert target user [27]. Furthermore, the modified kappa agreement for each item in S-MAUQ demonstrated excellent concordance (κ>0.74).

A total of 61 participants completed the web-based consent form and the S-MAUQ questionnaire for the purposes of the reliability study (the calculated sample size was 48). The participants were between the ages of 18 years and 22 years, and all were students (41 men, 20 women). The mean usability score (normalized to 1) for the Yazio app, as determined by the S-MAUQ, was 0.855 (SD 0.088). This indicates a high level of usability.

The Cronbach α value for the entire S-MAUQ was 0.87, and the values for the 3 subscales were 0.74 for the ease of use dimension (Q1-Q5), 0.74 for the interface and satisfaction dimension (Q6-Q12), and 0.64 for the usefulness dimension (Q13-Q16). The new numbering scheme was used for Q13-Q16, omitting the 2 deleted items.

Furthermore, the Cronbach α value for the complete questionnaire remained highly consistent when an item was deleted (see Table 4), indicating that the S-MAUQ exhibited good internal reliability.

Ensuring optimal usability of Spanish mobile apps is one of the most crucial factors for their use. Questionnaires are a common method for evaluating usability. This study outlined the process of translating, adapting, and validating the English version of the standalone MAUQ questionnaire for the patient into Spanish.

To this end, we followed the 4-step process proposed by Sousa and Rojjanasrirat [20], as it was outlined in the Methods section.

During the translation and adaptation phase, several meetings were held to ensure conceptual and linguistic equivalence for key terms such as interface, consistency, coherence, social setting, and health and well-being services. Final wording was established by consensus, often in consultation with the original questionnaire’s authors.

In the content validation phase, the expert panel initially did not reach full agreement on several items. A second round of review narrowed the discrepancies but left 2 items (#14 and #15) unresolved. Both items were judged to be conceptually ambiguous and difficult to interpret, even for experts. Following a series of email consultations with the original authors on the meaning of and convenience of these items and their overlap with others, we confirmed the experts’ results and their suggestions that these items could be removed from the final Spanish version.

During the face validation phase, the calculated FVI and κ coefficient yielded an S-FVI that substantially exceeded the recommended threshold, thereby ensuring the quality of the translation. The results of the study indicated that the items on the questionnaire were comprehensible and suitable for the target population.

Finally, the reliability analysis demonstrated strong internal consistency, with an overall Cronbach α of 0.87. The Spanish version maintained the original 3-dimensional structure of the English questionnaire, permitting direct cross-language comparison. Only the “usefulness” dimension was slightly affected, with an acceptable Cronbach α of 0.64 for exploratory studies. Overall, the S-MAUQ exhibited sound psychometric properties, indicating that it is a reliable and valid tool for assessing the usability of mobile health apps among Spanish-speaking patients. The final iteration of the S-MAUQ is presented in Table 5 for reference.

Our findings align with prior validations of the MAUQ conducted to assess a variety of mobile apps developed for different purposes and user groups, including the MyFitnessPal in Malaysia for evaluating caloric intake, weight goals, and nutritional guidance [16]; the Left-Handed Doctor mHealth app in China for medical consultations and public health guidance [33]; the Enable app in Germany for cancer patients to register electronic patient-reported outcomes and monitor treatment side effects [17]; and the DIGICOG-MS app in Italy for cognitive assessment of individuals with multiple sclerosis [18] and patients with osteoporosis in the Persian study [34]. Collectively, our findings suggest that both the MAUQ and S-MAUQ demonstrate stable content validity across diverse contexts and could be applied to a wide range of mHealth and eHealth applications designed for different populations.

It is noteworthy that a Spanish adaptation [19] developed in parallel with our own work has not adhered to the MAUQ standard; instead, it was adapted for breastfeeding. Several items have undergone alterations and been reordered, and one item was eliminated. The original dimensions were modified from 3 to 4, but the results indicate a significant lack of internal consistency in 2 final subscales. Furthermore, there was an erroneous interpretation and implementation of the formula applied to calculate the level of relevance and clarity of the questionnaire items. Consequently, it was not possible to achieve comparable outcomes in different languages.

We also compared our work with the aforementioned studies that focus on the “standalone, for patients” version of the MAUQ. The Malaysian version of the MAUQ [16] underwent an analysis similar to ours, maintaining the 3-dimensional structure but without eliminating any element. The German version of the MAUQ [17] also performed a study similar to ours and to that of the Malaysian version. However, the creators of the German version considered eliminating 1 item but finally decided not to do so in order to maintain consistency with the original. The Italian version [18] did not undergo content or face validation; they had a panel of 6 experts who directly compared a translated version with the original questionnaire and made modifications to have a more understandable Italian version.

In our study, as in other similar work [16], we surveyed a large sample of native Spanish-speaking students to provide a representative sample of the young adult demographic. In future research, criterion validation studies should be conducted to predict different outcomes for different domains and types of users of the S-MAUQ.

We addressed a significant gap in the field of health app evaluation in Spanish by translating and adapting the MAUQ, one of the most recognized usability questionnaires in this area. Despite the need to eliminate 2 items due to their lack of relevance and demonstrated confusion, S-MAUQ is a reliable tool in Spanish for evaluating the usability of mHealth apps.