This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
To date, several medication adherence apps have been developed. However, the existing apps have been developed without involving relevant stakeholders and were not subjected to mobile health app guidelines. In addition, the usability and utility of these apps have not been tested with end users.
This study aimed to describe the usability and utility testing of a newly developed medication adherence app—Med Assist—among ambulatory care patients in Malaysia.
The Med Assist app was developed based on the Theory of Planned Behavior and the Nielson usability model. Beta testing was conducted from March to May 2016 at a primary care clinic in Kuala Lumpur. Ambulatory care patients who scored ≥40% on the electronic health literacy scale, were aged ≥21 years, and were taking two or more long-term medications were recruited. Two rounds of in-depth interviews were conducted with each participant. The first interview, which was conducted upon participant recruitment, was to assess the usability of Med Assist. Participants were asked to download Med Assist on their phone and perform two tasks (register themselves on Med Assist and enter at least one medication). Participants were encouraged to “concurrently think aloud” when using Med Assist, while nonverbal cues were observed and recorded. The participants were then invited for a second interview (conducted ≥7 days after the first interview) to assess the utility of Med Assist after using the app for 1 week. This was done using “retrospective probing” based on a topic guide developed for utilities that could improve medication adherence.
Usability and utility testing was performed for the Med Assist app (version P4). A total of 13 participants were recruited (6 men, 7 women) for beta testing. Three themes emerged from the usability testing, while three themes emerged from the utility testing. From the usability testing, participants found Med Assist easy to use and user friendly, as they were able to complete the tasks given to them. However, the details required when adding a new medication were found to be confusing despite displaying information in a hierarchical order. Participants who were caregivers as well as patients found the multiple-user support and pill buddy utility useful. This suggests that Med Assist may improve the medication adherence of patients on multiple long-term medications.
The usability and utility testing of Med Assist with end users made the app more patient centered in ambulatory care. From the usability testing, the overall design and layout of Med Assist were simple and user friendly enough for participants to navigate through the app and add a new medication. From the participants’ perspectives, Med Assist was a useful and reliable tool with the potential to improve medication adherence. In addition, utilities such as multiple user support and a medication refill reminder encouraged improved medication management.
Medication adherence is defined as the “extent to which a person’s behaviour-taking medication, following diet, and/or executing lifestyle changes corresponds with agreed recommendations from a healthcare provider” [
Several interventions have been developed to improve medication adherence and reduce medication administration–related errors [
In recent years, mobile health (mHealth)—defined as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants, and other wireless devices” [
To date, several smartphone medication adherence apps have been developed [
This study aimed to assess the usability and utility of a newly developed medication adherence app—
A qualitative methodology was utilized to obtain an in-depth understanding of end users’ experiences when using
The development phase consisted of two parts: alpha and beta testing. Alpha testing involved testing the design of
The conceptual framework for the design and development of Med Assist based on the Theory of Planned Behavior and the Nielson Usability Model.
Summary of the preferred features and utilities of Med Assist.
Utility | Description |
Specific medication reminder | Users will be able to set specific medication reminders with a personalized tone |
Specific medication refill reminders | Users can set specific medication refill reminders, which prompt users to procure a prescription refill before running out of medications. |
Complex medication regime | Ability to aid patients in managing complex medication regime. |
Adherence scoring system | Users are able to calculate their adherence to medications. A 100% adherence to medications is displayed as five stars. |
Multiple user support | Users can enter another individual’s list of medications in addition to their own. |
Third party reminder or “pill buddy” | This function allows a family member to receive text message notification stating that the user has missed three consecutive medication reminders. |
Contact clinic or pharmacy by email | Users can contact the clinic receptionist via email to postpone an appointment or the outpatient pharmacy to check on medication availability. This maximizes appointment schedules and allocates last-minute vacant slots to other patients. |
Available in dual language |
Med Assist vP4 (
The start, registration, homepage, and adherence score of Med Assist (vP4).
Flowchart of Med Assist design and development.
We included participants who had an electronic health literacy score of ≥40%, who understood English or Malay, and were taking two or more prescribed medications for their chronic condition. We excluded participants aged <21 years or who had mental disabilities. Purposive sampling was used to recruit older (≥65 years of age) and younger (<65 years of age) participants, as we wanted the experiences of older participants who may have more comorbidities, but may not be comfortable in using mobile apps, as well as younger participants who may have lesser comorbidities (than older persons) but may be more comfortable with using mobile apps. The purpose of recruiting participants based on age was to obtain a wider perspective when using
A topic guide based on the Nielson usability model [
During the first interview, the usability of
During the second interview, the utility of
A
The electronic health literacy scale (eHEALS) was used to assess participants’ literacy skill in using their smart devices to find health-related information on the internet [
Ethics approval was obtained from the University of Malaya Medical Ethics Committee prior to the study (MECID no. 20143-12).
A researcher approached participants who were “using” their smartphone while waiting to see their doctor. Participants were asked several “screening questions” such as “how often do you use your smartphone?” “what do you use your smartphone for?” “do you use any smartphone applications?” and “if yes, what applications do you use?” These questions were asked, so that the researcher could identify participants who used their smartphone as more than just a telecommunication device. For those who agreed to participate, written informed consent was obtained. Participants were then asked to fill in the demographic form and the eHEALS.
Each participant was interviewed twice. During the first interview, participants were encouraged to “concurrently think aloud” [
Participants were “probed retrospectively” during the second interview [
Participants were recruited until data saturation occurred. Data saturation was defined as “no new themes or codes emerging from interviews.” This was established when the next three participants recruited provided perspectives that were previously highlighted by other participants [
All interviews were transcribed verbatim. One researcher (SC) immersed herself in the data. An interpretive-descriptive approach was used to identify the themes that emerged from the data. This approach was used to obtain a deeper understanding of the usability and utility of
A total of 13 participants (6 men, 7 women) were recruited (
Three themes emerged from usability testing. They were challenges encountered when adding a new medication, with regard to patients’ understanding of their complex medication regime, and on the medication summary page.
Several subthemes emerged under this theme: confusion by terms used when adding medications into
Demographic characteristics of participants recruited for beta testing.
IDa | Gender | Age (years) | Ethnicity | Level of education | Number of medication(s) | Patient/carer | iPhone/android user | eHEALSb score (%) |
P1 | Male | 66 | Chinese | Secondary | 3 | Patient/carer | iPhone | 84.4 |
P2 | Female | 29 | Eurasian | Secondary | –c | Carer | Android | 75.0 |
P3 | Female | 43 | Chinese | Tertiary | 4 | Patient | Android | 59.4 |
P4 | Female | 55 | Indian | Secondary | 2 | Patient | Android | 75.0 |
P5 | Female | 72 | Malay | Tertiary | 2 | Patient | Android | 75.0 |
P6 | Male | 56 | Indian | Tertiary | 3 | Patient | iPhone | 90.6 |
P7 | Male | 72 | Malay | Secondary | 4 | Patient | Android | 68.7 |
P8 | Male | 62 | Chinese | Tertiary | 6 | Patient | Android | 75.0 |
P9 | Female | 42 | Malay | Tertiary | 3 | Patient | Android | 46.9 |
P10 | Female | 64 | Malay | Secondary | 2 | Patient/carer | Android | 78.0 |
P11 | Female | 57 | Indian | Tertiary | 2 | Patient | Android | 50.0 |
P12 | Female | 27 | Malay | Tertiary | 4 | Patient | Android | 56.0 |
P13 | Male | 44 | Malay | Tertiary | 5 | Patient | Android | 75.0 |
aID: identification.
beHEALS: electronic health literacy scale.
cNot available.
Two terms—“timing” and “variable dosing”—confused most participants. In
Hmm..(Participant was initially confused by the term, but able to figure it out on her own) (taps on “timing) okay…once a day.. (participant thinks aloud)...
How do I specify the frequency to take my medication? What does “timing” mean? What do I put here? (taps on timing) Oh I see!
“Variable dosing” is a feature in
Okay...variable dose how...[taps on variable dose, switching it on]...850...take two tablets...no, strength is 850, strength supplied is 850 and strength to take is 850. I’m taking two tablets what...[after researcher helps him]...timing...twice a day, okay good...
When entering medication details into the app, most participants knew their medication by the brand name, but not by its generic name.
I’m not too sure what is the name of my medication. I know it by its brand name. I do know what the medication is for so having the indication automatically linked to the medication name is good to have.
This part, I am not sure of the name of my medication, for example my cholesterol tablet, simvas or something.
One participant was prescribed 75 μg levothyroxine daily but was supplied with 50 μg and 25 μg tablets. The participant was aware that she was supplied with two different strengths but was unaware that she had to enter the medication (levothyroxine) twice into
For strength prescribed it is 75 μg but I am given 50 μg and 25 μg by the pharmacy. Okay so for strength to take I should enter 75 μg because that is what I am taking, but the app won’t let me enter 75 μg. Why is that?
Most participants understood their medication regime and the reason for taking their medication.
Yes I do know my medication regime better after using Med Assist. I have previously been relying on my wife who takes care of my medication supply and gives me my tablets in the morning.
Once all the details of the medication were entered, a summary page appears. The medication summary page could be accessed from the home page by tapping on the medication icon. All participants found this feature useful, as a list of all the medications that they had to take was summarized onto one page.
Oh having a medication summary page makes sense. This is very useful, makes the information simplified and more organized. It’s nice to have this feature.
Two themes emerged from the utility testing of
Three subthemes emerged: a medication alarm reminder system, an adherence scoring system, and the pill buddy option.
All participants found the medication alarm reminder useful, including the customizable alarm tone. One participant expressed concern that if the device was not with the user when the alarm rang, the alarm reminder would not have reminded the user to adhere to his/her medication regime. However, a snooze option was available on the medication reminder, which would remind users to take their medication at a later time. Another participant was unsure what time he/she could fix the reminder, as he/she did not have a fixed daily schedule.
Oh yes this was helpful. It prompted me to remind my mum to take her medications.
Because I know… being ladies you tend to leave your handbag in the bedroom and you’re wandering around in the house and all that so I … the times that I actually took my phone with me, it does ring, it does ring that (short) tone beep. So if I were busy doing something else, I would have just missed it and of course it wouldn’t serve its purpose.
Oh dear, the problem with setting the reminder is that I don’t have set times. It would have to depend on what time I wake up in the morning and go to bed in the evening, which varies day to day.
Of the 13 participants, 7 were not aware that
I’m sorry what was that? Oh! I wasn’t aware of that. So this would show what medications I have missed and when? And I am rated based on my adherence?
Huh? I didn’t take my medicine? But...if I didn’t take my medicine, my husband also didn’t take the medicine? [This participant used the multiple user support feature] (After explanation) Oh that means each time I must tap on ‘take medicine.’ I didn’t know...”
The pill buddy system allowed family members of a user to check that they had taken their medications remotely. Family members added as a “pill buddy” in the app will be sent a text message when the user missed taking their medication three times consecutively. Users who were caregivers found this feature useful, as they were able to “monitor” family members remotely.
Oh this is good. It allows me to keep an eye on my mum even when I am not with her.
Oh I didn’t really explore that. I used the app for personal use only.
Only one subtheme emerged: multiple user support system.
Participants who were not only patients but also caregivers were pleased to be able to enter medication details of the person under their care in the same app.
Oh I used this function to enter a separate profile for my mum’s medications. And it was amazing. It had all the tablet icons on the homepage and its own alarm reminder.
From this study, three themes emerged from the usability testing of
The steps to add a medication were simplified and displayed in a hierarchical order to prevent cognitive overload [
The terms used in a medication adherence app should be self-explanatory. Although health care providers were involved in the design of
To our knowledge, no other study has reported the experiences of participants when using a medication adherence app. Participants also reported that the medication summary page, which was accessible through tapping on the medication icon on the homepage of
Several studies have shown that behavioral change is achievable through active reminders, which reinforces the benefits of medication adherence apps [
A review on mHealth apps found that most apps were based on a one-way reminder system (which sent reminders to users to take their medications) [
The adherence score serves as a gamification process (a process where the user is rewarded upon achieving the goal of the app) [
In the Asian community, it is common for working adults to look after their elderly parents [
This study used a two-phase in-depth interview for the data collection process, which allowed us to explore patients’ and caregivers’ experience and perspectives in using the app for the first time and its utility after a period of using the app. To the best of our knowledge, no other medication adherence apps have conducted a similar study [
The generalizability of this study is limited by the use of a local hospital medication database. Although this simplified the entry of new medications into
Our study found that the overall design and layout of
electronic health literacy scale
mobile health
We would like to thank all the participants and the University of Malaya in this study. Funding for this study was obtained from the University Malaya Research Program (RP015C-13HTM).
None declared.