This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Surgical patients are increasingly using mobile health (mHealth) platforms to monitor recovery and communicate with their providers in the postdischarge period. Despite widespread enthusiasm for mHealth, few studies evaluate the usability or user experience of these platforms.
Our objectives were to (1) develop a novel image-based smartphone app for postdischarge surgical wound monitoring, and (2) rigorously user test it with a representative population of vascular and general surgery patients.
A total of 9 vascular and general surgery inpatients undertook usability testing of an internally developed smartphone app that allows patients to take digital images of their wound and answer a survey about their recovery. We followed the International Organization for Standardization (ISO) 9241-11 guidelines, focusing on effectiveness, efficiency, and user satisfaction. An accompanying training module was developed by applying tenets of adult learning. Sessions were audio-recorded, and the smartphone screen was mirrored onto a study computer. Digital image quality was evaluated by a physician panel to determine usefulness for clinical decision making.
The mean length of time spent was 4.7 (2.1-12.8) minutes on the training session and 5.0 (1.4-16.6) minutes on app completion. 55.5% (5/9) of patients were able to complete the app independently with the most difficulty experienced in taking digital images of surgical wounds. Novice patients who were older, obese, or had groin wounds had the most difficulty. 81.8% of images were sufficient for diagnostic purposes. User satisfaction was high, with an average usability score of 83.3 out of 100.
Surgical patients can learn to use a smartphone app for postoperative wound monitoring with high user satisfaction. We identified design features and training approaches that can facilitate ease of use. This protocol illustrates an important, often overlooked, aspect of mHealth development to improve surgical care.
Telemedicine has begun to supplement, and in some cases supplant, postoperative care received in the clinic in many surgical practices. Existing platforms include Web and mobile phone–based portals for virtual follow-up after elective general surgery and telephone follow-up after laparoscopic cholecystectomy and open inguinal hernia repair [
As ownership of tablets and mobile phones becomes more common [
In addition, many existing telemedicine platforms designed for the postdischarge period are primarily text or audio based but transmit no visual information [
Those telemedicine protocols that do have a visual component are frequently asynchronous and episodic and have not been designed for ongoing monitoring of postoperative recovery. Most commonly, these protocols involve either digital images or videoconferencing intended to replace an in-person office visit [
We address these gaps by creating an image-based smartphone app aimed at increasing communication between patients and their caregivers after they leave the hospital as part of a forthcoming effort to detect wound complications at an early stage and to reduce hospital readmissions. We then evaluate its usability in a largely technology-naive population of patients undergoing general and vascular surgery. In constructing this project, we consulted 2 international standards: International Organization for Standardization (ISO) standard 9241-12 was used to optimize the design of our application and then ISO 9241-11 was used to guide usability testing of the app. ISO 9241-11, a widely used guideline for current usability testing methods, which focuses on effectiveness (ie, task completion), efficiency (ie, time within task), and user satisfaction of new technology, was used to assess the patient-centeredness and usability of this app to monitor postoperative wounds [
Eligible participants included inpatients 18 years of age or older on the vascular or general surgery service of a large, academic tertiary care hospital. Subjects were recruited during one of two usability sessions in November and December 2015. Participants were eligible if they had a surgical incision longer than 3 cm and were close to their baseline functional status.
User interface design dimensions from International Organization for Standardization (ISO) standard 9241-12 and corresponding WoundCheck design features.
Information display dimension | Definition | Method employed | Sample app design features |
Clarity | Content conveyed quickly and accurately | Physician review; focus group | All app language validated by physician review panel (for clinical usefulness) and lay focus group (for interpretation) |
Discriminability | Information is readily distinguished | Iterative redesign | Tap-only response options (no text entry or scrolls) |
Conciseness | No extraneous content | Focus group review of content | Yes or no questions for symptoms |
Consistency | Information is presented in the same way consistent with expectations | Focus group review of layout; Iterative redesign | All response screens are identical |
Detectability | Attention is directed to salient information | Multidisciplinary design team; physician review | Image review screens to ensure quality image |
Legibility | Easy to read content | Focus group test; iterative redesign | Readable Helvetica Neue bold font choice, size 26 or larger with high contrast display (black type on white background) |
Comprehensibility | Meaning is unambiguous and clear | Focus group review of content; physician review | 6th grade reading level |
Subjects with major cognitive or neurologic deficits prohibiting their independent use of the app were included only if they had a capable caregiver who consented to complete the app on their behalf. All subjects who met inclusion criteria were approached to participate. Participants were asked regarding their prior experience with smartphones, whether they owned their own smartphone, and whether they had used a smartphone to take a digital image.
We aimed for a sample size of at least 5 participants, a number based on evidence from the usability literature indicating that 5 participants make a sufficient sample size to detect 80-85% of an interface’s usability problems [
The University of Wisconsin Health Sciences Institutional Review Board approved the study protocol.
WoundCheck is an iOS app that enables patients to capture digital images of surgical wounds and sends them to their providers from home, along with brief updates on postoperative recovery. This app was developed internally through the University of Wisconsin Department of Surgery with the assistance of software programmers in our Information Technology division. In designing the app, we consulted ISO 9241-12, an international standard for screen layout and the visual display of complex information, and established guidelines on user interface design to ensure that the user interface was easily navigated by our target population of older adults and novice users [
Tenets of adult learning and memory and corresponding training design features.
Evidence-based dimension of adult learning | Sample training design features |
Require more time to learn new skills [ |
Let participant set the pace of training |
Need repetition and multiple formats of materials [ |
Repetition; supplementary flash cards; let participant develop own narrative around the device |
Challenged by complex, unusual material [ |
Emphasis on purpose of training; emphasize “why” of tasks |
Decline in motivation when not experiencing success [ |
Frequent positive feedback; opportunities to reflect and ask questions throughout |
Repeated exposure facilitates learning [ |
Primary training session + refresher training prior to discharge |
Cue-based recall [ |
Use of reminder alarm at the time of participant choosing as a cue to use app |
Task performance (not just observation) with teach-back [ |
Provide a device to participant to use throughout training |
The program is ultimately designed for use during the period between hospital discharge and the routine postoperative clinic visit. The app was designed to be linear with one pathway through the app to maintain simplicity and intuitiveness. There are 2 phases to the app: an image-taking phase where participants take digital images of their wound and have the ability to review or retake their images, and a brief survey with yes or no questions regarding their recovery. Screenshots of the app are provided in
To vet the content of the app and training and meet the burden of the ISO design standard, we conducted 2 focus groups to review the app with Community Advisors on Research Design and Strategies (CARDS). These are standing focus groups of community members from diverse racial, ethnic, socioeconomic, and educational backgrounds who are recruited from food pantries, senior meals, parenting programs, and other similar programs. They are trained to give constructive feedback to researchers, health educators, and outreach professionals. The CARDS members, the majority of whom are novice smartphone users, evaluated prototype screens of the app and all app language in the first focus group. The image capture training protocol was evaluated in the second focus group.
The app and transmission of patient data were developed to fully comply with the Health Insurance Portability and Accountability Act. A passcode is used to secure and encrypt the device. Each device is profiled, allowing us to remotely wipe the device, prevent the installation of additional apps, and limit other device features. No information is stored on the mobile phone itself; the app can only be used to submit information, not retrieve it. The app transmits data to the University of Wisconsin Department of Surgery research server using the Hypertext Transfer Protocol Secure (HTTPS;
Screenshots of the final app. A. Modified camera screen. B. Image review screen where participants can choose whether to keep the image they have taken or try again. C. Review screen of all added images; up to 4 images may be added. D. A series of yes or no questions follow. E. Participants can review their survey responses and have the option to change them prior to submission. F. Submission confirmation screen.
1. Have you have fevers or chills in the past 24 hours?
2. Have you changed how you take your medication in the last 24 hours?
2a. (If responded yes to 2) Is this change related to your pain medication?
2b. (If responded yes to 2a) Did you increase your pain medicine?
3. Has the area around your wound become red in the past 24 hours?
4. Has the area around your wound become swollen in the past 24 hours?
5. Is there a bad smell coming from your wound?
6. Is fluid leaking from your wound?
6a. (If responded yes to 6) Is the fluid white, yellow, or green?
6b. (If responded yes to 6) Do you change the dressing more than once because fluid soaks through?
Wound Check app data flow overview.
Following preliminary design, we formally tested the usability of the app with postoperative vascular and general surgery patients at a major academic medical center. The app was loaded onto a 5th generation iPod Touch running iOS8. We assessed patients’ baseline familiarity with smartphones prior to testing. A researcher introduced the device to participants with an overview of its general functions and how to operate it, if needed. User tasks included waking up the device, launching the app, image capture, review and retake or acceptance of captured images, question response, and submission. Following the first round of usability testing, an interim assessment of the app was performed and adjustments were made based upon the findings of the first round. The updated version of the app was then used for the second round of testing.
We consulted ISO 9241-11 in designing the format for formal usability testing of the app [
Following usability testing of the app, participants were asked to rate their performance and to provide feedback on the app itself. Participants also completed a system usability scale (SUS) to evaluate their satisfaction with the app (questions presented in
I think that I would like to use this app frequently
I found the app unnecessarily complex
I thought the app was easy to use
I think that I would need the support of a technical person to be able to use this app
I found the various functions of this app were well integrated
I thought there was too much inconsistency in this app
I would imagine that most people would learn to use this app very quickly
I found the app very awkward to use
I felt very confident using the app
I needed to learn a lot of things before I could get going with this app
Of the 14 patients who were approached to participate, 5 declined due to time constraints or disinterest. Nine participants completed usability testing, 3 of whom had caregiver assistance or proxy participation. Five participants owned their own smartphone, and 7 had used a smartphone to take a digital image at least once prior to this study, leaving 2 who had no prior experience with smartphones. Demographics and basic clinical information are presented in
Four participants (44.4%) had abdominal wounds (an aortic graft explantation and an axillary-bifemoral bypass, 1; an aortobifemoral bypass, 1; an open distal gastrectomy, 1; and an open distal pancreatectomy and splenectomy, 1). Four participants (44.4%) had groin wounds (an aortobifemoral bypass, 1; bilateral groin explorations and repair of a common femoral artery aneurysm, 1; a superficial femoral artery graft resection and interposition graft placement, 1; and an endovascular aortic aneurysm repair, 1). Two participants (22.2%) had lower extremity wounds (bilateral lower extremity fasciotomies, 1; and a superficial femoral artery to posterior tibial artery bypass, 1). One participant (11.1%) had an amputation stump above the knee. Two participants had 2 wounds, bringing the total number of wounds to 11.
Demographic and baseline characteristics.
Characteristic | n (%) or mean (SD) |
Female, n (%) | 5 (55.6) |
Age (years), mean (range) | 55.2 (19 - 80) |
White | 6 (66.7) |
African-American | 2 (22.2) |
Latino | 1 (11.1) |
Body mass index (kg/m2), mean (range) | 29.0 (17.4 - 43.65) |
Private | 4 (44.4) |
Medicare | 3 (33.3) |
Medicaid | 1 (11.1) |
Uninsured | 1 (11.1) |
Abdominal | 4 (44.4) |
Groin | 4 (44.4) |
Lower extremity | 2 (22.2) |
Amputation stump | 1 (11.1) |
Effectiveness, efficiency, and satisfaction results of usability testing.
Participant | Training time (min) | Time to complete app independently (min) | Total time (min) | Required assistance? | Image deemed usable by majority of raters | SUSa score | |
P1 | 12.8 | 1.6 | 14.4 | No | Yes (AKAb stump) | 82.5 | |
P2 | 2.7 | 3.1 | 5.8 | No | Yes (Abdomen) | 97.5 | |
Yes (Groin) | |||||||
P3 | 6.4 | 16.6 | 23.0 | Yes | No (Groin) | 72.5 | |
P4 | 2.2 | 2.4 | 4.6 | No | Yes (Abdomen) | 87.5 | |
Session 1 mean (SD) | 6.0 (4.9) | 5.9 (7.1) | 12.0 (8.6) | 85 (10.4) | |||
P5 | 2.4 | 1.4 | 3.9 | No | Yes (BLEc fasciotomies) | 82.5 | |
P6 | 3.2 | 6.2 | 9.4 | Yes | Yes (Groin) | 87.5 | |
P7 | 2.1 | 6.4 | 8.5 | Yes | Yes (Lower extremity) | 75 | |
No (Groin) | |||||||
P8 | 8.0 | 4.7 | 12.7 | Yes | Yes (Abdomen) | 70 | |
P9 | 2.9 | 2.2 | 5.1 | No | Yes (Abdomen) | 95 | |
Session 2 mean (SD) | 3.7 (2.4) | 4.2 (2.3) | 7.9 (3.5) | 82 (9.9) | |||
Overall | |||||||
mean (SD) | 4.7 (3.7) | 5.0 (4.7) | 9.7 (6.2) | 83.3 (9.6) |
aSUS: System Usability Scale (scored 0-100).
bAKA: above the knee amputation.
cBLE: bilateral lower extremity.
Effectiveness and efficiency data are presented in
The most difficult task in the initial round of testing was to take a digital image of the wound. Participants were confused about the flow through the image-taking portion of the app, and they also faced difficulty with button placement. Specifically, the placement of the image capture button directly next to the cancel button led to image capture attempts that resulted in cancellation. In addition, the cancel button looped back to restart the app rather than sending participants forward even if they had already captured an image. As a result of these difficulties, the image-taking portion of the app was redesigned to make it more intuitive, and the camera buttons were placed in more convenient locations on the screen to facilitate image capture (
Participants with groin wounds, and particularly obese participants with groin wounds, had considerable difficulty taking images of their wound independently due to inadequate exposure of the wound. At least one other person was required to fully expose the wound, and even then, it was difficult to achieve the optimal angle for image capture. Participants who had active caregivers present were better able to perform this task without requiring researcher’s assistance.
On assessment of image quality, 9 of 11 (81.8%) images were deemed sufficient for diagnostic purposes by a majority of rating physicians (
The survey task within the app was easy for all participants to use. On the initial round of testing, the screen for reviewing survey responses was scrollable, such that all responses appeared on a single screen, but some were not visible unless the participant scrolled to the bottom of the screen. This was confusing for some participants, as this was the only scrollable screen within the app, requiring mastery of a new functionality. The response review screen was revised in the second round of testing to be split into 2 screens to eliminate the need to scroll. After this adjustment, participants had no difficulty with this section.
Original and modified image-taking screen. On the left is the original camera screen with both the image-capture and cancel buttons at the bottom of the screen. On the right is the modified screen based on user feedback. The image-capture button takes up the whole bottom of the screen, but does not extend as far up into the screen, and the cancel button has been moved away from it to decrease button confusion.
The responses to the System usability scale (range: 0-100) are presented in
These challenges were also observed during usability testing, particularly with novice users who, in addition to learning to use the app, needed more time to become comfortable using the device itself. Four participants struggled with simply tapping the screen and alternating between tapping icons on the screen and pressing the home button; two came close to deleting the app by pressing the icon for too long rather than tapping it. As stated previously, novice users also struggled with using the camera, particularly with switching the direction of the camera to face them.
The most commonly cited concerns regarding the protocol were confidentiality of patient information and whether anyone in the care team would actually review the submitted images and survey responses. One participant was concerned “whether information (would be) followed through,” saying “you might have taken lots of pictures, but if no one looks at it, it’s all for nothing.” Other concerns raised were device battery life and difficulty being able to fully visualize the wound to take a digital image. Three participants stated they had no concerns. All 9 participants said they would be able to complete the app daily after discharge if they were given full instructions. One particularly enthusiastic participant said, “I wish I had it today.” All nine said they would benefit from a protocol using this app following hospital discharge. One participant said, “I think it’s really pretty neat...if you have a concern, you’ll get an answer like that.” Eight participants said they would recommend the app to a friend or family member if they had surgery, and one participant was neutral, saying “...that’s their decision.”
The current standard of care for the majority of surgical patients following hospital discharge involves little formal communication between patients and their care team until their routine clinic follow-up 2-3 weeks after discharge. This is a crucial time period during which many complications and setbacks to recovery occur, and is thus ripe for mHealth innovation [
To address these gaps, we have developed a smartphone app that allows patients to be in daily communication with their provider with both subjective symptom data and visual information in the form of digital images. We have demonstrated that most patients and their caregivers are able to learn to use our app, can use it to transmit meaningful clinical information, and have a high level of satisfaction and enthusiasm regarding the protocol. Additionally, studying patients during the immediate postoperative period allows for the most conservative estimate of usability given that patients are still in recovery and may not be at their functional baseline. Given that our participants were mostly older adults, seen during the vulnerable postoperative period, some with very limited prior smartphone experience, the wide success we observed is encouraging for the ability of the general population to use the app without difficulty once given protocol-based training and clinical support at the outset.
Insights from the field of systems engineering provide a helpful framework for the development of mHealth protocols, as well as their attendant training programs. Work focusing on universal access and assistive technology for persons with disabilities is especially relevant for creating mHealth protocols accessible to a diverse patient population, particularly patients recovering from surgery, who are elderly or have limited prior experience with the technology, as in this study. Vanderheiden [
For the purposes of our protocol, changing the individual involved tailored training, which we made modular so that portions could be added or skipped depending on the participant’s needs. As expected, the participants who struggled the most with the app were novice smartphone users and older participants. Most of this difficulty was in learning to navigate the smartphone itself and not necessarily related to the app. This was reflected in the responses to the system usability scale, where 11-20% of participants expressed needing to learn a lot before they could get going with the app or felt that they would need assistance of a technical person to complete it. Previous studies of mHealth apps have found similar results, with lack of familiarity with mobile devices and the need for assistance identified by participants as barriers to independent use [
Importantly, efficiency of training should not come at the expense of effectiveness. Protocol training will need to be performed at the pace of the learner, taking care to keep them engaged. Two participants expressed training fatigue, with one saying, “I’m glad you’re getting out of here; that was time consuming” after 27 minutes of training, despite her not having fully mastered the task. Another said, “you mean we’re not done?” after 25 minutes of training. Bearing this in mind, future training efforts may need to be spread over multiple sessions both to reinforce tasks and to avoid fatigue and boredom with a single session.
The second approach for improving accessibility is to provide adjunct tools to overcome particular barriers to use. For participants who struggled with tapping the screen, a stylus may be easier and more intuitive than using their finger. One participant opted to do this on her own based on her prior experience using a stylus with her tablet device. Another barrier we encountered in our protocol was the difficulty experienced by patients with wounds in certain locations that were difficult to take an image of, particularly groin and abdominal wounds as well as amputation stumps. Potential tools to aid these patients might include training them to use selfie-sticks or mirrors to improve their ability to independently take images of wounds in these locations. However, assistive devices or tools have the potential to add an additional layer of complexity for patients who are already uncomfortable with the device or the app, and this must be weighed against the potential benefit of their use. Because groin wounds are at increased risk of developing surgical site infection [
Finally, user accessibility may be improved by changing the environment to be accessible to all users without the need for specialized devices or tailoring to the individual, an approach termed “universal design.” Following the first round of testing, we made several subtle but significant improvements to the design of the app itself to improve its usability for a wide range of users. The reconfiguration of buttons on the camera screen made capturing images easier for participants with limited fine motor ability or who had difficulty with discrete touch. We eliminated screens that required scrolling up and down to preclude novice users or those with cognitive limitations from having to learn an additional skill. In making these changes, the app becomes more accessible to all users, including those who did not have difficulty completing it prior to these modifications, by making it as simple and straightforward as possible. mHealth platforms in the future should strive for universal accessibility in their design to maximize participation and benefit.
One aspect of universal design we did not achieve was making the app compatible with an Android device. For those participants more familiar with Android technology than iOS, learning to use the app first required learning to use the device, a barrier not experienced by those participants who had used an iOS device in the past. This is particularly important given key demographic differences in smartphone ownerships, specifically that minorities, those of lower income, and those with lower educational attainment are more likely to own an Android device [
However, despite our best efforts to incorporate these insights from systems engineering and develop a universal design for the app and for our training protocol, it is likely that some patients will still need the assistance of a caregiver to complete the app. Through usability testing, we identified several possible reasons why some might be unable to complete the app independently. Those patients who are novice smartphone users and are unable to learn to complete the app independently will by definition need assistance. Patients who have wounds in locations they cannot reach or cannot visualize sufficiently on their own will need a caregiver. Additionally, patients who have limited independence at baseline will need assistance, as with one of our participants who was a hemiparetic bilateral lower extremity amputee. In these cases, a competent caregiver or family member will need to be identified so that these patients may still benefit from mHealth protocols. These patients may already have a caregiver or involved family member due to their baseline functional status and reliance on others for aspects of their care.
Interestingly, participants consistently rated themselves as having successfully completed the app, even when their performance did not warrant such an assessment. When asked whether taking a digital image of their wound was easy to complete, only 2 participants were neutral, while all others agreed or strongly agreed. All 9 participants agreed or strongly agreed with the statement “I am confident I completed this task” in reference to taking a digital image of their wound, even the participants whose images were not sufficient for clinical decision making. Sonderegger et al [
Despite these barriers, there was substantial enthusiasm from most participants about the protocol. One participant told the research team he wished he could take the device home upon discharge and use it to stay in contact with the care team. All participants thought they would benefit from this protocol and would be willing to complete the app daily if they were instructed to do so. This is consistent with previous studies of mHealth [
In addition, the fact that many participants could ultimately complete the app independently or with caregiver’s assistance is encouraging. The overall usability score of 83.3 is above average for usability testing, indicating a level of comfort among first-time users of the app [
This is the first study, to our knowledge, to formally investigate usability of a medical device with digital image taking capability using the ISO 9241-11 standards [
The results of this study should be interpreted in the context of several limitations. Our study may be limited by its sample size. Considerable debate exists within the literature regarding the ideal sample size for usability testing. Historically, a sample of only 5 participants was thought to be of sufficient size, but more recent data suggests a larger sample is required to make accurate assessments [
As postoperative lengths of stay decrease, health systems will need to become creative in their methods of monitoring patients in the outpatient setting. Many telemedicine protocols have emerged to address this goal, but ours is the first to add an asynchronous visual component through the use of digital images, whose power to efficiently convey vast amounts of information is unparalleled in today’s standard of care. Additionally, by directly engaging with our patient population and making them active participants in their care, we participate in a growing movement toward patient-centered care and shared decision-making. We have demonstrated that the majority of patients can be taught to complete our app independently and that patients are enthusiastic about partnering with their providers in novel ways to optimize their recovery. Though the majority of participants had little difficulty completing the app, formal usability testing allowed us to identify components needing further improvement, providing invaluable information we could not have otherwise obtained. This argues strongly for the use of formal usability testing in the development of future novel protocols for patient-centered care.
above the knee amputation
bilateral lower extremity
surgical site infection
system usability scale
None declared.