This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Major surgery is associated with negative postoperative outcomes such as complications and delayed or poor recovery. Multimodal prehabilitation can help to reduce the negative effects of major surgery. Offering prehabilitation by means of mobile health (mHealth) could be an effective new approach.
The objectives of this pilot study were to (1) evaluate the usability of the Be Prepared mHealth app prototype for people undergoing major surgery, (2) explore whether the app was capable of bringing about a change in risk behaviors, and (3) estimate a preliminary effect of the app on functional recovery after major surgery.
A mixed-methods pilot randomized controlled trial was conducted in two Dutch academic hospitals. In total, 86 people undergoing major surgery participated. Participants in the intervention group received access to the Be Prepared app, a smartphone app using behavior change techniques to address risk behavior prior to surgery. Both groups received care as usual. Usability (System Usability Scale), change in risk behaviors 3 days prior to surgery, and functional recovery 30 days after discharge from hospital (Patient-Reported Outcomes Measurement Information System physical functioning 8-item short form) were assessed using online questionnaires. Quantitative data were analyzed using descriptive statistics, chi-square tests, and multivariable linear regression. Semistructured interviews about the usability of the app were conducted with 12 participants in the intervention group. Thematic analysis was used to analyze qualitative data.
Seventy-nine people—40 in the intervention group and 39 in the control group—were available for further analysis. Participants had a median age of 61 (interquartile range 51.0-68.0) years. The System Usability Scale showed that patients considered the Be Prepared app to have acceptable usability (mean 68.2 [SD 18.4]). Interviews supported the usability of the app. The major point of improvement identified was further personalization of the app. Compared with the control group, the intervention group showed an increase in self-reported physical activity and muscle strengthening activities prior to surgery. Also, 2 of 2 frequent alcohol users in the intervention group versus 1 of 9 in the control group drank less alcohol in the run-up to surgery. No difference was found in change of smoking cessation. Between-group analysis showed no meaningful differences in functional recovery after correction for baseline values (β=–2.4 [95% CI –5.9 to 1.1]).
The Be Prepared app prototype shows potential in terms of usability and changing risk behavior prior to major surgery. No preliminary effect of the app on functional recovery was found. Points of improvement have been identified with which the app and future research can be optimized.
Netherlands Trial Registry NL8623; https://www.trialregister.nl/trial/8623
Every year, approximately 635,000 adults in the Netherlands undergo major surgery [
Prehabilitation programs can be used to address these risks prior to major surgery. Prehabilitation is the process of improving an individual’s functional capacity by modifying risk behaviors to enable them to withstand the forthcoming stressor of major surgery [
As risk behaviors rarely occur alone, prehabilitation programs will often need to address multiple factors. These programs often consist of a combination of physical training (unsupervised or supervised by a physical therapist), nutritional support by a dietitian, smoking and alcohol cessation support, and/or psychological support [
Even though the preoperative period is considered a teachable moment, it is also a stressful period for many patients, which poses a challenge in terms of changing risk behavior [
In this study, we evaluate the first version of the Be Prepared (Beter Voorbereid, in Dutch) mHealth app. The content of the app was developed by a team of health care professionals and health care researchers collaboratively with patients to optimize the process of prehabilitation and overcome barriers. The app uses behavior change techniques to address risk behaviors and enhance patients’ health prior to surgery in order to achieve a better postoperative functional recovery [
The Centre for eHealth Research (CeHRes) roadmap, a 5-step development, evaluation, and implementation approach, was used as a guideline during the development and evaluation of the Be Prepared app [
This evaluation focuses on the usability of the app and the first effects on functional outcomes. Usability has been identified as an essential criterion for the evaluation of digital apps in health care since poor usability can influence the use of the app and thereby affect its effectiveness [
Therefore, the primary aim of this pilot study is to evaluate the usability of the Be Prepared app prototype. In addition, we explore whether the app is capable of bringing about a change in risk behaviors in people undergoing major surgery, and we estimate a preliminary effect of the Be Prepared app on functional recovery after major surgery.
This multicenter pilot randomized controlled trial (RCT) is part of the development approach following the CeHRes roadmap [
Patients scheduled for major elective surgery were recruited from the preoperative assessment outpatient clinic of two academic hospitals in the Netherlands between November 2018 and March 2019. Patients were informed about the study by the anesthetist, anesthesiology nurse, or anesthesiology assistant during their preoperative assessment. Patients who were interested in participation received written information about the study and were called the next day by the investigator (at least 24 hours after being informed about the study). During the phone call, the patient was asked whether they wanted to participate, and inclusion criteria were checked. Eligible patients signed the informed consent form provided with the written information and sent it back to the investigators. After completing the baseline questionnaires, participants were randomly assigned to either the intervention or control group using a web-based randomization system. The intervention group received a link to the Be Prepared app via email. Both groups received care as usual (eg, verbal information and information leaflets). Quantitative data were collected at baseline, 3 days before surgery, and 30 days after hospital discharge. Patient characteristics and perioperative factors were collected via the electronic health record. All other data were collected through questionnaires completed by the participant on a secured web-based system. Qualitative data were collected pre- or postoperatively from a selection of app users. Participants could indicate on the informed consent form whether they gave permission to be approached for a telephone interview.
The medical ethical committee of the Amsterdam University Medical Center approved this study (NL61503.029.18). No changes were made to the design after the study was approved. We followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines and the CONSORT EHEALTH (Electronic and Mobile Health Applications and Online Telehealth) checklist [
Patients scheduled for major elective surgery were eligible to be included if they were aged 18 years or older, had an indication for postoperative hospital stay of at least 2 nights, and had one or more risk behaviors. For the purpose of inclusion, we categorized risk behaviors into binary variables representing risk status based on the recommendations of the Dutch Health Council and evidence (ie, currently smoking, alcohol consumption 1 or more drinks every day, moderate intensity physical activity less than 30 minutes every day, muscle strengthening activities on fewer than 2 days per week, and/or unintentional weight loss of more than 3kg in the last month) [
The Be Prepared app prototype (Patient Journey platform by Interactive Studios BV) is a smartphone app which uses behavior change techniques to support patients in optimizing their health and risk behaviors prior to surgery. A behavior change technique is a strategy that helps an individual change their behavior to promote better health. Techniques like setting goals, advice on stop smoking medication, social support, feedback on behavior, and providing information on the health consequences of alcohol consumption and alcohol cessation were used in this app [
Participants in the intervention group received access to the Be Prepared app for use on their own mobile device. The introduction screen showed only basic information about the goal and use of the app. They could unlock the additional information and advice by entering a personal code they received via email.
The information and advice in the app were displayed on a dynamic timeline based on the patient’s operation date. Through this dynamic timeline, day-to-day information was offered in various ways to meet different needs. The timeline provided written information and videos, tips on healthy behavior and changing risk behaviors, quizzes, and exercise videos (
Screenshots of the Be Prepared app, translated from Dutch. From left to right, information about the purpose of the app, screening questions for app personalization, muscle strengthening exercise video, and check for progress with feedback video.
Usability of the mHealth intervention was assessed both quantitatively and qualitatively. Usability is the extent to which a product can be used by specified users to achieve specified goals with regard to effectiveness, efficiency, and satisfaction [
Whether a participant in the intervention group activated the app with their personal code was logged anonymously in the database of Interactive Studios (ISO 27001 and NEN7510 certified). In the online questionnaire, participants in the intervention group were asked whether they had used the app and, if applicable, were asked about reasons for nonuse.
The Dutch translation of the System Usability Scale (SUS) was used to assess usability among all app users. Participants completed the SUS 3 days prior to surgery. The SUS is a reliable and valid 10-statement usability scale suitable to assess a wide range of eHealth technologies. The total SUS score ranges from 0 to 100, and higher scores reflect higher usability [
For the qualitative data collection, semistructured telephone interviews were conducted with a selection of participants to gain more detailed insight into the usability of the app. Interviews took 20 to 30 minutes. During the interviews, a topic list was used based on the usability components described by Nielsen [
Risk behaviors were assessed through self-report at baseline and 3 days before surgery. Participants were asked to indicate on how many days per week they were physically active for at least 30 minutes, on how many days per week they performed muscle strengthening activities, and on how many days per week they consumed one or more alcoholic beverages. Participants were asked whether they were currently smoking. Current smokers were asked to indicate how many cigarettes they had smoked last week.
Participants were also asked to self-report their change in risk behaviors using a set of closed-ended questions at 3 days before surgery. All participants were asked whether they had made positive changes to their risk behaviors regarding smoking, alcohol intake, unintentional weight loss, physical activities, and/or muscle strengthening activities prior to surgery.
Functional recovery after surgery was assessed by the Patient-Reported Outcomes Measurement Information System physical functioning 8-item short form (PROMIS-PF) at baseline and 30 days after hospital discharge. The PROMIS-PF is derived from the Dutch PROMIS physical function item bank consisting of 121 items concerning daily activities. The short form consists of 8 questions which can be scored on a 5-point Likert scale from 1 (unable to do) to 5 (performed without any difficulty). The PROMIS-PF has a high reliability and validity and is applicable to patients with different conditions [
Descriptive statistics were used to describe patient characteristics and app use. No a priori level of statistical significance was set as this pilot study was not powered to assess effect. Point estimators and confidence intervals are given to estimate the effects, and
The total SUS score was calculated using the method by Brooke [
Qualitative analysis of interview data was done following the steps of thematic analysis: compiling, disassembling, reassembling, interpreting, and concluding [
Descriptive statistics were used to describe changes in risk behaviors. Bootstrapping methods (1000 samples) were used to calculate confidence intervals for medians. Chi-square tests for linear trend were performed to examine the relation between group allocation and change scores in days of performing physical activities and muscle strengthening activities. Chi-square tests or Fisher exact tests were used to test the difference in distribution of self-reported change of all risk behaviors between allocation groups.
Total raw scores on the PROMIS-PF were translated into a t score for each participant using the PROMIS score conversion table [
Between November 2018 and February 2019, 226 patients were screened for eligibility; 86 people were eligible and signed informed consent, with 45 participants randomized to the experimental group and 41 to the control group. In the control group, one participant did not complete baseline questionnaires and was therefore excluded from further analysis, and 4 participants were excluded from further analysis because their surgeries were cancelled. In the intervention group, 2 participants withdrew informed consent due to nursing home admission and start of palliative care. Thus, there were 40 evaluable participants in the intervention group and 39 in the control group (
Design of study and flow of participants through the trial.
The median age of participants was 61.0 (interquartile range [IQR] 51.0-68.0) years, 49% (39/79) were female; 34% (27/79) of participants had 1 risk behavior at baseline, 48% (38/79) had 2 risk behaviors, and 18% (14/79) had 3 or more. Of the participants, 81% (64/79) were insufficiently physically active, and the median waiting time for surgery was 28 (IQR 16-52) days. The groups were similar at baseline in terms of demographic and clinical characteristics (
Baseline characteristics of participants.
Characteristic | Total (n=79) | Intervention (n=40) | Control (n=39) | |
Age in years, median (IQR) | 61.0 (51.0-68.0) | 59.0 (43.8-64.0) | 63.0 (53.0-70.0) | |
Female, n (%) | 39 (49) | 22 (55) | 17 (44) | |
BMI (kg/m2), median (IQR) | 25.8 (23.9-28.3) | 25.5 (23.1-27.7) | 26.5 (24.5-28.8) | |
|
||||
|
I | 9 (12) | 7 (17) | 2 (5) |
|
II | 42 (53) | 21 (53) | 21 (54) |
|
III | 24 (30) | 11 (28) | 13 (33) |
|
IV | 3 (4) | 1 (2) | 2 (5) |
|
Unknown | 1 (1) | 0 (0) | 1 (3) |
|
||||
|
Neurosurgical | 16 (20) | 9 (23) | 7 (18) |
|
Cardiothoracic | 16 (20) | 7 (18) | 9 (23) |
|
Gastrointestinal | 15 (19) | 8 (18) | 7 (18) |
|
Oral and maxillofacial | 12 (15) | 4 (10) | 8 (20) |
|
Urologic and gynecologic | 13 (17) | 8 (20) | 5 (13) |
|
Orthopedic | 3 (4) | 0 (0) | 3 (8) |
|
Vascular | 3 (4) | 3 (8) | 0 (0) |
|
Other | 1 (1) | 1 (3) | 0 (0) |
Waiting time for surgery in days, median (IQR) | 28 (16-52) | 27 (16-46) | 29 (16-65) | |
|
||||
|
Smoking | 7 (9) | 4 (10) | 3 (8) |
|
Alcohol consumption | 12 (15) | 2 (5) | 10 (26) |
|
Physical activities | 64 (81) | 35 (88) | 29 (74) |
|
Muscle strengthening activities | 57 (72) | 29 (73) | 28 (72) |
|
Unintentional weight loss | 7 (9) | 3 (8) | 4 (10) |
|
||||
|
1 | 27 (34) | 12 (30) | 15 (39) |
|
2 | 38 (48) | 23 (58) | 15 (39) |
|
3 | 12 (15) | 5 (12) | 7 (17) |
|
4 | 2 (2) | 0 (0) | 2 (5) |
PROMIS-PFc (t score), median (IQR) | 47.8 (40.8-60.1) | 47.8 (42.3-60.1) | 46.7 (40.1-60.1) |
aASA PS: American Society of Anesthesiologists physical status.
bMultiple response options.
cPROMIS-PF: Patient-Reported Outcomes Measurement Information System physical functioning 8-item short form.
Of the participants in the intervention group, 73% (29/40) activated the app with their personal code. Reasons for nonuse were unable to access the app (2/29) and not seeing added value of app use (1/29). For the other participants, reasons for nonuse are unknown. There were no apparent differences in characteristics between app users and nonusers.
The SUS was completed by 80% (32/40) of participants from the intervention group. The usability of the Be Prepared app scored 68.2 (SD 18.4). The mean SUS scores per usability aspect were: learnability 69.8 (SD 22.5), efficiency 70.3 (SD 23.5), and satisfaction 65.4 (SD 21.9).
System Usability Scale scores per statement and usability aspect (higher scores reflect higher usability).
Of the participants in the intervention group, 33% (13/40) were approached for a telephone interview. One participant declined to participate because the interview would take too much time. After 12 interviews, data saturation was reached.
Characteristics of interviewees.
Code | Gender | Age (years) | App use (number of days) | Interview pre- or postoperative | Days of hospitalization |
1 | F | 35 | 15 | Pre | 4 |
2 | M | 65 | 3 | Pre | 2 |
3 | M | 76 | 4 | Pre | 3 |
4 | M | 68 | 3 | Pre | 5 |
5 | F | 63 | 17 | Post | 4 |
6 | M | 49 | 42 | Pre | Waitlisted |
7 | F | 43 | 34 | Post | 3 |
8 | F | 59 | 16 | Post | 7 |
9 | F | 63 | 24 | Post | 7 |
10 | M | 36 | 22 | Post | 1 |
11 | F | 52 | 38 | Pre | 6 |
12 | M | 77 | 84 | Post | 7 |
The log-in procedure was difficult for many interviewees, and some needed help from family or the research team to log in.
I had some trouble opening the app, but then I went back to the instructions and it clearly stated what I had to do.
The interviewees did not experience problems when using the app.
It is self-explanatory.
The interviewees were satisfied with the various ways in which information was presented in the app and appreciated the practical advice.
Suggestions are being made in the app that I found useful. For me it was combining walking with doing groceries. I thought that was a clever one, for me as well; if you go somewhere anyway, go walking. I even did that yesterday.
The app did not match everyone’s level of functioning, and interviewees saw the need for more personalization in the app.
There are, of course, very fit people and people who have been living unhealthy for a long time. So, it will be difficult to differentiate. There could be an option menu of some sort in the app.
For the majority of the interviewees, getting a reminder to exercise more or eat more protein-rich food was enough to change their behavior while some others preferred advice from a health care professional.
An app is useful because you read it. Only knowing something, does not mean you will do it, but when you read it in an app you are reminded that you have to do it.
The app emphasized the importance of changing risk behavior before major surgery. This motivated many interviewees. The push notifications supported most interviewees in their behavior change.
Some days I forgot or I was busy, so on those days the notifications came in handy.
The urgency to prepare for the upcoming surgery seemed to be the greatest motivation for the interviewees. Interviewees with a waiting time of a few weeks until their surgery felt the need to change their risk behavior.
I feel the need, I’m going into surgery next week. It has to stop raining, because this afternoon I have to work in my garden for at least 30 minutes as a physical activity.
Interviewees who had to wait more than 4 weeks for surgery or who didn’t know their surgery date did not feel the urge to change until a few weeks before surgery.
The longer you have to use something like that, the greater the chance that you will not finish it.
According to the majority of interviewees, a good preparation should benefit postoperative recovery.
So you can manage better when you get back home.
Being well-informed was also mentioned as a key factor for good preparation.
To know what you can expect, what you can and cannot do. That is important, because of course you don’t know, you don’t know what’s wise to do.
Of the participants, 81% (64/79) were physically inactive at baseline, of whom 69% (44/64) completed the presurgery follow-up questionnaire. At baseline, the median number of days on which participants were physically active for at least 30 minutes was 3 (95% CI 2.5 to 4.0) in the intervention group (n=28) and 4 (95% CI 2.0 to 5.0) in the control group (n=16). The intervention group became active on more days of the week after the intervention period (+1.0 day [95% CI 0.0 to +2.0]) compared with the control group (0.0 days [95% CI –0.5 to +1.0],
Self-reported change in risk behavior.
Of the participants, 72% (57/79) had risk behavior regarding muscle strengthening activities, of whom 65% (37/57) completed the presurgery follow-up questionnaire. At baseline, the median number of days on which participants performed muscle strengthening activities was 0 (95% CI 0.0 to 0.0) in both the intervention (n=21) and control group (n=16). The median number of days of muscle strengthening activities increased by +2.0 days (95% CI 0.0 to +3.0) in the intervention group and did not increase in the control group (0.0 [95% CI 0.0 to +1.0],
Of the participants, 15% (12/79) had risk behavior regarding alcohol consumption, and 92% (11/12) of those completed the presurgery questionnaire (intervention group [2/11], control group [9/11]). Both participants in the intervention group (100%) reported a reduction in alcohol consumption before surgery compared with 11% (1/9) in the control group (
Of the participants 9% (7/79) were current smokers at baseline, and 43% (3/7) completed the follow-up questionnaire (intervention group [2/3] and control group [1/3]). In each group, 1 participant indicated having stopped smoking or smoked less in the run-up to surgery (
Of the participants, 9% (7/79) had unintentionally lost 3 kg of weight in the last month. Due to an error in the online questionnaire, this outcome could not be evaluated.
Of the participants, 81% (64/79) completed the questionnaire on physical functioning at baseline and 30 days after hospital discharge. At baseline, the median PROMIS-PF score was 47.8 (95% CI 43.9 to 60.1) in the intervention group (n=33) and 50.8 (95% CI 41.6 to 60.1) in the control group (n=31). Compared with baseline, both the intervention group (–6.2 [95% CI –10.6 to –2.1]) and the control group (–3.6 [95% CI –7.2 to 0.0]) had a lower level of physical functioning 30 days after discharge from hospital. Of the participants, 27% (9/40) in the intervention group and 35% (11/39) in the control group could be considered functionally recovered at 30 days after discharge from hospital. Using the complete cases, between-group analysis showed no meaningful difference in functional recovery after correction for baseline values (β=–2.4 [95% CI −5.9 to 1.1]).
The primary aim of this pilot study was to investigate the usability of the Be Prepared app prototype in patients undergoing major surgery. Both quantitative and qualitative data support the usability of the app and provide insight into adjustments that can be made to improve the app. Data from the SUS showed that patients considered the Be Prepared app to have acceptable usability (mean 68.2 [SD 18.4]) [
Our results suggest the Be Prepared app to be capable of bringing about a change in risk behaviors prior to major surgery. Participants who used the app became physically more active and increased their number of days of performing muscle strengthening activities. Furthermore, a larger proportion of app users indicated that they drank less alcohol in the run-up to their operation. No difference was found in smoking behavior between groups, but the sample size of both groups was too small to draw any conclusions. In our study, risk behavior has been assessed by means of online questionnaires. Although this applies to both groups, it may lead to socially desirable answers and perhaps an underestimation of risk behavior at baseline or an overestimation of the positive change in risk behavior. In spite of that, our findings are consistent with other research suggesting that individuals awaiting surgery welcome support to increase physical activities and reduce alcohol consumption but are less positive about smoking cessation [
Despite the positive trend in our intermediate outcome, change of risk behaviors, no preliminary effect of the Be Prepared app prototype on functional recovery 30 days after major surgery was found. The fact that we have not been able to show an effect on functional recovery after surgery can have several causes. Based on the concept that improving an individual’s functional capacity before major surgery helps them withstand the forthcoming stressor of major surgery, it makes sense to target those at high risk of postoperative complications and functional decline after surgery [
Furthermore, the Be Prepared app was a home-based unsupervised intervention. Evidence suggests that supervised prehabilitation may have a greater effect and higher adherence than unsupervised prehabilitation programs [
This study is one of the first studies to explore the usability and preliminary effectiveness of an app for multimodal prehabilitation in patients undergoing major surgery. Our objective was to determine whether the Be Prepared app was usable and beneficial for a wide range of surgical patients. Therefore, our study sample was deliberately more heterogeneous than samples in other prehabilitation trials [
This study suffered from a large amount of missing data, especially at the presurgery follow-up. This can be explained, in part, by the timing of this measurement. Participants were invited to complete the presurgery follow-up questionnaire 3 days preoperatively, which might not have been an ideal time as patients have their minds on other things right before their surgery [
In this study, 73% of the possible app users activated the app. The difficult log-in procedure and differences in patient (digital) health literacy may have contributed to the substantial number of nonusers in the intervention group [
Overall, the results of this pilot RCT demonstrate that the Be Prepared app prototype for patients undergoing major surgery has potential in terms of usability and changing risk behavior prior to major surgery. The app seems to fit the needs of patients preparing for major surgery. Several points of improvement for the app and study procedures have been identified, which supports the further development of the Be Prepared app and adjustment of study procedures before evaluating its effectiveness in a large multicenter RCT. These include adaptation of the timing of follow-up measurements, additional support by the physiotherapist during the preoperative period, and expanding the preoperative content of the Be Prepared app.
Overview of app content.
Topic list for semistructured interviews.
CONSORT-eHEALTH checklist (V 1.6.1).
Centre for eHealth Research
Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth
interquartile range
mobile health
Patient-Reported Outcomes Measurement Information System physical functioning 8-item short form
randomized controlled trial
System Usability Scale
This work was supported by the Foundation Innovative Alliance—Regional Attention and Action for Knowledge circulation.
None declared.