A prospective, pragmatic, 2-arm, parallel, type 1 hybrid effectiveness-implementation trial [9] that assesses patients and professionals’ acceptability of a 3-month mHealth-enabled IC intervention as compared with that of usual care was conducted. The study was conducted from July 2018 to August 2019 in Lleida, which is a large rural area of more than 4300 km², including 2 tertiary hospitals, University Hospital Arnau de Vilanova and University Hospital Santa Maria, and a network of 23 primary care centers spread across the whole territory, providing services to 400,000 citizens.

The intervention was deployed for 2 different use cases (UCs): (1) home-dwelling patients, with chronic conditions and a history of visits to the emergency room (ER) leading to hospitalizations (UC1); and (2) home-dwelling patients, with chronic conditions, undergoing a major elective hip or knee arthroplasty surgery (UC2). The specific eligibility criteria included the following: age more than 55 years, having a hospital admission because of a respiratory or cardiovascular event (UC1), having a programmed major elective hip or knee arthroplasty surgery (UC2), not having dementia or cognitive impairment (Global Deterioration Scale<5 [10]), LACE index for readmission score>7 (UC1) [11], American Society of Anesthesiologists Physical Status Classification System II or III (UC2) [12], being assigned to a primary care center in the region, living at home and being discharged back to the community, and passing a basic technological test assessing home connectivity and patients and/or care givers’ competence with the use of technology (Multimedia Appendix 1 [13,14]).

Patients were recruited either during an unanticipated admission to the hospital through the ER (UC1) or at the time of surgery (UC2). All patients were identified based on data from electronic medical records (EMRs) and contacted by a case manager. Regardless of the UC, once a patient was recruited for the intervention arm, the search for a similar control began. All patients and their caregivers, regardless of the UC and study arm, received a face-to-face explanation about the study.

Patients in the intervention arm experienced an mHealth-enabled IC model, including (1) a preliminary assessment of the patient’s health status using several questionnaires, tests, and indices specific to their main chronic diseases and social needs done before hospital discharge for UC1 patients and at the time of scheduled surgery for UC2 patients; (2) access to a self-management app with status and performance reports, a virtual coach with customizable automated feedback, and full communication with the care team and guidance on its day-to-day use, taking into account that the app could be managed directly by the patients or indirectly by the informal caregivers or relatives; (3) a Fitbit Flex 2 (Fitbit) digital activity tracker [15] and additional sensors deemed necessary by the care team [16]: digital pulse-oximeter, digital scale, and digital blood pressure monitor, all of them fully integrated into the self-management app; (4) a patient’s profile in the SACM web-based platform, which would be accessible by all the involved professionals (hospital, primary, and social care) and used to coordinate and communicate with professionals in the different settings, control the patient’s evolution, and contact the patient if needed; and (5) assignment of a case manager in charge of supervising the whole process and being the main patient contact point. Additional details on the IC model and the required implementation efforts can be found in Multimedia Appendix 1. It must be noted that, as part of the CONNECARE project, the supporting technology used by patients and professionals was being developed and fine-tuned throughout the study. Usual care arm patients were managed from primary care.

Patients’ characteristics were collected at recruitment, including age, sex, main chronic diseases, Charlson Comorbidity Index [17], Barthel Index for Activities of Daily Living [18], and the Pfeiffer Mental Status Questionnaire [19]. The main patient outcomes were collected after 3 months and included patient’s perception of person-centeredness, assessed by the Person-Centered Coordinated Care Experience Questionnaire (P3CEQ) [20]; patient’s perception of continuity of care, assessed by questions G1 to G5 of the Nijmegen Continuity Questionnaire (NCQ) [21]; satisfaction with the IC platform in IC arm patients and staff, assessed by the Net Promoter Score (NPS) [22] and the System Usability Scale (SUS) [23]; and the actual use of the different elements of the IC platform by patients. The NPS was based on the question “How likely is it that you would recommend our system CONNECARE to a family member or friend?” to be answered in a 0 (not at all likely) to 10 (extremely likely) scale. Individuals scoring 9 or 10 were considered as promoters, individuals scoring 7 or 8 as passives, and individuals scoring 0 to 6 as detractors. The final NPS score was obtained by subtracting the proportion of detractors from the proportion of promoters, and it could range from −100% to +100% (a positive score is considered good, +50% is considered excellent, and anything more than +70% is exceptional) [24].

Participants’ baseline characteristics were described by the number (percentage), mean (SD), or median (P25-P75), as appropriate. Comparisons between IC and usual care patients’ baseline characteristics were performed using chi-square test, t test, or Kruskal-Wallis test, as appropriate. Comparisons between IC and usual care patients’ person-centeredness and continuity of care were performed using chi-square test or t test, as appropriate, excluding patients answering “Don’t know” or “No answer.” Satisfaction with the IC platform in IC patients and staff were described using mean (SD) or median (P25-P75), as appropriate. Finally, the actual use by patients of the different elements of the IC platform was described by reporting the proportion of times reported over times prescribed, using median (P25-P75). Data analyses were conducted using Stata, version 12.1 (StataCorp). The threshold for significance was set at .05.

The Consolidated Framework for Implementation Research (CFIR) [13] was used to assess implementation aspects. A detailed description of the implementation strategies and framework can be found in Multimedia Appendix 1.

This study was approved by the Ethics Committee of Hospital Arnau de Vilanova (CEIC-1685), and all participants provided written informed consent. All collected data were handled and stored in accordance with current National and International legislation.

The assessment of the patients and professionals’ acceptability of an mHealth-enabled IC program targeting home-dwelling CCP patients with a history of hospitalizations (UC1) or undergoing a major elective hip or knee arthroplasty surgery (UC2) showed 2 different perceptions. Although patients and/or informal caregivers or relatives reported a very high acceptability of the IC program and its supporting technology, professionals rated it as moderately poor. Patients reported very high perceptions of patient-centeredness, continuity of care, satisfaction, and usability of the IC platform, and matching with these positive perceptions, actual use of the different features of the IC platform was high. Although patients positively qualified their user experience [25], professionals felt the burden of a system under constant development, which in turn limited their experience and translated to moderately poor satisfaction with the IC platform.

This study has several strengths: (1) from day 1, an effort was made to involve all actors from different organizations who would participate in a large-scale deployment of the mHealth-supported IC program, which is key as the lack of cooperation between organizations and professionals is a well-known barrier for the implementation of IC [4]; (2) the involvement of informal carers as actors being important in facilitating the use of the patient’s app in older patients; (3) the prescription and monitoring of patients’ physical activity, as mobility impairment is frequent in people older than 65 years [26]; and (4) the geography of the implementation area, a large rural region of more than 4300 km², which could benefit the most from community-based integrated care initiatives that precluded unnecessary visits to primary care centers or hospitals. Similarly, there were several limitations: (1) the IC platform was in a constant process of refinement and addition of new functionalities; thus, the user experience was richer by the end of the implementation study compared with the very beginning; and (2) having a single entry point to the IC program, which was the hospital either after an ER admission (UC1) or at the time of surgery (UC2), as it is important that system-wide cross-organizational care pathways consider multiple entry points [27]. However, primary care centers are currently being considered as a potential additional entry point if the IC system is further implemented in the region.

The use of mHealth apps for patients with chronic conditions has been explored in the last decade, showing the potential for appropriate security level, effective monitoring, self-management, and communication [28]. However, a 2016 review of Apps for Heart Failure symptom monitoring and self-care reported that a minority of the available apps had the required quality, content, or functionality [29]. Our IC platform aimed to go beyond a patient’s app and established a comprehensive IC model, including a patient’s app, a portfolio of different sensors linked with the app, and a professional’s web-based platform for monitoring, communication with the patient, and collaboration among professionals in different health settings. The first indicator of the success of an mHealth intervention is the rate of dropouts. In our study, only 13% of patients in the IC arm decided to abandon the program. For instance, Bentley et al [30] reported that half of the participants in an mHealth-based self-management intervention for chronic obstructive pulmonary disease (COPD) withdrew from the study. The scores for acceptability (UC1: NPS +67%; UC2: NPS +45%) and usability (UC1: mean SUS score 79, SD 14; UC2: mean SUS score 68, SD 24) were outstanding. These results are better than most results obtained in existing mHealth tools targeting chronic patients, for example, mHealth tools aiming to improve quality of life in breast cancer survivors such as BENECA (NPS +7%) [31] or Oncokompas (NPS −36%) [32], an Intelligent Virtual Assistant for promoting behavior change and self-care in older people with type 2 diabetes (mean SUS 74, SD 13) [33], or a gait-monitoring mobile phone app for older users (mean SUS 60, SD 11) [34]. Similarly, interventions targeting surgical patients by means of education and communication apps have obtained positive results in terms of acceptability and usability [35,36], comparable with results obtained with automated phone messaging platforms [37,38]. Finally, preliminary results of variants of the CONNECARE IC model implemented in other European settings have been positive. For instance, the CONNECARE self-management app was rated positively by older patients with cancer who were offered remote home monitoring after surgery (mean SUS 74, SD 19; NPS +29%) [39].

The key factors for the success of our comprehensive IC model from the patients’ perspective have been (1) including a comprehensive set of features with the patient’s app acting as a hub of services including the integration of monitoring devices, in line with a 2016 review of Apps for Heart Failure symptom monitoring and self-care reporting that a minority of the available apps had the required content or functionality [29]; (2) the involvement of patients since early phases of development, as proposed by Lundell et al [40] in a recent qualitative analysis of the use of home telemonitoring in patients with COPD; (3) the flexibility of potential end users, as the app could be managed directly by the patients (most UC2 patients) or by a relative or informal carer (most UC1 patients); (4) enabled bidirectional communication with the care team, potentially avoiding unnecessary visits to primary care centers or hospital; (5) appropriate feedback on the daily monitoring and patients’ achieved goals, including personalized motivational advice; (6) push-up notifications to remind key events, tasks, or goals; and (7) ease of use and quality-of-life features, such as being translated to the different official languages in the region (Catalan and Spanish) or having several display settings including font size, as difficulty in using the technology is a common reason for withdrawal [30].

According to the Quadruple Aim, the improvement of the work life of clinicians and staff is a key factor for the adoption of new health programs [8]. In this study, the scores for professionals’ acceptability were low (UC1: NPS −25%; UC2: NPS −35%), whereas scores for usability were moderately high (UC1: mean SUS score 63, SD 20; UC2: mean SUS score 62, SD 19). However, these ratings were directly related to the temporal constraints of the study setting, as professionals were required to use a system in a dynamic development and implementation process rather than a fully developed one. However, having the opportunity to directly participate in the development of the IC model and platform allowed professionals to feel engaged and propose changes and new features to be developed, which ultimately resulted in great engagement (no professionals dropped out of the implementation study). Understanding the factors influencing professionals’ adoption of eHealth is complex [41], but the ability to provide quality care is key [42]. On the one hand, the potential of the IC platform to provide quality care was the key to professionals’ engagement; being able to monitor key aspects of chronic diseases or monitoring pain after surgery, enabling communication between carers in different care settings; having the option to prescribe and monitor physical activity; or case managers having access to a geographical representation of patients in a map, with the possibility of selecting patients based on predefined characteristics or generating optimized routes for home visits, showed professionals the game-changing features of the platform. On the other hand, a system under constant development, not achieving a full integration with legacy EMR systems and the coexistence of 2 management systems (usual care and IC) at the same time (which implied some duplicity of tasks) were the main barriers to adoption.

Although usability and acceptability are key for the adoption of mHealth-enabled IC, large-scale adoption requires cost-effectiveness and an adequate reimbursement and payment model. On the one hand, regarding cost-effectiveness, the implementation of our IC model reduced unplanned contacts with the health system, reduced health costs, and was cost-effective, as reported elsewhere. On the other hand, designing of an imbursement and payment model capable of accommodating the costs of new roles and required technologies, while fully benefiting from the savings in terms of reductions in the use of health and social care resources, can be challenging, especially when the model involves different organizations and providers. Therefore, a firm positioning of the involved health authorities and governing bodies is required to fully fulfill the ambition of our mHealth-enabled IC model. The use of the CFIR framework highlighted the key barriers and facilitators for large-scale adoption (Multimedia Appendix 1).

The assessment of the patients and professionals’ acceptability of an mHealth-enabled IC program showed outstanding results from the patients’ perspective. However, the web-based professionals’ platform needs to be fully matured and fully integrated into legacy systems before moving forward toward large-scale deployment. This paper, thus, provides useful lessons learned through the development and assessment process and may be of use for organizations willing to develop or implement mHealth-enabled IC for older adults.