Evaluating the Implementation of a Mobile Phone–Based Telemonitoring Program: Longitudinal Study Guided by the Consolidated Framework for Implementation Research

Background: Telemonitoring has shown promise for alleviating the burden of heart failure on individuals and health systems. However, real-world implementation of sustained programs is rare. Objective: The objective of this study was to evaluate the implementation of a mobile phone–based telemonitoring program, which has been implemented as part of standard care in a specialty heart function clinic by answering two research questions: (1) To what extent was the telemonitoring program successfully implemented? (2) What were the barriers and facilitators to implementing the telemonitoring program? Methods: We conducted a longitudinal single case study. The implementation success was evaluated using the following four implementation outcomes: adoption, penetration, feasibility, and fidelity. Semistructured interviews based on the Consolidated Framework for Implementation Research (CFIR) were conducted at 0, 4, and 12 months with 12 program staff members to identify the barriers and facilitators of the implementation. Results: One year after the implementation, 98 patients and 8 clinicians were enrolled in the program. Despite minor technical issues, the intervention was used as intended. We obtained qualitative data from clinicians (n=8) and implementation staff members (n=4) for 24 CFIR constructs. A total of 12 constructs were facilitators clustered in the CFIR domains of inner setting (culture, tension for change, compatibility, relative priority, learning climate, leadership engagement, and available resources), characteristics of individuals (knowledge and beliefs about the intervention and self-efficacy), and process (engaging and reflecting and evaluating). In addition, we identified other notable facilitators from the characteristics of the intervention domain (relative advantage and adaptability) and the outer setting (patient needs and resources). Four constructs were perceived as minor barriers— the complexity of the intervention, cost, inadequate communication among high-level stakeholders, and the absence of a formal implementation plan. The remaining CFIR constructs had a neutral impact on the overall implementation. Conclusions: This is the first comprehensive evaluation of the implementation of a mobile phone–based telemonitoring program. Although the acceptability of the telemonitoring system was high, the strongest facilitators to the implementation success were JMIR Mhealth Uhealth 2018 | vol. 6 | iss. 7 | e10768 | p. 1 http://mhealth.jmir.org/2018/7/e10768/ (page number not for citation purposes) Ware et al JMIR MHEALTH AND UHEALTH


Introduction
Heart failure telemonitoring systems are developed with the objective of reducing mortality and hospitalizations and improving patients' quality of life [1]. However, despite patient acceptance [2][3][4], the diffusion of these services is lagging [5]. Meta-analyses tend to support claims of the positive impact on heart failure outcomes [1,[6][7][8][9]; however, important inconsistencies in the evidence persist [10]. Inconsistencies are believed to result from the heterogeneity of the intervention and patient populations used in primary studies and the lack of consistency with which interventions are used (ie, fidelity) in clinical trials [10]. Much remains to be learned on how to best implement telemonitoring interventions such that true effectiveness can be measured and their benefits be fully realized.
Systematic reviews present extensive lists of various barriers and facilitators to telemonitoring implementation [11][12][13]. External barriers include the lack of a clear business model in single-payer health systems [14] and the lack of acceptable reimbursement methods for clinician users [15]. Furthermore, clinician adoption is influenced by the quality and usability of the technology, compatibility of the intervention with existing work processes, and intrinsic clinician motivation to adopt telemonitoring as part of their practice [11][12][13]. In fact, a recent review found that the challenge presented by new and often ill-defined clinician roles within changing workflows was a key factor in leading to the failure of eHealth interventions [16]. One multisite qualitative study similarly highlights the importance of contextual factors for clinician adoption, including the degree of support clinicians receive in their new roles and alignment with organization objectives [17]. However, most telemonitoring implementation studies have been conducted retrospectively, which does not allow for a robust analysis of how these barriers and facilitators exert their influence over the entire implementation period. In addition, few studies report on quantitative outcomes to justify judgments of the implementation success or failure.
A mobile phone-based telemonitoring program called Medly was implemented as part of the standard of care at a specialty heart function (HF) clinic in Toronto, Canada. This program features a system that has previously demonstrated improvements in clinical outcomes, patient self-care, and quality of life [18]. The objective of this study was to evaluate the implementation of the Medly program by answering two research questions: (1) To what extent was the Medly program successfully implemented? (2) What were the barriers and facilitators to implementing the Medly program?

Study Design
This study used a longitudinal single case study design. The case was defined as the telemonitoring intervention and the implementation site as described below for one year following the enrollment of the first patient (August 23, 2016). The units of analysis for this evaluation were the Ted Rogers Centre of Excellence for Heart Function (HF clinic) and program staff with data for determining the implementation success being collected through a document review. In addition, barriers and facilitators were assessed using semistructured interviews with program staff guided by the Consolidated Framework for Implementation Research (CFIR) [19]. Notably, the patient perspective, including reasons for use, adherence, and withdrawal will feature in an upcoming publication. In this study, data collection was conducted within the context of a larger quality improvement program evaluation [20], which has been approved by the University Health Network (UHN) Research Ethics Board .

Intervention
The Medly program consists of two components: (1) the technology (hardware and software) and (2) the human-dependent interactions and services.

Medly Technology
The patient-facing technology includes the Medly mobile phone app, which works by allowing patients with heart failure to record the following three parameters: (1) weight; (2) blood pressure; and (3) symptoms. Based on these data inputs by patients at home, the Medly app, which contains a rule-based algorithm customized according to patient-specific target ranges, displays self-care messages and generates alerts that are automatically relayed to a clinician when signs of clinically significant health status deterioration occur. Patients were instructed to record the three parameters daily, and they would receive an automated phone call if they had not done so before 10 am; this was intended to assist with compliance. For the launch of the program, each patient was provided with all the required equipment, which includes a mobile phone with a data plan, a Bluetooth-enabled weight scale, and a Bluetooth-enabled blood pressure monitor.
The clinician-facing technology seeks to support the management of the patient alerts; this is primarily conducted through a Web-based interface (ie, the dashboard) containing a list of patient alerts, graphs showing patient-level trends of the three clinical parameters monitored, and heart failure-specific lab results. In addition, clinicians have the option of receiving alerts through automated emails, which contain the latest weight, blood pressure, and symptoms. Furthermore, the email contains the patients' current medication list, heart failure-related laboratory results, and contact information.

Medly Services
Enrollment into the program was based on clinical judgment. After discussing the program with patients, a clinician, ie, a cardiologist, a nurse practitioner (NP), or a resident, fills out a form to indicate the desired target ranges needed to customize the algorithm. Then, a telehealth analyst (THA) provides patients with the Medly technology and training on how to use it. When alerts are triggered, they are viewable by patients' treating cardiologist and NPs. The clinicians might act independently or communicate among themselves by email or in person to determine the best course of action. If required, a clinician will follow up with patients either by phone or email, documenting all actions and decisions in the hospital electronic medical record (EMR). Furthermore, patients and clinicians are instructed to contact the THA to receive the technical support, if required.

Implementation Site
The HF clinic, part of UHN, is a high-volume specialty care clinic for patients with heart failure in Toronto. The intervention was developed by UHN's Centre for Global eHealth Innovation (eHI) in close collaboration with clinicians from the HF clinic. The THA is employed by the UHN Telehealth Department with 25% of their time dedicated to supporting the Medly program. The HF clinic, UHN telehealth services, and eHI are physically located in the same building.

Implementation Strategy
Preparations for the program launch included the development of training materials for patients (user manual and training checklist). In addition, clinician users were provided with a user guide and a training session lasting approximately 1 hour. Moreover, members of the eHI team followed a service design methodology, consisting of mapping clinic workflows and producing a service blueprint for the Medly program, which sought to minimize the disruption to existing HF clinic processes.

Implementation Outcomes
We selected 4 implementation outcomes from Proctor et al's Implementation Outcomes Framework as measures of the implementation success [21]. In addition, data on the outcomes, defined below, were collected after 4 and 12 months of the launch through a document review process and semistructured interviews.
• Adoption: The number of clinicians deciding to monitor patients using the Medly system. • Feasibility: The extent to which the Medly program can be successfully used by patients.
• Fidelity: The extent to which the Medly program is being used as initially intended.

Barriers and Facilitators to Implementation
Semistructured interviews were developed based on the constructs of the CFIR, which provides a pragmatic organization of theory-informed constructs known to impact the implementation success across the following 5 domains: (1) intervention characteristics; (2) outer setting (eg, patient needs and resources, external policy and incentives, etc); (3) inner setting (networks and communication, implementation climate, readiness for implementation, etc); (4) characteristics of individuals; and (5) process [18]. Further interview probes were developed to explain the quantitative implementation outcome indicators. Moreover, interviews were conducted prior to the program launch, and again after 4 and 12 months, each session lasting 30-60 minutes. Of note, all adopting clinicians and eHI Medly program staff were invited for participation. In addition, clinicians who had not adopted the system by 12 months were also invited to participate. All interviews were recorded and transcribed for later qualitative analysis.

Data Analysis
The interview transcripts were analyzed by two independent investigators (PW and KG) using the Framework Method [22]; this involved a largely deductive thematic analysis using a codebook based on the CFIR constructs [19]. PW and KG independently coded the transcripts and then met to discuss contradictory codes and passages. The management of source documents and coding was done with the help of NVivo version 11 (QSR International, Doncaster, Victoria, Australia). To determine the degree to which the barriers and facilitators impacted the implementation, valence ratings were attributed by PW and KG to each construct according to the criteria outlined by Damschroder et al [23]. Qualitative findings and valence ratings were validated during a meeting with key members of the clinician and eHI program staff (n=6).

Study Participants
In this study, 8 clinicians participated. One cardiologist, who was the only clinician who had not adopted the technology before the end of the study period, did not respond to requests to be interviewed prior to completion of the manuscript. Table  1 shows the interview schedule and the role of each participant. Table 2 presents the results of implementation outcomes, which are discussed below.

Adoption and Penetration
The program was launched with 3 clinician users (1 cardiologist and 2 NPs). By the 12-month time point, 5 additional cardiologists were monitoring patients using Medly, representing an increase in the penetration of the Medly program in the HF clinic from 38% to 89%, a diffusion pattern that is explained in the interviews.  All participants described how the Medly program was initially only open to the 3 clinicians who were most actively involved in its development. By the 11 th month of the program, the clinical lead of the HF clinic decided to open its availability as a resource to other cardiologists. Many of the later adopting clinicians had always expected to be involved and were simply waiting to be invited. Although all cardiologists who adopted Medly after the initial launch had a similar perspective, some were concerned about the time it would add to their workday. They ultimately decided to participate because they felt a responsibility to share in the workload being taken on by their colleagues. Another important factor swaying their decision was a concern that they could be excluded from their patients' circle of care.

Feasibility
By 12 months, 98 patients were enrolled in the Medly program; this was a lower number than initially anticipated and is partially explained by a low initial penetration within the clinic. In addition, throughout the implementation, clinicians began to realize that patients benefited differently depending on their disease severity, ability to use the technology, ability to adhere to taking measures, and receptivity to self-care messages, which led to clinicians becoming more selective of which patients were enrolled.
I also think and I respect that they're doing their due diligence…about actually finding the right patients. The clinicians need to make sure they're only targeting patients that would benefit and not someone that they'll just take off after a week…So of course, it's a little difficult on their side. They have to do a lot, you have to think a lot more about it. But I feel like they're being more mindful about it. [eHealth 4] Feasibility is also demonstrated by the relatively low number of patients who chose to stop using the system (n=5). Additionally, 5 patients passed away during the evaluation period. These deaths were determined to be unrelated to the Medly program and were explained by clinicians as being reflective of the severe disease state of the patient population.

Fidelity
Overall, the intervention is generally being used as intended with clinicians reviewing all alerts generated by Medly, following up with patients when necessary, and documenting all actions in EMR. The Medly program was launched with the idea that both the system and the service would continue to improve and evolve over time. Throughout the implementation, the THA received 159 calls from patients and 36 calls from clinicians related to problems with the system (eg, receiving adherence calls when they had taken their readings, usability issues, and general connectivity problems between the phone and the peripheral Bluetooth equipment), all representing examples of when the system was not working as intended. However, these, as well as the 72 documented feature requests by patients and clinicians, are evidence of a properly functioning quality improvement mechanism. Table 3 describes the barriers and facilitators of the implementation along with a valence rating signifying the degree to which it had an impact on the implementation of the Medly program. Unless otherwise discussed, valences were relatively consistent throughout the entire 12-month implementation period.

Evidence Strength and Quality
Clinician participants acknowledged ambiguity in the literature of the impact of telemonitoring for heart failure. However, this did not impact the implementation for reasons identified in the construct knowledge and beliefs about the intervention.

Relative Advantage
The Medly program was perceived as having a relative advantage over alternative telemonitoring options. Unlike many telemonitoring systems, Medly measures multiple clinical parameters and offers algorithm-based self-care instructions with structured telephone support when necessary.

Complexity
Several statements revealed the complex nature of the Medly program, giving this construct a negative valence in the early stages of the implementation. Examples of complexity include (1) need for extensive documentation, (2) relying on engagement from patients, (3) challenges in identifying program candidates, (4) communicating patient information between Medly clinicians, and (5)   Perception of the quality and validity of the evidence supporting the use of telemonitoring for heart failure.
Evidence strength and quality +2 Perception of the advantage of implementing the Medly program versus an alternative solution.
Relative advantage

+2
The degree to which the Medly program can be adapted to meet the needs of the HF clinic c .
Adaptability -1 Perceived complexity of the Medly program as reflected by the degree of disruptiveness to existing workflows and number of steps involved in using the intervention as intended.
Complexity (reverse rated) 0 Perceived quality of the Medly program (technology and service components) and how well these components are bundled and work together.
Design quality and packaging -1 Financial and opportunity costs of implementing the Medly program. Cost

+2
The degree to which heart failure patients' needs are known and prioritized by the HF clinic (ie, patient-centeredness). The degree to which the HF clinic and UHN have a climate that provides time and space for reflective thinking and that allows team members to feel essential, valued, and safe to try new methods. Knowledge and beliefs about the intervention +2 Clinicians' and telehealth analyst's belief in their own capabilities to execute their role within the Medly program and achieve implementation goals.

-1
The degree to which a plan for implementing the Medly program was developed in advance and the quality of that plan.

+1
Individuals in the HF clinic who have a formal or informal influence on the attitudes and beliefs of their colleagues with respect to implementing the Medly program. Executing +2 Feedback about the progress and quality of the implementation along with regular debriefing about progress and experience.
Reflecting and evaluating a The constructs and operational definitions are adapted from the Consolidated Framework for Implementation Research [19]. b Definitions of valence ratings are adapted from Damschroder et al's study [23]: -2, the construct had a strong negative influence on the implementation effort; -1, the construct had a minor negative influence on the implementation effort; 0, the construct had a neutral influence on the implementation effort. Alternatively, different aspects of the construct had a positive influence, while others had negative influence: +1, the construct had a minor positive influence on the implementation effort; +2, the construct is a strong positive influence on the implementation effort. c HF clinic: Ted Rogers Centre of Excellence for Heart Function. d UHN: University Health Network.

[Another challenge] is this idea of trying to guess what your range of variation is going to be for each patient [when setting parameter thresholds for the algorithm]. I think there are probably more accurate mathematical models to try to come up with that rather than me sort of flipping a coin and deciding, "Okay, it's going to be 3 pounds + or -, or 2.5 pounds." [Clinician 6]
As these challenges were identified and solutions to mitigate their impact implemented, the complexity of the program was no longer viewed as negatively impacting the program's continued growth.

Design Quality and Packaging
General satisfaction with the Medly program and its design were a pervasive theme throughout the interviews. However, some design-related factors were perceived as barriers to continued growth of the program and sustainability even if they did not significantly impact clinician adoption. For example, clinicians expressed a desire for a more seamless integration with existing technologies and workflows by (1) integrating the dashboard with the hospital EMR to facilitate documentation, (2) making the dashboard available on mobile devices, (3) providing more patient details in the dashboard, and (4) allowing for multiple patient-clinician communication modalities (eg, short message service text messaging).

Cost
Most of the costs associated with implementing the Medly program were related to the equipment, which were perceived as high and unsustainable. However, plans are being made to reduce costs by having patients use their own devices. The other major cost involved the THA's time, which was higher than initially anticipated.
Currently the system is CAD $2200 a pop for the phone, the data plan, the weight scale and the cuff so obviously, we don't yet have a mechanism to pay for that and thankfully through [ This construct received a ranking of -1 because although the equipment cost was not a large initial barrier due to philanthropic funding, it was a barrier to program sustainability that would need to be addressed. Furthermore, although the additional THA time requirements did not significantly impact the Medly implementation, it did have important opportunity costs related to that individual's ability to work on other projects. A formal economic evaluation of the impact of the Medly program from the perspectives of patients, HF clinic, and public payer will be published subsequently.

Patient Needs and Resources
This construct represents the idea that the implementation site comprehends and seeks to address patients' needs. Numerous examples of the HF clinic's patient-centered approach to care made this a strong facilitator.
It's a very supportive environment for patients and families, and that is something that I repeatedly hear in clinic, especially patients who have been in the clinic for a long period of time, how thankful and how grateful they are for the care that they receive. You hear this a lot, people just…they don't want to go somewhere else. [Clinician 2]

External Policy and Incentives
This construct had a neutral impact because although the program was perceived as compatible with government policies seeking to encourage more comprehensive chronic disease care outside of the hospital, factors like regulation, funding, and clinician reimbursement were flagged as crucial barriers to the program's sustainability.

Networks and Communication
Three networks of communication were identified as having an influence on the implementation success. First, communication within the HF clinic was described as generally good and had a positive influence. Second, communication between clinicians and the THA involved in the day-to-day operations of the Medly program was also perceived as positive for the implementation. Finally, communication among high-level stakeholders was described as an early barrier to the implementation, particularly as it is related to decision making about the program and having clear channels to operationalize those decisions. This barrier was identified and notable improvements were made by the 4-month time point.

Tension for Change
Clinician participants were proud of the quality of care they offered to patients. However, a perceived gap existed between the current and ideal state. Coupled with a busy clinic with limited staff and space resources, this created a tension for change.
[Clinic capacity] is an ongoing concern for me and I think we're at a bit of a crux where we couldn't handle somebody not coming to work and we can't handle any more volume. [Clinician 1]

Compatibility
The Medly program was perceived as compatible with the values of the organization and complimentary to existing services offered in the HF clinic. Early apprehension about increased clinician workloads speaks of the incompatibility of the telemonitoring program with existing clinic workflows. However, by the end of the first year, evidence exists that a new normal has been created such that this initial incompatibility did not significantly impact the overall implementation success.

Relative Priority
The implementation of the Medly program was perceived as having a high priority by all participants.

Learning Climate
Participants describe a work environment that values ongoing quality improvement. They feel the climate offers a safe space for learning and trying new things, making this construct a strong positive influence on implementation success.

Available resources
Important to the success of this implementation was the availability of financial and human resources. No new clinicians were employed; rather, existing NPs were expected to perform Medly-related tasks within their salaried hours. Although this was possible, the added NP workload should not be underestimated.

[I am] not complaining about [responding to alerts] because that is part of why I'm hired. It's just that there needs to be, in order for Medly to work, you have to have a clinician who is devoting time to do all of that, to answer alerts, to document, and to see patients that are unwell in clinic. [Clinician 3]
The THA was an additional resource that was hired to support this program. Flexibility with respect to this resource, both in terms of quantity of time and time during the day, was an important facilitator that might not be realistic in other sites.

It makes it a lot easier when they call me down to the clinic or they have a patient come to the clinic and I am available and I can just run down and be there in five minutes. My worry would be if it was a different site and they need that kind of instant support. It may be difficult getting someone there. [eHealth 4]
Funding for the equipment came from philanthropic donations, thereby mitigating the potential barrier of nonavailability of funds common in many real-world implementations.

The cost, although improved, is still an issue, because right now Medly is being funded [by philanthropy] and obviously, we're not here to fund it for the province. [Clinician 1]
Finally, insufficient physical space is a challenge for the HF clinic and was likely an indirect barrier as clinic rooms are required for patient onboarding.

Access to Knowledge and Information
The availability of the THA to provide on-call and personalized information about how to use the intervention was an important facilitator. However, although clinicians perceive the training they received to be sufficient, some felt that more comprehensive training around understanding the algorithm was required. In addition, the novelty of this program meant that no clear medical-legal guidelines existed on exactly what information needed to be documented. Therefore, this is a challenge that clinician users needed to navigate on their own.

Knowledge and Beliefs About the Intervention
Clinician knowledge and positive beliefs about the Medly program likely helped overcome the potentially negative influence of the equivocal scientific evidence.

Self-Efficacy
Despite initial apprehensions about increased workload, the clinician and eHI teams were confident that they would be successful in implementing the program.
I don't think that there's any doubt that we will be implementing it I think as intended. While I may be apprehensive, it doesn't mean that I don't think that we still actually need to try and actually see.

Process Domain
Planning Despite user training being planned and the initial service design work leading up to the program launch, an overarching implementation plan was never explicitly developed at the outset; this was perceived as having a negative impact during the initial months of the implementation. However, after realizing this deficiency, ongoing plans were formalized by the team; this was perceived as having a positive influence on the current and future program.

Engaging Champions and Opinion Leaders
The presence of a clinician champion or opinion leader was an important facilitator for both the development and implementation of the Medly program. Importantly, the fact that this champion and opinion leader set a positive example appears to have had more of an impact on the implementation success than this individual's role as a formal leader.

Reflecting and Evaluating
Embedded within the Medly program was a mechanism to facilitate the ongoing quality improvement. All participants spoke positively of the benefits of being able to quickly identify and evaluate problems and implement solutions when possible.
[We meet] every two weeks to discuss the recruitment in the program, how things are going, any issues or problems that people have faced. And then we discuss those issues, identify solutions and come up with a plan for how we want to address them. We also talk about achievements that have happened, so recruitment milestones, things like that. [eHealth 2]

Principal Findings
This longitudinal implementation evaluation found that the Medly program had been successfully implemented, as demonstrated by the steady growth in patient enrollment and clinician adoption and the fidelity with which the intervention was being used in clinical practice. Costs were relatively high because of the decision to initially supply patients with all the telemonitoring equipment. That said, these costs were not estimated to have significantly impacted the implementation and are expected to dramatically decrease, as the program shifts to a bring your own device (BYOD) model. This study also identified 24 CFIR constructs that explain these measures of implementation success. Fifteen constructs were facilitators predominately clustered in the domains of inner setting and characteristics of individuals. Four CFIR constructs were minor barriers in the earlier phases of implementation-complexity, cost, networks and communication, and planning. Five additional constructs had a mixed valence and therefore were determined to have a neutral impact on the implementation.

Comparison with Prior Work
The implementation barriers and facilitators identified in this study are very much in line with results from other telemonitoring implementation studies. Systematic reviews have concluded the importance of characteristics of the technology, people involved, extraorganizational environment, and implementation setting [11][12][13]. In addition, the literature suggests that having undefined roles in the context of new workflows is a common barrier to eHealth implementations [16]. This study provides concrete examples of these barriers as they relate to the CFIR constructs of complexity and compatibility. For example, in the absence of clear guidelines for documentation, identifying ideal patient candidates and setting parameter thresholds needed for the algorithm, clinicians are forced to develop experiential knowledge to be able to perform these tasks. The development of this tacit knowledge can often only happen over time and might be challenging in a fast-paced clinic environment. The learning climate in this study was perceived as being an important facilitator, which likely helped overcome this challenge. However, clear guidelines for clinician staff roles will likely be required to ensure implementation success where learning climates might not be as favorable.
In addition to providing a framework that allows for the easier transferability of study results, using CFIR-guided methods allowed this research to make two additional contributions to the field of implementation science. To the best of our knowledge, this is the first study to demonstrate the feasibility of using the CFIR for evaluating complex telehealth interventions [24]. However, the CFIR lacks granularity for identifying factors that might be unique to health information systems. Researchers wanting an in-depth understanding of the impact of the technology (as opposed to the full intervention) should consider informing their methods using an additional framework that will help open the black box of design quality and packaging. For example, the Clinical Adoption Framework could be useful for designing probes around the quality of the system, quality of the information within the system, and quality of the services supporting the system [25]. Another limitation of the CFIR is that we consider most software updates to be an inherent quality of software-based health interventions. As such, we do not think that a technology's capacity to iterate is adequately captured in the CFIR construct of adaptability, which relates more to the components of an intervention that can be adapted or tailored.
Unlike studies that present a list of barriers and facilitators, the CFIR guides the classification of these factors into broader domains. For example, the strongest influencing factors on the Medly program were in the CFIR domains of inner setting, characteristics of individuals, and process. This is not to say that the characteristics of the intervention were not important, but it makes the point that the implementation context cannot be ignored.

Limitations
This study was conducted at a single implementation site. Therefore, we acknowledge that the characteristics of the HF clinic might differ compared with other settings in terms of the availability of resources, structure of care delivery, and characteristics of the individuals involved. In addition, we acknowledge the absence of the patients' perspective in this study. That said, a mixed method study of factors that influence patient adoption, use, and adherence to the Medly program will feature in a future publication. Finally, one cardiologist did not agree to participate in an interview; therefore, barriers to adoption for this individual are unknown.

Recommendations
We offer the following recommendations based on key study findings to facilitate the transferability of results to other implementation settings: 1. Evaluate contextual barriers: This study highlights the importance of contextual factors. Early identification of potential barriers as part of a readiness assessment would allow for the development of mitigating strategies. Using a framework such as the CFIR could facilitate this task; our results provide an example of how the CFIR constructs can be operationalized for telehealth interventions. 2. Define all components of the intervention: Complexity is an important barrier for the successful implementation of eHealth interventions [16]. However, this negative influence can be mitigated by an explicit definition of each intervention component. In this study, contextual facilitators helped overcome the lack of protocolized clinician roles; however, better definition of nontechnology program components and roles could facilitate clinician adoption in future implementations. 3. Plan and document the implementation strategy: The lack of clearly defined implementation plan was identified as an early barrier in this study, which was moderated by contextual facilitators and other process factors, including the presence of a strong clinical champion and a robust mechanism for ongoing reflecting and evaluation. The Quality Implementation Framework [26] offers a prescriptive approach that can help formulate an implementation strategy that incorporates an assessment of many of the constructs outlined in the CFIR [19].

Conclusions
This study presents results from the real-world implementation evaluation of a mobile phone-based telemonitoring program for patients with heart failure. The overall success of the implementation, as determined by the four implementation outcomes, was explained by the presence of several facilitators and relatively few barriers. Although the results are consistent with other telemonitoring implementation studies, this study also demonstrates how barriers and facilitators are dynamic and can influence the implementation success differently over time.
Finally, we highlight a previously undescribed challenge-telemonitoring interventions often rely on clinicians' ability to build experiential knowledge to use the system as intended. The results from this research can inform the development of telemonitoring programs and their implementation strategies. Hence, evidence-based implementation is important to ensure the success of real-world telemonitoring deployments as well as for ensuring that telemonitoring studies can yield unambiguous evidence of effectiveness, which will be required for the wider diffusion of telemonitoring.