This paper is in the following e-collection/theme issue:

Wearables and MHealth Reviews (259) mHealth for Telemedicine and Homecare (364) Telehealth and Telemonitoring (1566) Digital Health Reviews (1186) Evaluation and Research Methodology for mHealth (194)

Published on in Vol 9, No 8 (2021): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/29381, first published .
Domains and Methods Used to Assess Home Telemonitoring Scalability: Systematic Review

Domains and Methods Used to Assess Home Telemonitoring Scalability: Systematic Review

Domains and Methods Used to Assess Home Telemonitoring Scalability: Systematic Review

Authors of this article:

Salome Azevedo1, 2, 3 Author Orcid Image ;   Teresa Cipriano Rodrigues3 Author Orcid Image ;   Ana Rita Londral1, 2 Author Orcid Image
Article Authors Cited by (6) Tweetations (4) Metrics

    Review

    1Value for Health CoLAB, NOVA Medical School, Lisbon, Portugal

    2Comprehensive Health Research Centre, NOVA Medical School, UNL, Lisbon, Portugal

    3CEG-IST, Centre for Management Studies of Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal

    *all authors contributed equally

    Corresponding Author:

    Salome Azevedo, MSc

    Value for Health CoLAB

    NOVA Medical School

    Rua do Instituto Bacteriológico, no 5, 5-A e 5-B

    Lisbon, 1099-085

    Portugal

    Phone: 351 218803110

    Email: salome.azevedo@vohcolab.org


    Background: The COVID-19 pandemic catalyzed the adoption of home telemonitoring to cope with social distancing challenges. Recent research on home telemonitoring demonstrated benefits concerning the capacity, patient empowerment, and treatment commitment of health care systems. Moreover, for some diseases, it revealed significant improvement in clinical outcomes. Nevertheless, when policy makers and practitioners decide whether to scale-up a technology-based health intervention from a research study to mainstream care delivery, it is essential to assess other relevant domains, such as its feasibility to be expanded under real-world conditions. Therefore, scalability assessment is critical, and it encompasses multiple domains to ensure population-wide access to the benefits of the growing technological potential for home telemonitoring services in health care.

    Objective: This systematic review aims to identify the domains and methods used in peer-reviewed research studies that assess the scalability of home telemonitoring–based interventions under real-world conditions.

    Methods: The authors followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines and used multiple databases (PubMed, Scopus, Web of Science, and EconLit). An integrative synthesis of the eligible studies was conducted to better explore each intervention and summarize relevant information concerning the target audience, intervention duration and setting, and type of technology. Each study design was classified based on the strength of its evidence. Lastly, the authors conducted narrative and thematic analyses to identify the domains, and qualitative and quantitative methods used to support scalability assessment.

    Results: This review evaluated 13 articles focusing on the potential of scaling up a home telemonitoring intervention. Most of the studies considered the following domains relevant for scalability assessment: problem (13), intervention (12), effectiveness (13), and costs and benefits (10). Although cost-effectiveness was the most common evaluation method, the authors identified seven additional cost analysis methods to evaluate the costs. Other domains were less considered, such as the sociopolitical context (2), workforce (4), and technological infrastructure (3). Researchers used different methodological approaches to assess the effectiveness, costs and benefits, fidelity, and acceptability.

    Conclusions: This systematic review suggests that when assessing scalability, researchers select the domains specifically related to the intervention while ignoring others related to the contextual, technological, and environmental factors, which are also relevant. Additionally, studies report using different methods to evaluate the same domain, which makes comparison difficult. Future work should address research on the minimum required domains to assess the scalability of remote telemonitoring services and suggest methods that allow comparison among studies to provide better support to decision makers during large-scale implementation.

    JMIR Mhealth Uhealth 2021;9(8):e29381

    doi:10.2196/29381

    Keywords

    telemonitoring; scalability; home telecare; systematic review 


    The Universal Health Coverage commitment aligned with the emergence of COVID-19 reinforced the value of telemedicine services and elected these services crucial to coping with the pandemic’s challenges in the health care sector. Since the pandemic reached the western countries, home telemonitoring offered an alternative to control the health status of infected nonsevere patients at their homes to avoid unnecessary visits to the hospital [1].

    During the early part of 2020, from a social perspective, the fast-track solution to prevent the spread of COVID-19 focused on social distancing [2]. Governments forced people to stay at home, canceled mass gatherings, imposed teleworking, and closed all educational institutions [3]. From a health care perspective, governments took some extreme measures to increase the capacity to cope with the virus, namely reduction or deferral of nonurgent care and hands-on visits, and postponement of nonurgent surgeries [4]. These measures exposed high-risk groups, such as the elderly at home, people at long-term care facilities, patients with chronic conditions, and hidden diseases [5]. Inevitably, physicians started following-up with their patients through video calls and remote monitoring to continue treatment and avoid long-term complications [6]. In parallel, health care providers launched new telehealth services to assist patients in their homes [7]. Policy makers and practitioners did not have enough information to decide which pilot intervention they should disseminate into real-world settings, considering different financial reimbursement strategies, health care system organizations, and workforce acceptance levels [8].

    With technological progression and decreasing equipment costs, remote patient monitoring emerged as a telemedicine application. It comprises interactive and noninteractive technologies to support health care and monitor patients’ health status in their homes [9].

    Home telemonitoring is one type of remote patient monitoring, which has shown and is showing potential to improve clinical and patient-reported outcomes and ensure cost reductions for health care practices [10]. In this work, the authors consider the definition given by Paré and colleagues [11] for home telemonitoring. A service based on home telemonitoring consists of health care professionals monitoring the patient's health status at a distance. Patients or caregivers transmit their health-related data to a responsible health care professional through information and telecommunication technologies. Research on home telemonitoring showed benefits concerning health care systems’ capacity constraints [12], patient empowerment, and treatment commitment [13]. It revealed significant improvement in clinical outcomes even in some diseases [11]. Despite the considerable investment in accelerating health information technology [14], there is not enough information on determining whether home telemonitoring is appropriate and feasible for implementation in a real-world context [15]. Scaling up a health intervention requires wise and efficient spending of resources [16]. Therefore, it is crucial to assess the suitability of scaling up home telemonitoring interventions with proven efficacy to provide answers to the following two questions [17]: Does it work in practice? Is it worth it?

    To answer these questions and decide which technology-based health intervention can be scaled up for mainstream care delivery, one must assess its scalability (ie, the ability to be expanded to real-world conditions without compromising on effectiveness and access to the eligible population) [18].

    Most of the studies focus only on assessing the effectiveness and costs of a health intervention. Nevertheless, these are two of many considerations to address when evaluating the potential of scaling up an intervention [19]. Other domains such as the feasibility and adaptability of the health intervention and the political or strategic contexts are rarely analyzed. As emphasized by Milat and his colleagues [15] in their recently proposed Intervention Scalability Assessment Tool (ISAT), assessing a health intervention’s scalability involves considering multiple domains, such as the political and strategic contexts, workforce, and infrastructure, among others.

    There is a need to conduct evidence-based studies that assess pilot interventions’ potential to achieve population-wide benefits [20]. Scalability studies that also consider the intervention’s suitability to the socioeconomic context in question are important to estimate the success of deploying these interventions in different contexts [15].

    Owing to the lack of research on scalability analysis, in this paper, the authors present a systematic review, based on Milat and colleagues’ domains [15], to identify and characterize methods used to assess the potential to scale-up home telemonitoring interventions in the context of a growing telehealth service in the industry. This study focuses on peer-reviewed studies conducted to evaluate the scalability of follow-up interventions based on home telemonitoring. The authors aim to provide a comprehensive overview of these studies concerning the domains and methods used and identify gaps for future research to address when evaluating the potential to implement or scale-up home telemonitoring interventions. As the authors are not aware of other systematic reviews focusing on this aspect, they believe that this review will enlighten researchers, practitioners, and policy makers regarding the most used strategies to assess the scalability of home telemonitoring interventions.


    The search strategy followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines to conduct the review [21]. The population, intervention, comparison, outcome (PICO) framework [21] allowed the identification of key concepts such as “Home Telemonitoring,” “Follow-up,” “Scalability,” and “Assessment” to formulate a well-focused question and facilitate the literature search. To optimize the search through effective queries, the authors used PubMed’s Medical Subject Headings (MeSH) to identify indexed terms [22]. This step was fundamental as this review emerges from the combination of research fields with different terms for the same concept. Textbox 1 presents the rationale used to build the final query used in each database.

    Queries used to search each database.

    1. (((Telemonitoring) OR (Home remote monitoring)) AND (Mobile Health OR health OR mHealth OR eHealth OR Telehealth OR Telemedicine)) OR (Telehomecare)

    2. (Scalability) OR (Feasibility) OR (Scaling up OR scale up OR upscale OR up-scale OR scale-up) OR ((Deployment OR Implementation OR Application) OR (Broad-scale OR Wide-scale OR Widespread OR Mainstream)) OR (((Efficienc*) AND (Program OR Intervention)) OR Economic* Viability)

    3. (Follow-up Care* OR Follow Up Care* OR Care*) OR (Case Management OR Patient Care Planning)

    4. ((Appraisal* OR Evaluation* OR Assessment* OR Appropriateness) AND ((Impact) OR (Cost-Effective* OR Qualitative OR Quantitative OR Index* OR Methodolog*) OR (Clinical Trial* AND (Pragmatic OR Naturalistic Randomized OR Practical OR Real World)) OR (Sustainability) OR (Profitability) OR (Risk*)))

    5. #1 AND #2 AND #3 AND #4

    Textbox 1. Queries used to search each database.

    Figure 1 illustrates the search performed in PubMed, Scopus, Web of Science, and EconLit covering studies from 2000 to 2020 (Figure 1 - Set #1). The authors chose to explore EconLit owing to the economic evaluation required to assess a health care intervention’s scalability. The authors selected full-text and peer-reviewed papers written in English (Figure 1 - Set #2).

    After removing the duplicates and references without abstracts (Figure 1 - Set #3), two authors independently scanned the titles and abstracts identified in the literature search and applied the selection criteria presented in Textbox 2 (Figure 1 - Set #4).

    To guarantee that the article’s topic aligned with the research question, the same authors scanned the 49 full-text articles, which reduced the number of studies considered for review to 13 (Set #5).

    The authors analyzed 13 full-text articles, corresponding to 13 studies, in detail and registered all the observations in a literature matrix [23]. First, to better explore each intervention and summarize relevant, well-specified, and secure data, the authors conducted an integrative synthesis. The main variables were the country of origin, publication year, sample size, setting, duration of follow-up, comparator arms, type of technology, and study outcomes [24].

    Second, the authors assessed the strength of each eligible study’s evidence according to the 9-level classification system proposed by Jovell and Navarro-Rubio [25].

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram showing the included studies.
    View this figure

    Eligibility criteria for screening titles, abstracts, and full-text papers.

    Inclusion criteria

    • Health interventions shown to be efficacious on a small scale or under controlled conditions
    • Assessment of the health intervention’s ability to be expanded to real-world conditions to reach a more significant proportion of the eligible population while retaining effectiveness.
    • Studies assessing at least one domain of scalability through the evaluation of feasibility, acceptability, costs, sustainability or, adaptability (ie, to suit the needs of the context in which it is to be scaled up)
    • Described methods to assess the scalability of a health intervention

    Exclusion criteria

    • Telemonitoring involving invasive medical devices
    • Studies that use telemonitoring “not involving the patients, their relatives, or informal caregivers, their relatives, or informal caregivers
    • Studies that described the concept of scalability without providing an assessment method
    • Studies just focusing on describing disease risk patterns or intervention efficacy testing
    • Study protocols or medical testing procedures for potential scalability assessment and possible scale-up
    • Statistical or conceptual modeling without a real-world study
    • Facilitators and barriers to scale-up within specific interventions or general experiences of scale-up that did not provide a scalability assessment method
    • Studies recommending an assessment method (of feasibility or acceptability or costs or sustainability or adaptability), but that did not assess the potential to scale-up a telemonitoring-based health intervention
    Textbox 2. Eligibility criteria for screening titles, abstracts, and full-text papers.

    Finally, they conducted narrative and thematic analyses to identify themes and patterns in the eligible articles and outline the findings under specific headings [24] to better examine how each study assessed the potential of scaling up an intervention. When disagreements occurred, the authors reached a consensus via discussion. One author extracted data from the studies and completed quotes, and the second author validated the data according to the definition of each category. The authors conducted this analysis based on the work undertaken by Milat and colleagues [18] in the development of a tool to perform systematic assessments of the suitability of health interventions for scale-up (ISAT). ISAT comprises three parts: setting the scene, planning the intervention implementation, and summarizing the scalability assessment. The first two parts made it possible to classify each study according to the stage of scale-up, context, and focus area. Moreover, Milat and colleagues’ domains enabled the authors to identify the methods and instruments used by the researchers to assess the intervention’s scalability [18].

    The research conducted for each domain assessed in the eligible papers was classified as qualitative or quantitative. The research was classified as qualitative if it was based on the description of experiences, emotions, behaviors, events, or actions [26] and quantitative when the respective authors used numerical data to measure, categorize, or identify patterns, relationships, or generalizations through statistical analysis [26].


    Country of Origin and Year of Publication

    From 2009 to 2020, the authors analyzed 13 studies in 7 countries, which focused on the potential to scale-up home telemonitoring health care interventions; however, more than half (n=7) were published between 2018 and 2020. Most of the articles (n=8) were from Canada and the United States, whereas the rest were from 5 European countries—Denmark (n=1), Italy (n=1), Lithuania (n=1), Netherlands (n=1), and Spain (n=1).

    Population and Home Telemonitoring Intervention Assessment

    Target Condition or Disease

    The studies addressed either chronic or acute conditions, with a higher number of studies addressing only chronic conditions (n=8). The full spectrum of chronic conditions covered were cardiovascular diseases (n=4), chronic obstructive pulmonary diseases (n=2), cerebrovascular diseases (n=1), chronic obstructive sleep apnea (n=1), cystic fibrosis (n=1), and diabetes mellitus (gestational [n=1] and type 1 and 2 [n=1]). Further, one study only characterized the patients’ condition as chronic or acute, and the remaining studies addressed multiple conditions (eg, surgical patients, cardiovascular and pulmonary diseases, diabetes mellitus).

    Duration and Setting of Home Telemonitoring Intervention

    Home telemonitoring was integrated into a follow-up service in the 13 studies and required a responsible health care professional (or a team) to manage the patient’s care. The minimum duration of the follow-up was 3 consecutive nights (sleep apnea [27]). However, the 1-year (n=4) and 6-month (n=4) follow-up interventions were the most implemented. In particular, authors reporting the secondary prevention of cerebrovascular disease [28] defined the intervention according to recommended monitoring protocols, assuming a 20-year time horizon for the modeling strategy. Moreover, 10 studies had 2 dedicated teams for executing the intervention; one was responsible for the patient’s holistic care management and the other for telecare management. In two studies, the conventional care team was accountable for usual care and telecare management, and in the other, there was no traditional care team.

    Types of Technologies

    The technologies used in the studies ranged from a kit with just one regular telephone (1) to an integrated communication and data collection system with mobile devices (5). Moreover, six studies conducted home telemonitoring interventions with an integrated clinical data system, remote monitoring digital technology (mobile devices that collect physiological signs), and a telephone.

    Study Design Assessment

    Study Characteristics

    The average total sample size of the studies was approximately 436 (maximum: 3086, minimum: 34), with an average treatment and control group size of 260.

    To better understand the type of research conducted, it is essential to highlight that 6 out of the 13 studies were experimental. Therefore, the authors of these studies allocated participants to different treatment groups. As the other 7 studies were observational, there was no allocation of the participants. Most of the studies (n=10) were comparative studies (control group) with conventional care services, and the other 3 were single-arm studies.

    Study Design Classification

    According to the 9-level classification system proposed by Jovell and Navarro-Rubio [25], the studies conducted by Padwal and colleagues [28], and Vestergaard and colleagues [29] were classified as “very good,” as they conducted randomized controlled trials with large samples. The studies by Lugo and colleagues [27], and Paré and colleagues [30] were classified as “good” as these studies were randomized controlled trials with small samples. Furthermore, the studies of Ware and colleagues [31], as well as Zaliūnas and colleagues [32], were classified as “poor” because they consisted of noncontrolled clinical series or descriptive studies. The other 7 were classified as fair and included nonrandomized controlled prospective studies (n=3), cohort studies (n=3), and case-control studies (n=1).

    Scalability Assessment

    Table 1 displays the scalability assessment domains for each study.

    Table 1. Scalability assessment domains for each study.
    Application fieldStage of scale-upDomains for scale-upDomains for implementation planning


    		ProblemInterventionContextEffectivenessCosts and benefitsFidelity and adaptabilityReach and acceptabilitySetting and workforceInfrastructureSustainability
    Improved health outcomes in a rural area [33]Pre–scale-upYesYesYesYesYesNoYesNoNoNo
    Diabetes [34]Pre–scale-upYesYesYesYesYesYesYesYesYesYes
    Cystic fibrosis [35]Pre–scale-upYesYesNoYesYesNoNoYesYesYes
    Chronic heart failure [36]Pre–scale-upYesNoNoYesYesNoNoNoNoNo
    Obstructive sleep apnea [27]Pre–scale-upYesYesNoYesYesYesYesYesYesYes
    Secondary prevention of cerebrovascular disease [28]Pre–scale-upYesYesYesYesYesYesNoNoNoYes
    Heart failure [29]Pre–scale-upYesYesYesYesYesNoNoNoNoNo
    Gestational diabetes mellitus [37]Pre–scale-upYesYesYesYesYesYesYesYesYesYes
    Rural home health agencies [38]Scale-upYesYesYesYesYesYesYesNoYesYes
    Chronic obstructive pulmonary disease [30]Scale-upYesYesYesYesYesNoYesYesYesYes
    Ischemic heart disease [32]ImplementationYesYesYesYesNoYesYesYesYesYes
    Heart failure [31]ImplementationYesYesYesYesNoYesYesYesYesYes
    Chronic obstructive pulmonary disease [39]ImplementationYesYesYesYesNoNoYesYesYes Yes
    Scale-Up Stages

    The authors classified eight studies as being in the pre–scale-up stage because their descriptions consisted of steps or activities conducted before scaling up the evidence-based home telemonitoring intervention. Two studies described steps or actions involved in the dissemination of the intervention. The authors classified the other three studies as being in the implementation stage because their descriptions indicated using or integrating the evidence-based intervention within a setting.

    Domains Considered for Scale-Up

    Although all the studies described the problem under intervention and the target population, one study [36] did not provide details concerning the proposed home telemonitoring intervention to address the issue. All studies referred to the level of evidence available to support the proposed intervention’s scale-up, either by referring to their work or other scientific literature., Three studies did not consider the known costs and benefits of delivering the intervention [31,32,39], and three more did not consider the strategic/political/environmental contexts that influence the scaling up of the intervention [27,35,36].

    Domains Considered for Implementation Planning

    Seven studies considered intervention changes when assessing fidelity, and nine studies assessed the level of acceptability perceived by the program deliverers or recipients of the intervention. Further, 9 studies referred to the definition of the intervention settings and the workforce required to scale-up, and 10 described the necessary infrastructure.

    All the studies accounted for the sustainability of the home telemonitoring service by either referring to the long-term outcomes of the scale-up or the medium- and long-term sustainability of the intervention following scale-up.

    Methods for Scalability Assessment

    This section explains the research foci and methods used by the eligible studies in each domain of scalability assessment. When describing the problems, interventions, and contexts of their studies, all the researchers adopted qualitative research methods, as Table 2 shows. The definitions of the domains and research foci are given in Multimedia Appendix 1. We have included six publications [40-45] in this appendix.

    Table 2. Qualitative studies on scalability assessment considering the problem, intervention, and context domains for scale-up.
    DomainResearch focusResearch typeData collection techniqueData analysis techniqueStudies, nReference
    ProblemProblem descriptionQualitativeDocument analysisNarrative summary13[27-39,46-50]
    InterventionIntervention descriptionQualitativeDocument analysisNarrative summary12[27-35,37-39]
    ContextContext descriptionQualitativeDocument analysisNarrative summary10[28-34,37-39]

    All the studies adopted quantitative research methods to assess clinical outcomes namely surveys or questionnaires (n=10), published databases (n=2), and observations (n=1) (Table 3). To assess humanistic and satisfaction outcomes, the researchers chose surveys or questionnaires; however, for assessing for usage outcomes, they either conducted observations (n=9) or used published databases (n=3). As for validated instruments, only one was used in one study [27] to assess clinical outcomes, namely the Epworth Sleepiness Scale (ESS) [51]. For assessing humanistic outcomes, three validated questionnaires were used: EuroQol 5-Dimensions 5-Levels (EQ-5D-5L) [52] in the contexts of heart failure [29] and obstructive sleep apnea [27]; Quebec Sleep Questionnaire (QSQ) [53] for obstructive sleep apnea; and Chronic obstructive pulmonary disease Assessment Test (CAT) [54] for chronic obstructive pulmonary disease [39]. In the context of ischemic heart disease [32], two more validated questionnaires were used: Patient Satisfaction Questionnaire Form III (PSQIII) [55] and Thought Control Questionnaire (TCQ) [56].

    Table 3. Quantitative research studies involving data analyses using descriptive and inferential statistics for scalability assessment considering the effectiveness domain for scale-up.
    Research focus and data collection technique

    		Studies, nReference
    Clinical outcome assessment

    		Observation; published databases3[28,36,38]

    		Nonvalidated surveys or questionnaires9[29-35,37,39]

    		Validated surveys or questionnaires1[27]
    Humanistic outcome assessment

    		Nonvalidated surveys or questionnaires3[32-34]

    		Validated surveys or questionnaires3[27,29,39]
    Satisfaction assessment

    		Nonvalidated surveys or questionnaires7[27,29,30,33,34,37,39]

    		Validated surveys or questionnaires1[32]

    For the domains of fidelity and acceptability, quantitative research methods involving observations were more predominantly used as the main data collection methods, as shown in Tables 4 and 5. Contrarily, for analyzing infrastructure, setting, and workforce, most of the studies chose qualitative techniques (n=8).

    Table 4. Studies on scalability assessment concerning the reach and acceptability domain for implementation planning involving data analyses using descriptive and inferential statistics.
    Research focus and typeData collection techniqueStudies, nReference
    Acceptability assessment

    		QuantitativeObservation7[27,30,32-34,37,39]

    		QualitativeSemistructured interviews1[38]
    Compliance assessment

    		QuantitativeNonvalidated surveys or questionnaires1[31]

    		QuantitativeValidated surveys or questionnaires1[32]
    Penetration assessment

    		QuantitativeObservation2[31,37]
    Table 5. Research focus and methods found in the studies for scalability assessment concerning the fidelity and adaptability domain for implementation planning.
    Research focus and type

    		Data collection techniqueData analysis techniqueStudies, nReference
    Adaptability assessment

    		QuantitativeObservationDescriptive statistics; inferential statistics1[27]

    		QualitativeObservations; oral history or life storiesNarrative summary2[32,38]
    Feasibility assessment

    		QuantitativeObservationDescriptive statistics; inferential statistics2[31,37]

    When conducting economic evaluation (Table 6), the authors found 7 different types of techniques used across 10 studies (see Multimedia Appendix 2 for the main results of the studies that conducted economic evaluation of home telemonitoring). The most popular technique was cost-effectiveness analysis used in three studies with different fields of application. These three studies were able to show outcome improvements and cost savings. Table 7 presents the scalability assessment studies concerning the setting and workforce, infrastructure, and sustainability domains for implementation planning

    Table 6. Quantitative research studies focusing on data collection using document screening and published databases for scalability assessment considering the costs and benefits domain for scale-up (research focus: economic evaluation).
    Data analysis techniqueStudies, nReference
    Cost analysis2[34,35]
    Cost-benefit1[38]
    Cost-effectiveness3[27,33,37]
    Cost minimization1[30]
    Cost utility2[28,29]
    Cost-saving simulation1[35]
    Value of information analysis1[36]
    Table 7. Studies on scalability assessment concerning the setting and workforce, infrastructure, and sustainability domains for implementation planning.
    Domain and research focus and typeData collection techniqueData analysis techniqueStudies, nReference
    Setting and workforce

    		Setting and workforce assessment


    		QualitativeObservations; oral history or life storiesNarrative summary8[27,28,31,32,34,35,37,39]


    		QuantitativeObservationDescriptive statistics1[30]
    Infrastructure

    		Infrastructure assessment


    		QualitativeObservations; oral history or life storiesNarrative summary9[27,28,30,32,34,35,37-39]


    		QualitativeSemistructured InterviewsDescriptive statistics1[31]
    Sustainability

    		Opportunity and challenge assessment


    		QualitativeObservations; oral history or life storiesNarrative summary12[27-30,32-39]


    		QualitativeSemistructured interviewsNarrative summary1[31]
    Scalability Assessment

    All the 13 articles assessed scalability based on the results achieved in the respective studies. Table 8 summarizes the assessments obtained through narrative analysis. On the one hand, two studies provided positive assessments regarding the potential to scale-up the intervention. On the other hand, eight studies highlighted the need for cost-effectiveness or cost-benefit analysis before proceeding to scale-up the intervention.

    Table 8. Scalability assessment based on the authors’ conclusions in each study.
    Scalability assessmentStudies, NReference
    Not able to be expanded1[27]
    Able to be expanded, but the diffusion and sustainability will depend on a supportive policy environment1[34]
    Able to be expanded but requires cost-benefit analysis for reimbursement planning3[28,36,38,39]
    Able to be expanded but requires cost-effectiveness analysis3[29,30,35]
    Able to be expanded but requires some technical changes, cost-benefit analysis for reimbursement planning, and solutions for regulatory issues2[32,33]
    Able to be expanded under real-world conditions2[31,37]

    Principal Results

    Despite the rapid growth of telemedicine applications in the last few years, particularly after the emergence of COVID-19, scientific studies assessing the scalability of these health interventions are scarce [19].

    In this review, all the eligible studies are from developed countries, particularly the United States and Canada. The absence of such studies in developing countries could be owing to the lack of specialized human resources, information and communications technology (ICT) infrastructure, and equipment [46]. Besides, the significant difference found between North America and Europe might be related to the requirement of evidence to justify private payer reimbursement for health care interventions [47] or the investment in developing strategies to encourage telemedicine adoption [48]. Nevertheless, this review has not identified studies from countries that invested significantly in telehealth solutions, such as the United Kingdom or Australia [46]. The justification for this might be the frequent research focus of health interventions on clinical effectiveness [11], instead of assessing their scale-up potential. More than half of the studies were published between 2018 and 2020. Thus, this research area is receiving more attention from the scientific community as a logical next step after demonstrating robust evidence regarding the effectiveness and technological maturity of such interventions.

    The use of one of the most recent scalability assessment frameworks [18] granted the opportunity to compare the strategies used to assess the scale-up potentials of interventions in each study. This advantage of this framework is that it allows the analysis of different domains considering the stage of the transference process of an intervention from a research setting into the practical implementation stage.

    This review suggests an agreement in some analyzed domains, such as problems, interventions, effectiveness, costs, and benefits, to support the decision to scale-up interventions. However, this is not the case for the methods and instruments used. For example, although cost-effectiveness was the most common approach across the 13 studies, researchers used 7 different cost analysis methods. Moreover, to demonstrate effectiveness, studies provided evidence of different outcomes, such as clinical, humanistic, and utilization outcomes. This inconsistency leads to different scalability assessments and does not enable comparing interventions with home telemonitoring technologies.

    There is a recognized methodological gap in understanding other relevant domains such as the sociopolitical context, setting, workforce, and implementation infrastructure to provide the home telemonitoring intervention to the target population. A common framework will allow determining if interventions demonstrated as effective are appropriate and feasible in other settings [18,49].

    Lastly, another relevant result obtained from this systematic review was that researchers assigned different weights to the analyzed domains when concluding the intervention scalability. On the one hand, 12 studies concluded their ability to scale- up based on the costs and outcomes of the interventions, although they had analyzed other domains. On the other hand, one study restrained the decision to scale-up the intervention based on the policy environment. Future research should address the influence that each domain has on the final decision to scale-up the interventions with sound and transparent methods, avoiding mistakes reported in the literature [50].

    Limitations

    This relevant limitation of this review might be associated with the low maturity of this research area, despite its recent growth. Additionally, one database filter concerned peer-reviewed journals, which influenced the rejection of studies with no statistical significance but could have been relevant in this review with respect to the domains and methods used when assessing scalability. This review only considered studies published in English, which might have influenced the number of eligible studies. Moreover, the authors did not conduct a meta-analysis owing to the limited number of studies on this subject. Finally, the domains used to analyze the scalability assessment strategies were predefined, thus limiting the spectrum of domains studied.

    Conclusions

    Studies on home telemonitoring interventions integrated into follow-up care have already proved their efficacy. Although some studies focused on including domains such as effectiveness, costs, and benefits, these are not enough to assess the potential of scaling up these interventions. As technology progresses and the need for providing care to more people in their homes increases, it is extremely important to conduct more studies on scalability assessment considering domains such as workforce and infrastructure characteristics and the strategic context. Future research should establish rigorous study designs and scientific methods to assess scalability based on the results of this systematic review. Further understanding of the usage of health services and medium- and long-term sustainability of interventions would yield more robust evidence to support their future integration into mainstream care delivery systems. This research area, although still emerging, will advance knowledge on the factors that influence the successful scale-up of interventions.

    Acknowledgments

    We are thankful to Professor Mónica Oliveira for her support and guidance in the ideation process of this systematic review. In addition, we are grateful for the funding provided by Portugal 2020, under the scope of PROJETOS DE I&DT EMPRESAS EM COPROMOÇÃO - COVID-19 (15/SI/2020) and the project Fast Track Covid, LISBOA-01-02B7-FEDER-069202.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Glossary of research methods and scalability assessment domains used to systematically review the eligible studies in this work.

    PDF File (Adobe PDF File), 82 KB
    ‎
    Multimedia Appendix 2

    Main results of the economic evaluations conducted in each eligible study that addressed the domains of costs and benefits.

    PDF File (Adobe PDF File), 77 KB
    1. Annis T, Pleasants S, Hultman G, Lindemann E, Thompson J, Billecke S, et al. Rapid implementation of a COVID-19 remote patient monitoring program. J Am Med Inform Assoc 2020 Aug;27(8):1326-1330 [FREE Full text] [CrossRef] [Medline]
    2. Adolph C, Amano K, Bang-Jensen B, Fullman N, Wilkerson J. Pandemic politics: timing state-level social distancing responses to COVID-19. J Health Polit Policy Law 2021 Apr;46(2):211-233. [CrossRef] [Medline]
    3. Anderson M, Mckee M, Mossialos E. Developing a sustainable exit strategy for COVID-19: health, economic and public policy implications. J R Soc Med 2020 May;113(5):176-178. [CrossRef] [Medline]
    4. European Centre for Disease Prevention and Control (ECDC) 2021.   URL: https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases [accessed 2021-08-02]
    5. Carter P, Anderson M, Mossialos E. Health system, public health, and economic implications of managing COVID-19 from a cardiovascular perspective. Eur Heart J 2020 Jul;41(27):2516-2518 [FREE Full text] [CrossRef] [Medline]
    6. Srivatana V, Liu F, Levine DM, Kalloo SD. Early use of telehealth in home dialysis during the COVID-19 pandemic in New York City. Kidney360 2020 Apr;1(6):524-526. [CrossRef]
    7. Fisk M, Livingstone A, Pit SW. Telehealth in the context of COVID-19: changing perspectives in Australia, the United Kingdom, and the United States. J Med Internet Res 2020 Jun;22(6):e19264 [FREE Full text] [CrossRef] [Medline]
    8. Smith AC, Thomas E, Snoswell CL, Haydon H, Mehrotra A, Clemensen J, et al. Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020 Mar;26(5):309-313. [CrossRef]
    9. Strode SW, Gustke S, Allen A. Technical and clinical progress in telemedicine. JAMA 1999;281(12):1066-1068. [CrossRef] [Medline]
    10. Meystre S. The current state of telemonitoring: a comment on the literature. Telemed J E Health 2005 Feb;11(1):63-69. [CrossRef] [Medline]
    11. Pare G, Jaana M, Sicotte C. Systematic review of home telemonitoring for chronic diseases: the evidence base. J Am Med Inform Assoc 2007 May;14(3):269-277. [CrossRef]
    12. Mengden T, Vetter H, Tislér A, Illyés M. Tele-monitoring of home blood pressure. Blood Press Monit 2001 Aug;6(4):185-189. [CrossRef] [Medline]
    13. Riley JP, Gabe JP, Cowie MR. Does telemonitoring in heart failure empower patients for self-care? a qualitative study. J Clin Nurs 2013 Sep;22(17-18):2444-2455. [CrossRef] [Medline]
    14. Makhni S, Atreja A, Sheon A, Van Winkle B, Sharp J, Carpenter N. The broken health information technology innovation pipeline: a perspective from the NODE health consortium. Digit Biomark 2017 Aug;1(1):64-72 [FREE Full text] [CrossRef] [Medline]
    15. Milat AJ, King L, Bauman AE, Redman S. The concept of scalability: increasing the scale and potential adoption of health promotion interventions into policy and practice. Health Promot Int 2013 Sep;28(3):285-298. [CrossRef] [Medline]
    16. European C. Defining value in "value-based healthcare": report of the expert panel on effective ways of investing in health (EXPH). Publications Office of the European Union. 2019 Jul.   URL: https://op.europa.eu/s/psov [accessed 2021-08-02]
    17. Haynes B. Can it work? Does it work? Is it worth it? the testing of healthcare interventions is evolving. BMJ 1999 Sep;319(7211):652-653 [FREE Full text] [CrossRef] [Medline]
    18. Milat A, Lee K, Conte K, Grunseit A, Wolfenden L, van Nassau F, et al. Intervention scalability assessment tool: a decision support tool for health policy makers and implementers. Health Res Policy Sys 2020 Jan;18(1):1 [FREE Full text] [CrossRef] [Medline]
    19. Kidholm K, Ekeland AG, Jensen LK, Rasmussen J, Pedersen CD, Bowes A, et al. A model for assessment of telemedicine applications: mast. Int J Technol Assess Health Care 2012 Jan;28(1):44-51. [CrossRef] [Medline]
    20. Ben Charif A, Zomahoun HTV, LeBlanc A, Langlois L, Wolfenden L, Yoong SL, et al. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implement Sci 2017 Nov;12(1):139 [FREE Full text] [CrossRef] [Medline]
    21. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    22. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al, editors. Cochrane Handbook for Systematic Reviews of Interventions. 2nd Edition. Chichester, United Kindgom: John Wiley & Sons; 2019.
    23. Kruse CS. Writing a systematic review for publication in a health-related degree program. JMIR Res Protoc 2019 Oct;8(10):e15490 [FREE Full text] [CrossRef] [Medline]
    24. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005 Jan;10(1):45-53. [CrossRef] [Medline]
    25. Jovell AJ, Navarro-Rubio MD. Evaluation of scientific evidence. Med Clin (Barc) 1995 Dec;105(19):740-743. [Medline]
    26. University of Newcastle Library Guides - Research Methods. University of Newcastle Library Guides.   URL: https://libguides.newcastle.edu.au/researchmethods [accessed 2021-08-02]
    27. Lugo VM, Garmendia O, Suarez-Girón M, Torres M, Vázquez-Polo FJ, Negrín MA, et al. Comprehensive management of obstructive sleep apnea by telemedicine: clinical improvement and cost-effectiveness of a virtual sleep unit. a randomized controlled trial. PLoS One 2019;14(10):e0224069 [FREE Full text] [CrossRef] [Medline]
    28. Padwal RS, So H, Wood PW, Mcalister FA, Siddiqui M, Norris CM, et al. Cost-effectiveness of home blood pressure telemonitoring and case management in the secondary prevention of cerebrovascular disease in Canada. J Clin Hypertens (Greenwich) 2019 Feb;21(2):159-168 [FREE Full text] [CrossRef] [Medline]
    29. Vestergaard AS, Hansen L, Sørensen SS, Jensen MB, Ehlers LH. Is telehealthcare for heart failure patients cost-effective? an economic evaluation alongside the Danish TeleCare North heart failure trial. BMJ Open 2020 Jan;10(1):e031670 [FREE Full text] [CrossRef] [Medline]
    30. Paré G, Poba-Nzaou P, Sicotte C, Beaupré A, Lefrançois É, Nault D, et al. Comparing the costs of home telemonitoring and usual care of chronic obstructive pulmonary disease patients: a randomized controlled trial. European Research in Telemedicine / La Recherche Européenne en Télémédecine 2013 Jun;2(2):35-47. [CrossRef]
    31. Ware P, Ross HJ, Cafazzo JA, Laporte A, Gordon K, Seto E. Evaluating the implementation of a mobile phone-based telemonitoring program: longitudinal study guided by the consolidated framework for implementation research. JMIR Mhealth Uhealth 2018 Jul;6(7):e10768 [FREE Full text] [CrossRef] [Medline]
    32. Zaliūnas R, Benetis R, Vanagas G, Slapikas R, Vainoras A. Implementation of international transtelephonic ECG platform for patients with ischemic heart disease. Medicina (Kaunas) 2009;45(2):104-110 [FREE Full text] [Medline]
    33. Hicks LL, Fleming DA, Desaulnier A. The application of remote monitoring to improve health outcomes to a rural area. Telemed J E Health 2009 Sep;15(7):664-671. [CrossRef] [Medline]
    34. Nundy S, Dick JJ, Chou C, Nocon RS, Chin MH, Peek ME. Mobile phone diabetes project led to improved glycemic control and net savings for Chicago plan participants. Health Aff (Millwood) 2014 Feb;33(2):265-272 [FREE Full text] [CrossRef] [Medline]
    35. Tagliente I, Trieste L, Solvoll T, Murgia F, Bella S. Telemonitoring in cystic fibrosis: a 4-year assessment and simulation for the next 6 years. Interact J Med Res 2016 May;5(2):e11 [FREE Full text] [CrossRef] [Medline]
    36. Grustam AS, Buyukkaramikli N, Koymans R, Vrijhoef HJM, Severens JL. Value of information analysis in telehealth for chronic heart failure management. PLoS One 2019;14(6):e0218083 [FREE Full text] [CrossRef] [Medline]
    37. Lemelin A, Paré G, Bernard S, Godbout A. Demonstrated cost-fffectiveness of a telehomecare program for gestational diabetes mellitus management. Diabetes Technol Ther 2020 Mar;22(3):195-202. [CrossRef] [Medline]
    38. Hansen D, Golbeck AL, Noblitt V, Pinsonneault J, Christner J. Cost factors in implementing telemonitoring programs in rural home health agencies. Home Healthc Nurse 2011 Jun;29(6):375-382. [CrossRef] [Medline]
    39. Farias R, Sedeno M, Beaucage D, Drouin I, Ouellet I, Joubert A, et al. Innovating the treatment of COPD exacerbations: a phone interactive telesystem to increase COPD action plan adherence. BMJ Open Respir Res 2019 Apr;6(1):e000379 [FREE Full text] [CrossRef] [Medline]
    40. Walton MK, Powers JH, Hobart J, Patrick D, Marquis P, Vamvakas S, et al. Clinical outcome assessments: conceptual foundation—report of the ISPOR clinical outcomes assessment – emerging good practices for Outcomes Research Task Force. Value Health 2015 Sep;18(6):741-752 [FREE Full text] [CrossRef] [Medline]
    41. Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, editors. Chapter 6-Outcome definition and measurement. In: Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Rockville, United States: Agency for Healthcare Research and Quality; Jan 2013.
    42. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Fourth edition. Oxford: Oxford University Press; 2015.
    43. Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Rev 1991 Jan;11(3):247-266. [CrossRef]
    44. Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One 2016;11(3):e0150205 [FREE Full text] [CrossRef] [Medline]
    45. Haynes RB, Taylor DW, Sackett DL, editors. Compliance in Health Care. Baltimore: Johns Hopkins University Press; 1979.
    46. Combi C, Pozzani G, Pozzi G. Telemedicine for developing countries. a survey and some design issues. Appl Clin Inform 2016 Nov;7(4):1025-1050 [FREE Full text] [CrossRef] [Medline]
    47. Whitten P, Buis L. Private payer reimbursement for telemedicine services in the United States. Telemed J E Health 2007 Feb;13(1):15-23. [CrossRef] [Medline]
    48. Gheorghiu B, Ratchford F. Scaling up the use of remote patient monitoring in Canada. Stud Health Technol Inform 2015;209:23-26. [Medline]
    49. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008 Sep;337:a1655 [FREE Full text] [CrossRef] [Medline]
    50. Keeney RL. Value-focused thinking: Identifying decision opportunities and creating alternatives. Eur J Oper Res 1996 Aug;92(3):537-549. [CrossRef]
    51. Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep 1991 Dec;14(6):540-545. [CrossRef] [Medline]
    52. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011 Dec;20(10):1727-1736 [FREE Full text] [CrossRef] [Medline]
    53. Lacasse Y, Bureau M, Sériès F. A new standardised and self-administered quality of life questionnaire specific to obstructive sleep apnoea. Thorax 2004 Jun;59(6):494-499 [FREE Full text] [CrossRef] [Medline]
    54. Jones PW, Harding G, Berry P, Wiklund I, Chen W, Kline Leidy N. Development and first validation of the COPD assessment test. Eur Respir J 2009 Sep;34(3):648-654 [FREE Full text] [CrossRef] [Medline]
    55. Ware JE, Snyder MK, Wright W, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Program Plann 1983 Jan;6(3-4):247-263. [CrossRef] [Medline]
    56. Wells A, Davies MI. The thought control questionnaire: a measure of individual differences in the control of unwanted thoughts. Behav Res Ther 1994 Nov;32(8):871-878. [CrossRef] [Medline]

    ‎
    CAT: Chronic Obstructive Pulmonary Disease Assessment Test
    EQ-5D-5L: EuroQol 5-Dimensions 5-Levels
    ESS: Epworth Sleepiness Scale
    ICT: information and communications technology
    ISAT: Intervention Scalability Assessment Tool
    MeSH: Medical Subject Heading
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analysis
    PSQIII: Patient Satisfaction Questionnaire Form III
    QSQ: Quebec Sleep Questionnaire
    TCQ: Thought Control Questionnaire

    Edited by L Buis; submitted 05.04.21; peer-reviewed by L Trieste, S Pit; comments to author 11.06.21; revised version received 25.06.21; accepted 14.07.21; published 19.08.21

    Copyright

    ©Salome Azevedo, Teresa Cipriano Rodrigues, Ana Rita Londral. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 19.08.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.