Improving Adherence to Web-Based and Mobile Technologies for People With Psychosis: Systematic Review of New Potential Predictors of Adherence

Introduction

E-mental health interventions, defined as “the use of information and communication technology to support or improve mental health care” [1,2], have been proposed as promising alternatives to traditional interventions. Proposed benefits include ease of use, accessibility, and the potential to be less stigmatizing [3-5]. This may be particularly appealing for service users with psychosis who tend to have high relapse rates yet limited access to psychological therapies [4,6]. Psychosis is a debilitating mental health disorder that includes symptoms such as hallucinations, delusions, disorganized thoughts, and speech, as well as diminished emotional expression and lack of volition [7]. Dropout and nonadherence rates for traditional psychological and psychopharmacological interventions are high for people with psychosis. “Dropout” is defined as noncompletion of the study protocol or the study assessments, and “adherence” is defined as the extent to which a participant experiences or engages with a mobile or Internet-based intervention [8]. Dropout rates of 25% for people with psychosis [9-11] and 30-57% for people with first episode psychosis (FEP) are commonly found [12]. Some have suggested that e-mental health technologies may provide a more acceptable therapy format than traditional face-to-face therapy [13]. Rates of adherence across different types of e-mental health interventions for people with psychosis have not been systematically examined.

A recent review of 12 studies highlighted that a specific examination of adherence, the extent to which a participant engages with an intervention, would be helpful for the field of e-mental health [14]. The study demonstrated that service users with psychosis varied in their engagement with the technological interventions; some showed regular or intermittent use and approximately 25-30% of participants did not engage or dropped out [14]. We seek to update this 2013 review for two main reasons. First, since 2013, there has been a dramatic increase in peer-reviewed publications examining Web-based or mobile technologies for a variety of mental health conditions. When reviewing the publication rate of e-mental health papers over the past 20 years, 57% were published in the last 5 years and the number of publications tripled between 2009-2014 [15]. Higgins and Green (2011) recommend that review updates should be carried out every 2 years, especially in a rapidly growing field [24]. Second, examining use and adherence to these new technologies is increasingly important as the benefits are limited if service users do not use them.

In order to obtain an overview of the rates of adherence, two types of adherence rates were collected: (1) mean percentage of the intervention completed and (2) percent of participants that complete the intervention [16]. Previous systematic reviews have developed four main approaches to examining adherence to mobile or Web-based interventions for treatment of depression and anxiety [8,16] (see Table 1 for an overview). The first is to examine factors that contribute to dropout from a study; for example, a comparison of baseline symptomology or demographic factors in participants who stay in the study and those who drop out. The second is to conduct statistical analyses, including correlational or regression analyses within a study to identify potential predictors of adherence. Specific service user factors (eg, demographics and clinical severity) and intervention factors (eg, week 1 vs week 2 of intervention) are most commonly explored. The third is to use questionnaires to retrospectively examine participants’ experiences of adherence and perspectives on continued use. The fourth approach is to experimentally manipulate factors within a study to impact upon adherence; for example, to compare different technological interfaces, frequency of use, or behavioral interventions.

In addition to these four approaches to studying adherence, we evaluated two theoretically proposed predictors of adherence: (1) level of social presence or contact with a researcher, clinician or peer, and (2) servicer user involvement in the development of the intervention or app. The level of social presence or contact refers to the frequency and quality of clinician, researcher, or peer presence or contact throughout the intervention [14]. Several studies have identified that contact and support from clinicians or peers in the form of telephone, email, Web-based forums, or e-chats can help improve adherence to mobile and Internet-based interventions; people with psychosis may particularly benefit from this support [17,18]. Mohr et al [19] outline a “supportive accountability model” whereby a supportive social presence may positively influence accountability, expectations, and bond during a mobile or Web-based intervention. This predictor has some credibility as Day et al [20] found that for acute inpatients with psychosis, a positive relationship with a clinician was related to adherence to medication and a positive attitude toward treatment. In addition, LeClerc et al [11] established that a good therapeutic alliance improved adherence to psychosocial treatment. This review conducted a preliminary examination of the level of social presence and support that is offered in each intervention.

The second potential predictor of adherence is the level of service user involvement in the development of the intervention. This has been highlighted as vital for effectiveness and adherence to interventions [21]. The sense of involvement in the project may promote self-efficacy and therefore accountability to the intervention [19]. Recently, Wykes and Brown [21] emphasized the importance of providing service users with choice, for example, the choice of digital or face-to-face interventions, or a combination of the modalities [22]. Choice leads to a greater feeling of control; this may tap into intrinsic motivation that is important for adherence to interventions [19]. This review highlights any studies that involve service users in the development and improvement of the interventions and the potential impact on adherence. This review updates Alvarez-Jimenz et al’s [14] findings; we examined rates of adherence to mobile or Internet-based interventions, trials, or observational studies for people with psychosis.

Methods

This systematic review was conducted following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines and recommendations for conducting and reporting systematic reviews (see Multimedia Appendix 1) [23].

Eligibility Criteria

The following PICOS criteria [24] were adopted for study inclusion: (1) population: adults (18-65 years); at least 75% of participants have a diagnosis of schizophrenia spectrum disorder according to diagnostic and statistical manual of mental disorders (DSM)-IV or the international classification of diseases (ICD)-10; (2) interventions, trials, or observational studies involving Web-based, mobile, e-technology or Web-based interfaces enabling peer-to-peer contact, patient-to-expert communication, or interactive psycho education or therapy; flexible, accessible monitoring, self-help, and symptom management; (3) comparison group: none were specified; (4) outcomes: at least one measure of adherence or dropout; and (5) study design: as this study aimed to provide an overview of the current state of the field, generous inclusion criteria were adopted. Types of studies included all primary group studies including RTCs; cross-sectional, longitudinal, and comparison studies with and without a control group; cross-over trials, case controls or cohort studies; observational studies with experience sampling components (ESM); and feasibility or acceptability studies. The following exclusion criteria were included: (1) publications written in a language other than English, (2) conference abstracts and theses not published in a peer-reviewed journal, and (3) book chapters.

Information Sources and Search Strategy

The following databases were systematically searched from August 2013 to November 2016: OVID including MedLine, EMBASE and PsychInfo, Pubmed and Web of Science. The following terms were used in the keyword search of abstracts and titles: (Internet or online or Web-based or website or mobile) AND (bipolar disorder or manic depression or manic depressive illness or manic-depressive psychosis or psychosis or schizophr* or psychotic). Additionally, hand-searching was performed on 5 key journals (Schizophrenia Bulletin, Schizophrenia Research, Journal of Medical Internet Research, Telemedicine and E-health, and Psychiatric Services) along with the reference lists of included primary studies. The term “adherence” was purposely not included in the search terms as most studies do not include references to reported adherence in the title or abstract [16].

‎View this figure

Study Selection

Titles and abstracts of articles were scanned independently by 2 researchers (CK and ZH). Articles deemed potentially eligible were retrieved in full and independently reviewed (CK and ZH). Disagreement between researchers was dealt with by consensus with a senior member of the research team (TW).

Data Collection Process

A standard form was used to extract data from selected studies to create 7 results tables. Tables 2-7 comprise: (1) randomized intervention studies, (2) feasibility or acceptability studies, and (3) observational studies. Tables 2-4 include the following study characteristics: study source, sample size, gender, age, diagnosis, study design, purpose of intervention, and control group. Tables 5-7 include characteristics of interventions: levels of adherence, dropout, type of social presence, service user involvement, and measurement of participant feedback.

Assessment of Methodological Quality and Procedures

The study quality was assessed separately for RCTs, feasibility studies, and observational studies (nonrandomized studies). The RCTs and feasibility studies were separately assessed using the clinical trials assessment measure (CTAM) [25]. The CTAM was designed to assess trial quality specifically in trials of psychological interventions for mental health. It contains 15 items grouped into six areas that are important for assessing bias in psychological interventions that include sample size, recruitment method, allocation to treatment, assessment of outcome, control groups, and description of treatments and analysis. Each study is rated out of a total of 100. This scale has good interrater reliability (.96) and high concurrent validity (.97) [26]. Observational studies were assessed using the Downs and Black scale [27]. This scale comprises 27 questions assessing key areas of methodological quality for nonrandomized studies for systematic reviews. It includes questions on reporting, external validity, bias, confounding, and power. This scale was modified slightly for this study. The question on power (see question 27 of the scale) was simplified to a rating of 1 or 0 following the practice in other reviews [2,28]. Each study is rated out of a total of 28 points. Scores are classified in the following ranges: excellent 26-28, good 20-25, fair 15-19, and poor less than 15. Two reviewers (CK and ZH) independently assessed the quality for all of the included studies. All of the first authors of the included articles were contacted to approve their CTAM or Downs and Black rating and if necessary provide further information to ensure that the quality of the study was not confused with the quality of the reporting.

Results

Study Selection

The search strategy returned 2639 titles and abstracts. After removal of 797 duplicates, 1842 titles and abstracts were screened and 108 full text papers were assessed for inclusion. In total, 20 studies met the inclusion criteria (see summary in Figure 1; PRISMA flowchart).

Study Characteristics

Study characteristics are summarized in Tables 2-4. Six were randomized controlled interventions; 7 were feasibility, acceptability studies; and 7 were observational studies. In total, 656 participants with a diagnosis of schizophrenia spectrum disorders and a mean age ranging from 20-48 years participated. Sixteen studies included individuals with schizophrenia or schizo-affective disorder, 1 study included people with first episode psychosis, 1 included individuals with a dual diagnosis of schizophrenia and substance misuse, and 2 included people with nonaffective psychosis.

Quality Assessment

Trial quality assessment scores are summarized in Tables 8 and 9. The mean study quality score for the RCTs on the CTAM was 77.3 (range 62-88). Five [31-35] of the RCT studies were deemed to be of adequate trial quality (rating of 65+), except for Palmier-Claus et al [29], which received ratings of 62. As expected due to the lack of randomization, feasibility studies (n=7) had a lower mean score (44.7). The mean quality rating for the observational studies, was 20 and ranged from 17- 24. Three studies fell into the “good” classification range and 4 were “fair.”

Adherence: Types of Measurement Across Studies

The most common measures of adherence were percent of intervention completed by participants and percentage of participants completing the intervention. Figure 2 displays the types of adherence measure used and the level of adherence for each study. For the 12 studies reporting mean % of the intervention completed by participants, adherence ranged from 70.7-98.0% with a mean of 83.4%. For the 8 studies reporting the percentage of participants completing the intervention, adherence ranged from 28- 100% with a mean of 74.3%. All of the studies also listed the number of participants that dropped out of the study. This ranged from 0-55% with a mean of 12.3% dropout across both observational and intervention studies.

‎View this figure

Approach 1: Analysis of Dropout

See Tables 5-7 for details of rates of dropout. Five studies analyzed the relationship between specific variables and dropout. In terms of the variables of age or gender and dropout, most of the studies found no relationship [29,41,42]; however, Van der Krieke et al [31] found that the dropouts tended to be younger and male. Hartley et al [41] and So et al [42] did not find a relationship between symptom severity and dropout; however, Palmier-Claus et al [29] (also reported in the original study [30]) found that higher severity on the positive and negative syndrome scale’ (PANSS) positive symptom subscale predicted dropout. Finally, Sanchez et al., [43] found that the level of cognitive functioning did not predict completion of the study. See Table 10 for a summary.

Approach 2: Analysis of Within Study Predictors of Adherence

Six studies conducted within-study analyses to examine adherence predictors and found few significant predictors of adherence. Van der Krieke et al [31] analyzed the chronicity of symptoms and reported that service users with first episode psychosis used a Web-based decision aid autonomously more often than service users with chronic psychosis. For those who required assistance from the research team to complete the intervention, 56% were service users in long-term care. However, the report does not provide specific statistical data.

In terms of intervention specific factors, Palmier-Claus et al [29,30] found no relationship between the length of time taken to complete an entry and the number of entries completed by an individual. They also examined number of entries completed across the number of weeks of the study. They found that more entries were completed in the first week than the second week of the intervention and participants rated more highly the question, “were there times when you felt like not answering?” during the second week.

Approach 3: Poststudy Questionnaires on Participants’ Perspectives on Adherence

11 studies retrospectively asked participants to provide questionnaire-based qualitative or quantitative feedback about their experience of the study or intervention. All the studies used different rating scales (eg, Treatment Experience Questionnaire in Smith et al [33], idiosyncratic quantitative feedback questionnaire in Palmier-Claus et al [29], and idiosyncratic SocialVille program rating in Nahum et al [36]) and therefore it is difficult to draw comparisons across studies. Four studies specifically asked whether participants would continue to use the intervention [33,35-37]; see Figure 3). For 4 studies, the mean percent of participants who agreed to continue to use the intervention was 73.1%.

‎View this figure

Approach 4: Analysis of Specific Intervention Manipulations and Effect on Adherence

Two studies were designed to manipulate conditions that may have an impact on adherence. Palmier-Claus et al [29,30] compared two different types of interventions: SMS text-only (short message service, SMS) interface or a mobile phone–based graphical app. They assessed the acceptability and feasibility of each device and found that participants completed more data points when using the mobile phone interface (mean entries=16.5) compared with the SMS text-only interface (mean entries=13.5; P=.002). Schlosser et al [40] increased the frequency and intensity of contact from a research coach from once a week to 5 times a week. This led to improved rates of adherence, for example, number of logins increased from 3.51 days/week to 4.69 days/week.

Interestingly two interventions found that adherence significantly affected the intervention efficacy. Smith et al [33] found that completing more training trials of a virtual reality job interview training correlated with fewer weeks searching before securing a job (P<.001) and greater self-confidence (P=.03).

Ben-Zeev et al [38] analyzed symptom change throughout the intervention and any related association to adherence and found that change in participants’ Beck Depression Inventory (BDI) scores were significantly correlated with use of mobile intervention; less frequent use of the FOCUS mobile intervention was associated with a the greater the reduction in depression score. Change in PANSS scores was not associated with use of the FOCUS app.

New Potential Predictors of Adherence

Potential Predictor: Social Presence Analysis

To assess Mohr et al’s [19] “supportive accountability” model (social presence leads to better adherence), we examined the amount of contact for each study and the level of adherence to the intervention. As there is heterogeneity across the studies, we provide a narrative synthesis. Across all 20 studies the mean number of contacts per week from a researcher or clinician was 4.4 and it ranged from 0-28 contacts per week. This included face-to-face, mobile, Web-based or telephone-based contacts.

As presented in Figure 4, regardless of level of support there is still a moderate to high rate of adherence across all 20 studies. Interestingly, it appears that studies with very high contact have almost 10% higher rates of adherence (83.8%) than those with no support (71.1%), but studies with minimal contact also had high adherence ratings (82.5%). Anecdotally, the importance of social presence is confirmed from participant reports. Gleeson et al [37] found that 90% of participants cited the use of a Web-based facilitator contributed to their sense of safety when using the Web-based program. All participants either agreed or strongly agreed with statements such as they always felt supported by the Web-based facilitator and 60% reported an increase in feelings of social connectedness. Recently, Schlosser et al [40] found that increasing the frequency of contact with a research coach increased use of the mobile app PRIME significantly. They found that when service users were able to tailor the amount of social support they received, they engaged more with the app.

‎View this figure

Potential Predictor: Service User Involvement

Of the 20 studies included, only 3 described service user involvement in terms of the development or initial piloting of the intervention.

Coproduction, meaning the collaboration of service users and researchers, in the beginning phases of intervention development has a potential influence on participants’ perception and adherence to the intervention. Ben-Zeev et al [38] used feedback and recommendations from a pilot with service users to develop a mobile intervention, FOCUS, to facilitate real-time mobile illness self-management. They found that participants rated the intervention highly with 90% acceptability and the mean percent of entries completed was 86.5%. Gleeson et al’s [37] HORYZONs program was developed with a service user focus group. It was found that 95% of participants used the social media component, 60% completed the therapy modules, and 75% reported a positive experience with the program. Schlosser et al [40] used an iterative service user feedback process called user centered design (UCD) process. After using the mobile app for 1 week, service users were consulted by means of in-depth interviews about their experience and identified key areas for improvement. The recommended changes were incorporated into the design of the device and this led to a 2 to 3-fold increase in use of the app in week 2. Service users also rated the app at 8 out of 10 in terms of satisfaction. In this case, service users were directly involved in the design, development, and implementation of the new device. When compared with adherence ratings (mean rate of adherence across studies that used different types of adherence ratings) to feasibility studies or RCTs that did not involve service users (mean adherence rate of 78%), service user involvement was associated with higher adherence (mean of 89%), though this is a small number of studies (n=3).

Additional Potential Predictor: Duration of Study

Interestingly, a comparison of the duration of the study (number of days participants are expected to be active in the study) and levels of adherence (averaged across both types of adherence ratings) revealed that the studies with the shortest duration had the highest mean rates of adherence (see Figure 5). The duration of the ESM-based studies ranged from 1 day to 14 days and the mean rate of adherence for these studies was 82.7%. Conversely, the duration of the RCT studies ranged from 6-161 days with a mean adherence rating of 76.4%; the duration of feasibility studies ranged from 7 -84 days with a mean adherence rating of 79.7%.

‎View this figure

Discussion

Principal Findings

This is the first review to document rates of adherence and to explore predictors of adherence to mobile and Web-based interventions for people with psychosis. Overall, from the examination of the four approaches to studying adherence across these diverse studies, we conclude that adherence to mobile and Web-based interventions is not necessarily predicted by service user specific factors such as age, symptoms, or gender. However, people with FEP may prefer an intervention that they can independently access [31]. Additionally, adherence is moderate to high across specific intervention factors such as amount of time to complete an entry and across different study designs. However, service users may prefer the mobile phone interface and may adhere more in the first week of an intervention [29]. This review has important implications for the acceptability and use of current interventions and the development of new ones. For example, offering service users choice in terms of the duration of the intervention and also the mode of delivery may have an important influence on adherence. Some service users may prefer a mobile app whereas others prefer a Web-based platform. Two potential new predictors of adherence were explored: (1) more frequent social support and (2) service user involvement in the intervention development. Providing service users with more input and control may add to the value and use of these interventions.

The Measurement of Adherence

Overall, adherence rates (whether measuring mean percentage of the intervention completed or percent of participants that complete the intervention) to mobile and Web-based interventions for people with psychosis are in line with adherence rates for similar technology-based interventions for other mental health disorders. Rates of adherence to interventions for depression and anxiety are approximately 66% for self-care interventions [16], and a median 56% for a computerized cognitive behavioral therapy (CBT) intervention [44]. Rates for completion of a Web-based site for personality disorder ranged from 80-100% completion; social phobia reported 70-90% completion and the only post-traumatic stress disorder (PTSD) intervention reported completing rate of 64%. In terms of adherence across different types of interventions (eg, face-to-face; medication-based interventions), completion rates of a one-to-one CBT intervention for psychosis was 55% [45] and 68.3% for a one-to-one CBT intervention for FEP [46]. Overall, the current review found moderate to high levels of adherence to Web-based or mobile interventions for psychosis with a range of 60-100% and a mean of 83%.

In terms of the four approaches to studying adherence, the studies in this review varied in terms of the within-study predictors that are associated with adherence, questionnaires used to assess participants’ perspectives on factors impacting adherence, and whether or not they conducted any experimental manipulations to impact on adherence.

Predictors of Adherence and Dropout

Only 2 studies found specific predictors of adherence: less chronic symptoms [31] and a higher rate of adherence was found in the first intervention week than the second [29,30]. Although other predictors of adherence were examined (age, gender, cognition, negative symptoms, and persecutory delusions), none were found to have a significant effect. Two studies also found significant predictors of dropout: severity of symptoms [39], younger age, and male gender [31].

Complex analyses, such as the multiple regression analysis performed by Palmier-Claus et al [39] of specific predictors such as service-user factors (symptoms, socioeconomic factors, interpersonal factors, and cognitive factors) along with e-mental health intervention factors (complexity of the interface, cost, and access) should be a priority for future studies. This will inform which service-user group may adhere to different type of interventions.

One interesting area of future research would be to examine the duration, frequency, and intensity of the intervention and the effect that this may have on adherence. Studies that last for several months may have more variable adherence than those that last only 1 week. Additionally, longer adherence is not always synonymous with better outcomes. Palmier-Claus et al [29] found that the longer participants used the app, the greater the increase in their depression symptoms. This has important implications for future research; it could be that people will stop using the app as they improve and should therefore be given the opportunity to stop using the app when they have exceeded the benefit. Ultimately, it may be most effective to allow service users choice of the duration, frequency, or intensity of interventions. With supportive guidance, service users may best be able to decide whether or not a technology is helpful and supportive in their recovery.

Poststudy Questionnaires

Several studies used participant feedback questionnaires, however, they were all different; some were previously published but most were idiosyncratic and this variability also hindered comparison. A standard questionnaire specifically for Web-based and mobile interventions could provide detailed and comparable information on the service user perspective and experience. Additionally, more independent data collection, perhaps from service user researchers not associated with the study, may provide a more unbiased and critical view of the interventions (eg, [47]). The use of posttrial feedback should be a priority for future research studies.

Experimental Manipulation

Only 2 studies specifically manipulated variables in an attempt to influence adherence or use of the intervention. Both successfully improved adherence to the intervention (eg, mobile phone rather than text message based delivery; higher frequency supportive contact). Experimental manipulation of variables is vital particularly in terms of the types of technologies service users would prefer, the content of interventions and the level of independence, or clinician involvement in use of the intervention.

New Predictors of Adherence

“Support” in this review was defined liberally as any type of contact with a clinician or researcher involved in the study. Of the 20 studies, 14 reported some level of clinician or researcher contact. This ranged from very limited initial interaction with a researcher to multiple daily support calls from a dedicated mobile interventionist. It should be noted that 7 of the studies were designed as observation studies with ESM components. In this case, researcher or clinician contact may only occur if service users stop filling in the data. Additionally, ESM studies are usually very short so there is less time for absolute dropout. As evidenced by our comparison with adherence ratings grouped by the duration of the study, ESM studies tended to be the shortest studies with the highest adherence ratings.

At present, it is difficult to draw clear conclusions about the importance of support, as only 2 studies specifically reported data on the effect of the Web-based interventionists [37]. However, as demonstrated by Schlosser et al [40], when the amount of coaching support was increased during the second half of the intervention, it led to increased engagement. In the future, it would be interesting for studies to experimentally manipulate the level of support and then measure the impact on adherence, or correlate the ratings of therapeutic alliance in the intervention and the level of adherence. This will clarify the impact of social presence.

Alvarez-Jimenez et al [14] and Wykes and Brown [21] recommend that service user involvement in intervention development might be an important predictor of adherence. However, in the current dataset, only 3 studies included service users in the development of the intervention, so it is difficult to draw conclusions about the impact on adherence. However, adherence to these interventions was very high (84.9%, 86.5%, and 95%). This is an important area requiring future study.

Quality of Studies

As expected, the RCT studies were rated more highly (77.3%) than feasibility studies (44.7%). All of the studies had interventions carried out by independent assessors and had adequate handling and assessment of dropouts. Only 4 studies had outcome assessments conducted by assessors blind to group allocation. In terms of observational studies, these studies were classified as either fair or good in terms of the quality. Few studies (n=4) used a method of blind rating of outcomes. This is particularly important when assessing service user satisfaction with the intervention, as researcher involvement may unintentionally bias the ratings. Finally, it is difficult to compare study quality across feasibility, RCT, and observational studies. Currently there is no measure to assess the quality of feasibility studies. The CTAM and Downs and Black scales provide a useful reference point; however, direct comparisons are not possible. In the future, RCTs should be developed from the feasibility studies discussed here, to provide further, high quality support for these initial findings.

Strengths and Limitations of the Review and Recommendations

One of the main limitations of this study is the difficulty of comparing rates of adherence across studies with different interventions and different outcomes. Although most studies provided data either as percent of individuals completing an intervention or the mean percentage of an intervention completed, these two measures may not provide as accurate information when directly combined. A universal measure of adherence should be adopted in addition to more detailed information on the quantity or quality of adherence. For example, Simco et al [16] recommended including not just the percentage of an intervention completed but the number of exercises per week or log-ins per week to get a more qualitative perspective on use. Along these lines, it will be important for future reviews to separate and compare the modes of delivery in their analysis of baseline adherence levels. For example, the baseline rate of adherence to a mobile phone intervention may be different than for a Web-based intervention; comparisons across and within modes of delivery may provide insight into the types of technologies that are preferred. Finally, it will be important for future reviews to carefully document and unpick any potential risks of harm that service users may experience when using these remote technologies. Reviews should provide an unbiased account of both the benefits and disadvantages of remote interventions, for example, as highlighted by the finding by Ben-Zeev et al (2014b) that participants’ BDI scores were significantly correlated with use of mobile intervention; less frequent use of the FOCUS mobile intervention was associated with a greater reduction in depression score. This is an important finding that should guide further use of this intervention (eg, Ben-Zeev et al, 2016). Any potential negative effects should be carefully explored and documented.

The review provides a comprehensive, up-to-date review of adherence across a variety of intervention types and platforms. The strengths include assessing a broad range of different novel technological interventions from text message-based to Web-based to virtual reality-based programs. This allowed us to demonstrate that adherence across different types of studies and a diverse range of interventions is moderate to high. Although the choice between face-to-face and remote intervention was not examined, this result at least demonstrates potential clinical utility. This review is timely as we included up-to-date literature from the past 3 years to ensure that the reader is informed of the most recent developments. The review also provides an innovative exploration of theoretically proposed predictors of adherence. This is the first review of its kind to explore the importance of service user involvement and support in facilitating adherence.

We conclude that specific service user factors such as age or symptom severity may not have a significant influence on adherence; however, the experience of the service user in terms of the development of these technologies and interventions may be an important factor that requires care and consideration.