Usage and Dose Response of a Mobile Acceptance and Commitment Therapy App: Secondary Analysis of the Intervention Arm of a Randomized Controlled Trial

Background Mobile phone apps offer a promising medium to deliver psychological interventions. A mobile app based on Acceptance and Commitment Therapy (ACT) was developed and studied in a randomized controlled trial (RCT). Objective To study usage metrics of a mobile ACT intervention and dose-response relationship between usage and improvement in psychological flexibility. Methods An RCT was conducted to investigate the effectiveness of different lifestyle interventions for overweight people with psychological stress. This paper presents a secondary analysis of the group that received an 8-week mobile ACT intervention. Most of the analyzed 74 participants were female (n=64, 86%). Their median age was 49.6 (interquartile range, IQR 45.4-55.3) years and their mean level of psychological flexibility, measured with the Acceptance and Action Questionnaire II, was 20.4 (95% confidence interval 18.3-22.5). Several usage metrics describing the intensity of use, usage of content, and ways of use were calculated. Linear regression analyses were performed to study the dose-response relationship between usage and the change in psychological flexibility and to identify the usage metrics with strongest association with improvement. Binary logistic regression analyses were further used to assess the role of usage metrics between those who showed improvement in psychological flexibility and those who did not. In addition, associations between usage and baseline participant characteristics were studied. Results The median number of usage sessions was 21 (IQR 11.8-35), the number of usage days was 15 (IQR 9.0-24), and the number of usage weeks was 7.0 (IQR 4.0-8.0). The participants used the mobile app for a median duration of 4.7 (IQR 3.2-7.2) hours and performed a median of 63 (IQR 46-98) exercises. There was a dose-response relationship between usage and the change in psychological flexibility. The strongest associations with psychological flexibility (results adjusted with gender, age, and baseline psychological variables) were found for lower usage of Self as context related exercises (B=0.22, P=.001) and higher intensity of use, described by the number of usage sessions (B=−0.10, P=.01), usage days (B=−0.17, P=.008), and usage weeks (B=−0.73, P=.02), the number of exercises performed (B=−0.02, P=.03), and the total duration of use (B=−0.30, P=.04). Also, higher usage of Acceptance related exercises (B=−0.18, P=.04) was associated with improvement. Active usage was associated with female gender, older age, and not owning a smart mobile phone before the study. Conclusions The results indicated that active usage of a mobile ACT intervention was associated with improved psychological flexibility. Usage metrics describing intensity of use as well as two metrics related to the usage of content were found to be most strongly associated with improvement. Trial Registration ClinicalTrials.gov NCT01738256; https://clinicaltrials.gov/ct2/show/NCT01738256 (Archived by WebCite at http://www.webcitation.org/6iTePjPLL)

If this is a JMIR submission, please provide the manuscript tracking number under "other" (The ms tracking number can be found in the submission acknowledgement email, or when you login as author in JMIR. If the paper is already published in JMIR, then the ms tracking number is the four-digit number at the end of the DOI, to be found at the bottom of each published article in JMIR) no ms number ( Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms. 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The intervention was delivered through a stand-alone mobile application. One introductory face-to-face session was provided, but the meeting did not contain intervention content in itself.
Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial 1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The target group was not of great importance for this analysis.
1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.
Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis and the study design has been previously described, this is only briefly described. "An RCT was conducted to investigate the effectiveness of different lifestyle interventions for overweight people with psychological stress. This paper presents a secondary analysis of the group that received an 8-week mobile ACT intervention." Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "...of the group that received an 8-week mobile ACT intervention." There was human involvement in the introductory session, but no actual intervention was provided there, only an introduction to ACT principles and the mobile application.
Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional offline trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, we feel this level of detail is not needed in the abstract. The procedures will be summarized in the methods section of the main article. A more detailed description is provided in (Lappalainen et al. 2014).
Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Use data are reported in the abstract. "The median number of usage sessions was 21 , the number of usage days was 15 (IQR 9.0-24), and the number of usage weeks was 7.0 (IQR 4.0-8.0). The participants used the mobile app for a median duration of 4.7 (IQR 3.2-7.2) hours and performed a median of 63 (IQR 46-98) exercises." Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, we only report results within the mobile intervention group.

INTRODUCTION 2a) In INTRODUCTION: Scientific background and explanation of rationale
Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program? Intended for a particular patient population? Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions? (Note: Details about the intervention are provided in "Methods" under 5) 1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Problem: "..., it is still unclear how the usage of interventions should be measured, how usage is associated with benefits, and how much interventions should be used in order to gain health benefits." Scientific background, rationale: What is known about the (type of) system that is the object of the study (be sure to discuss the use of similar systems for other conditions/diagnoses, if appropiate), motivation for the study, i.e. what are the reasons for and what is the context for this specific study, from which stakeholder viewpoint is the study performed, potential impact of findings [2]. Briefly justify the choice of the comparator.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This study focuses on investigating the associations between different metrics and outcomes, and therefore we provide definitions and literature on usage and adherence: "The definitions and metrics used for describing usage vary between studies. Many studies describe usage in terms of adherence, which has been adopted from health care and means the extent to which a person's behavior follows instructions or recommendations from a health care provider [7]. Christensen et al [8] defined adherence as "the extent to which individuals experience the 2b) In INTRODUCTION: Specific objectives or hypotheses * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study All objectives are presented at the end of the introduction: "The aims of this study were (1) to investigate the dose-response relationship between usage and the change in psychological flexibility, (2) to identify the usage metrics that were most strongly associated with improvement in psychological flexibility, and (3) to study the associations between usage and baseline participant characteristics." METHODS 3a) Description of trial design (such as parallel, factorial) including allocation ratio * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, this level of detail is not needed.
3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Bug fixes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug fixes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have influenced study design such as staff changes, system failures/downtimes, etc. [2].
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The mobile application was a stand-alone application, running locally in the mobile phones borrowed to participants. No changes were made during the 8-week intervention period.
4a) Eligibility criteria for participants * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, only summary provided here: "The inclusion criteria were 1) age between 25 and 60 years, 2) body mass index (BMI) 27-34.9 kg/m2, 3) psychological stress (at least 3/12 points in General Health Questionnaire (GHQ-12; [27]), and 4) the possibility to use a computer and internet connection." A more detailed description is provided in (Lappalainen et al. 2014).
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clarified.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study One of the inclusion criteria for the RCT was the possibility to use a computer and internet connection. We measured and reported the participants' prior experiences with using technologies.
Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, only a summary is provided here.
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis of the main trial, only a summary is provided here.
4b) Settings and locations where the data were collected * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, only summary provided here.
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The measurements were collected during visits at study centers as well as through Web-based questionnaires.
Report how institutional affiliations are displayed to potential participants [on ehealth media], as affiliations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results) 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/1KlxFl4iTrxRIWADX-jCukJwHv4lE...
As this is a secondary analysis, this is not reported in the article. However, the institutional affiliations were displayed to the participants in the recruitment advertisements and informed consent forms.
5) The interventions for each group with sufficient details to allow replication, including how and when they were actually administered Mention names, credential, affiliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conflict of interest" section or mentioned elsewhere in the manuscript).

2 3 4 5 subitem not at all important essential
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The authors involved in the development of the application are listed in Conflicts of Interest.
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/1KlxFl4iTrxRIWADX-jCukJwHv4lE...
We have described the history of the application and previous evaluations in an earlier article and a reference is provided in this article. "The mobile intervention group received their ACT intervention through a mobile app called Oiva, which has been previously described by Ahtinen et al [37]." Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Researchers had no access to the app during the intervention, and the app and its contents remained constant during the intervention." Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The application was based on Acceptance and Commitment Therapy (ACT) and the contents were designed by psychologists specializing in ACT. Majority of the exercises were based on previously published and widely used and studied ACT exercises. Examples and references are provided.
Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientific reporting.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We provide a description of the application with screenshots, reference an earlier article that describes the application in detail, and provide links to the current versions of the application.
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The web version of the current Oiva application is archived with webcitation.
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers /readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "In the beginning of the study, the participants were invited to a 1.5-hour group meeting where a trained psychologist gave a 30-minute presentation about the principles of ACT. The features of the mobile app were introduced by a researcher. The participants were given Android mobile phones (ZTE Blade or ZTE Skate; ZTE Corporation, Shenzhen, China) with the app preinstalled along with printed user instructions. The participants were allowed to use the phones as their personal phones, but as the intervention period was only 8 weeks, it was not expected of subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study A summary is provided, as a detailed description is provided in an earlier article (Ahtinen et al., 2013). The theoretical framework (Acceptance and Commitment Therapy) is described in the Introduction section. The mobile intervention is briefly presented in the Methods setion.
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/1KlxFl4iTrxRIWADX-jCukJwHv4lE...
"The participants were encouraged to use the app independently and actively, a few times a week, for the following 8 weeks." Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study In the beginning of the study, the participants were invited to a 1.5-hour group meeting where a trained psychologist gave a 30-minute presentation about the principles of ACT.
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The reminder mechanism has been presented in Ahtinen et al. 2013. As it seemed to be unnoticeable to most users, we have not highlighted it here.
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "...a trained psychologist gave a 30-minute presentation about the principles of ACT." 6a) Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As this is a secondary analysis, we only provide a summary of the measures used in this study.
If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].
All used psychological questionnaires are described with example items and scoring, and references are provided. Questions regarding prior technology use are described.
Describe whether and how "use" (including intensity of use/dosage) was defined/measured/monitored (logins, logfile analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from manuscript text "The mobile app recorded usage log files locally in the mobile phone. The log files were obtained from the phones at the end of the intervention period. The log files were analyzed to extract altogether 15 usage metrics, detailed in Table 1." Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups). 6b) Any changes to trial outcomes after the trial commenced, with reasons indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No changes were made. 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed Describe whether and how expected attrition was taken into account when calculating the sample size.

2 3 4 5 subitem not at all important essential
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This item is out of scope for this secondary analysis. Sample size determination was described in the article on the trial design (Lappalainen et al. 2014).
7b) When applicable, explanation of any interim analyses and stopping guidelines * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No such analyses or guidelines were in place.
CONSORT-EHEALTH (V 1.6.1) -Submission/Publication 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

2 3 4 5
subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

See details in Lappalainen et al. 2014.
Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator". 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Out of scope for this article, as this article is a secondary analysis.
12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The dose-response relationship between the usage metrics and the change in psychological flexibility was analyzed using linear regression analyses on continuous variables. First, univariate linear regression models were used, and then potential confounders (age, gender, baseline GHQ-12 score, and baseline AAQ-II and AAQW scores) were added to the models." "To assess the effects of usage between those whose psychological flexibility improved and those whose flexibility did not improve, binary logistic regression analyses were conducted. The

X26) REB/IRB Approval and Ethical Considerations
[recommended as subheading under "Methods"] (not a CONSORT item) 1 2 3 4 5 subitem not at all important essential CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/1KlxFl4iTrxRIWADX-jCukJwHv4lE... indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The RCT study was approved by the ethics committee of the Central Finland Health Care District" Outline informed consent procedures e.g., if consent was obtained offline or online (how? Checkbox, etc.?), and what information was provided (see 4a-ii). See [6] for some items to be included in informed consent documents.

2 3 4 5 subitem not at all important essential
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "...written informed consent was obtained from all participants. The approval and consent also covered the secondary analyses performed:" Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study These are reported for the full RCT: "Out of the 645 individuals who responded to the advertisements, 339 fulfilled the inclusion criteria and consented to participate. After baseline examinations, further 41 participants declined to participate or were excluded because of findings in baseline measurements. Thus, 298 participants started the actual study. " And the analysed mobile intervention group: "Altogether, 85 of 339 (25.1%) participants were randomized to the mobile intervention group.
13b) For each group, losses and exclusions after randomisation, together with reasons * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study These are reported both for the full RCT and the mobile intervention group. Full RCT: "After baseline examinations, further 41 participants declined to participate or were excluded because of findings in baseline measurements." Mobile group: "...Of these, 78 (92%) received the intervention, and 75 (88%) participated in postintervention measurements. One participant did not provide postintervention results on psychological flexibility and Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement. 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Usage parameters are reported in the article for the whole group and for improvers and nonimprovers.
14a) Dates defining the periods of recruitment and follow-up * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The RCT consisted of a 10-week period during which the interventions were delivered, followed by a 26-week follow-up period. The recruitment began in August 2012 and ended in February 2013. The last follow-up measurement was performed in December 2013.
Indicate if critical "secular events" fell into the study period, e.g., significant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No important secular events.
14b) Why the trial ended or was stopped (early) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The trial was run as planned.

15) A table showing baseline demographic and clinical characteristics for each group
NPT: When applicable, a description of care providers (case volume, qualification, expertise, etc.) and centers (volume) in each group * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Relevant baseline characteristics of the analysed mobile intervention group are presented in Table 2.
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The characteristics (age, gender, education and prior experiences in using technologies) are described in Table 2. 16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Denominators reported in each analysis.
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This analysis is only secondary analysis, and only mobile intervention group participants are analysed.
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Levels and changes of psychological flexibility, confidence intervals, and effect sizes are reported for the whole group and subgroups.
In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a).
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Usage metrics are reported in detail, and several different metrics were utilized and studied in the article, as studying usage was the main aim of the study. Tables 3 and 4 present the detailed metrics and their associations to improvement in psychological flexibility.
17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable.
18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study All analyses were detailed in 12 b, A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not study only users, but all mobile intervention group participants.

19) All important harms or unintended effects in each group
(for specific guidance see CONSORT for harms) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study
but also incidents such as perceived or real privacy breaches [1], technical problems, and other unexpected/unintended incidents. "Unintended effects" also includes unintended positive effects [2].
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

None
Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Out of scope for this paper.

22)
Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study All principal findings are discussed. "A dose-response relationship between usage and the change in psychological flexibility was found. The usage metrics that were the most strongly associated with the change in psychological flexibility were related to the intensity of use, that is, the number of usage sessions, the number of usage days, the number of usage weeks, the total number of exercises performed, and the total duration of use." "Logistic regression analyses confirmed these findings. However, the R2 Highlight unanswered new questions, suggest future research.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Future research should strive to study the usage of mobile apps in a more natural way, including using the participants' personal mobile phones and not limiting usage time. The results provided in this study about the associations between usage of different types of exercises and outcomes highlight the capabilities of digital interventions to enable detailed analyses of what the participants are actually exposed to during interventions." 20) Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
1 2 3 4 5 subitem not at all important essential * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Limitations are addressed in Discussion. "The study population consisted of participants who volunteered to take part in the RCT. Although they were randomly allocated to different interventions and therefore did not know which intervention, if any, they would receive, the group is likely to be biased and the results cannot be generalized. Volunteering in a research study often requires a lot of effort from the participants and they may therefore be highly motivated. Also, the large proportion of female participants, which is not uncommon in health trials, 21) Generalisability (external validity, applicability) of the trial findings NPT: External validity of the trial findings according to the intervention, comparators, patients, and care providers or centers involved in the trial Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations 1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Generalizability is discussed in Discussion: "The study population consisted of participants who volunteered to take part in the RCT. Although they were randomly allocated to different interventions and therefore did not know which intervention, if any, they would receive, the group is likely to be biased and the results cannot be generalized. Volunteering in a research study often requires a lot of effort from the participants and they may therefore be highly motivated. Also, the large proportion of female participants, which is not uncommon in health trials, Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Regarding the design of the RCT, the duration of the mobile intervention (8 weeks) was arbitrary and does not correspond to real-life use of mobile apps. Because the participants had to return the phones at the end of the intervention period, there is no way of knowing how the usage would have evolved over time and whether additional benefits would have been gained as a result. On the other hand, the knowledge of the limited availability may also have motivated the participants for such active use. Also, the fact that the app did not run on the OTHER INFORMATION 23) Registration number and name of trial registry * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The trial was registered at ClinicalTrials.gov with the identifier NCT01738256." 24) Where the full trial protocol can be accessed, if available * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

25)
Sources of funding and other support (such as supply of drugs), role of funders * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

X27) Conflicts of Interest (not a CONSORT item)
In addition to the usual declaration of interests (financial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
1 2 3 4 5 subitem not at all important essential Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The mobile ACT intervention app was developed in collaboration between VTT Technical Research Centre of Finland and University of Jyväskylä. The authors involved in the development of the app were EM, PV, ME, ES, PL, and RL.
About the CONSORT EHEALTH checklist * yes, major changes yes, minor changes no Providing links to the current versions of the mobile intervention application. Mostly the changes included making small clarifications throughout the article.