Usability of a Mobile App for Real-Time Assessment of Fatigue and Related Symptoms in Patients With Multiple Sclerosis: Observational Study

Introduction

Background

Multiple sclerosis (MS) is an inflammatory, demyelinating disorder of the central nervous system that currently affects nearly 750,000 people in the United States [1]. Fatigue is one of the most disabling symptoms, affecting more than 65% of patients [2]. Fatigue has been associated with disease progression [3], and its pharmacological management remains challenging [4,5].

The pathogenesis of MS-related fatigue is not well understood. Neurogenic, inflammatory, endocrine, and metabolic mechanisms have been proposed [6]. In addition, several other factors, such as comorbid depression, anxiety, and sleep abnormalities, as well as physical activity and medications, may interfere with the perceived level of fatigue [2,6]. In fact, different fatigue phenotypes may exist in different patients. The ability to distinguish patient phenotypes might impact the understanding of underlying mechanisms and enable personalized management of this debilitating symptom. As an example of the need to capture the circadian nature of mood symptoms, including fatigue, it has been suggested that most people may experience lower energy levels in the evening than in the morning [7], whereas in a subset of patients affected by major depressive disorder, mood symptoms might be worse in the morning than in the evening [8]. In addition, in a recent study, we demonstrated the advantages of classifying patients according to temporal patterns of fatigue when associating damage to select brain circuitries with fatigue in patients with MS [9-14]. Although these studies highlighted the relevance of assessing temporal patterns of fatigue, retrospective data only included long-interval (every 1-2 years), repeated measures of fatigue, which might not reflect pathophysiologically relevant patterns.

Objectives

We developed a mobile app to enable circadian assessment of fatigue and other mood symptoms, with the longer-term goal of identifying clinically and pathophysiologically relevant phenotypes of fatigue. As an example of its potential significance, this mobile app will enable us to test the hypothesis that diverse fatigue phenotypes may respond to different mechanism-specific treatments. Although several drugs have demonstrated efficacy in improving wakefulness in other conditions (such as narcolepsy [15]), none have been proven effective in treating fatigue in MS [4,5].

Several tools have been developed to assess various symptoms (including fatigue), monitor clinical status [14,16-20], or improve the perceived level of fatigue in patients with MS [16,19,21-23]. We believe that the presented mobile app meets a specific need for a tool to characterize fatigue phenotypes in MS by assessing the temporal patterns of fatigue and its comorbid mood symptoms.

In this study, we aim to test and use this mobile app in the framework of a prospective study that was designed to investigate the association between MS-related fatigue and structural brain damage. The overarching goal of this study is to identify brain magnetic resonance imaging (MRI) predictors of persistent and treatment-resistant fatigue in MS. The aim of this study is to describe the design of the mobile app and to report the first findings regarding patient compliance and the relationship between compliance and patient characteristics, including MS disease severity.

Methods

Study Population and Study Design

Patients with MS were recruited from the Comprehensive Longitudinal Investigation of Multiple Sclerosis at Brigham and Women’s Hospital (CLIMB) study [24] (n>2400), a large-scale, long-term study of patients with MS, using the following selection criteria: (1) availability of at least one previously recorded Modified Fatigue Impact Scale (MFIS) [25].

r Quality of Life in Neurological Disorders (Neuro-QoL) [26] score, (2) brain MRI scan acquired within 1 month of recruitment into the study, and (3) absence of clinical exacerbation within 3 months before their MRI. Selected patients were mailed a recruitment letter at least 2 weeks before their scheduled clinical visit at Brigham and Women’s Hospital (BWH). The letter contained a brief description of our study procedures and invited patients who were interested in participating in our research study for an in-person study initiation session (SIS) following their scheduled clinical appointment or at another scheduled time at BWH. The aim of the SIS was to (1) provide patients with a detailed description of our study; (2) obtain a written informed consent form; and (3) teach patients how to use the following 3 study devices: the abovementioned mobile app installed on a mobile device, a wrist-worn actigraphic MotionLogger watch [27], and a Nox T3 home sleep test (HST) device [28]. The actigraphic watch assessed physical activity during the daytime and sleep quality at night throughout the entire study (ie, for 2 weeks), whereas the HST device was used to assess sleep apnea and periodic limb movements at one night in the patient’s home. Data collected using the actigraphic watch and the HST device are not presented in this paper. Between May 2018 and September 2019, 64 patients with MS were recruited into the study and provided written informed consent in accordance with our study protocol approved by the institutional review board of BWH. Demographic data of these patients are presented in Table 1. Participants who answered all questions and returned all study devices to our laboratory received a remuneration of US $100.

Design of the Mobile App

Our research team, comprising physician scientists, designed and developed a mobile app in collaboration with the software developer team of Mobilengine [29]. Our teams communicated through virtual workshops. The app was installed on an Android smartphone with a 5-inch high-definition display that was provided to the participants. The participants used the smartphone for 2 weeks, during which they followed their normal daily routine. They then returned their smartphones to our research laboratory. The mobile app features 3 modules, which we describe in detail below, and in Multimedia Appendix 1: (1) a series of one-time questionnaires, to be answered within 3 days of enrollment (Table 2); (2) Visual Analogue Scales (VASs) for self-reporting of fatigue, depression, anxiety, and pain levels, every 4 hours while awake; (3) a sleep diary (SLD) with separate items to be completed before and after each sleep or nap, containing a series of questions regarding perceived duration and quality of sleep, as well as the same VAS described earlier. VAS and SLD data were consistently collected for 14 days, starting after all one-time questionnaires were answered. Therefore, the duration of the data collection varied between 14 and 17 days depending on how much time it took for the participant to answer all one-time questionnaires. Of note, subjects were allowed to pause their entries and return to complete the remainder of the questions at a later time.

The following measures were taken to prevent slip errors: (1) user control and freedom: the selected (or entered) answer was submitted only when the user tapped the forward (or backward) arrow on the screen; (2) flexibility: users can freely navigate forward and backward between questions within the same block (ie, the same questionnaire) and make corrections as needed; (3) consistent and standard user interface: every scale had the same size and all questions, answer options, and forward and backward arrows were located on the same part of the screen. In addition, the app also has a landscape mode, in which the distance between the points of the VAS is larger; and (4) visibility and system status: the number of completed questions out of the total number of questions was shown at the bottom of the screen.

One-Time Questionnaire Module

We included 10 questionnaires (Table 2) that were previously validated. In addition, we created 22 new questions to assess the relevant aspects of fatigue that were not captured by existing fatigue questionnaires, such as (1) circadian differences in fatigue severity and (2) effects of caffeine and nicotine use on the perceived level of fatigue, and (3) we incorporated open-ended questions aimed at exploring other potential aggravators and alleviators of fatigue (questions are presented on pages 16-20 of Multimedia Appendix 1). The one-time questionnaire module consisted of 168 questions. We preserved wording and the type of response (single choice, multiple choice, free text, analog scale, and selection of date) applied in the original questionnaires for each question included in our app (Figure 1). Questions were grouped into separate blocks, reflecting the original questionnaires. Patients were presented 1 question at a time but were able to freely navigate forward and backward between questions within the same block. Participants were notified at the end of each block and cautioned to be confident with their previous responses before final submission because they would no longer be able to return to a previous block of questions.

‎View this figure

VAS Module

The VAS [35-39] module was used to assess a subject’s current level of fatigue, anxiety, depression, and pain. Each of these 4 VASs is scored between 0 and 10, with 0 representing absence (none) of the symptom (eg, no fatigue) and 10 representing extreme presence of the symptom (Figure 2). Previous studies have shown a very high correlation between VAS and a series of line drawings of faces with expressions of increasing distress [40,41]. We added a faces scale under each VAS under the assumption that the combination of the 2 scales may yield more robust results. Reminder functions were implemented in the mobile app based on the results of previous studies that associated reminders with positive effects on engagement with digital behavior change interventions [42]. One of our main aims was to assess circadian changes in fatigue, anxiety, depression, and pain by measuring the level of these symptoms at least once in the morning, once in the afternoon, and once in the evening or night. Therefore, while awake, patients were prompted every 4 hours to complete the 4 VASs. Prompting was achieved through an acoustic and haptic (vibratory) alarm as well as a visual notification. In the absence of a response, the alarm and notification would be presented again after 30, 60, 90, and 120 minutes. If a subject did not answer a VAS within the allotted 2-hour period (between 4 and 6 hours after the previous VAS was completed), the subject would not be able to complete the VAS since beyond the allotted 2-hour period, the pending measurement would be closer in time to the next scheduled VAS assessment. Rather, they would be presented with notifications for a new VAS during an additional 2 hours (8 hours after the previous VAS had been completed).

‎View this figure

SLD Module

The SLD contains items to be completed before sleep and items required after awakening from night sleep or daytime naps. At the time of recruitment, patients were instructed to activate the SLD immediately before any planned sleep period. Before sleep, the patient was prompted to indicate whether they were taking a nap or going to sleep for the night and their intended wake-up time (Figure 3). Then, the 4 VASs were presented to the patient for completion before the sleep episode. Sound alarms and notifications were deactivated in the mobile app until 2 hours following the intended wake-up time provided by the patient. Patients were at liberty to wake up spontaneously or prompted by a preprogrammed device, such as an alarm clock. On awakening, the patient was required to select the “I just woke up” icon on their phone (Figure 3). If this option was not activated 2 hours after the intended wake-up time, sound, haptic, and visual alerts were activated. The “I just woke up” icon gives access to the wake items of the SLD that were developed by the Division of Sleep Medicine at BWH [43] and assesses final wake time, time awake before scheduled rising time, sleep latency, number of awakenings, sleep duration, subjective sleep quality, as well as current feeling of refreshed, sleepiness, and tenseness (wake items of the SLD are presented on page 21 of Multimedia Appendix 1). These items were followed by the day’s first VAS. Figure 4 shows the algorithm for the SLD and VAS assessments. If no sleep or nap option was selected on a given day, subjects were prompted by the app at 4 AM of the following calendar day, “We noticed that you did not indicate when you went to sleep last night. Please select the appropriate answer below:” with options for “I forgot to indicate when I went to sleep” and “I did not go to sleep.” If “I forgot to indicate when I went to sleep” was selected, the participant was prompted to enter an answer for “Time into bed” and to fill out an SLD.

‎View this figure

‎View this figure

The sequence of the questionnaires and questions was the same every time. All questions had to be answered before the questionnaire was submitted. At the bottom of the screen, a status bar indicated progress of the participant in the one-time questionnaires or VASs (Figures 1 and 2). The opening and submission dates and times were recorded for each question. All participants received an anonymized unique subject ID. No other identifying information was entered into the app.

Monitoring the Database

All functions of the app were available offline (ie, with no internet connection) without limitations. The app saved and stored all subject responses locally in the smartphone’s internal storage space. Each smartphone was provided with Wi-Fi and 3G (Third Generation) internet access to upload the collected data to Mobilengine’s designated server as soon as each item was completed. If the network connection was inconsistent, the data were transferred as soon as the connection was adequate. This system allowed us to monitor data collection in real time on demand. In addition, an email was sent daily at 4 PM Eastern time to designated investigators, listing the currently active devices with their IDs, as well as either (1) the progress of the subject in the one-time questionnaires or (2) whether the subject had filled out at least one VAS or SLD during the previous 24 hours. Subjects who did not complete their daily tasks are highlighted in red.

Following the completion of 14 days of VASs and SLDs, the subject’s monitoring period was concluded, and they no longer had the option to answer further questions. The subject code was removed from daily emails. The ability to monitor our subjects’ responses in this way allowed the investigators to promptly notify subjects who were not completing the study in the recommended time frame rather than waiting for the mobile device to be returned at the end of a 14- to 17-day period. We consider this monitoring feature to be an important part of mobile apps. All results in the database were easily exportable as a table with rows containing subject ID, individual question ID, individual question response, and time of response. Every answer or item created a new row in the table (ie, a long data format).

Assessment of Patient Compliance

Patient compliance was defined as adherence to the study protocol and was measured by the number of completed questions. We aimed to assess the association between the number of completed questions and demographic and clinical data. To this end, we grouped the recruited patients into the following 4 compliance groups: group 1: completed all one-time questionnaires, VASs, and SLDs; group 2: completed all one-time questionnaires but did not answer all VASs and SLDs; group 3: answered some, but not all one-time questionnaires and did not answer any VAS or SLD; group 4: did not answer any one-time questionnaire, VAS, or SLD. These groups were defined after the completion of patient recruitment. The following variables were compared between the 4 groups: age, sex, ethnicity, disease category (ie, relapsing-remitting multiple sclerosis [RRMS] and secondary-progressive multiple sclerosis [SPMS] ratio), disease duration, and physical disability (assessed using the Expanded Disability Status Scale [44]). These variables were assessed as part of the patients’ routine clinical visits. We also compared the Fatigue Severity Scale (FSS) [30] score between the abovementioned groups 1 to 3 (group 4 was not included in this analysis because patients in group 4 did not complete any questions) to assess whether baseline fatigue level was associated with patient compliance.

Statistical Analysis

Differences in demographic and clinical variables between the patient compliance groups were assessed using analysis of variance or Kruskal-Wallis tests for continuous variables (depending on the distribution of the data) and the chi-square test or Fisher exact test (when n<5) for categorical variables. The threshold for statistical significance was set at P<.05. All statistical analyses were performed using Stata Statistical Software: Release 13 (StataCorp).

Results

Patient Compliance

Of the 64 patients, 56 (88%) began the study (ie, answered at least one question), whereas 8 (13%) patients did not complete any questions (Figure 5). Of the 56 patients who began the study, 51 (91%) completed the one-time questionnaires and 47 (84%) completed both the one-time questionnaire as well as the 14-day VAS and SLD modules (Figure 5).

‎View this figure

Among the 51 out of 64 patients who completed the one-time questionnaires, 44 (86%) met the target of 72 hours, and the median time to completion was 6.19 hours, with a range from 21 minutes to 10.9 days (Figure 3). Of the 51 patients, 14 (28%) completed the one-time questionnaires within an hour, 20 additional patients (39%) completed it on the same day that they started it, and 5 additional patients (10%) completed the questionnaire within a 48-hour period (Figure 6).

‎View this figure

All subjects who completed the one-time questionnaire module also completed at least 2 days of VAS modules. Of the 51 subjects, 3 (6%) stopped after 8 days or less.

Study subjects submitted close to 4 VASs per day with a mean of 3.9 (SD 1.3).

Demographic and Clinical Comparison of Compliance Groups

There were no significant differences in age, sex, ethnicity, disease duration, and Expanded Disability Status Scale scores between the 4 patient compliance groups (Table 1). The RRMS/SPMS ratio was significantly higher in group 1 than in group 2 (P=.01; Table 1). There was no significant difference in the RRMS/SPMS ratios between the other groups. There was no significant difference in FSS scores between groups 1 to 3.

Discussion

Principal Findings

We developed and used a mobile app that is innovative because of the following aspects: (1) assessment of fatigue and its associated mood symptoms (ie, depression and anxiety) and pain using the VAS every 4 hours while patients are awake for 2 weeks; (2) patient prompting to assess their symptoms multiple times a day; (3) real-time symptom monitoring of patients with MS by researchers or treating physicians through a web-accessible portal; and (4) single time point assessments of fatigue and its significant confounders, such as depression, anxiety, physical activity, sleep problems, and motivation level by commonly used questionnaires along with several other questions that we created to assess the relevant aspects of fatigue that were not captured by existing fatigue questionnaires. Here, we report good patient compliance with our mobile app–based assessments, that is, 91% (51/56) completed all one-time questionnaires and 84% (47/56) completed all one-time questionnaires, VASs, and SLDs out of those patients who started using the app (n=56). Patients reported no issues with the usage of the app, and there were no technical issues with our web-based data collection system.

Limitations

The limitations of the study are as follows: (1) we did not systematically collect feedback about the app itself, its ease of use, or any patient suggestions about its features. However, our coinvestigators asked patients regarding their user experience and satisfaction when they contacted us (eg, because of the questions on the study protocol or the use of the study devices) or when our coinvestigators contacted the patients (eg, when the daily, system-generated email showed that a patient did not complete all questions on that day). Our high completion rate (47/56, 84% of those who began the study completed it) shows that many found it navigable and usable for the 2-week monitoring period. (2) The data collection and monitoring period was only 2 weeks. The observed patient compliance may change if the mobile app is used for a longer period (ie, beyond the novelty period) [45]. (3) The mobile app was installed on a smartphone that was provided to each participant. The use of a new device might have influenced the participants’ behavior. (4) The current version of our app was not gamified. We may consider using gamification strategies when developing the next version of the app to further increase patient compliance and improve user experience.

Comparison With Previous Work

Recently published studies underlined the relevance of delivery of health care through mobile devices (ie, mobile health) in MS [46,47]. Although several tools have been developed to assess various symptoms (including fatigue) and monitor clinical status [14,16-20,48] or to improve the perceived level of fatigue in MS [16,19,21-23], our app is the first that was specifically designed to conduct a state-of-the-art characterization of MS-related fatigue, including high-frequency fatigue assessments and real-time symptom monitoring.

To assess fatigue, 3 different self-assessment questionnaires (ie, MFIS [25], FSS [30], and Neuro-QoL fatigue [26]) and repeated daily VAS were implemented in our app. The MFIS has cognitive, physical, and psychosocial domains and addresses the effect of fatigue on daily activities [49]. The FSS addresses mainly physical fatigue with only one question related to cognitive fatigue [30]. The add-on value of the Neuro-QoL fatigue questionnaire is that it uses semantics to better characterize fatigue (eg, “felt exhausted,” “had no energy,” “felt fatigued,” or “felt tired”). Both FSS and Neuro-QoL assess fatigue in the past week, whereas MFIS refers to fatigue in the past 4 weeks. In addition, we formulated several new questions to assess the relevant aspects of fatigue that were not captured by existing fatigue questionnaires (eg, circadian differences in fatigue severity as well as potential aggravators and alleviators of fatigue).

Mood symptoms (ie, depression and anxiety) are highly intercorrelated with fatigue in MS [50,51], and their prevalence is over 20% in MS [52]. However, the relationship between MS-related fatigue and depression is not well understood [6,53]. Clinical symptomatology and, accordingly, clinical assessments or diagnosis of fatigue and depression show considerable overlap, as exemplified by (1) fatigue or loss of interest and diminished ability to think or concentrate, or indecisiveness are part of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, diagnostic criteria of major depression; (2) tiredness or fatigue, loss of energy, and concentration difficulty are items on the Beck Depression Inventory [54]; and (3) impairment in concentration, thinking, and decision making are items in the MFIS [25]. Overlap in the clinical symptomatology of fatigue and depression may raise the question of whether these symptoms reflect separate entities with different etiologies and pathophysiologies or share a similar pathogenesis. To investigate the association between fatigue and depression and anxiety, we included two one-time questionnaires (ie, Neuro-QoL depression and anxiety batteries [26]) in our app along with daily VAS for depression and anxiety. In addition, we included those questions from the Symptoms of Depression questionnaire [31], which specifically assesses vegetative symptoms of depression under the hypothesis that the presence of vegetative symptoms distinguishes clinically significant depression from phenotypes of fatigue that are not associated with depression.

Other relevant confounders of fatigue include sleep abnormalities [6], physical activity [55], and motivation [56]. Patients with MS have significantly more sleep disturbances than the general population [6]. The prevalence of sleep abnormalities is approximately 50% in patients with MS [6]. To assess sleepiness and alterations in sleep quality, a one-time questionnaire (ie, Epworth sleepiness scale [32]) and daily SLD were incorporated into our mobile app.

Patients with MS may have a sedentary lifestyle because of associated disabilities (eg, muscle deconditioning or pain), diminished physical endurance, and deterioration of symptoms that could follow physical exertion [55]. However, a growing body of evidence suggests that nonpharmacological approaches (eg, exercise or psychobehavioral interventions) might be more effective in reducing MS-related fatigue than medications [57]. We included a one-time questionnaire (ie, Godin Leisure-Time Exercise Questionnaire [33]) to assess physical activity and created targeted questions to assess the effect of physical activity, occupational, and psychological therapies on the patients’ perceived level of fatigue.

The central fatigue model of Chaudhuri and Behan [58] hypothesized that central fatigue is caused by damage to the nonmotor components of the cortico-striato-thalamic system. Indeed, a more recent study associated MS-related fatigue with alterations in reward responsiveness [59]. Furthermore, several neuroimaging studies have found damage to reward-related brain regions in fatigued patients with MS [2,56]. Therefore, our app includes a self-assessment questionnaire (ie, behavioral approach system or behavioral avoidance system scale [34]) to measure reward responsiveness, drive, and fun seeking.

Previous studies have shown that fatigue fluctuates over time, even throughout the day [13,14]. Temporal fluctuations of fatigue are understudied in MS. Our recently published studies showed that taking into account temporal fatigue dynamics may improve the characterization of brain pathological correlates of MS-related fatigue in MS [9-11,60]. We included 4 VASs in our mobile app to assess circadian changes in fatigue, depression, anxiety, and pain levels under the hypothesis that the circadian evolution of these symptoms may show different patterns, potentially representing different fatigue phenotypes with different etiologies and pathogeneses.

To increase patient compliance, most of the abovementioned apps have built-in reminder functions, similar to our mobile app. In addition, our system provides real-time monitoring of data collection for researchers and patient-treating physicians. None of the abovementioned apps or platforms provided real-time monitoring of data collection.

Conclusions

We developed a mobile app with reasonable patient compliance to assess fatigue and its confounders in patients with MS. This tool contains a battery of commonly used questionnaires and has the capability to monitor the level of subjective symptoms, such as fatigue, anxiety, depression, and pain, through VAS. We hope that it will facilitate remote monitoring of symptoms and adverse events and may allow more timely intervention than is possible with scheduled face-to-face visits.