Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 19.03.19 in Vol 7, No 3 (2019): March

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/10855, first published Apr 23, 2018.

This paper is in the following e-collection/theme issue:

    Review

    Training Cognitive Functions Using Mobile Apps in Breast Cancer Patients: Systematic Review

    1Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy

    2Applied Research Division for Cognitive and Psychological Science, European Institute of Oncology (Istituto di Ricovero e Cura a Carattere Scientifico), Milan, Italy

    *these authors contributed equally

    Corresponding Author:

    Laura Vergani, MSc

    Department of Oncology and Hemato-Oncology

    University of Milan

    Via Festa del Perdono 7

    Milan, 20122

    Italy

    Phone: 39 029 4372099

    Email:


    ABSTRACT

    Background: Breast cancer is an invalidating disease and its treatment can bring serious side effects that have a physical and psychological impact. Specifically, cancer treatment generally has a strong impact on cognitive function. In recent years, new technologies and eHealth have had a growing influence on health care and innovative mobile apps can be useful tools to deliver cognitive exercise in the patient’s home.

    Objective: This systematic review gives an overview of the state-of-the-art mobile apps aimed at training cognitive functions to better understand whether these apps could be useful tools to counteract cognitive impairment in breast cancer patients.

    Methods: We searched in a systematic way all the full-text articles from the PubMed and Embase databases.

    Results: We found eleven studies using mobile apps to deliver cognitive training. They included a total of 819 participants. App and study characteristics are presented and discussed, including cognitive domains trained (attention, problem solving, memory, cognitive control, executive function, visuospatial function, and language). None of the apps were specifically developed for breast cancer patients. They were generally developed for a specific clinical population. Only 2 apps deal with more than 1 cognitive domain, and only 3 studies focus on the efficacy of the app training intervention.

    Conclusions: These results highlight the lack of empirical evidence on the efficacy of currently available apps to train cognitive function. Cognitive domains are not well defined across studies. It is noteworthy that no apps are specifically developed for cancer patients, and their applicability to breast cancer should not be taken for granted. Future studies should test the feasibility, usability, and effectiveness of available cognitive training apps in women with breast cancer. Due to the complexity and multidimensionality of cognitive difficulties in this cancer population, it may be useful to design, develop, and implement an ad hoc app targeting cognitive impairment in breast cancer patients.

    JMIR Mhealth Uhealth 2019;7(3):e10855

    doi:10.2196/10855

    KEYWORDS



    Introduction

    Cancer is a major public health problem worldwide and the second leading cause of death in the United States. In 2017, Siegel and colleagues [1] reported 1,688,780 new cases and estimated 600,920 deaths. Breast cancer is one of the most common cancers around the world [2]. It is the most common cancer diagnosed in women, affecting about 1 in 8 women in the United States during their lifetime (12.4%), and it is the second most common cause of death by cancer in women [3]. The incidence rate is strictly associated with age: 95% of new cases are registered in women older than 40 years; only 1.5 cases per 100,000 are registered in young women (20 to 24 years). The incidence is higher for older women: 421.3 cases per 100,000 women are registered in women 75 to 79 years old; the median age at diagnosis is 61 years [3]. Prognosis is related to cancer stage: 5-year survival rate is 100% for stage 0 and I, 93% for stage II, 72% for stage III, and 22% for stage IV [3]. In the last 20 years, we have witnessed a decline in overall cancer mortality. This is especially true for breast cancer mortality: the death rate steadily decreased by 38% from 1989 to 2014. This is due especially to decreases in smoking and advances in early detection and treatment [1].

    The advances made in breast cancer treatment offer women greater prospects of cure and a better quality of life. However, cancer treatment has some deleterious acute or long-term side effects that impact women’s physical, functional, emotional, financial, and social lives [4]. Along with depression and anxiety [5], cognitive impairment is a short- or long-term outcome or side effect of breast cancer and its treatment and the treatment of other diseases such as stroke [6], HIV and hemophilia [7], and multiple sclerosis [8]. In recent years, empirical evidence has risen regarding significant cognitive impairment following breast cancer treatments in survivor women. The American Cancer Society/American Society of Clinical Oncology Breast Cancer Survivorship Care Guideline [9] showed that 75% and 35% of patients report cognitive impairment during treatment and after treatment, respectively. In addition, in everyday clinical practice, women with breast cancer often complain about cognitive difficulties in different domains of cognition. For example, these cognitive impairments include problems with concentration, executive function, memory [9] and, especially in patients treated with chemotherapy, problems with visual memory, information processing speed, and verbal memory [10]. Cognitive impairments could also reduce the quality of life, lead to distress, and have a negative impact on women’s working, societal, and family life [9].

    Cognitive impairment can have multifactorial causes. Runowicz and colleagues [9] reported that insomnia, depression, fatigue, surgery, anesthesia, different type of cancer treatments and cancer itself could cause cognitive impairment. For example, a meta-analysis conducted by Jim and colleagues [11] showed that breast cancer survivors treated with chemotherapy suffer small and limited observed cognitive deficits. This is especially true for the domains concerning verbal and visuospatial ability. However, other difficulties also concern memory and attention deficits due to chemotherapy. These symptoms, generally known as chemobrain or chemofog, are experienced by a lot of women treated with chemotherapy [4]. Bakoyiannis and colleagues [12] conducted a systematic review on the impact of endocrine therapy on cognitive functions of breast cancer patients. They especially investigated difficulties in 5 cognitive domains: verbal memory, verbal fluency, attention and working memory, motor speed, and psychomotor speed. They concluded that endocrine therapy may alter cognitive functions in these women.

    However, the exact mechanism underlying cognitive impairment is not clear. For example, empirical evidence exists on the role of stress and coping styles [13], direct neurotoxic injury, telomere shortening, oxidative stress, cytokine dysregulation, estrogen-mediated effects, genetic polymorphism [14], peripheral proinflammatory cytokines [15], decreased estrogen levels, and structural brain changes [16] in cognitive impairment following cancer treatment.

    Moreover, there are uncertainties regarding the nature and magnitude of cognitive impairment in breast cancer patients [17] and also regarding the most effective treatment to target these kinds of cognitive difficulties [9]. In their review on pharmacological and nonpharmacological interventions to manage cognitive alterations after chemotherapy, Chan and colleagues [18] concluded that pharmacological interventions to manage cognitive alterations after chemotherapy for breast cancer are not well supported by current empirical evidence. On the other hand, some kind of cognitive training—computerized cognitive training, cognitive behavioral therapy, memory training, speed of processing training, psychoeducation, Tibetan sound—and physical interventions may be useful. However, further studies are needed in order to provide guidelines and concrete recommendations for clinical practice [18].

    Overall, these psychological interventions for cognitive impairment are time- and money-consuming. They may result in a huge burden for patients because they are often delivered in an in-patient context [17]. The development and delivery of home-based or Web-based interventions may have advantages over traditional clinic-based ones. However, there is a scarcity of these kinds of innovative interventions for cognitive training.

    Starting with the introduction of the World Wide Web in our daily lives, the internet has increasingly become an essential part of modern living. A recent report by the Pew Research Center states that in the developed nations, the number of people who use the internet or a mobile phone remains high throughout the years (around 86% from 2015 to 2018) and that in the developing world the rate is increasing constantly (from 62% of the population using the internet or owning a mobile phone in 2013 to 64% in 2018) [19]. A common misunderstanding is that the elderly, compared to younger people, are less interested in technology. However, older people are rapidly gaining more interest in the subject [20], and the rate of people aged 65 years and above using the internet grew from 14% in 2000 to 58% in 2015. Internet use for other age groups is growing as well: internet use from 2000 to 2015 increased from 70% to 96% for people aged 18 to 29 years, from 61% to 93% for people aged 30 to 49 years, and from 46% to 81% for people aged 50 to 64 years. These statistics are especially interesting because they could indicate the promise of development of innovative Web-based interventions also targeting elderly people. Traditional health care interventions have been delivered through face-to-face meetings with clinicians. However, eHealth and mHealth uses have increasingly spread in the last decades [21].

    The two leading platforms for health-related mobile apps are iOS and Android. As of 2014, more than 100,000 mHealth apps had been released [22,23]. mHealth apps permit real-time and bidirectional interaction with the patient [23]. This transformation brings changes even in the diagnosis and treatment of cancer.

    Several mHealth apps have been developed to inform patients; enhance communication, consulting, and symptom self-management and monitoring; and improve health record access and maintenance and clinical decision making [24]. However, little is known about apps directly aimed at improving cognitive functioning in patients reporting cognitive difficulties. No apps have been specifically developed to counteract cognitive difficulties in breast cancer survivors. Thus, we aimed to identify effective apps to train cognitive function and counteract cognitive impairment in both clinical and nonclinical populations. Another aim was to evaluate their efficacy and assess whether they could be also used to counteract cognitive impairment in breast cancer patients. Thus, we focused on their distinctive features in terms of targeted populations and the specific cognitive domains being trained.


    Methods

    Our search strategy was designed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [25]. A flowchart of the systematic review is shown in Figure 1.

    To identify articles for our review, we searched PubMed and Embase databases for the terms smartphone, mobile, and app. The searched cognitive domains were memory, attention, concentration, verbal fluency, motor speed, psychomotor speed, problem-solving, executive function, visuospatial function, language, and cognitive control. In particular, the string of terms included:

    • Smartphone app or mobile app
    • Cognitive function or memory or attention or concentration or verbal fluency or motor speed or psychomotor speed or problem solving or executive function or visuospatial function or language or cognitive control

    The search was limited to the English language. No restriction was placed on the year of publication. The search was completed in February 2018.

    Two authors independently reviewed the titles and abstracts to identify relevant papers. We extracted only those papers with the main focus on the use of a mobile-based app (mobile phone or tablet-based) to deliver cognitive training. About 10% of the papers were double screened, and disagreements in data extraction were resolved through discussion with a third author.

    The second step consisted of a full article screening. We excluded papers that were (1) reviews, case reports, or protocols; (2) abstracts for conferences; or (3) related to apps that assess cognitive functions without training them.

    Figure 1. Flowchart of the systematic review.
    View this figure

    Data collection included information on authors, title, publication date, study aim, sample characteristics, clinical or nonclinical populations, cognitive domains trained, app exercise descriptions, assessment and feedback, app evaluation and efficacy, training with the experimenters, and whether the device was given to the study participants.


    Results

    Studies Selected

    A flowchart of the systematic review is shown in Figure 1. We retrieved a total of 493 articles. We selected 11 articles after a full-text screening. We excluded 2 reviews (eg, [21,26]), 5 case reports (eg, [27-31]), and 4 protocols (eg, [32,33]). The other studies were excluded for content reasons, for example: some studies did not use apps to train cognitive functions [22,34-47], and others were excluded because they used apps only to assess—and not to train—cognitive functions [48-58]. Our 11 selected studies focus on the cognitive domains of attention, memory, problem solving, cognitive control, executive function, visuospatial function, and language (reported in Table 1).

    Sample Characteristics

    Our search returned 11 studies that include 819 participants. The number of participants in these studies ranged from 9 to 626. Weighted by sample size, the mean age of participants was 36.2 years. However, there is a huge heterogeneity in participant ages among these 11 studies: ages ranged from 50 months to 96 years. In particular, 1 study focused on preschool children with a mean age of 60 months [59], while 4 papers focused especially on older adults with a mean age over 68 years [60-63].

    Nearly all of the studies were performed in developed countries. Almost half of the studies (5/11, 45%) were conducted in North America: 4 were performed in the United States [60,61,64,65] and 1 in Canada [63]. Three were conducted in Europe: 2 in Italy [59,66], and 1/11 in Norway [67]. Two studies were conducted in Asia: 1 in Taiwan [62] and 1 in Jordan [68]. Only 1 study was conducted in Oceania (Australia [69]). In addition to this heterogeneity in cultural provenance, the 11 identified studies have a moderate variability in the kind of population in which they evaluated the app for the cognitive training. No studies involved breast cancer patients. Ten of 11 studies involved adults or elderly people. One study focused on adults affected by brain injuries [65], 1 on cognitively impaired patients with multiple sclerosis [66], and 1 on patients with early stages of Alzheimer disease [68]. Three papers focused on healthy elderlies [60-62] while 1 study chose a population of older adults with and without subjective cognitive complaints and mild cognitive impairment [63]. One study focused on adults with mild to moderate depression [64] and 1 on overweight or obese adult [69]. Just 1 paper from our search specified healthy young adults [67]. Only 1 study focused on children with mild to severe language impairments or delays [59]. Inclusion and exclusion criteria were different in all of the studies. Only 3 studies [65,67,69] included control groups.

    Aim

    The majority of the selected papers had a primary aim to evaluate the apps in terms of usability, acceptability, feasibility, and user satisfaction [59-61,63,66-68]. Only 2 articles [62,65] chose to describe the development and improved design of the apps. Only 1 paper [64] had as the main aim to assess and improve depressive symptoms by comparing 3 different apps, 1 of which is explicitly aimed at improving depression through a cognitive control exercise. Just 1 study had the explicit aim of investigating the efficacy of the app in improving cognitive functioning [69].

    App Characteristics

    App availability on the market was checked by searching in the main app stores (App Store and Google Play); however, none are available to the general public. The same search was conducted on online test catalogs, such as Pearson Assessments or Psychological Assessment Resources Inc, but no apps from the studies were found. In all of the studies, app users were asked to respond with a touchscreen.

    Table 1. App developers, app names, and cognitive domains being targeted in each study.
    View this table

    Bless and colleagues [67] explored the feasibility and effectiveness of an app that enables people to train auditory attention. They developed the app in-house, but they did not give it a name. Participating in this study were healthy adults; they were given an iPod touch and asked to perform a consonant-vowel dichotic listening task. According to the forced-attention condition of the standard consonant-vowel dichotic listening paradigm, during this task each participant was presented simultaneously with 2 different syllables via earphones: while one was presented to the left ear, the other was delivered to the right. Syllables, made by consonants and vowels, were read by a male, native Norwegian speaker with constant voice intonation and intensity. Before the beginning of the task, each participant was told via message on the display to pay attention only to the syllable delivered to the left or the right ear. After hearing the syllables, participant had to choose the correct syllable from 6 multiple choice answers. Each training session, about 6 minutes long, included 6 blocks with 5 pairs of syllables. Syllables were presented in 400 to 500 ms with an interval of 4000 ms between each pair. At the end of each session, participants were shown feedback of their correct answers. Results were stored on the device and available to the researcher at the end of the study.

    Lorusso and colleagues [59] developed an app to improve semantic competence and structural knowledge in children with learning impairments. It was part of an integrated system. The aim of the study was to evaluate learnability, usability, user satisfaction, and the quality of the interaction in children playing with this integrated system. The app was used on a supplied tablet; it scanned the tag that the experimenter had placed under some toys (plastic animals). When the children scanned the tag under the toys, the app displayed a menu with 5 different activities that were all related to this specific animal. The first was an informative activity in which children were given some interesting information about this specific animal. The second one was a storytelling activity that directly connected to a website with stories and tales related to the animal. The third was a visual activity with several picture and photos of this specific animal. The fourth presented a song strictly related to the chosen animal. The last one was a puzzle activity in which children were asked to put together pieces of a puzzle representing this specific animal. The authors chose all these activities because they enrich semantic knowledge and organization. After the completion of each the task, the child was given feedback by picture or sound. The app also provided a text-to-speech tool to listen to written information.

    Characteristics of the included studies and related apps are presented in Multimedia Appendix 1 and 2.

    Shellington and colleagues [63] examined the feasibility and utility of an app called HealtheBrain, which was focused on improving visuospatial executive function and working memory. Participants of this study were older adults with or without subjective cognitive complaints and mild cognitive impairment. They received HealtheBrain to improve cognitive functioning through a specific physical exercise called Square Step Exercise. Before the beginning of the exercise, the app offered participants the opportunity to have a tutorial about the use of the app and then to calibrate the step length. The square step exercise had 35 progressive stages and the participants had to start to form the first one. In order to go from 1 level to another, the subject had to complete at least 80% of each task. For each step, participants had to memorize 4 to 8 walking and stepping sequences and repeat them. When they reached the goal, an item was added to their virtual garden and they could get to the next level. They were asked to exercise at least 3 times per week over the next 3 weeks, and subjects were invited to contact the researchers with any question or technical issue.

    In 2015, Hill and colleagues [60] developed an app to train attention. More recently [61], they decided to improve it considering the problematic issues identified in their previous study. Both studies focused on the usability, acceptability, and feasibility of the Attention Training Application. The experimental design of the study in 2018 was very similar to the pilot study in 2015. Before the start of exercise, 2 preliminary sessions were established: in the first session an examiner met with a participant in order to introduce them to the device (iPad or iPad Mini). Prior to the beginning of the trial, participants could exercise using the device for a week to become familiar with its functioning. The app was then introduced in the second session. Written instruction was given to participants, and they were able to contact the experimenter to receive further information and instructions on the use of the app and device. This app delivered a program targeting attention and was based on the dual n-back training paradigm. This cognitive domain was improved by different exercises: in the first one a visual stimulus was presented, in the second one an auditory stimulus was given, and in the third step visual and auditory stimuli were presented together. The visual stimulus was a grid split in 8 parts with a square in 1 of them. The auditory stimuli were spoken letters. Given that the authors follow the n-back paradigm, the exercise requested subjects to report the correct answer of the exercise presented n trial before. At the end of each trial, the app sends a visual stimulus so the participants know immediately if the answer is the correct one. The Attention Training Application was adaptive: if participants performed well, the exercise increased in difficulty. The differences between the 2 versions of the app consisted mostly in modified elements to help the user better comprehend how to use the app and correctly perform the exercise. In 2018, the authors added some preliminary training features: a first session with the experimenter that presented the iPad and a week to become familiar with the device. Moreover, feedback was modified in the second version. While the 2015 app gave negative feedback to users (a red X indicated each wrong answer), this was removed in 2018 because participants generally reported frustration and confusion with it. Finally, the response time changed from 3 seconds to 5 seconds. The paradigm followed, and the exercise and adaptive style of exercise presentation remained the same.

    Lu and colleagues [62] developed a prototype of an app called Brain Win. The app was directed to older people and evolved through 2 cycles of design and evaluation. The app involved 4 tasks and 6 games to train 5 cognitive domains: attention, executive function, memory, language, and visuospatial function. The 6 implemented games directly relate to real-life experience and daily activities of older adults:

    1. My Calendar: discrimination task to train attention, executive function, and memory. Participants identify the correct date and time.
    2. Go to Market: visuomotor task to train attention, executive function, memory, and visuospatial function. Participants draw the route to the market place.
    3. Shopping in the Market: calculation task to train attention, executive function, and memory. Participants buy items with a limited budget and make calculations using the item prices.
    4. Finding Objects During a Phone Call: discrimination task to train attention, memory, language, and executive function. Participants listen and find correct items on the screen.
    5. Super Singer: respelling task to train attention, memory, language, visuospatial function, and executive function. Participants reorganize character cards with song lyrics, listening to or reading them.
    6. Go to the Zoo: discrimination task to train attention, memory, and executive function. Participants recognize and remember the noise made by the zoo animals and identify them.

    Each game was supported by visual and auditory instructions. Buttons and icons had a realistic appearance and participants were asked to test the app in their home. Feedback sounds were presented at the end of each task. There was also the possibility to check the game scores or participant’s position in a ranking table.

    The Arean and colleagues [64] study aimed at testing the efficacy of 3 apps in relieving depressive symptomatology. These 3 apps were called Project: EVO, iPST, and Health Tips. In our review, we are mainly interested in Project: EVO, which trained cognitive control in order to improve cognitive symptoms. The other 2 apps were used to control treatment (Health Tips) and conduct psychotherapy intervention to improve depression (iPST). The participant had to use just 1 of the 3 apps: if they owned a mobile phone but not an iPad, the experimenter gave them one to use. Project: EVO is an exercise to train cognitive control and is designed as a video game. As the participants improved their proficiency with the game over time, the app increased the difficulty of the exercises. The app had an internal system to remind individuals to complete the exercise.

    Blackburne and colleagues [69] tested the efficacy of the NoGo app that considered cognitive restraint—and specifically inhibitory control—to be used for weight control in obese adults. In particular, the app trained 3 domains: unhealthy eating, smoking, and alcohol consumption. Each game had 2 distinct tasks: Go and NoGo trials, in which a timer appeared next to an image that remained the same and started to countdown, and the Stop trial, in which participants had to choose the healthy food image as fast as possible. In this exercise, the image changed from healthy to unhealthy after the countdown began. Participants had to tag the correct answer if the Go tone was produced; if the NoGo tone was reproduced instead, participants had to hold back from answering. The stimuli order changed in each game so that the individuals could not anticipate and predict the next image and could not respond based on their previous experience with the game. The difficulty level was modulated by the timer and by the number of images presented (with a maximum of 12 images). The app collected data such as reaction time, game level, correct responses, and errors. The researcher could gain access to the data at the end of the training session.

    Powell and colleagues [65] aimed at developing and testing an app called ProSolv, a problem-solving app developed by the authors after focus groups and interviews. In this study, the device was not provided; an inclusion criterion was that each participant must own a mobile phone and have access to the internet. The ProSolv program included 3 steps plus FAQ and help pages. The first was a face-to-face meeting with a coach; during this session, each participant could learn and test the app with the help of the expert and the program manual. In the second, a Web-based tutorial introduced the conceptual model of problem solving and its usefulness in everyday activities. Next, use of the app was explained in a video. The third step consisted of participants using the app to create a problem-solution list and remember each step to solve problems in a more effective way. The app comprised 4 pages: Welcome to the ProSolv app, My problem, My solution, and My contact. With the app, individuals could evaluate each problem-solution on the list by rating it with 1 to 5 stars.

    Tacchino and colleagues [66] described the Cognitive Training Kit (COGNI-TRAcK). In their study, the app was used as a cognitive rehabilitation intervention based on working memory exercises in a sample of patients with multiple sclerosis. The authors stated that COGNI-TRAcK could be used to implement 3 types of working memory. The first task targeted visuospatial working memory and presented a sequence of visual stimuli, with participants asked to touch the corresponding location where the stimuli had appeared on the screen. The second task was an operation n-back exercise in which participants were presented with 2 numbers on the screen. If the instruction on the screen said N=0, participants had to touch the sum of the 2 numbers as the right answer. When the instruction said, for example, N=1, they had to touch on the screen the sum of the numbers previously presented. The third was a dual n-back exercise in which each patient was presented with a single number on the screen. Similar to the first n-back task, if the instruction said, for example, N=1, participants had to remember and recall the number presented in the previous exercise by touching it on the screen. All the information collected by the app was directly stored in a database with 3 sections: (1) Patient contained participant data; (2) Exercise and Treatment contained information about the assignment, workload, record of the exercise, and length of the intervention; and (3) Setting contained the characteristics of the configuration of the app. The main feature of the app was the possibility to implement the workload and regulate its intensiveness.

    The study by Zmily and colleagues [68] focused on the usability of a subtask of an integrated app named ADcope. The app was developed for mobile devices and aimed to support Alzheimer disease patients in their daily routines. During the study, patients were given a tablet with the app along with clear instructions on how to use the device and app. Patients were asked to perform exercises while sitting together in a room. The app had sections for improving user quality of life and a support module, but for our review, we focused on the third section, which aimed to exercise patient memory in 2 ways. Audio-Assisted Memory Training played audio files of patient biographical information and then quizzed them about the information. The Spaced Retrieval exercise, whose usability was the main focus of the study, comprised 2 phases. The first assessed users’ current memory recall ability by presenting them with information followed, after some delay, by a question with 4 multiple choice answers. This exercise could use text information or a simple figure. The difficulty of the exercise could be operationalized by measuring the length of the delay between the information and the quiz. If the individual gave the right answer after a certain period of time, the app would increase the difficulty by increasing the time delay. The training phase (the same as the assessment) comprised 10 questions with a delay length based on the assessment results. After the participant completed the trials, written feedback appeared on the display. The app also has text-to-speech tools enabling patients to have the information read aloud.

    Cognitive Domains

    Overview

    As shown in Table 1, the cognitive domains trained by the identified apps are attention, problem solving, memory, cognitive control, executive function, visuospatial function, and language. Nine out of 11 studies considered the usefulness of the app for the training of just 1 cognitive domain at a time (cognitive control [64,69], attention [60,61,67], language [59], problem solving [65], and memory [66,68]). The remaining 2 studies considered more than 1 cognitive function: Lu and colleagues [62] investigated attention, memory, executive function, visuospatial function, and language; Shellington and colleagues [63] investigated memory, executive function, and visuospatial function.

    Cognitive Control

    Cognitive control was trained in studies by Arean and colleagues [64] and Blackburne and colleagues [69]. Both of these authors stated that app usage could improve cognitive control as a mean to enhance other conditions: depression for Arean and colleagues [64] and obesity for Blackburne and colleagues [69]. Cognitive control in the paper by Arean and colleagues [64] was trained with an app that is designed as a video game that modulates cognitive control abilities. Blackburne and colleagues [69] instead reported the efficacy of cognitive control training in food consumption in an obese population.

    Problem Solving

    People with cognitive impairment following brain injury often lack problem-solving skills, and a Web-based approach could be useful in rehabilitation [65]. The ProSolv app from Powell and colleagues [65] could be a useful tool to help solve this issue. The deficit in problem-solving was trained through the creation of a personalized problem-solution list and with the possibility to use the app as a resource for remembering the steps to effective problem solving.

    Memory

    This cognitive dimension was analyzed in different samples and with different meanings by the authors. Tacchino and colleagues [66] and Zmily and colleagues [68] explored how to train memory in patients with diseases that cause cognitive impairments: multiple sclerosis and Alzheimer disease, respectively. As they referred to different kinds of populations, the authors explored different kinds of memory. Tacchino and colleagues [66] focused on working memory training, which can lead to changes in a healthy individual’s brain structures that can improve cognitive function, useful in the daily life. The improvement is demonstrated in cognitively impaired patients as well. Their study focused on multiple sclerosis, as it is a cognitively disabling condition and affects all age groups. Another disease that has serious effects on cognitive impairment is Alzheimer disease. Memory impairment in Alzheimer disease patients was the focus of the work of Zmily and colleagues [68], which focused on recall ability training to help individuals retain critical information longer and consequentially improve their quality of life.

    The research of Lu and colleagues [62] and Shellington and colleagues [63] investigated memory impairment in older populations. Shellington and colleagues [63] considered older adults with and without subjective cognitive complaints and mild cognitive impairment. In the study, the authors tried to train memory through physical activities. They stated that physical exercise was associated with higher cognitive function and used a square step exercise that was proven to train memory skills. Lu and colleagues [62] studied age-related memory decline and its effect on recollection ability during information finding and retrieval in healthy older adults. Memory was trained with 4 tasks: discrimination, visuomotor, respelling, and calculation.

    Executive Function

    Lu and colleagues [62] reported some evidence on age-related decline in executive function and performance; these abilities are generally affected by decreases in working memory functioning and by the perception of time. Deficiency in this domain is related to future functional impairment. Neurochemical, localized, and process aging theories indicate that age-related cognitive changes also affect executive functioning. Brain Win, the app developed by authors, stimulated executive function in all game contexts with 4 types of tasks (discrimination, visuomotor, respelling, and calculation). Shellington and colleagues [63] studied executive function related to physical exercise. They considered physical activities as a means to train executive function. Their app, Healthe Brain, suggested a series of exercises, called square step exercises, to implement the cognitive domain.

    Visuospatial Function

    Brain Win, the app developed by Lu and colleagues [62], stimulated the visuospatial function with 2 tasks: Go to the Market and Super singer. Brain Win was specifically developed to improve visuomotor ability in older adults; visuospatial function is affected by age-related cognitive changes including visuospatial attention, memory, and orientation decline. Visuospatial function is the focus of the work by Shellington and colleagues [63] as well. Their study combined physical activities with cognitive training. In particular, they focused on a series of activities called square step exercises. These exercises could be described as a visuospatial working memory with a cued stepping response also known as mind-motor exercise.

    Attention

    Four studies considered the cognitive domain of attention. In particular, Bless and colleagues [67] focused on an app to train auditory attention, based on the forced-attention conditions of the consonant-vowel dichotic listening paradigm. Individuals have to listen to different auditory stimuli simultaneously in both ears while paying attention to only 1 of the sounds. According to the authors, this paradigm could be considered as an analog task in some everyday life situations in which people are asked to effectively master different and confounding auditory events. The authors reported that some deficits in auditory attention could also be found in clinical conditions like schizophrenia, preterm–born adolescents, dyslexia, and aging. In their study, the authors tried also to see if the trained task has transfer effects on cognitive interference and attentional task in daily visual and auditory domain activities.

    The Attention Training Application by Hill and colleagues [60,61] is based on the dual n-back training paradigm and targets attention in elderly people. The authors reported that through aging, some aspects of attention could tend to decline—for example, with increasing age, the ability to divide or switch attention could decline. Attention is a cognitive function that has a pervasive influence on several daily activities. Thus, its training may also lead to improvements in other cognitive performance aspects and enhance the appropriateness of several daily life activities. Specifically, the authors highlighted that the effect of the dual n-back task could be transferred to other cognitive abilities.

    Also, the Brain Win app, developed in a working prototype for older adults by Lu and colleagues [62], targets the cognitive domain of attention. This app directly stimulates attention with 4 types of games addressing discrimination, visuomotor, respelling, and calculation tasks.

    Language

    Two studies directly focused on training the cognitive domain of language. Lu and colleagues [62] started by reporting evidence of a decline of language functioning in elderly people. Specifically, neurochemical, localized, and process aging theories demonstrate age-related cognitive changes affecting language. Language is connected with various cognitive aspects; a decline in this domain could also affect sentence understanding and text recalling. In Brain Win, the app developed by the authors, language is stimulated through 2 context games (Finding Objects During a Phone Call and Super singer) that were connected to 2 types of abilities, discrimination and respelling, respectively. Lorusso and colleagues [59] focused on improving language ability in children with language impairment and typically developing children. Their system aimed to improve semantic competence and structural knowledge with various activities. The system is made up as an integrated combination of a tablet, a group of plastic toys, the near field communication technology, and a custom app that allows children to play with various activities to train specific cognitive processes and abilities such as mental representation, conceptual networks, and semantic versus structural world knowledge. These aspects are especially relevant in speech and language development and functioning.

    Efficacy

    Only 3 out of 10 studies directly evaluate the apps in improving cognitive abilities. Blackburne and colleagues [69] evaluated the efficacy of the NoGo app in having an impact on cognitive control and in particular on inhibitory control. Compared to a control group, the authors found in people using the app an increase of cognitive restraint evaluated with a self-report questionnaire. Regarding the exercise conveyed by the app, authors showed a significant effect in improving inhibitory control in the training group. Other measured outcomes were self-reported food consumption and attitudes toward food and diet. Results demonstrated the effectiveness of the app in improving even these aspects.

    Bless and colleagues [67] concluded that the training of auditory attention through an app is feasible and successful. After 21 days of practicing, participants were found to have an increase in auditory attention. This is supported by the fact that the training group showed an increase in the performance of the exercise itself and in brain activation measured with functional magnetic resonance imaging.

    Finally, Powell and colleagues [65] focused on the efficacy of their app and evaluated its effectiveness in improving problem-solving ability, but no significant effects of the cognitive training were found in comparing an intervention group using ProSolv and a control group performing traditional cognitive training. Other authors focused attention on subjective evaluations of their apps [59,61-63,66,68], but these studies lack of data regarding the effects produced by the apps on cognitive abilities.


    Discussion

    General Considerations

    This systematic review provides a useful, clear, and comprehensive overview of the current state of the art in available apps for cognitive training. However, this work also raises some critical issues regarding the application of this kind of training in clinical and research practice for breast cancer patients. While some issues are broader and relevant to the field of cognitive training apps, others are more specific and directly related to their application in a cancer patient population.

    General Issues in Cognitive Training Apps

    The majority of the articles included in this review have the primary aim of evaluating app usability, acceptability, feasibility, and user satisfaction; only a few of them directly focus on demonstrating efficacy in improving cognitive functioning. Only 2 [67,69] out of 11 studies report significant and quantitative amelioration in cognitive performance. Even if feasibility is important, it should be noted that the objective efficacy of the apps has not been measured in the majority of the relevant articles. For several apps, we could not establish whether the training effectively improved the targeted cognitive domains.

    In addition, some definitional and conceptual issues about cognitive functions also emerge. Specifically, there is a vast heterogeneity in the definition and conceptualization of cognitive function across studies, with similar cognitive domains being labeled in different ways (eg, executive functions may include visuospatial attention, attention, and so on). Moreover, targeted cognitive functions are not unique or consistent across studies, with only marginal overlap between studies. Some cognitive functions (eg, memory) cover heterogeneous abilities (eg, working memory, short-term memory, long-term memory) encompassing specific functions that can be selectively impaired in some diseases and not in others. Thus, it is difficult to draw conclusions on which cognitive domain could benefit from app training. Given the aforementioned issues, it is critical to properly evaluate the clinical efficacy and effectiveness of these apps.

    Specific Issues in Cognitive Training Apps Applied to Cancer Patients

    Taken together, these results show that there are apps that are best candidates to target cognitive functions generally impaired in women with breast cancer. However, the feasibility, usefulness, and efficiency of cognitive training in this particular population should not be automatically taken for granted.

    Relevant articles included in our review employed highly heterogeneous samples of subjects: only a few apps have been tested on patients, such as people with Alzheimer disease [68] or multiple sclerosis [66]; most have been evaluated on a healthy general population sample of adults or children. This heterogeneity is also indicative of the different needs of these diverse populations.

    People with breast cancer may benefit from using available interventions or to-be-developed apps for this specific category of patients. In fact, breast cancer patients somehow reside in a class between mildly or severely cognitively impaired patients and healthy general population subjects because they display objective cognitive impairments or report cognitive problems [9] but they still can properly work and have an active and preserved social life. On one hand, they do not display a degenerative disease with a progressive worsening of global or selective cognitive functioning. On the other hand, they are aware of and worried that their cognitive worsening could interfere with daily routines and everyday life. Thus, they are strongly motivated to be involved in a cognitive training program. Worthy of note, breast cancer patients usually display a high level of distress symptoms and serious psychological side effects [5,9] that can influence subjective perception of cognitive functioning and could potentially interfere with training activities.

    Finally, the summarized papers, apart from Lu and colleagues [62] and Shellington and colleagues [63], reported training interventions on a single cognitive domain. However, women with breast cancer often display difficulties and deficits in several cognitive domains. Thus, it could be relevant to design, develop, and implement an ad hoc app targeting the various cognitive function domains for breast cancer patients.

    Conclusion

    Our study highlights the fact that cognitive training apps are becoming more present in rehabilitation of different diseases. It is noteworthy that none of them has been developed to counteract cognitive impairment in breast cancer patients, a specific population in which short- and long-term cognitive difficulties have been underlined. Thus, currently there are no available cognitive training apps that meet the needs of breast cancer survivor women. Available apps lack strong specificity for oncological breast patients both from the point of view of the cognitive functions that should be addressed and for the psychological complexity that these patients display. In fact, the psychological and physical impact of breast cancer on cognitive impairment should be taken into account as well. As a specific population with specific needs, it is necessary to create an app that considers their deficit as different from the deficit that another population of patients could have (eg, neurodegenerative conditions). Medicine is evolving to consider not only the patient’s physical safety but also a personalized approach to disease [70,71]. From the patient empowerment perspective, it is very important to give breast cancer survivors reliable means to improve and train their cognitive functioning because of the huge impact of cognitive complaints on the quality of life and patient empowerment [72]. Moreover, women with breast cancer may benefit from using a mobile or Web-based tool to improve cognitive functioning, effectively manage their daily activity, and properly cope with everyday difficulties. This would be especially helpful to foster breast cancer patient perceived self-efficacy and manage their anxious and depressive symptomatology. We conclude that further studies should test the feasibility, usability, and effectiveness of available cognitive training apps in women with breast cancer. Because of the complexity and multidimensionality of the cognitive difficulties affecting this cancer population, it may be useful to design, develop, test, and implement an app with the specific aim to train cognitive impairment in breast cancer patients.

    Acknowledgments

    This project has received funding from the European Union’s Horizon 2020 Research and Innovation Program under grant agreement number 643529. This manuscript reflects the authors’ views; the European Commission is not responsible for any use that may be made of the information it contains.

    Authors' Contributions

    GM and LV reviewed the literature and wrote the main part of the review. SFMP, DM, KM, and GP supervised the work, reviewed the manuscript, and provided conclusions.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Summary of study characteristics in terms of study type, aim, and sample.

    PDF File (Adobe PDF File), 30KB

    Multimedia Appendix 2

    Summary of main aspects of apps and their evaluation.

    PDF File (Adobe PDF File), 29KB

    References

    1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2015. CA Cancer J Clin 2015;65(1):5-29 [FREE Full text] [CrossRef] [Medline]
    2. Ghoncheh M, Pournamdar Z, Salehiniya H. Incidence and mortality and epidemiology of breast cancer in the world. Asian Pac J Cancer Prev 2016;17(S3):43-46 [FREE Full text] [Medline]
    3. Alkabban F, Ferguson T. StatPearls: Breast Cancer. Treasure Island: StatPearls Publishing; 2018.
    4. Fallowfield L, Jenkins V. Psychosocial/survivorship issues in breast cancer: are we doing better? J Natl Cancer Inst 2014 Nov;107(1):335 [FREE Full text] [CrossRef] [Medline]
    5. Mitchell AJ, Ferguson DW, Gill J, Paul J, Symonds P. Depression and anxiety in long-term cancer survivors compared with spouses and healthy controls: a systematic review and meta-analysis. Lancet Oncol 2013 Jul;14(8):721-732. [CrossRef] [Medline]
    6. Douiri A, Rudd A, Wolfe C. Prevalence of poststroke cognitive impairment: South London Stroke Register 1995-2010. Stroke 2013 Jan;44(1):138-145. [CrossRef] [Medline]
    7. Riva S, Cutica I, Krampe C, Reinecke L, Russell-Edu W, Santoro C, et al. A Cohort Pilot Study on HIV-Associated Neuropsychological Impairments in Hemophilia Patients. Front Hum Neurosci 2015;9:313 [FREE Full text] [CrossRef] [Medline]
    8. Schulz D, Kopp B, Kunkel A, Faiss J. Cognition in the early stage of multiple sclerosis. J Neurol 2006 Aug;253(8):1002-1010. [CrossRef] [Medline]
    9. Runowicz C, Leach C, Henry N, Henry K, Mackey H, Cowens-Alvarado R, et al. American Cancer Society/American Society of Clinical Oncology Breast Cancer Survivorship Care Guideline. J Clin Oncol 2016 Feb 20;34(6):611-635. [CrossRef] [Medline]
    10. Jansen CE, Miaskowski C, Dodd M, Dowling G, Kramer J. A metaanalysis of studies of the effects of cancer chemotherapy on various domains of cognitive function. Cancer 2005 Nov 15;104(10):2222-2233 [FREE Full text] [CrossRef] [Medline]
    11. Jim H, Phillips K, Chait S, Faul L, Popa M, Lee Y, et al. Meta-analysis of cognitive functioning in breast cancer survivors previously treated with standard-dose chemotherapy. J Clin Oncol 2012 Oct 10;30(29):3578-3587 [FREE Full text] [CrossRef] [Medline]
    12. Bakoyiannis I, Tsigka E, Perrea D, Pergialiotis V. The impact of endocrine therapy on cognitive functions of breast cancer patients: a systematic review. Clin Drug Investig 2016 Feb;36(2):109-118. [CrossRef] [Medline]
    13. Reid-Arndt SA, Cox CR. Stress, coping and cognitive deficits in women after surgery for breast cancer. J Clin Psychol Med Settings 2012 Jun;19(2):127-137. [CrossRef] [Medline]
    14. Ahles T, Saykin A. Candidate mechanisms for chemotherapy-induced cognitive changes. Nat Rev Cancer 2007 Mar;7(3):192-201 [FREE Full text] [CrossRef] [Medline]
    15. Janelsins M, Mustian K, Palesh O, Mohile S, Peppone L, Sprod L, et al. Differential expression of cytokines in breast cancer patients receiving different chemotherapies: implications for cognitive impairment research. Support Care Cancer 2012 Apr;20(4):831-839 [FREE Full text] [CrossRef] [Medline]
    16. McDonald B, Saykin A, McAllister T. Functional MRI of mild traumatic brain injury (mTBI): progress and perspectives from the first decade of studies. Brain Imaging Behav 2012 Jun;6(2):193-207 [FREE Full text] [CrossRef] [Medline]
    17. Damholdt MF, Mehlsen M, O'Toole MS, Andreasen RK, Pedersen AD, Zachariae R. Web-based cognitive training for breast cancer survivors with cognitive complaints: a randomized controlled trial. Psychooncology 2016 Nov;25(11):1293-1300 [FREE Full text] [CrossRef] [Medline]
    18. Chan RJ, McCarthy AL, Devenish J, Sullivan KA, Chan A. Systematic review of pharmacologic and non-pharmacologic interventions to manage cognitive alterations after chemotherapy for breast cancer. Eur J Cancer 2015 Mar;51(4):437-450 [FREE Full text] [CrossRef] [Medline]
    19. Poushter J, Bishop C, Chwe H. Social media use continues to rise in developing countries but plateaus across developed ones. Washington: Pew Internet and American Life Project; 2018 Aug 19.   URL: http:/​/www.​pewglobal.org/​wp-content/​uploads/​sites/​2/​2018/​06/​Pew-Research-Center_Global-Tech-Social-Media-Use_2018.​06.​19.​pdf [accessed 2018-09-28] [WebCite Cache]
    20. Perrin A, Duggan M. Americans' internet access: 2000-2015. Washington: Pew Internet and American Life Project; 2015 Jun 26.   URL: http:/​/www.​pewresearch.org/​wp-content/​uploads/​sites/​9/​2015/​06/​2015-06-26_internet-usage-across-demographics-discover_FINAL.​pdf [accessed 2018-09-28] [WebCite Cache]
    21. East ML, Havard BC. Mental health mobile apps: from infusion to diffusion in the mental health social system. JMIR Ment Health 2015;2(1):e10 [FREE Full text] [CrossRef] [Medline]
    22. Law GC, Neihart M, Dutt A. The use of behavior modeling training in a mobile app parent training program to improve functional communication of young children with autism spectrum disorder. Autism 2018 May;22(4):424-439. [CrossRef] [Medline]
    23. Zhao J, Freeman B, Li M. Can mobile phone apps influence people's health behavior change? An evidence review. J Med Internet Res 2016 Oct 31;18(11):e287 [FREE Full text] [CrossRef] [Medline]
    24. Ventola C. Mobile devices and apps for health care professionals: uses and benefits. P T 2014 May;39(5):356-364 [FREE Full text] [Medline]
    25. Moher D, Liberati A, Tetzlaff J, Altman D, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    26. Bateman D, Srinivas B, Emmett T, Schleyer T, Holden R, Hendrie H, et al. Categorizing health outcomes and efficacy of mhealth apps for persons with cognitive impairment: a systematic review. J Med Internet Res 2017 Aug 30;19(8):e301 [FREE Full text] [CrossRef] [Medline]
    27. Visser L, Sinkeviciute I, Sommer IE, Bless JJ. Training switching focus with a mobile-application by a patient suffering from AVH, a case report. Scand J Psychol 2018 Feb;59(1):59-61. [CrossRef] [Medline]
    28. Ruiz-Manrique G, Tajima-Pozo K, Montañes-Rada F. Case Report: ADHD Trainer: the mobile application that enhances cognitive skills in ADHD patients. F1000Res 2014;3:283 [FREE Full text] [CrossRef] [Medline]
    29. O'Connor S, Bouamrane M, O'Donnell CA, Mair FS. Barriers to co-designing mobile technology with persons with dementia and their carers. Stud Health Technol Inform 2016;225:1028-1029. [Medline]
    30. DeLaughter K, Sadasivam R, Kamberi A, English T, Seward G, Chan S, et al. Crave-Out: a distraction/motivation mobile game to assist in smoking cessation. JMIR Serious Games 2016 May 26;4(1):e3 [FREE Full text] [CrossRef] [Medline]
    31. Ganz J, Hong E, Goodwyn F, Kite E, Gilliland W. Impact of PECS tablet computer app on receptive identification of pictures given a verbal stimulus. Dev Neurorehabil 2015 Apr;18(2):82-87. [CrossRef] [Medline]
    32. Noone C, Hogan M. A protocol for a randomised active-controlled trial to evaluate the effects of an online mindfulness intervention on executive control, critical thinking and key thinking dispositions in a university student sample. BMC Psychol 2016 Apr 12;4:17 [FREE Full text] [CrossRef] [Medline]
    33. Granich J, Dass A, Busacca M, Moore D, Anderson A, Venkatesh S, et al. Randomised controlled trial of an iPad based early intervention for autism: TOBY playpad study protocol. BMC Pediatr 2016 Dec 19;16(1):167 [FREE Full text] [CrossRef] [Medline]
    34. Roncero M, Belloch A, Doron G. A novel approach to challenging OCD related beliefs using a mobile-app: an exploratory study. J Behav Ther Exp Psychiatry 2018 Jun;59:157-160. [CrossRef] [Medline]
    35. Chan SCC, Chan CCH, Derbie AY, Hui I, Tan DGH, Pang MYC, et al. Chinese calligraphy writing for augmenting attentional control and working memory of older adults at risk of mild cognitive impairment: a randomized controlled trial. J Alzheimers Dis 2017;58(3):735-746. [CrossRef] [Medline]
    36. Mansbach W, Mace R, Clark K. The efficacy of a computer-assisted cognitive rehabilitation program for patients with mild cognitive deficits: a pilot study. Exp Aging Res 2017;43(1):94-104. [CrossRef] [Medline]
    37. Choo D, Dettman S, Dowell R, Cowan R. Talking to toddlers: drawing on mothers’ perceptions of using wearable and mobile technology in the home. Stud Health Technol Inform 2017;239:21-27. [Medline]
    38. Brown W, Liu C, John RM, Ford P. Developing an eBook-integrated high-fidelity mobile app prototype for promoting child motor skills and taxonomically assessing children's emotional responses using face and sound topology. AMIA Annu Symp Proc 2014;2014:333-342 [FREE Full text] [Medline]
    39. Rabin C, Bock B. Desired features of smartphone applications promoting physical activity. Telemed J E Health 2011 Dec;17(10):801-803. [CrossRef] [Medline]
    40. Li N, Chen W. A mobile game for the social and cognitive well-being of elderly people in China. Stud Health Technol Inform 2017;242:614-621. [Medline]
    41. Boyd A, Synnott J, Nugent C, Elliott D, Kelly J. Community-based trials of mobile solutions for the detection and management of cognitive decline. Healthc Technol Lett 2017 Jun;4(3):93-96 [FREE Full text] [CrossRef] [Medline]
    42. Gálvez JA, Lockman JL, Schleelein LE, Simpao AF, Ahumada LM, Wolf BA, et al. Interactive pediatric emergency checklists to the palm of your hand: how the Pedi Crisis app traveled around the world. Paediatr Anaesth 2017 Aug;27(8):835-840. [CrossRef] [Medline]
    43. Dennis-Tiwary TA, Denefrio S, Gelber S. Salutary effects of an attention bias modification mobile application on biobehavioral measures of stress and anxiety during pregnancy. Biol Psychol 2017 Jul;127:148-156. [CrossRef] [Medline]
    44. Crane D, Garnett C, Brown J, West R, Michie S. Factors influencing usability of a smartphone app to reduce excessive alcohol consumption: think aloud and interview studies. Front Public Health 2017;5:1-19 [FREE Full text] [CrossRef] [Medline]
    45. Garnett C, Crane D, Michie S, West R, Brown J. Evaluating the effectiveness of a smartphone app to reduce excessive alcohol consumption: protocol for a factorial randomised control trial. BMC Public Health 2016 Jul 08;16:1-10 [FREE Full text] [CrossRef] [Medline]
    46. Torous J, Staples P, Fenstermacher E, Dean J, Keshavan M. Barriers, benefits, and beliefs of brain training smartphone apps: an internet survey of younger US consumers. Front Hum Neurosci 2016;10:1-8 [FREE Full text] [CrossRef] [Medline]
    47. Brown HR, Zeidman P, Smittenaar P, Adams RA, McNab F, Rutledge RB, et al. Crowdsourcing for cognitive science--the utility of smartphones. PLoS One 2014;9(7):e100662 [FREE Full text] [CrossRef] [Medline]
    48. Michel J. Mobile application removes societal barriers to P4 medicine. J Frailty Aging 2017;6(4):216-218. [CrossRef] [Medline]
    49. Price E, Moore G, Galway L, Linden M. Validation of a smartphone-based approach to in situ cognitive fatigue assessment. JMIR Mhealth Uhealth 2017 Aug 17;5(8):e125 [FREE Full text] [CrossRef] [Medline]
    50. Kim H, Sung JE, Kim J, Park EJ, Yoon JH, Yoo HJ, et al. Response time measurements in the iOS app-based Token Test, the Brief Token Test in the elderly. Geriatr Gerontol Int 2014 Oct;14(4):969-974. [CrossRef] [Medline]
    51. Suffoletto B, Goyal A, Puyana J, Chung T. Can an app help identify psychomotor function impairments during drinking occasions in the real world? A mixed-method pilot study. Subst Abus 2017;38(4):438-449. [CrossRef] [Medline]
    52. Jongstra S, Wijsman LW, Cachucho R, Hoevenaar-Blom MP, Mooijaart SP, Richard E. Cognitive testing in people at increased risk of dementia using a smartphone app: the iVitality proof-of-principle study. JMIR Mhealth Uhealth 2017 May 25;5(5):e68 [FREE Full text] [CrossRef] [Medline]
    53. Atkins AS, Tseng T, Vaughan A, Twamley EW, Harvey P, Patterson T, et al. Validation of the tablet-administered Brief Assessment of Cognition (BAC App). Schizophr Res 2017 Dec;181:100-106 [FREE Full text] [CrossRef] [Medline]
    54. Ocklenburg S, Ströckens F, Bless JJ, Hugdahl K, Westerhausen R, Manns M. Investigating heritability of laterality and cognitive control in speech perception. Brain Cogn 2016 Dec;109:34-39. [CrossRef] [Medline]
    55. Pal R, Mendelson J, Clavier O, Baggott M, Coyle J, Galloway G. Development and testing of a smartphone-based cognitive/neuropsychological evaluation system for substance abusers. J Psychoactive Drugs 2016;48(4):288-294. [CrossRef] [Medline]
    56. Sangha S, George J, Winthrop C, Panchal S. Confusion: delirium and dementia: a smartphone app to improve cognitive assessment. BMJ Qual Improv Rep 2015;4(1) [FREE Full text] [CrossRef] [Medline]
    57. Robbins R, Gouse H, Brown H, Ehlers A, Scott T, Leu C, et al. A mobile app to screen for neurocognitive impairment: preliminary validation of neuroscreen among HIV-infected South African Adults. JMIR Mhealth Uhealth 2018 Jan 05;6(1):e5 [FREE Full text] [CrossRef] [Medline]
    58. Robbins RN, Brown H, Ehlers A, Joska JA, Thomas KGF, Burgess R, et al. A smartphone app to screen for HIV-related neurocognitive impairment. J Mob Technol Med 2014 Feb 01;3(1):23-26 [FREE Full text] [CrossRef] [Medline]
    59. Lorusso ML, Biffi E, Molteni M, Reni G. Exploring the learnability and usability of a near field communication-based application for semantic enrichment in children with language disorders. Assist Technol 2018 Feb;30(1):39-50. [CrossRef] [Medline]
    60. Hill N, Mogle J, Colancecco E, Dick R, Hannan J, Lin F. Feasibility study of an attention training application for older adults. Int J Older People Nurs 2015 Sep;10(3):241-249 [FREE Full text] [CrossRef] [Medline]
    61. Hill NL, Mogle J, Wion R, Kitt-Lewis E, Hannan J, Dick R, et al. App-based attention training: incorporating older adults' feedback to facilitate home-based use. Int J Older People Nurs 2018 Mar;13(1):1-10 [FREE Full text] [CrossRef] [Medline]
    62. Lu M, Lin W, Yueh H. Development and evaluation of a cognitive training game for older people: a design-based approach. Front Psychol 2017;8:1837 [FREE Full text] [CrossRef] [Medline]
    63. Shellington EM, Felfeli T, Shigematsu R, Gill DP, Petrella RJ. HealthBrain: an innovative smartphone application to improve cognitive function in older adults. Mhealth 2017;3:17 [FREE Full text] [CrossRef] [Medline]
    64. Arean PA, Hallgren KA, Jordan JT, Gazzaley A, Atkins DC, Heagerty PJ, et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res 2016 Dec 20;18(12):e330 [FREE Full text] [CrossRef] [Medline]
    65. Powell LE, Wild MR, Glang A, Ibarra S, Gau JM, Perez A, et al. The development and evaluation of a web-based programme to support problem-solving skills following brain injury. Disabil Rehabil Assist Technol 2017 Oct 24:1-12. [CrossRef] [Medline]
    66. Tacchino A, Pedullà L, Bonzano L, Vassallo C, Battaglia MA, Mancardi G, et al. A new app for at-home cognitive training: description and pilot testing on patients with multiple sclerosis. JMIR Mhealth Uhealth 2015 Aug 31;3(3):e85 [FREE Full text] [CrossRef] [Medline]
    67. Bless J, Westerhausen R, Kompus K, Gudmundsen M, Hugdahl K. Self-supervised, mobile-application based cognitive training of auditory attention: a behavioral and fMRI evaluation. Internet Interv 2014;1(3):102-110. [CrossRef]
    68. Zmily A, Mowafi Y, Mashal E. Study of the usability of spaced retrieval exercise using mobile devices for Alzheimer's disease rehabilitation. JMIR Mhealth Uhealth 2014 Aug 14;2(3):e31 [FREE Full text] [CrossRef] [Medline]
    69. Blackburne T, Rodriguez A, Johnstone SJ. A serious game to increase healthy food consumption in overweight or obese adults: randomized controlled trial. JMIR Serious Games 2016 Jul 13;4(2):e10 [FREE Full text] [CrossRef] [Medline]
    70. Kondylakis H, Koumakis L, Tsiknakis M, Marias K, Genitsaridi E, Pravettoni G, et al. Smart recommendation services in support of patient empowerment and personalized medicine. Multimed Serv Intell Environ Springer 2013:39-61. [CrossRef]
    71. Kondylakis H, Koumakis L, Genitsaridi E, Tsiknakis M, Marias K, Pravettoni G, et al. IEmS: A collaborative environment for patient empowerment. 2012 Presented at: IEEE 12th Int Conf Bioinforma Bioeng; 2012; Larnaca p. 535-540. [CrossRef]
    72. Kondylakis H, Kazantzaki E, Koumakis L, Genitsaridi I, Marias K, Gorini A, et al. Development of interactive empowerment services in support of personalised medicine. Ecancermedicalscience 2014;8:400 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    COGNI-TRAcK: Cognitive Training Kit


    Edited by G Eysenbach; submitted 23.04.18; peer-reviewed by C Păsărelu; comments to author 06.06.18; revised version received 02.07.18; accepted 16.07.18; published 19.03.19

    ©Laura Vergani, Giulia Marton, Silvia Francesca Maria Pizzoli, Dario Monzani, Ketti Mazzocco, Gabriella Pravettoni. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 19.03.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.