In this comprehensive analysis of registered mHealth studies of interventions for disabling neuropsychiatric conditions, there were 6 key findings. First, there has been a large increase in the number of clinical trial registrations in the past 2 years; almost half (44.5%) of the trial registrations were registered in the past 2 years. Thus, despite the increasing additions of health-based smartphone apps [1], this snapshot of ClinicalTrials.gov suggests that only a few such apps for high-burden neuropsychiatric conditions are being clinically evaluated in trials. Second, the studies were generally located in the United States, but a few of the studies were funded by the NIH. Third, the study characteristics were such that they would not generally be considered as high-quality evidence and for use in guideline recommendations because of small sample sizes and heterogeneous interventions. Fourth, the mean study duration has not changed with time, suggesting that long-term outcomes are still not the focus of research. Despite the myriad of ways in which results can be reported, a few trials had results reported either as entered on ClinicalTrials.gov or as study results automatically indexed to ClinicalTrials.gov. As stated earlier, only 6.7% (9/135) of such trials that have been registered on ClinicalTrials.gov have reported results. Overall, 18.5% (25/135) of trials had results reported in some capacity (either under the results section of ClinicalTrials.gov or through publications indexed in databases, which were automatically associated with the ClinicalTrials.gov identifier). Fifth, initially specified outcomes were changed after trials commenced in a quarter of all trials. Finally, study duration and sample size of those enrolled are associated with the reporting results of studies.

More trials are being registered on ClinicalTrials.gov, given the requirements for prospective registration as a condition of publication. A previous study examined clinical trial registration for 3 groups of disorders (ie, cardiovascular disorders, mental health disorder, and oncologic disorders) that comprise the largest number of DALYs lost in the United States. The authors found that the number of trials submitted for registration between October 2004 and September 2007 and then October 2007 and September 2010 increased by about 140% from 28,881 to 40,970 [9]. A study assessing trial registration revealed that a few trials still reported on the trial registries [154]. However, more researchers are learning about this requirement, hence the expected increase in trial registrations.

Despite the increase in trial registrations, only 135 trials met the criteria for this study. Thus, a few apps for high-burden neuropsychiatric conditions are being clinically evaluated in the trials reported on ClinicalTrials.gov. A recent study of top-funded digital health companies examined the number of research studies collectively undertaken and found that of the top 10 disease categories examined, 7 were neuropsychiatric. However, none of these industry studies reported on the clinical effectiveness of the digital health tools for these high-burden conditions. Thus, although there is high interest in digital health toward neuropsychiatric conditions, there is little registered evidence that such apps on the market work [155].

A majority of the studies were based in the United States, followed by Europe. This is not because ClinicalTrials.gov is based in the United States, as ClinicalTrials.gov accounts for more than 80% of all the clinical studies in the WHO portal [9]. There may be fewer studies in Europe because of the newly implemented General Data Protection Regulation in 2018, which is a European Union (EU) law for the protection of data and privacy for all individuals within the EU and the European Economic Area. It also concerns the export of personal data outside the EU.

A few (26%) of the trials were NIH funded. This is not surprising as much of the research in digital health has occurred in the private sector. In 2016, 296 private digital health companies received venture funding that totaled to more than US $4.2 billion and approached US $6 billion in 2017 [155]. The 20 top-funded, private US-based digital health companies were studied to analyze their products and services, peer-reviewed evidence, and the potential for impact on patients with high-burden conditions. Less than one-third (27.9%) of the studies targeted patients with a high-burden condition. Only 16 (15%) studies assessed the clinical effectiveness of the product or service, and only 8 (8%) studies assessed the clinical effectiveness of the product or service in a high-burden or high-risk factor population. Only a small number of studies published data, and interestingly, journals without impact factors were the most common (31%) source of publications [155].

There was tremendous heterogeneity in the purpose of the use of the apps not only across conditions but also within a given condition. The most common purpose across all conditions was symptom tracking and medication adherence. There were also some studies based on skills learning, for example, cognitive behavioral therapy or progressive muscle relaxation, and an app was used to help with the delivery of these skills. A small number of studies were designed to detect physical symptoms or signs, such as atrial fibrillation and seizure, or enhance communication with clinicians using apps connected to sensors and/or devices. This latter intervention was to provide support to patients, in some cases with the intent to prevent relapse or indicate the degree of symptoms to potentially lead to a change in medication management if warranted.

In terms of study design, the studies were generally small (samples sizes<100). This study’s results were comparable with the study examining the ClinicalTrials.gov registration of interventional studies for cardiovascular disease, mental health, and oncology, which found that 62% of the trials had an anticipated enrollment mean of less than 100 [9]. This study’s findings were also comparable with a study of trials conducted by the top 20 funded digital health companies, which found that 51% of the studies had less than 100 participants [155]. Thus, these studies will not be able to help in creating major guidelines with high-quality evidence related to these disabling neuropsychiatric conditions.

Although this study was not designed to assess why there is low registration of trials, the potential reasons may include lack of awareness and different prioritization in the app development industry. “Given the diversity of stakeholders involved in mHealth research, competing outcomes, priorities, funding, and publication requirements may potentially mean some studies are less likely to be registered” [156].

This study’s results are in line with previous work showing that few studies reported published results on ClinicalTrials.gov. In a study examining trial registration compliance in publications related to headache, only 26% of all the studies that should have been registered were indeed registered [154]. A recent study of 556 trial registrations on ClinicalTrials.gov showed that out of all the trials in the study, 150 (27%) trials remained unpublished 5 years after the study completion dates [157]. There are a number of potential reasons for the low publication rates. First, the lack of reporting of negative results is a well-known phenomenon in academic medicine [6,158,159]. For example, some trials of triptans [160] and gabapentin [154,161] were never published. Second, in mHealth, there are likely specific challenges to publication, including high attrition rates, usability issues, and lack of sufficient previous formative research [162]. Third, previous research has shown that clinical trials with large sample sizes were more likely to be published [154,158,162,163]. Many of the mHealth clinical trial registrations in this study’s sample sizes were considerably small.

As noted previously, the study outcomes were changed. In a systematic review in 2011 assessing the transparency of outcome reporting and trial registration of randomized controlled trials and top psychosomatic and behavioral health journals, of the 63 articles meeting the study criteria, only 25 (39.7%) articles had adequately declared primary or secondary outcomes [164]. Thus, this study’s results are in line with previous research in the field of mental health. This is an especially prevalent concern in digital health, where researchers could easily change their outcomes and conduct selective analyses [165].

Finally, this study’s findings suggest that study status is not associated with either estimated or actual sample size or length of the intervention. Study status may not be updated in real time; thus, it may not reflect the true status of the study. This study’s findings that studies never reporting any results have a longer duration than those reporting results are logical in the sense that shorter studies may be easier to complete. Similarly, this study’s findings that studies never reporting any results have a higher mean number of participants compared with studies that did report results make sense, as larger studies are more likely to be difficult to complete. In sum, these findings that shorter and smaller studies are associated with reporting results compared with longer and larger studies are intuitive and reflect that digital health studies have the same challenges that all clinical studies face in terms of reaching reporting status.

One of the limitations of this study is that we may have failed to identify some mHealth studies due to the sampling methodology. The primary search terms that we used were “mHealth,” “smartphone,” “electronic diary,” and “mobile technology,” while specifying subcategories of the different neuropsychiatric conditions (ie, migraine, migraine with aura, migraine without aura, and migraine disorders). Other search terms such as “digital,” “ecological momentary assessment,” “experience sampling,” or “log” may have captured additional studies. We only searched ClinicalTrials.gov, and other trial registration websites exist. Second, many of the studies are recent, and thus authors may not have completed their studies and posted results yet. Third, the potential usefulness of different behavioral interventions administered through smartphone apps is not listed on ClinicalTrials.gov and is likely unknown; it is generally the point of the studies to assess whether these interventions may be effective. Finally, the information on ClinicalTrials.gov will always be incomplete for two main reasons: (1) Individual studies may not be registered in the database. Second, information entered on ClinicalTrials.gov may be incomplete, for example, certain data elements may have had a different format or structure or may have been optional when the study information was initially entered. (2) There are few incentives to motivate responsible parties to update their studies [150] registered on ClinicalTrials.gov. One study found that 17% to 20% of the studies on ClinicalTrials.gov were observational and only 7% had posted results [150,151]. Inferential statistics were limited based on reports from individual studies.

The study information on ClinicalTrials.gov is helpful for understanding the landscape of smartphone-based studies for neuropsychiatric conditions. However, the information listed does not offer enough detail to fully understand the nature of smartphone apps and sensor data collection. Future solutions may include the posting of web-based demonstrations of apps being studied or links to the version of the app used in trials. In addition, it is clear that many of the health-based apps are being developed by the private sector. Efforts need to be made to encourage commercial companies to register their studies on ClinicalTrials.gov and to adhere to the trial registration guidelines, for example, report trial results in a timely manner, as described by ClinicalTrials.gov. For health-based apps that make claims of efficacy, there needs to be stringent oversight of the registered clinical trials.

Despite the increasing use of health-based smartphone apps by the general public, only a few such apps are rigorously evaluated in clinical settings. Similar to other research on the studies registered on ClinicalTrials.gov, studies of the top neuropsychiatric conditions involving mHealth, registered on ClinicalTrials.gov, tended to be small, and there was a large amount of heterogeneity in the methods (types of interventions), duration, and reporting methods [9]. Moreover, very few registered studies (6.7%) reported their results, raising the question of whether the burgeoning creation of mHealth-based interventions is efficacious, despite these apps being widely downloaded and used. There were few studies for the most disabling neuropsychiatric conditions that typically use electronic diaries for the self-management of migraine and epilepsy. Future work should focus on studying the efficacy of these mHealth interventions for neuropsychiatric conditions if they are to be used by patients with these disabling conditions. Such studies should be registered on ClinicalTrials.gov to ensure transparency and so that the public can also learn about the research being conducted using these interventions.