Compared with data from the Health Survey for England (2011) [23], a greater proportion of participants in this study were women (80.52%, 5129/6370 compared with 55.61%, 4782/8599 expected). The age bands 35 to 64 years were over-represented in this study (73.11%, 4657/6370 compared with 51.18%, 4401/8599 expected; Table 2), with fewer participants in the extremes of age (<35: 14.65%, 933/6370 compared with 24.7%, 2126/8599 expected; ≥75: 1.19%, 76/6370 compared with 11.13%, 957/8599 expected).

Following inspection of the log-likelihood plot (Figure 3), a four-cluster solution was retained. The clusters (Figure 4) were allocated names based on the best description of their engagement patterns: high engagement (14%, 865/6370; red), moderate engagement (22%, 1384/6370; purple), low engagement (39%, 2503/6370; green), and tourists (25%, 1618/6370; teal).

The proportion of days on which data were entered and rates of data completion varied substantially between clusters (Table 3).The median days “in study” ranged from 175 days (IQR: 152-177) in the high-engagement cluster to 1 day (IQR: 1-1) in the tourist cluster. Participants in the moderate-engagement cluster stayed in the study 10 times longer than those in the low-engagement cluster (88 days, 42-163 vs 8 days, 4-16).

Those in the high-engagement cluster provided data on most days throughout follow-up (Figure 4). The high-engagement cluster reported complete motifs on 89.13% (106,360/119,332) of the days that they provided data, and the moderate-engagement cluster provided complete motifs on 87.5% (67,704/77,368) of all data-entry days. Rates of completion were slightly lower in the other clusters, with the low-engagement cluster and tourists recording complete motifs on 82.88% (13,415/16,186) and 64.85% (1947/2848) of the days on which any data were reported, respectively (Table 3).

Higher engagement was associated with increased age, with a difference of more than 5 years between the median age of those who were in the low-engagement (47, IQR: 39-57), or tourist clusters (49, IQR: 40-58), and those who were in the high-engagement cluster (median 56 years, IQR: 47-63). A substantially lower proportion of those in the tourist cluster were women (76.27%, 1234/1618; 95% CI 74.2-78.3) than any other cluster (high engagement: 82.31%, 712/865; 95% CI 79.6-84.7; moderate engagement: 84.10%, 1164/1384; 95% CI 82.1-85.9; low engagement: 80.66%, 2019/2503; 95% CI 19.1-82.2).

There were no differences between clusters with respect to the site of pain or in the prevalence of rheumatic disease diagnoses (eg, rheumatoid arthritis, fibromyalgia). The proportion of people in the tourist cluster (17.74%, 287/1618; 95% CI 15.88-19.60) who reported “other” pain conditions was also lower than in the high-engagement (23.70%, 205/865; 95% CI 20.87-26.55), moderate-engagement (24.49%, 339/1384; 95% CI 22.22-26.76), and low-engagement (24.41%, 611/2503; 95% CI 22.73-26.09) groups.

No differences were observed between the clusters regarding the use of analgesics and steroids. Only the use of synthetic DMARDs differed substantially between the clusters, with less of those in the tourist cluster (16.63%, 269/1618; 95% CI 14.82-18.44) reporting taking the medication than those in the other engagement clusters (high engagement: 22.20%, 192/865; 95% CI 19.43-24.97; moderate engagement: 21.10%, 292/1384; 95% CI 18.95-23.25; low engagement: 21.13%, 529/2503; 95% CI 19.53-22.73). Comparable proportions were using biologic or other DMARDs.

There were no differences in the strength of belief that the weather affected pain, but fewer of those in the high-engagement cluster believed the cold affected their pain (62.31%, 539/865; 95% CI 59.08-65.54) when compared with those in the low-engagement and tourist clusters (71.87%,1799/2503; 95% CI 70.11-73.63 and 68.29%, 1105/1618; 95% CI 66.02-70.56, respectively). Conversely, more of those who were highly engaged (35.49%, 307/865; 95% CI 32.30-38.68) believed that changes in barometric pressure were associated with pain that those in the low-engagement and tourist clusters (28.35%, 714/2503; 95% CI 26.76-30.30 and 28.99%, 469/1618; 95% CI 26.78-31.20, respectively). There were no observed differences in the proportion of participants who believed their pain is associated with damp or rain, heat, or changes in temperature (Table 4).

Cloudy with a Chance of Pain is the first mHealth study to demonstrate successful and rapid mass recruitment of a largely representative sample of highly engaged participants. Among our sample, patterns of ongoing engagement showed that around 1 in 7 participants provided data on most days in the first 6 months, completing full data entry on 89% of those days.

A major strength of Cloudy with a Chance of Pain is the rapid mass recruitment of eligible participants. Our study benefitted from wide promotion by the UK national media at the time of the study launch, which emphasizes the power of national media to promote. Indeed, as a result of coverage including, among others, the BBC2 television show Trust Me I’m a Doctor on January 20, 2016 and BBC Breakfast on January 26, 2016, 90% of participants enrolled in the study by July 20 were recruited within 1 month of the study launch.

Furthermore, ongoing engagement within Cloudy with a Chance of Pain was high. More than 30% of participants were in the high-engagement or moderate-engagement cluster, entering data on at least half of days throughout the 6 months. In comparison, fewer than 25% of participants in Apple’s ResearchKit studies were active by 10 weeks [29], with similar proportions active in a physical-activity study by 42 days [14]. In one of the largest mHealth studies reported to date (mPower study of people with Parkinson disease and healthy controls), less than 10% of enrolled participants completed 5 or more days within the first 6 months of the study [5]. One in 7 participants were in the high-engagement cluster and provided data on most days throughout the 6 months; we are not aware of other mHealth studies that have reported such high levels of ongoing engagement to date.

Previous analyses have used arbitrary definitions that fail to capture the patterns of use through time and may ignore the importance of continuity of data entry [5,14-16]. In contrast, this analysis attempted to account fully for data complexity and made no a priori decisions to define engagement. Thus, this study has improved understanding of the extent to which participants remain engaged over time and provides a promising method for future engagement studies.

Our recruitment strategy enrolled a sample which comprised an under-representation of males and persons at the extremes of age (<35 years and ≥75 years) than would have been expected from the general population data of the Health Survey for England (2011) [23]. Although women are more likely to respond to more traditional population surveys [30-33], we recruited a much higher proportion of women than would have been expected using traditional recruitment methods. A possible explanation is that women recruited to this study more commonly viewed the television programs or may have perceived the potential for additional benefits for participation. However, we also note that self-selection likely accounts for the observed differences in our population. For example, not only are women known to use social media [34] and health apps more than men [35], but they also use digital content differently [36,37].

Therefore, future mHealth studies may benefit from the use of supplementary and targeted recruitment strategies used by other digital health interventions [38] in which it would be possible to oversample men, such as the use of health professionals, friends, and families, or work-based campaigns, as well as outreach programs designed to access hard-to-reach groups. Although similar methods could also promote the recruitment of younger adults, other opportunities to promote participation among this group include the use of social networks, community components, and app gamification [39-41].

Nevertheless, the internal validity of the study results (ie, the relationship between the weather and pain within our sample) is unlikely to be influenced by the excess of women entering the study, as there is no reason to suspect the relationship differs by sex. Analysis of the relationship between the weather and pain, and whether the relationship differs between the sexes, is underway and will be reported separately.

The impact on external validity (ie, the generalizability) is unclear, as people with a particular belief may have been more inclined to participate (which may, in turn, differ by sex). That said, our findings about beliefs align with prior research that suggests that as many as 92% of patients with arthritis believe in an association between weather and pain [42].

The reasons for the unprecedented rates of engagement observed in this study are worth exploring, particularly as we sought to collect a large amount of daily data, and this burden on the participants might well have been expected to result in a higher loss to follow-up through time and over such a long period. This study found that older participants were more likely to remain engaged in the study. One possible explanation for this is that older persons are less likely to use smartphone apps [1] and therefore may be less likely to experience “app fatigue” than younger participants. They may also feel a greater responsibility to complete the ongoing data entry once registered or have more time to give to the study. Furthermore, functionalities such as geolocation consume battery power, which may have a greater impact on younger persons, who use their smartphones for a greater number of varied tasks, than on older persons [1]. We note, however, that reasons for declining engagement are likely numerous.

Earlier studies have sought to examine possible mechanisms of engagement, including the complexity of tasks [3], the time of day data are entered [43], and various functionality features such as reminders, interactivity, tailored content, and delivery of feedback [14,15]. In a feasibility study [22], we reported that key motivators for ongoing engagement were the simple graphical user interface, automated reminders for data entry, a desire to contribute to answering an understandable and engaging research question, and visualization of data. However, limited information was available in this larger study to delineate the motivators of engagement in this population. We also acknowledge that the study did not capture education and income, which would have enabled this study to investigate the potential impact of the digital divide on recruitment and engagement.

In summary, Cloudy with a Chance of Pain demonstrates a rapid and successful recruitment of a large and engaged sample of people with chronic pain. Although there may be selection bias toward older females in our study, younger men are also less likely to participate in studies using traditional data-collection methods. Thus, our study provides strong evidence to suggest that smartphones could provide a viable alternative to traditional data collection methods, particularly for collecting daily data over long periods.