Advantages associated with real-time (or near real-time) methods of assessing alcohol consumption behavior for research purposes have been widely documented in recent years [1-3]. Such methods—which are increasingly electronic in nature—are advocated on the basis of how they allow data to be captured repeatedly in the natural environment and in the absence of the researcher [2,3]. This facilitates the collection of actual intake information rather than the summary data commonly elicited from more traditional retrospective methods of assessing drinking. As such, recall and response biases are thought to be minimized and the ecological validity of the data is consequently enhanced [3]. Real-time methods additionally reduce the quantity of missing data and can yield a diverse and potentially very large sample [4,5]. Crucially, real-time data enable variations in behavior to be examined across time and in concert with cognitive, affective, environmental, and physiological factors. In this way, the antecedents and consequences of drinking behavior can also be investigated [2,3].

Real-time electronic methods of collecting alcohol consumption information for research purposes have evolved rapidly over the last two decades. Early studies typically featured hand-held electronic devices [6,7] or interactive voice response systems [8,9]. In the case of the former, participants used the device as an electronic diary; in the latter, the interactive voice response system was programmed to call participants on researcher-supplied cellular phones so that an automated questionnaire could be administered. Short message service (SMS) text messaging protocols have also been employed. In such studies, SMS text messages direct participants to follow a link to complete a Web-based survey via their mobile browser [10] or ask them to respond to simple questions about their alcohol consumption via text [11]. With the advent of smartphones, researchers have increasingly been programming study-specific apps for use in such studies. In most cases, this involves either providing participants with a phone preloaded with the app [12,13] or loading the app onto participants’ own phones and downloading the data at the end of the experimental period [14].

There are a number of disadvantages associated with utilizing the aforementioned electronic protocols. There are costs, for instance, associated with programming, supplying, and training participants to use electronic devices that are not their own [15-18]. Participants must also be provided with some means whereby they can recharge these devices [16]. If they have their own mobile, they will be carrying 2 devices during the experimental period, which might prove unduly burdensome and reduce compliance. Questionnaires completed via cell phone browsers do not always scale well to all mobile devices and require internet connectivity [19]. Surveys conducted via SMS text messages are necessarily limited in scope and, in the absence of internet connectivity, potentially costly to participants [20]. In all cases, participants are required to visit the lab at least twice during the experimental period; they must collect the device or have the program loaded onto their own phone plus undergo training, and they must return the device or have the data downloaded off their phone [18,21]. These disadvantages potentially diminish some of the benefits of using electronic devices to collect real-time alcohol intake behavior; specifically, sample size and diversity may be a function of the cost of supplying the device to those willing and able to attend the lab, and data may go unrecorded where internet connectivity or cost to the participant is an issue.

Using apps participants download to their own smartphones, without ever visiting a lab, might enable researchers to more fully realize the benefits of collecting alcohol consumption information electronically and in real time. Smartphone penetration currently stands at upward of 70% across many developed nations and is growing rapidly across developing ones [22]. Across the United States, United Kingdom, and Australia, more than 93% of 18 to 34 year olds own a smartphone [23]. Taking advantage of high smartphone ownership rates by using participants’ own devices represents a substantial cost reduction both in terms of equipment and training. Asking individuals to download apps via marketplace vendors and having data stored on the phone until it can be automatically uploaded to a server via a Wi-Fi connection alleviates the participants’ need to visit the lab and reduces the likelihood of data being lost or going unrecorded. Advantages of real-time assessment related to completeness of data, sample size, and diversity might therefore be preserved in studies employing apps downloaded to participants’ smartphones.

Nonetheless, there are a number of development and methodological concerns pertaining to assessing alcohol consumption behavior via smartphone apps that warrant further investigation. Given that any app must be programmed for 2 different, frequently updating operating systems with distinct deployment protocols, researchers are likely to require the assistance of 1 or more app programmers in the development phase. The literature offers little guidance regarding the software development process as it relates to the behavioral sciences [24]. Examples of how to effectively manage and integrate the requirements and expectations of multiple stakeholders when developing apps for behavioral studies are consequently required.

Although several recent studies have reported promising results with regard to the validity of app-based methods of capturing alcohol intake information [25-27], issues related to app protocol compliance and reactivity have received less attention. Compliance refers to the extent to which participants adhere to study requirements and protocols throughout the experimental period, whereas reactivity describes a process whereby the monitoring of a behavior results in a change in that behavior [1-3]. Simply completing traditional alcohol intake assessment and screening measures has been found to decrease consumption among heavy drinkers and reduce self-reported hazardous drinking behaviors [28-30]. However, it is unclear if this reactivity arises as a function of drinking behavior. Individuals characterized by more risky consumption patterns may, more likely than others, alter their behavior in response to monitoring. Although commentators generally report little evidence of reactivity in real-time studies of alcohol intake, they also note further investigation is required [1,21,31-33]. Reactivity is a phenomenon that can emerge for several reasons. Participants might become more aware of the behavior and become consequently motivated to implement change, or the demands of the protocol might provoke a tendency to satisfice [31]. In any real-time study of alcohol intake, a decrease in drinking could be evidence of the former, whereas a decrease in responding “yes” to the behavior (and therefore in having to respond to further questions) might suggest the latter.

In this paper, we describe the process used to guide the development of a custom-built smartphone app designed to capture alcohol intake behavior in the healthy population for research purposes. We also evaluate methodological issues related to compliance with and reactivity to study protocols as a function of hazard and nonhazard drinker status. We expect individuals in both hazard and nonhazard groups will be equally compliant. Reactivity in the form of a decrease in drinking will be minimal or confined to the hazard group, whereas individuals in both groups will be susceptible to reactivity in the form of reduced responding.

The CNLab-A app represents the outcome of an iterative development process that included elements of agile software design, namely requirements analysis, feature and interface design, and app implementation [24,34].

This study was based on data from 671 participants (mean age 23.12 years, SD 7.24; range 16-56 years, 70% female) that form a subset of an ongoing project—entitled CheckMyControl—investigating the relationship between alcohol use and various social and cognitive factors in the healthy population. This subsample completed the app component of the project (Figure 2). Participants were recruited via adverts posted in and around the University of Melbourne, researcher networks, and social media posts; as such, they formed a convenience sample. Individuals were eligible to take part in this study if they were fluent in English and aged 13 years or older. The University of Melbourne Human Research Ethics Committee approved the study in accordance with the standards for ethical research of the National Health and Medical Research Council.

After reading a plain language statement and providing informed consent, participants answered a Web-based researcher-devised demographic survey and the Alcohol Use Disorders Identification Test (AUDIT) [39]. The demographic survey included questions pertaining to country of birth, first language, and place of residence. Participants were also asked if they had ever been diagnosed with an alcohol or substance use disorder. Participants were then required to download and use the CNLab-A smartphone app to record alcohol use for 21 days.

They were compensated Aus $10 for time spent completing Web-based surveys and Aus $0.50 each day information about alcohol consumption was submitted via the app (regardless of whether alcohol had been consumed or not). Participants received a bonus Aus $9.50 if app data were submitted on all 21 days. The maximum participants could be reimbursed was Aus $30.

The AUDIT is a 10-item screening measure that asks participants to respond to questions assessing alcohol intake, problems, and dependence with reference to the preceding 6 months. Participants were categorized into hazard (n=286) and nonhazard (n=385) groups based on their score, with scores of 8 or more indicative of hazardous alcohol consumption [45]. Previous studies have shown the AUDIT is a valid instrument for assessing alcohol misuse among adolescents—sensitivity (Sn)=.71; specificity (Sp)=.84—university students (Sn=.75-.89; Sp=.73-.82), and the general population (Sn=.92; Sp=.94) in various countries, including Australia (Sn=.93; Sp=.82) [39,46-48].

CNLab-A is a freely available custom-built app that can be used to record alcohol intake for research purposes. Once downloaded, CNLab-A requires participants to allow it to send them notifications. One notification is preset to 8 am, whereas the other can be set to suit the user. Although participants are directed at the outset to record alcohol consumption as it happens (or as soon thereafter as possible), notifications serve to prompt individuals to input information twice daily in case they neglect to do so when drinking. Thus, alcohol intake data can be submitted at any time, either in response to notifications or while drinking. A unique identification code, provided to participants during the Web-based component of the study, is required before the app opens. CNLab-A has previously been found to be a valid measure of alcohol intake [27].

Independent t tests and Chi-square analyses were conducted to determine whether hazard and nonhazard groups were matched demographically. Homogeneity of variance was assessed using Levene Test, where this assumption was violated, adjusted t values and associated degrees of freedom were reported. Effect sizes were computed for t tests using r values; they were interpreted according to Cohen guidelines: 0.10=small, 0.30=moderate, and 0.50=large effect [49].

Participants were considered compliant with app protocols if they responded to at least 1 notification each day for the 21-day experimental period. We assessed both number of days of use and submissions per day as a function of gender, age bracket (13-19 years, 20-29 years, and ≥30 years), and hazard and nonhazard group status using t tests and 1-way analysis of variance. Reactivity was assessed using linear growth model analyses, conducted using SPSS 24 (IBM). These analyses examined (1) change in drinks consumed each day and (2) change in daily (“yes”) responding over the 21-day experimental period as a function of AUDIT group membership. In the first analysis, drinks per day (within persons; level 1) were nested within individuals (between persons; level 2); in the second, daily (“yes”) response rate (within persons; level 1) was nested within individuals (between persons; level 2). Time was centered on the first day of data collection (Day 0); each unit of time represented an interval of 1 day [50]. All mixed models were estimated using restricted maximum likelihood [51]. Initially, slope variance and autocorrelation were included in both models, but they were removed when parameter estimates for these effects were found to be very small (<0.01).

At the time of testing, 39.9% (268/671) of the sample was aged under 20 years, 46.2% (310/671) was aged 20 to 29 years, and 13.9% (93/671) was aged 30 years or over. In Australia, individuals in the 18 to 29-year-old age bracket are more likely than others to consume alcohol in a manner that increases their short-term risk of alcohol-related injury, whereas almost 19% are at risk of long-term alcohol-related harm [52]. Most participants were born in Australia (66.8%, 448/671), spoke English as their first language (80.0%, 537/671), and lived in urban regions (88.1%, 591/671). Census data show 67% of the Australian population is locally born and 79% speak English as their first language [53]. Most Australians (86%) reside in urban regions [54]. A small number of participants indicated they identified as an Indigenous Australian (0.6%, 4/671). This was less than what the census data suggest as typical for the state (Victoria) in which this study took place (0.8%) [53]. However, this difference is likely a product of our recruitment campaign. As advertisements for the study were posted in and around the University of Melbourne, the sample contained a large number of young tertiary-aged participants (85.7%, 575/671). The proportion of indigenous students studying at this institute in 2016 stood at 0.6% [55].

Before commencing the 21-day recording period, the minority of participants (2.2%, 15/671) indicated they had never consumed alcohol; the majority (92.8%, 623/671) described themselves as regular drinkers. A small proportion indicated they had been diagnosed with an alcohol (0.3%, 2/671) or substance (0.4%; 3/671) use disorder. Hazard and nonhazard groups did not differ significantly with regard to age, t₆₆₉=1.02, P=.31 or years of education, t_654.11=1.65, P=.10. There was a significant association between gender and AUDIT group membership, χ²₆₇₁=5.96, P=.02. The odds of being a hazardous drinker were 1.51 times higher for males than females (95% CI 0.22-2.01). There were significant differences between the AUDIT scores of the hazard (mean 12.45, SD 4.30) and nonhazard (mean 4.64, SD 3.59) groups, t_547.67=24.94, P<.001. Mean alcohol intake indices recorded via CNLab-A are detailed in Table 2.

On average, participants used CNLab-A 20.27 (SD 1.88) days out of 21 (96.5%); 96.0% (644/671) of participants completed at least 1 submission per day for the entire 21-day experimental period. There were no significant differences as a function of gender t₆₆₉=0.83, P=.41, or age bracket, F_2,668=0.34, P=.71, with regard to number of days of use. Moreover, there were no differences between hazard and nonhazard groups, t₆₆₉=0.69, P=.49. As data submission was either event- or notification-contingent, there was no upper limit to the number of drinking sessions participants could report using the app. Participants received a maximum of 42 notifications asking them to record information about their drinking. They submitted data, on average, 2.00 (SD 0.41) times per day.

With regard to the average number of responses per day, there were no significant differences as a function of gender, t₆₆₉=0.65, P=.51, or age bracket, F_2,668=1.08, P=.34. In addition, there were no differences on the basis of AUDIT group membership, t₆₆₉=0.61, P=.54. There were 27,355 data points captured via the app in total.

Table 3 details parameter estimates for fixed and random effects of the linear growth analysis model for change in drinks consumed each day. With regard to fixed effects, the nonhazard group reported consuming significantly fewer standard drinks (0.67) on Day 0 than the hazard (0.67+1.48=2.15) group. The nonhazard group showed no significant decrease in consumption over the 21-day experimental period, whereas the hazard group demonstrated a slight but significant decrease in drinking over the same period (0.04 standard drinks per day). Within-person variance (7.54) equates to 2.75 SD units; thus, 95% of observed residuals were between ±5.49 units of their fitted values. The intercept variance (0.74) corresponds to 0.86 SD units; therefore, 95% of the population varied between ±1.72 units of the typical intercept for their group (Table 3).

Table 4 shows parameter estimates for fixed and random effects of the linear growth model for change in daily (“yes”) responses. The nonhazard group responded (“yes”) significantly less often (0.32) on Day 0 than the hazard (0.32+0.15=0.47) group. A slight but significant decrease in responding was evident for both the nonhazard (–0.003 per day) and hazard (–0.003+(– 0.004)=–0.007 per day) groups. Within-person variance (0.14) equates to 0.37 SD units; thus, 95% of observed residuals were between ±0.75 units of their fitted values. The intercept variance (0.08) corresponds to 0.28 SD units; therefore, 95% of the population varied between ±0.57 units of the typical intercept for their group (Table 4).

In this study, we aimed to describe the development and implementation of a custom-built smartphone app devised to measure real-time (or near real-time) alcohol consumption behavior in the healthy population. Designed for use in the research arena, the app was a product of an iterative process that included elements typical of agile software design. Decisions made during each phase of development were informed by a desire to create an app that the participants could download onto their own smartphones without ever having to visit the lab. We anticipated this might minimize disadvantages—such as equipment and training costs, participant burden, and missing data—often associated with using some types of electronic protocols, while simultaneously enhancing benefits pertaining to sample size and diversity. We additionally explored methodological factors related to app protocol compliance and reactivity as a function of hazard and nonhazard drinker status.

Compliance with app protocols was high. Participants were required to submit data about their drinking (regardless of whether they had consumed alcohol or not) at least once per day for 21 days. On average, they uploaded data on 96.5% of days, and there were no differences between hazard and nonhazard groups with regard to the number of days of app use or the number of responses per day. Presumably, the use of daily payments and an end-of-study bonus (for 21 consecutive days of data submissions) incentivized responding. In previous alcohol intake-based studies utilizing various electronic methods of collecting data—including SMS text messaging [56], hand-held electronic devices [57], and interactive voice response systems [8]— incentivized responding resulted in similarly high rates of compliance. It should be noted that when the measure is freely available via marketplace vendors, the size of the sample can quickly balloon. As such, it is important to not only balance the burden of protocol compliance with the incentive offered but it is also necessary to consider overall budget constraints and ensure there are some swift means of limiting access to the app if required.

Our results suggest there was some slight degree of reactivity—particularly among hazardous drinkers—to the app protocol. Though the effect was small, the hazard group decreased their intake significantly over the experimental period: 0.80 standard drinks in 21 days, which represented a 2% decrease in total standard drinks for this group. By contrast, nonhazard drinkers showed no significant reduction in consumption over the same period. This accords with evidence from other studies demonstrating some reduction in alcohol consumption only because of measurement among hazardous drinkers [28,29]. Even though participants received no feedback about their drinking during the assessment period, it is possible that those in the hazard group were motivated to modify their intake because the act of recording it made them more aware of their behavior. Equally, the knowledge that they were being monitored may have induced them to drink less. Considered a manifestation of the Hawthorne effect, social desirability is thought to underpin this type of assessment reactivity [58]. It is also possible that reductions in consumption were the result of satisficing; that is, participants may have responded “yes” to drinking less often over time to avoid having to submit further information via the app. Both hazard and nonhazard groups showed a significant reduction in the frequency of responding “yes” to drinking over the 21-day period. However, the rate of this reduction was very small: nonhazard participants decreased “yes” responding by 0.06 and hazard participants by 0.14 in 21 days. As such, this reduction might be a reflection of increased familiarity with the app over time, rather than satisficing; that is, participants may have summarized their drinking across a day into fewer submissions once they became more familiar with the app.

There is some debate in the literature regarding reactivity to real-time measures. Several investigators postulate such assessment reduces the likelihood of reactivity related to social desirability as participants record data in the absence of the researcher [31]. Bates and Cox found participants were, for example, more likely to reveal lifetime alcohol consumption details when they completed surveys outside, as opposed to inside, the lab [59]. Other researchers speculate real-time methods are particularly susceptible to reactivity effects because assessments are completed in close proximity to the behavior, giving participants time to consider their actions [1], though it has also been suggested that repeated surveying may reduce reactivity via habituation [60,61]. Nonetheless, there is consensus that reactivity is generally an overlooked facet of real-time research and further investigation is required [21,31,62]. It is possible, for instance, that reactivity differs according to the population. Our finding that hazardous drinkers reacted to the app protocol to a greater extent than nonhazard drinkers would certainly support this supposition. Our data also suggest app-based alcohol-related intervention studies would benefit from the inclusion of a measurement-only control condition to disentangle the effects related to reactivity from those linked to the intervention.

Several limitations to this study must be noted. Although representative of the population at large in terms of country of birth, first language, and usual place of residence, young female university students predominated in our sample. Although current research suggests men and women are equally likely to download and use health-orientated smartphone apps [63,64], women in Australia typically drink less than men and are consequently at decreased risk of alcohol-related harm [52]. Nonetheless, our hazard group reported drinking alcohol a third of the time during the experimental period and, on average, consumed in excess of 5 standard drinks per episode. According to Australian guidelines, this pattern of consumption places them at increased risk of both short-term alcohol-related injury and lifetime harm from alcohol [65]. Notwithstanding, future studies would be required to determine if similar rates of app compliance and reactivity are evident in a sample dominated by young men. Similarly, older individuals might respond in different ways to app-based protocols designed to assess alcohol consumption behavior. A recent survey found 59% of Australians over 65 years are willing to use or already utilize technology designed to track health [66]. In addition to research showing age has not been identified as a barrier to participation in mental health studies [67], this suggests older individuals might respond well to alcohol-related research apps. Moreover, as age-related illness and associated difficulties pertaining to condition severity, transportation, and inconvenience do inhibit participation in research [67], apps people access via marketplace vendors and download to their own smartphones may possibly boost participation in this age group. Future studies would nonetheless be required to fully consider app compliance and reactivity among older individuals. Finally, we did not examine if different assessment periods impact compliance and reactivity in diverse ways. A shorter experimental period might, for instance, diminish the effect of reactivity, though this may limit how effectively the app captures variability of intake. This is another potential area for further research.

In conclusion, we examined the feasibility of developing and employing an app—downloaded by participants to their own smartphones—designed to collect alcohol intake information for research purposes. We demonstrate how utilizing apps such as CNLab-A can yield a potentially large sample representative of the population. Both hazard and nonhazard participants appeared highly compliant when using app protocols. Although there was some evidence of reactivity in our study, especially among hazardous drinkers, effect sizes were small. Our findings suggest the CNLab-A app—or potentially 1 similar—is a methodologically sound means of examining alcohol consumption behavior across time. In future, such apps can be paired with those that chart cognition, affect, or social and environmental factors in real time to facilitate a more thorough investigation of the antecedents and consequences of drinking behavior.