This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
There are disadvantages—largely related to cost, participant burden, and missing data—associated with traditional electronic methods of assessing drinking behavior in real time. This potentially diminishes some of the advantages—namely, enhanced sample size and diversity—typically attributed to these methods. Download of smartphone apps to participants’ own phones might preserve these advantages. However, to date, few researchers have detailed the process involved in developing custom-built apps for use in the experimental arena or explored methodological concerns regarding compliance and reactivity.
The aim of this study was to describe the process used to guide the development of a custom-built smartphone app designed to capture alcohol intake behavior in the healthy population. Methodological issues related to compliance with and reactivity to app study protocols were examined. Specifically, we sought to investigate whether hazard and nonhazard drinkers would be equally compliant. We also explored whether reactivity in the form of a decrease in drinking or reduced responding (“yes”) to drinking behavior would emerge as a function of hazard or nonhazard group status.
An iterative development process that included elements typical of agile software design guided the creation of the CNLab-A app. Healthy individuals used the app to record alcohol consumption behavior each day for 21 days. Submissions were either event- or notification-contingent. We considered the size and diversity of the sample, and assessed the data for evidence of app protocol compliance and reactivity as a function of hazard and nonhazard drinker status.
CNLab-A yielded a large and diverse sample (N=671, mean age 23.12). On average, participants submitted data on 20.27 (SD 1.88) out of 21 days (96.5%, 20.27/21). Both hazard and nonhazard drinkers were highly compliant with app protocols. There were no differences between groups in terms of number of days of app use (
Smartphone apps participants download to their own phones are effective and methodologically sound means of obtaining alcohol consumption information for research purposes. Although further investigation is required, such apps might, in future, allow for a more thorough examination of the antecedents and consequences of drinking behavior.
Advantages associated with real-time (or near real-time) methods of assessing alcohol consumption behavior for research purposes have been widely documented in recent years [
Real-time electronic methods of collecting alcohol consumption information for research purposes have evolved rapidly over the last two decades. Early studies typically featured hand-held electronic devices [
There are a number of disadvantages associated with utilizing the aforementioned electronic protocols. There are costs, for instance, associated with programming, supplying, and training participants to use electronic devices that are not their own [
Using apps participants download to their own smartphones, without ever visiting a lab, might enable researchers to more fully realize the benefits of collecting alcohol consumption information electronically and in real time. Smartphone penetration currently stands at upward of 70% across many developed nations and is growing rapidly across developing ones [
Nonetheless, there are a number of development and methodological concerns pertaining to assessing alcohol consumption behavior via smartphone apps that warrant further investigation. Given that any app must be programmed for 2 different, frequently updating operating systems with distinct deployment protocols, researchers are likely to require the assistance of 1 or more app programmers in the development phase. The literature offers little guidance regarding the software development process as it relates to the behavioral sciences [
Although several recent studies have reported promising results with regard to the validity of app-based methods of capturing alcohol intake information [
In this paper, we describe the process used to guide the development of a custom-built smartphone app designed to capture alcohol intake behavior in the healthy population for research purposes. We also evaluate methodological issues related to compliance with and reactivity to study protocols as a function of hazard and nonhazard drinker status. We expect individuals in both hazard and nonhazard groups will be equally compliant. Reactivity in the form of a decrease in drinking will be minimal or confined to the hazard group, whereas individuals in both groups will be susceptible to reactivity in the form of reduced responding.
The CNLab-A app represents the outcome of an iterative development process that included elements of agile software design, namely requirements analysis, feature and interface design, and app implementation [
In the requirements analysis phase, the research team determined key variables of interest. To this end, empirical definitions of excessive and binge drinking and variables derived from validated retrospective measures of drinking were examined [
In this phase, the research team also considered assessment design. Studies reliant on real-time (or near real-time) monitoring of behavior tend to employ event- or time-based sampling [
Finally, the content for each assessment was based on the key variables of interest. Drink type formed the central element of each assessment as, once selected, all subsequent items were dependent on this choice (
Decisions made during the requirements analysis phase were documented via an iterative storyboard process. This ensured that the research team and programmer had access to a common record.
Feature and interface design was informed by app software capabilities and in consultation with the programmer. During this phase, iOS app prototypes were developed and deployed in response to design decisions and feedback from the research team. Deployment was via Test Flight, which allowed invited users to test beta versions of iOS apps.
Flowchart of assessment pathway and screenshots from the CNLab-A app. On opening, CNLab-A asks users if alcohol has been consumed in the last 24 hours. Thereafter, participants are asked if they have consumed alcohol since their last submission. If they indicate (by pressing “No”) that no drinking has occurred, the app can be closed. If participants indicate drinking has occurred (by pressing “Yes”), images of common alcoholic beverages (including beer, wine, cider/premix, spirit/liqueur, and cocktail) are displayed (1). Type of beverage consumed is selected by touching the appropriate image on the screen. Quantity and size consumed for each beverage is indicated via a simple scroll option menu (2). Alcohol content as a function of beverage type is prefilled. This process is repeated by tapping “Back” in order to add as many drink types as required. Erroneously entered data can be deleted by swiping left. Prior to submitting data, the start and end time of drinking must be specified, again using a scroll option menu (3). Participants are able to either report drinking in separate sessions or they can leave the app open so as to record beverages as they are consumed. The latter option still allows participants to use other features on their phone. Participants can access a history of their submission dates and times (but not their drinking data) via the “History” button. At the conclusion of the experimental period, an automated message thanks participants; gives them simple feedback regarding the number of days they consumed alcohol, total standard drinks consumed, and average daily consumption; and, asks them to remove the app from their smartphone.
We determined response selection would be via touch screen and scroll menus. At the commencement of each assessment, for instance, when participants were asked if they have consumed alcohol since their last submission, they indicated “Yes” or “No” by touching either option on the screen. Similarly, it was decided that drink types would be presented on 1 touch screen as a set of images with captions so that participants could readily identify and select their beverage. Images were designed using Inkscape, a freely available vector graphic software package. In the case of beer, images for mid and light-strength were opaque versions of the full-strength visual. Quantity and size options were available via a simple scroll option menu. Likewise, a scroll menu presented dates and times (in 15-min intervals) for recording of drinking start and end time. Participants pressed “Submit” to upload (to the server) and end the assessment. Any submission that failed to upload because of lack of internet connectivity would automatically be uploaded when connectivity was reestablished.
One of the potential advantages of using electronic means of assessment in the behavioral sciences relates to penetration. Such methods can provide investigators with large, diverse samples if facilitated by appropriate implementation decisions related to development and deployment. To that end, the research team decided to develop a native app to be run locally on iOS and Android platforms with marketplace deployment via Apple iTunes and Google Play, respectively, to optimize app availability. However, because of funding limitations, only an iOS version of the app was initially alpha and beta tested.
Drink types, alcohol content, and serving size options available in the CNLab-A app.
Drink type | Alcohol content (%) | Serving sizes | |
Full strength | 4.8 | Glass 200 mL; pot/middy 285 mL; can/bottle 375 mL; schooner 425 mL; pint 570 mL | |
Mid strength | 3.5 | As above | |
Light strength | 2.7 | As above | |
White/Champagne | 12.0 | Glass 150 mL | |
Red | 13.0 | As above | |
Port, sherry, marsala, madeira, vermouth, etc | 18.0 | Glass 60 mL | |
Most ciders and premix drinks including alcopops | 5.0 | Bottle 300 mL; bottle 330 mL; can 375 mL; bottle 500 mL | |
Rum, gin, vodka, brandy, tequila, whiskey, liqueurs, etc | 40.0 | Standard 30 mL; double 60 mL | |
Various | 40.0 | 1 shot 30 mL; 2 shots 60 mL; 3 shots 90 mL; 4 shots 120 mL |
Alpha testing involved a small sample of individuals (N=8, mean age 38.13 years, SD 16.54; range 18-68 years, 37.5% female) and was designed to identify programming bugs and oversights. Beta testing comprised a slightly larger group (N=19, mean age 37.37 years, SD 9.73; range 22-68 years, 68.4% female) and focused on eliciting user feedback (via email) post testing. During both the test studies, individuals were asked to keep a hardcopy record of any data submissions so that app data could be checked for accuracy. Average time to submit drinking information during testing was 34 seconds.
Suggestions provided by individuals involved in this phase and that were incorporated into the final version of the app included making an instructional video explaining how to use CNLab-A available to participants [
The requirements analysis and feature and interface design phases took almost 4 months, whereas the alpha and beta testing of the app implementation phase spanned approximately 8 weeks. We had 3 meetings with the programmer in the early stages of development and then a further meeting toward the completion of the feature and interface design phase. All other communication was via email. The final production version of the app was eventually made available on both iOS (8.4+) and Android (Kitkat 4.1+) platforms. As operating systems evolve, both apps will be audited to ensure they continue to function as required.
This study was based on data from 671 participants (mean age 23.12 years, SD 7.24; range 16-56 years, 70% female) that form a subset of an ongoing project—entitled CheckMyControl—investigating the relationship between alcohol use and various social and cognitive factors in the healthy population. This subsample completed the app component of the project (
After reading a plain language statement and providing informed consent, participants answered a Web-based researcher-devised demographic survey and the Alcohol Use Disorders Identification Test (AUDIT) [
They were compensated Aus $10 for time spent completing Web-based surveys and Aus $0.50 each day information about alcohol consumption was submitted via the app (regardless of whether alcohol had been consumed or not). Participants received a bonus Aus $9.50 if app data were submitted on all 21 days. The maximum participants could be reimbursed was Aus $30.
Flow diagram following Consolidated Standards of Reporting Trials guidelines of study participation.
The AUDIT is a 10-item screening measure that asks participants to respond to questions assessing alcohol intake, problems, and dependence with reference to the preceding 6 months. Participants were categorized into hazard (n=286) and nonhazard (n=385) groups based on their score, with scores of 8 or more indicative of hazardous alcohol consumption [
CNLab-A is a freely available custom-built app that can be used to record alcohol intake for research purposes. Once downloaded, CNLab-A requires participants to allow it to send them notifications. One notification is preset to 8 am, whereas the other can be set to suit the user. Although participants are directed at the outset to record alcohol consumption as it happens (or as soon thereafter as possible), notifications serve to prompt individuals to input information twice daily in case they neglect to do so when drinking. Thus, alcohol intake data can be submitted at any time, either in response to notifications or while drinking. A unique identification code, provided to participants during the Web-based component of the study, is required before the app opens. CNLab-A has previously been found to be a valid measure of alcohol intake [
Independent
Participants were considered compliant with app protocols if they responded to at least 1 notification each day for the 21-day experimental period. We assessed both number of days of use and submissions per day as a function of gender, age bracket (13-19 years, 20-29 years, and ≥30 years), and hazard and nonhazard group status using
At the time of testing, 39.9% (268/671) of the sample was aged under 20 years, 46.2% (310/671) was aged 20 to 29 years, and 13.9% (93/671) was aged 30 years or over. In Australia, individuals in the 18 to 29-year-old age bracket are more likely than others to consume alcohol in a manner that increases their short-term risk of alcohol-related injury, whereas almost 19% are at risk of long-term alcohol-related harm [
Before commencing the 21-day recording period, the minority of participants (2.2%, 15/671) indicated they had never consumed alcohol; the majority (92.8%, 623/671) described themselves as regular drinkers. A small proportion indicated they had been diagnosed with an alcohol (0.3%, 2/671) or substance (0.4%; 3/671) use disorder. Hazard and nonhazard groups did not differ significantly with regard to age,
On average, participants used CNLab-A 20.27 (SD 1.88) days out of 21 (96.5%); 96.0% (644/671) of participants completed at least 1 submission per day for the entire 21-day experimental period. There were no significant differences as a function of gender
With regard to the average number of responses per day, there were no significant differences as a function of gender,
Average alcohol intake indices as recorded via CNLab-A app (21 days), including hazard (n=286) and nonhazard (n=385) group totals and differences.
Drinking indices | Total, mean (SD) | Hazard, mean (SD) | Nonhazard, mean (SD) | 95% CI | Pearson correlation |
|
Days drinking | 5.32 (4.17) | 6.90 (4.02) | 4.15 (3.90) | 8.91 | 2.14-3.35 | .33 |
Total drinksb | 24.26 (25.41) | 37.45 (28.09) | 14.47 (17.75) | 12.16 | 19.27-26.70 | .43 |
Drinks per day | 1.20 (1.25) | 1.86 (1.38) | 0.71 (0.86) | 12.45 | 0.97-1.33 | .43 |
Drinks per day drinking | 3.98 (3.02) | 5.53 (2.92) | 2.83 (2.56) | 12.45 | 2.27-3.12 | .43 |
Hourly rate of drinking | 2.20 (2.09) | 2.60 (2.14) | 1.89 (2.00) | 4.41 | 0.40-1.04 | .17 |
Highest drink count in 2 hours | 4.14 (3.15) | 6.03 (3.37) | 2.79 (2.09) | 14.34 | 2.80-3.69 | .48 |
4/4+ intakec | 2.16 (2.58) | 3.48 (2.83) | 1.19 (1.86) | 11.89 | 1.91-2.67 | .42 |
6/6+ intake | 1.31 (1.92) | 2.26 (2.27) | 0.61 (1.20) | 11.14 | 1.35-1.94 | .40 |
8/8+ intake | 0.85 (1.44) | 1.51 (1.70) | 0.36 (0.94) | 10.41 | 0.94-1.38 | .37 |
12/12+ intake | 0.31 (0.80) | 0.58 (1.06) | 0.10 (0.43) | 7.26 | 0.35-0.61 | .27 |
20/20+ intake | 0.07 (0.31) | 0.12 (0.41) | 0.03 (0.20) | 3.55 | 0.04-0.15 | .14 |
a
bDrinks refer to self-reported alcohol consumption in Australian standard drinks (1 drink=10 g alcohol).
c4/4+ (and so forth) intake refers to occasions where 4 or more drinks were consumed in 1 episode.
Parameter estimates for linear growth model of drinks per day as a function of hazard and nonhazard Alcohol Use Disorders Identification Test group membership.
Parameter | Estimate (SE) | 95% CI | ||||
Intercept (Day 0) | 0.67 (0.08) | 8.68 | 2301.73 | <.001 | 0.52 to 0.82 | |
Time | –0.01 (0.01) | –1.80 | 13418.00 | .07 | –0.02 to 0.001 | |
Hazard | 1.48 (0.11) | 13.26 | 2301.73 | <.001 | 1.26 to 1.70 | |
Hazard by time | –0.03 (0.01) | –3.50 | 13418.00 | <.001 | –0.04 to–0.01 | |
Between-person (level 2) intercept | 0.74 (0.06) | 12.28 | —b | <.001 | 0.63 to 0.87 | |
Within-person (level 1) residual | 7.54 (0.09) | 81.91 | — | <.001 | 7.37 to 7.73 |
a
bNot applicable.
Parameter estimates for linear growth model of daily (“yes”) responses per day as a function of hazard and nonhazard AUDIT group membership.
Parameter | Estimate (SE) | 95% CI | ||||
Intercept (Day 0) | 0.32 (0.02) | 18.24 | 982.85 | <.001 | 0.28 to 0.35 | |
Time | –0.003 (0.001) | –3.63 | 13418.00 | <.001 | –0.004 to –0.001 | |
Hazard | 0.15 (0.03) | 5.99 | 982.85 | <.001 | 0.10 to 0.20 | |
Hazard by time | –0.004 (0.001) | –3.53 | 13418.00 | <.001 | –0.006 to –0.002 | |
Between-person (level 2) intercept | 0.08 (0.01) | 16.86 | —b | <.001 | 0.07 to 0.09 | |
Within-person (level 1) residual | 0.14 (0.002) | 81.91 | — | <.001 | 0.14 to 0.15 |
a
bNot applicable.
In this study, we aimed to describe the development and implementation of a custom-built smartphone app devised to measure real-time (or near real-time) alcohol consumption behavior in the healthy population. Designed for use in the research arena, the app was a product of an iterative process that included elements typical of agile software design. Decisions made during each phase of development were informed by a desire to create an app that the participants could download onto their own smartphones without ever having to visit the lab. We anticipated this might minimize disadvantages—such as equipment and training costs, participant burden, and missing data—often associated with using some types of electronic protocols, while simultaneously enhancing benefits pertaining to sample size and diversity. We additionally explored methodological factors related to app protocol compliance and reactivity as a function of hazard and nonhazard drinker status.
Compliance with app protocols was high. Participants were required to submit data about their drinking (regardless of whether they had consumed alcohol or not) at least once per day for 21 days. On average, they uploaded data on 96.5% of days, and there were no differences between hazard and nonhazard groups with regard to the number of days of app use or the number of responses per day. Presumably, the use of daily payments and an end-of-study bonus (for 21 consecutive days of data submissions) incentivized responding. In previous alcohol intake-based studies utilizing various electronic methods of collecting data—including SMS text messaging [
Our results suggest there was some slight degree of reactivity—particularly among hazardous drinkers—to the app protocol. Though the effect was small, the hazard group decreased their intake significantly over the experimental period: 0.80 standard drinks in 21 days, which represented a 2% decrease in total standard drinks for this group. By contrast, nonhazard drinkers showed no significant reduction in consumption over the same period. This accords with evidence from other studies demonstrating some reduction in alcohol consumption only because of measurement among hazardous drinkers [
There is some debate in the literature regarding reactivity to real-time measures. Several investigators postulate such assessment reduces the likelihood of reactivity related to social desirability as participants record data in the absence of the researcher [
Several limitations to this study must be noted. Although representative of the population at large in terms of country of birth, first language, and usual place of residence, young female university students predominated in our sample. Although current research suggests men and women are equally likely to download and use health-orientated smartphone apps [
In conclusion, we examined the feasibility of developing and employing an app—downloaded by participants to their own smartphones—designed to collect alcohol intake information for research purposes. We demonstrate how utilizing apps such as CNLab-A can yield a potentially large sample representative of the population. Both hazard and nonhazard participants appeared highly compliant when using app protocols. Although there was some evidence of reactivity in our study, especially among hazardous drinkers, effect sizes were small. Our findings suggest the CNLab-A app—or potentially 1 similar—is a methodologically sound means of examining alcohol consumption behavior across time. In future, such apps can be paired with those that chart cognition, affect, or social and environmental factors in real time to facilitate a more thorough investigation of the antecedents and consequences of drinking behavior.
Alcohol Use Disorders Identification Test
blood alcohol content
short message service
sensitivity
specificity
The authors wish to thank Jiajie Li and Jemie Effendy who assisted with upgrades to the smartphone app as well as Cameron Patrick from the Melbourne Statistical Consulting Platform for advice in reviewing the analyses. This research was supported by an Australian National Health and Medical Research Council grant (1050766), and an Australian Research Council fellowship (FT110100088). The funding bodies had no role in the study design, collection, analysis or interpretation of data, writing the manuscript, or in the decision to submit the paper for publication.
None declared.