Published on 15.03.16 in Vol 4, No 1 (2016): Jan-Mar
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/4470, first published Mar 27, 2015.
Ecological Momentary Assessment of Illicit Drug Use Compared to Biological and Self-Reported Methods
Background: The use of mHealth methods for capturing illicit drug use and associated behaviors have become more widely used in research settings, yet there is little research as to how valid these methods are compared to known measures of capturing and quantifying drug use.
Objective: We examined the concordance of ecological momentary assessment (EMA) of drug use to previously validated biological and audio-computer assisted self-interview (ACASI) methods.
Methods: The Exposure Assessment in Current Time (EXACT) study utilized EMA methods to assess drug use in real-time in participants’ natural environments. Utilizing mobile devices, participants self-reported each time they used heroin or cocaine over a 4-week period. Each week, PharmChek sweat patch samples were collected for measurement of heroin and cocaine and participants answered an ACASI-based questionnaire to report behaviors and drug using events during the prior week. Reports of cocaine and heroin use captured through EMA were compared to weekly biological or self-report measures through percent agreement and concordance correlation coefficients to account for repeated measures. Correlates of discordance were obtained from logistic regression models.
Results: A total of 109 participants were a median of 48.5 years old, 90% African American, and 52% male. During 436 person-weeks of observation, we recorded 212 (49%) cocaine and 103 (24%) heroin sweat patches, 192 (44%) cocaine and 161 (37%) heroin ACASI surveys, and 163 (37%) cocaine and 145 (33%) heroin EMA reports. The percent agreement between EMA and sweat patch methods was 70% for cocaine use and 72% for heroin use, while the percent agreement between EMA and ACASI methods was 77% for cocaine use and 79% for heroin use. Misreporting of drug use by EMA compared to sweat patch and ACASI methods were different by illicit drug type.
Conclusions: Our work demonstrates moderate to good agreement of EMA to biological and standard self-report methods in capturing illicit drug use. Limitations occur with each method and accuracy may differ by type of illicit drugs used.
JMIR mHealth uHealth 2016;4(1):e27
The detection of biochemical markers of illicit drugs in biological samples of hair, urine, sweat, or blood is considered the gold standard for assessing illicit drug use and is widely used in drug treatment and employment drug testing settings . The utility of these methods lies in their ability to detect metabolites of illicit drugs used within a specific window of time that varies depending on the biological specimen. Despite being the gold standard for the assessment of drug use, biological samples are often difficult to collect in the field, may be cost prohibitive, and can require greater participant engagement (eg, frequent urine screens at a treatment facility). Additionally, biologic samples typically only assess whether an individual has used drugs rather than quantifying how much and how often the drug was consumed [ , ].
In epidemiological studies, the most feasible method of assessing illicit drug use is self-report, which is recounted over extended periods of recall (eg, 6 to 12 months or longer) [- ]. The benefit of self-report includes the ease of use, convenience, and low cost. However, whether assessed via study interviewer or audio-computer assisted self-interview (ACASI), these methods may involve recall, response, or social desirability biases [ , ]. Additionally, these methods require participants to return to the clinic or study site at regular intervals, which not only requires participants to have reliable transportation options but also disrupts their daily routines. Despite these potential issues, assessment of illicit drug use through self-report has been repeatedly shown to be a valid and reliable measure of drug use [ , - ].
Yet, self-report and biological testing methods of capturing drug use lack the ability to assess real-time drug use, miss varying periods of intense or intermittent drug use, and cannot ascertain the proximate context of an individual’s drug using experience . Ecological momentary assessment (EMA) is a mobile health (mHealth) method that is capable of collecting participant-level data in real time over notably shorter time intervals.
Mobile devices that employ mHealth strategies (eg, smartphones or other handheld devices) can utilize EMA methods for remote data collection and monitoring as well as health education and intervention . EMA methods have been utilized in smoking cession studies [ - ] and among methadone-maintained outpatient drug users [ - ] but have yet to be validated as a reliable method for assessing drug use. By capturing drug-using events in real-time, outside of the study clinic and in a participant’s natural settings, a more robust, vibrant, and comprehensible understanding of drug use can be generated beyond periodic biological detection or infrequent self-reports of drug use.
Prior studies have examined concordance between the assessment of drug use by biological measures and self-report [- ]. The aim of the current study was to investigate the concordance of assessing drug use via EMA methods compared to biological (eg, sweat patch) and ACASI methods. Additionally, we identified correlates of discordance and examined the feasibility, strengths, and weaknesses of these methods in assessing drug use among a community sample of drug users in Baltimore, Maryland.
EXACT Study Participants
Exposure Assessment in Current Time (EXACT) study participants were recruited from the AIDS Linked to the IntraVenous Experience (ALIVE) study, an on-going, community-recruited, observational cohort of people with a history of injecting drugs in Baltimore, Maryland . The ALIVE cohort is community-based, rather than clinic-based, and thereby avoids selection bias toward people seeking or accessing health care. Details of the EXACT study have been previously described [ , ] and included 4 successive trials conducted from November 2008 through May 2013. Each trial was planned to follow 30 participants for 30 days.
Eligibility criteria for the EXACT study included current enrollment in ALIVE and the ability to understand and follow directions on a personal digital assistant (PDA) or mobile phone. Convenience sampling was utilized to identify individuals for participation in EXACT. The specific inclusion criteria regarding drug use and HIV status were varied slightly to ensure a diverse overall sample; both injection and noninjection drug users were included. Individuals were excluded if they had any medical conditions that would prevent them from operating the handheld device (eg, vision impairment) or failed to attend the screening appointment where they were trained on device use.
For Trials 1-3, participants were provided with a Palm Z22 PDA running applications developed using Satellite Forms software. All PDA programs were disabled except for study-required applications. In Trial 4, participants were provided with a Motorola Droid X2 Android mobile phone running an application developed using the electronic MObile Comprehensive Health Application platform, which was created at Johns Hopkins School of Medicine .
The Johns Hopkins Bloomberg School of Public Health Institutional Review Board approved the study protocol. All participants provided written informed consent. Participants were informed that involvement (or noninvolvement) in EXACT would not affect their participation in ALIVE.
EMA Data Collection
For 30 days of observation, participants were asked to self-initiate a survey on their handheld device and self-report each time they either used heroin or cocaine (or both) in any manner (ie, smoked, snorted, or injected). For each event, participants answered questions concerning their drug use, mood, and social, physical, and activity environment, using survey instruments adapted from previous EMA studies [- ]. To ensure responses were recorded in real-time, participants were required to indicate that drug use had occurred within 30 minutes of completing this survey. The device also delivered an end-of-day (around 9 pm) survey that asked if there was any drug use that was not reported earlier in the day. The surveys were designed to be completed in less than 3 minutes.
PharmChek Drugs of Abuse Patches were collected weekly for the assessment of heroin or cocaine use. These patches detect traces of cocaine or heroin secreted in sweat during the period it is worn. Drugs captured via PharmChek sweat patches represent “parent” drugs (ie, same chemical compound that was taken by the drug user) and drug metabolites (ie, breakdown products of the parent drug) excreted through sweat. The patch can be worn for up to 10 days and is able to capture any drug use that occurred during the period of wear as well as 24 hours prior to patch application . To ensure the patch stayed in place, an additional overlay made of the same adhesive material was worn atop the sweat patch. Once removed, patches were sent to a commercial laboratory for drug evaluation. Specimens were initially screened using an enzyme immunoassay technique with positive patches undergoing confirmation using liquid chromatography/mass spectrometry/mass spectrometry [ ].
Cocaine predominates in sweat after cocaine use, however, the most common metabolite of cocaine is benzoylecgonine (BZ). A positive sweat patch result for cocaine use is confirmed by the presence of both BZ and cocaine at or above the limit of detection of 10 ng/mg. Topical analgesics, such as lidocaine or novocain, contain BZ and are used in various surgical procedures, however cocaine is structurally unique and does not resemble any of these products .
Opiate metabolites detectable by sweat patch include heroin, 6-monoacetylmorphine (6-MAM), codeine, and morphine. The presence of 6-MAM can only come from the use of heroin. A positive sweat patch for heroin includes the presence of the parent drug (heroin) and morphine above the limit of detection of 10 ng/ml, 6-MAM and morphine above the limit of detection of 10 ng/ml, or 6-MAM alone above the limit of detection of 10 ng/ml. The presence of morphine alone may be due to the use of other opiate-containing legal medications (eg, oxycodone, hydrocodone) or the consumption of certain foods (eg, poppy seeds). Therefore, the presence of morphine alone does not indicate a positive sweat patch for heroin .
Self-Report by ACASI
Participant baseline characteristics were obtained from ACASI completed at enrollment into EXACT and/or from the prior ALIVE study visit. Additionally, at the conclusion of each study week, participants returned to the study site to answer an ACASI that included questions concerning activities, behavior, and drug use frequency during the prior week. In addition to sociodemographic variables (eg, age, sex, race, education, marital status, employment, income, homelessness, and health insurance status), baseline data collection included self-reported alcohol, tobacco, and illicit drug use; an index of drug abuse (Drug Abuse Screening Test (DAST)) ; and depressive symptoms (Center for Epidemiologic Studies-Depression Scale (CES-D)) in the prior 6 months [ ]. Clinical characteristics (eg, HIV/antiretroviral therapy status, CD4 T-cell count, HIV RNA levels, and hepatitis B and C status) were obtained from the existing ALIVE database.
Data from all 4 trials were aggregated and analyzed together. To ensure accurate comparisons between each method of capturing drug use, all analyses were assessed by week (this was necessary as the ACASI and sweat patch data were only collected weekly). The day during which the sweat patch was placed on the participant’s arm and when the handheld device was provided represented the start of the study week 1. Seven days later, when the ACASI was completed, marked the end of the week. At this time, the sweat patch was removed and replaced with a new patch. This process was repeated for all 4 weeks of the study. Drug use reported by ACASI and sweat patch indicated use or no use within the prior week.
Real-time heroin or cocaine use, reported via the EMA entries or the end-of-day survey were summed by day and week for each participant. For analysis, an individual was considered to have used drugs if at least 1 report of drug use (eg, heroin or cocaine use by any manner) was reported in real-time within a given study week. Heroin-only and cocaine-only reports incorporated any reports of heroin or cocaine use (including those jointly used with another drug).
Because the sweat patch was able to capture drug use that included the 24 hours prior to adhesion, the EMA week was offset by 1 day to ensure concurrent periods of time were evaluated when comparing the methods. There was no adjustment for time for comparisons between EMA- and ACASI-assessed drug use.
To examine the concordance of drug use reported by EMA to sweat patch or to ACASI methods, percent agreement and concordance correlation coefficients were calculated. The concordance correlation coefficient is a measure of the level of agreement that takes into account the agreement occurring by chance for repeated measures (much like the kappa statistic is for categorical variables measured at 1 point) . Using the same scale for kappa values for comparing categorical variables, concordance correlation coefficients of less than 0.2 are considered poor; 0.21-0.40 fair; 0.41-0.60 moderate; 0.61-0.80 good; and 0.81-1.00 very good [ ].
If the number of EMA events in any week was greater than the number of ACASI or sweat patch responses it was considered EMA overreporting, while EMA underreporting was determined if the number of EMA reports were fewer than those reported by sweat patch or ACASI. To determine correlates of discordance between methods of assessing drug use, logistic regression models with generalized estimating equations (GEE) were examined. GEE methods adjusted for the correlation of repeated measures within each subject ID over the 4 weeks of follow-up. Variables selected for the final multivariable models were chosen through step-wise logistic regression with inclusion of significant variables (P<.1) from the univariate analyses. The different technologies used (eg, PDA vs mobile phone) were not accounted for analytically because the mobile platforms were very similar and we had no a priori reason to suspect that mobile platform type would in any way influence measurement. All analyses were performed using Stata release 12 (StataCorp LP, College Station, Texas).
Among 109 EXACT participants contributing 436 weeks of observation (), the median age was 48.5 years (interquartile range (IQR) 43-53 years), 98 (90%) were African American, 58 (52%) were male, and 64 (59%) were HIV infected. In the 6 months prior to baseline assessment, 23% of participants reported recent methadone treatment and 83% reported smoking cigarettes.
|Type of Variable||Characteristics||All Trials (N=109)|
|Sociodemographic||Median age, y (IQR)||48.5 (43.3-52.9)|
|African American||98 (90)|
|Completed high school||44 (41)|
|Never married||66 (61)|
|Annual income < $5000||83 (78)|
|Medical insurance||93 (85)|
|Substance Use||Cigarettes||91 (83)|
|Cocaine, any route||50 (46)|
|Heroin, any route||49 (46)|
|Drug Abuse Screening Test, DAST>16||6 (18)|
|Clinical||Have primary care doctor||97 (89)|
|Emergency room visit||28 (26)|
|Depressive symptoms, CES-D>23||26 (23)|
|Methadone treatment||26 (23)|
|Hepatitis C virus seropositive||94 (86)|
|HIV positivec||64 (59)|
|Median CD4 (IQR)c||360.5 (239-529)|
|HIV viral load > 500 copies/mLc||35 (55)|
|Any retroviral therapy||42(65)|
aAll baseline characteristics represent behavior within the 6 months prior to the start of EXACT.
bA speedball is defined as the simultaneous injection of a mixture of cocaine and heroin.
cHIV-positive status was an inclusion criterion for Trials 3 and 4; CD4 and viral load were tested on HIV-positive participants only.
Comparison of Methods to Capture Illicit Drug Use
Out of a possible 436 weeks of follow-up (109 individuals followed for 4 weeks), 12 weeks did not have evaluable EMA assessments of drug use, resulting in 424 weeks (97%) of observable data. Over 436 weeks, 396 (91%) sweat patches were returned and 410 (94%) ACASI surveys were completed. Missing data was the result of 22 individuals who were unable to return 29 (7%) sweat patches that were damaged or removed prematurely and 12 individuals failing to complete 14 (3%) ACASI surveys. Reports of drug use by EMA represent any report in a week and not the number of individuals or the total amount of uses in a week.
Total weeks of drug use obtained from sweat patch, ACASI, and EMA methods are described in. Over 436 weeks of study follow-up, 212 (49%, green bars) sweat patches, 192 (44%, blue bars) ACASI surveys, and 163 (37%, orange bars) weeks of EMA reports were positive for any cocaine use. For heroin use, 103 (24%) sweat patches, 161 (37%) ACASI surveys, and 145 (33%) weeks of EMA reports were captured over follow-up. Seventy-seven (18%) sweat patches, 117 (27%) ACASI surveys, and 96 (22%) weeks of EMA reports captured both cocaine and heroin use. The proportion of sweat patches with heroin and cocaine detected remained stable by study week.
For cocaine use, the overall percent agreement between EMA and sweat patch methods was 70% (a, blue bars) ( ) and for EMA and ACASI methods was 77% ( a, green bars). For heroin use, the percent agreement between EMA and sweat patch methods ( b, orange bars) was 72% and for EMA and ACASI methods ( b, yellow bars) was 79%. With heroin or cocaine use, the percent agreement was slightly higher between the EMA and ACASI methods compared to EMA and sweat patch assessments.
Percent agreement does not take into consideration the agreement between 2 methods solely due to chance. The concordance correlation coefficient is a measure of inter-rater reliability that takes into account the agreement occurring by chance for repeated measures. The concordance correlation coefficients were slightly lower for comparisons of drug use between EMA and sweat patch methods than observed for EMA and ACASI methods. The concordance correlation coefficients for the comparison of EMA and sweat patch methods were in the moderate agreement range for both cocaine 0.51 (0.48-0.59) and heroin 0.48 (0.40-0.55) use. The agreement in reports between EMA and ACASI methods for cocaine use was 0.59 (0.53-0.65) and for heroin use was 0.61 (0.55-0.67), with the former representing moderate agreement and the latter representing good agreement. The agreements in reports between ACASI and sweat patch methods were not as consistent between heroin and cocaine use. The concordance correlation coefficient for the comparison of ACASI and sweat patch methods for cocaine use was 0.72 (0.67-0.77), representing good agreement, while the concordance correlation coefficient for heroin use was 0.58 (0.51-0.64), representing moderate agreement.
Misreporting of responses by EMA relative to sweat patch and ACASI methods were assessed for cocaine and heroin separately (). Relative to sweat patch results, underreporting of drug use by EMA methods was more likely for heroin than cocaine use (19% vs 9%), but overreporting by EMA methods was greater for cocaine than heroin use (21% vs 8%). Misreporting was identified less commonly between EMA and ACASI methods. Compared to ACASI reports, underreporting by EMA was infrequent and similar for both cocaine and heroin use (8% vs 9%). Overreporting by EMA relative to ACASI was slightly greater for cocaine than heroin use (15% vs 13%).
|n (%)||n (%)|
|Sweat Patch||Reported yes on EMA/sweat patch negative (overreport)||30 (9%)||70 (19%)|
|EMA & sweat patch concordant||298 (70%)||307 (72%)|
|Reported no on EMA/sweat patch positive (underreport)||79 (21%)||28 (8%)|
|ACASI||Reported yes on EMA/ACASI negative (overreport)||29 (8%)||33 (9%)|
|EMA & ACASI concordant||327 (77%)||335 (79%)|
|Reported no on EMA/ACASI positive (underreport)||58 (15%)||49 (13%)|
aSweat patches and ACASI captured both cocaine and heroin use (ie, there is double counting); therefore, numbers do not add up to 100%.
Correlates of EMA Misreporting
We sought to identify sociodemographic, behavioral or clinical factors associated with over- or underreporting of drug use by EMA. Relative to sweat patch results, there were no significant correlates of EMA overreporting of cocaine use (). Underreports of cocaine use by EMA were almost 2-fold less likely among females (odds ratio (OR) 0.47, 95% confidence interval (CI): 0.23-0.98) and 80% less likely among individuals who reported injecting once per day or more at baseline (OR 0.21, 95% CI: 0.05-0.87). Although only marginally significant, individuals under 50 years of age were found to be less likely to underreport cocaine use as well (OR 0.51, 95% CI: 0.25-1.07).
EMA overreports of heroin use relative to sweat patches were twice as likely if heroin was used at baseline (OR 2.10, 95% CI: 1.0-4.56), but baseline heroin use was also the only factor significantly associated with EMA underreporting of heroin use (OR 5.56, 95% CI: 1.37-22.46). Female gender also achieved marginal significance in being less likely to underreport heroin use via EMA methods (OR 0.31, 95% CI: 0.08-1.10).
Compared to ACASI methods (), EMA overreports of cocaine and heroin use were twice as likely if a participant reported sharing injection needles at baseline (OR cocaine 2.79, 95% CI: 1.03-7.5; OR heroin 2.96, 95% CI: 1.17-7.51). Underreporting of cocaine use by EMA was marginally associated with being married and was 6-fold more likely if individuals reported having medical insurance at baseline (OR 6.62, 95% CI: 1.16-37.76). EMA overreports of heroin use were positively associated with baseline heroin use (OR 3.04, 95% CI: 1.33-6.95) as well as over 4-fold more likely if the participant was HIV infected (OR 4.56, 95% CI: 1.80-11.58).
|Cocaine EMA overreport||Cocaine EMA underreport||Heroin EMA overreport||Heroin EMA underreport|
|OR (95% CI)||OR (95% CI)||OR (95% CI)||OR (95% CI)|
|Sweat patch||Female||0.47 (0.23-0.98)b||0.31 (0.08-1.10)c|
|Never Married||0.51 (0.25-1.07)c||0.68 (0.21-2.19)|
|Alcohol Use||1.55 (0.72-3.36)|
|Any heroin||2.10 (1.0-4.56)b||5.56 (1.37-22.46)b|
|Any cocaine||1.55 (0.72-3.36)||1.68 (0.79-3.62)|
|Same doctor for at least 2 years||0.60 (0.30-1.23)|
|Never Married||0.47 (0.21-1.02)c||0.53 (0.24-1.18)|
|Cigarette use||2.73 (0.64-11.72)||2.88 (0.61-13.57)||2.76 (0.46-16.55)|
|Alcohol use||1.73 (0.69-4.29)|
|Any heroin||0.77 (0.29-1.99)||3.04 (1.33-6.95)b|
|Any cocaine||1.78 (0.76-4.20)||1.86 (0.77-4.51)||1.19 (0.53-2.68)|
|HIV infected||0.52 (0.18-1.5)||4.56 (1.80-11.58)b|
|Shared needles||2.79 (1.03-7.5)b||2.96 (1.17-7.51)b|
|Primary care physician||0.57 (0.15-2.14)|
|Yearly income<$5000||1.69 (0.45-6.34)|
aCorrelates included in multivariable models had P<.1 in univariate analyses and represent behaviors occurring within the 6 months prior to the start of EXACT.
bStatistical significance, P<.05.
cMarginal statistical significance, P<.1.
This analysis demonstrated moderate to good concordance and inter-rater reliability of reported drug use by EMA when compared to either biological measures of sweat patches or more conventional ACASI self-report methods. However, our data raised concerns regarding the use of sweat patches as a gold standard for drug use assessment due to the notably lower prevalence of heroin use defined by biological detection compared to the prevalence of heroin use we determined in this study based on self-reported methods and to the expected prevalence based on prior data in our ALIVE cohort [, ]. Even relative to imperfect gold standards, we provide evidence that researchers should be confident that EMA methods can accurately capture and characterize illicit drug use comparable to currently used methods. Given the relative benefits of daily real-time assessments of drug use in terms of reductions in recall bias, social desirability bias (reductions in participant need to please the interviewer), participant burden, and follow-up time, EMA methods for assessing drug use may have broad applications in settings ranging from epidemiological studies to behavioral interventions.
EMA Compared to Sweat Patch Assessment
Biological samples serve as the gold standard for assessing drug use because they are able to capture the biochemical components of drug use as the body excretes them. Sweat patches are often used to detect longer-term drug use and can continuously capture drug metabolites in sweat until the patch is removed. It is designed to be flexible, waterproof, and safe from environmental contaminates . However, once the patch comes off the skin, it cannot be put back on to resume drug use capture. Current applications of sweat patch testing include use in drug treatment for monitoring drug relapse and for determining the effectiveness of medical and psychological therapy [ , ].
Our results suggest moderate concordance between EMA and sweat patch methods for assessing drug use. Prior studies have shown substantial discordance between self-report of drug use and biochemical tests results across out-of-treatment populations . A 10-week outpatient clinical trial in which participants wore sweat patches and provided urine samples and self-reports of cocaine use thrice weekly demonstrated the concurrent validity of urine and sweat patches to be reasonable (correlation: 0.76, P<.001), but the correlation between self-report and the patches was lower (correlation: 0.40, P<.05) [ ]. A separate outpatient study examining the utility of sweat testing for monitoring drug use also found the level of agreement between positive sweat test results and positive urine results to be 33% for heroin and 92% for cocaine [ ].
The results of our sweat patch analyses demonstrated a notably greater number of cocaine-positive sweat patches compared to heroin-positive sweat patches. This finding was unexpected as our prior analyses with this EXACT population demonstrated heroin to be the predominant drug of use over 30 days of follow-up . Additionally, recent estimates indicate that Baltimore suffers from a far greater public health burden of heroin abuse compared to cocaine abuse [ ] and this is mirrored in the participants of the ALIVE study [ , ]. Upon consultation with PharmChek, manufacturers of the sweat patches, it was suggested this difference may have been the result of our heroin-using participants using such small amounts of heroin that, even after a week of wearing the patch, they did not secrete enough heroin metabolites to be detected at the limit of detection of 10 ng/ml (Matthew Hartley, personal communication).
Although concerns have been raised that sweat patches may serve as a deterrent to drug use , there was no incentive in this study to modify behavior and our self-reported drug use data did indicate greater heroin use. To explain our findings, there would have to be a differential effect resulting in heroin misreporting relative to cocaine in response to sweat patch placement, which seems implausible. Yet, despite these problems of relatively lower heroin detection via sweat patches, the inter-rater reliability of EMA methods compared to sweat patch analysis remained moderate for both heroin and cocaine use as evidenced by concordance correlation coefficients.
EMA Compared to ACASI Assessment
ACASI methods have now become the standard approach for collecting sensitive data in epidemiologic research studies. The use of ACASI has resulted in greater disclosure of sensitive behaviors such as drug and sexual risky behaviors in some population [- ], thereby reducing social desirability bias and improving accuracy of self-report.
Although the best time interval for assessing drug use exposure remains unknown, several studies have found that reporting sensitive sexual behaviors can be accurately recalled for intervals of 1-3 months [, ]. Longer time frames may be more representative of a person’s behavior patterns, but can be more difficult to recall. It is likely that participants asked to recall behaviors over longer time periods may rely on a strategy such as “guestimation” of the average number of days per week they have been with a specific partner or used drugs [ ]. Despite the potential problems with accuracy of information collected over longer periods of time, ACASI assessments are rarely done in shorter intervals due to practical issues.
In this analysis, the inter-rater reliability and concordance of EMA methods compared to ACASI methods for assessing drug use appear stronger for heroin use (concordance correlation coefficient for heroin use had good agreement, 0.61 (0.53-0.69)) than for cocaine use. Both methods involve self-report in settings with increased privacy over traditional face-to-face interviews providing greater anonymity when disclosing sensitive information. In the current analysis, the ACASI reports captured more drug use than EMA methods. It is hard to differentiate between “fuzzy” recall that may have been reported via ACASI (leading to overreports) from participants that may have been impaired from drug use when answering the EMA survey (leading to underreports).
While we document good concordance between EMA methods with both ACASI and sweat patch approaches, this analysis neglects to consider a primary analytical strength of EMA methodology, namely the examination of real-time drug use. EMA methods allow for the examination of the variation and amount of drug use by day. ACASI methods are unable to examine daily drug-using patterns due to feasibility issues (eg, study visits, etc). Despite these differences in methodologies, our EMA results remained reliable when compared to ACASI-based methods and could prove extremely useful in understanding drug use among chronic drug users.
Misreporting by EMA
Relative to sweat patch reports, there were no demographic or behavioral correlates of overreporting of cocaine use by EMA whereas having a stable partner, male gender, and daily injection at baseline were associated with underreporting of cocaine use by EMA. Baseline heroin use was the only significant correlate of misreporting of heroin by EMA relative to the sweat patch. In total, these data suggest that more regular heroin users were more likely to both overreport and underreport heroin, raising concerns regarding misclassification due to the limitations of sweat patch detection of heroin as highlighted above. In-depth or timeline follow-back interviews could be used to better understand discordant events, as we do not know if the dynamics of overreporting and underreporting are the same.
Relative to ACASI, sharing injection needles was positively associated with EMA overreporting of both cocaine and heroin use, while underreports of cocaine use was positively associated with having medical insurance. In contrast, EMA underreporting of heroin use were positively associated with baseline heroin use and HIV status. Stable factors, such has having insurance, may lead to underreporting of cocaine use as a result of a social desirability bias. The associations of misreporting with sharing needles, baseline heroin use, and HIV status most likely reflects the recruitment criteria of EXACT, which included a large proportion of HIV-infected individuals with more recent and intense drug use.
Substance abuse is commonly associated with a chaotic or disordered life, mental illness, financial and legal difficulties, and inadequate housing or transportation [- ]. As members of the ALIVE cohort, all EXACT participants had a history of abusing illicit drugs and most were HIV infected. Long-term chronic drug abuse has physical ramifications but also cognitive effects. Working memory deficits are also prominent among HIV-infected individuals. During periods of intoxication, heroin users suffer a slow drop-off in attention and often fall asleep [ ]. Baseline cocaine and heroin use heavily impacted EMA misreporting of heroin use but not cocaine use. It is possible that inattention or sleep may have contributed to heroin users (more so than cocaine users) having difficulties in recalling drug use on a weekly basis (both as overreports and underreports) as was required of drug use assessed by ACASI. With respect to EMA underreporting, we found no evidence of exhaustion from using the handheld devices in reporting drug use across study week or by drug type. Notwithstanding these differences in reporting by drug type, EMA methods captured much of the drug use reported by ACASI methods.
There are several strengths of EMA methods that make them desirable to capture illicit drug use among community-dwelling populations. Because they are assessed in essentially real-time, EMA does not require individuals to recall or remember behaviors for prolonged periods. Social desirability bias is a reporting bias that arises when individuals underreport specific behaviors or actions because they believe these actions are sensitive and not socially acceptable to report . ACASI methods have been shown to decrease social desirability bias by allowing greater respondent privacy since questions are administered audibly and in text on a computer screen in a private room without the direct participation of a study interviewer [ ]. However, EMA methods may allow for even greater respondent privacy since participants are able to answer questions in their natural environment, which allows participants to calmly respond to questions where they feel most comfortable, away from a study site. ACASI interviews and sweat patches require participants to visit study sites at regular intervals.
EMA methods can provide more intensive follow-up opportunities compared to sweat patch or ACASI assessment. Daily outcome assessments over extended periods of time are feasible when using EMA methods because participants can carry the devices 24 hours a day, 7 days a week. Prior studies have involved participants using the devices for up to 6 months . Real-time data capture allows for the daily context and “situatedness” of drug use to be assessed, including information on the number of days used, frequency and amount used in a day, as well as participant behaviors that occur due to specific cultural, organizational, and structural environments [ ]. Historically, ACASI has resulted in greater disclosure of sensitive information [ - ] and previous EMA analyses by ourselves and others have demonstrated that EMA is capable of capturing this type of information as well [ , , , , ].
Assessing drug use in epidemiologic studies ideally involves an approach that is unobtrusive, does not rely on recall, has limited requirements for participant participation, and is accessible and affordable. mHealth may provide an excellent solution for assessing drug use in the field. The level of concordance between EMA and traditional biological and self-report methods suggests that utilizing EMA mHealth strategies are feasible for assessing drug use among community dwelling, nontreatment-seeking drug users. Future studies integrating EMA methods with the use of sensors for assessing drug use will likely provide both the biological and environmental cues of illicit drug use as well as provide a more complete picture of drug-using behaviors.
The authors acknowledge the substantial commitment of EXACT study participants and staff. Funding for the study was provided through the National Institute on Drug Abuse (Grants U01-DA-04334 and R01-DA-12568). LWC is supported by the National Institutes of Mental Health at the National Institutes of Health (K23MH086338). RPW is supported by the National Institutes of Health/National Institute on Drug Abuse (K23DA032306).
Conflicts of Interest
- Wasserman DA, Korcha R, Havassy BE, Hall SM. Detection of illicit opioid and cocaine use in methadone maintenance treatment. Am J Drug Alcohol Abuse 1999 Aug;25(3):561-571. [Medline]
- Dillon FR, Turner CW, Robbins MS, Szapocznik J. Concordance among biological, interview, and self-report measures of drug use among African American and Hispanic adolescents referred for drug abuse treatment. Psychol Addict Behav 2005 Dec;19(4):404-413 [FREE Full text] [CrossRef] [Medline]
- Harrison L, Martin S. , Enev, T.,& Harrington, D. Comparing drug testingself-report of drug use among youthsyoung adults in the general population. Substance AbuseMental Health Services Administration. Rockville, MD. Studies OoA. 2007. S URL: https://www.cdhs.udel.edu/content-sub-site/Documents/Publications/Comparing%20Drug%20Testing%20and%20Self- Report%20of%20Drug%20Use%20Among%20Youths%20and%20Young%20Adults%20in%20the%20General%20Population.pdf [accessed 2016-02-28] [WebCite Cache]
- Knowlton AR, Arnsten JH, Eldred LJ, Wilkinson JD, Shade SB, Bohnert AS, et al. Antiretroviral use among active injection-drug users: the role of patient-provider engagement and structural factors. AIDS Patient Care STDS 2010 Jul;24(7):421-428. [CrossRef] [Medline]
- Bell C, Metsch L, Vogenthaler N, Cardenas G, Rodriguez A, Locascio V, et al. Never in care: characteristics of HIV-infected crack cocaine users in 2 US cities who have never been to outpatient HIV care. J Acquir Immune Defic Syndr 2010 Aug;54(4):376-380 [FREE Full text] [CrossRef] [Medline]
- Schroeder JR, Epstein DH, Umbricht A, Preston KL. Changes in HIV risk behaviors among patients receiving combined pharmacological and behavioral interventions for heroin and cocaine dependence. Addict Behav 2006 May;31(5):868-879. [CrossRef] [Medline]
- Vlahov D, Anthony JC, Munoz A, Margolick J, Nelson KE, Celentano DD, et al. The ALIVE study, a longitudinal study of HIV-1 infection in intravenous drug users: description of methods and characteristics of participants. NIDA Res Monogr 1991;109:75-100. [Medline]
- Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet 2002 Jan 19;359(9302):248-252. [CrossRef] [Medline]
- Paganini-Hill A, Ross RK. Reliability of recall of drug usage and other health-related information. Am J Epidemiol 1982 Jul;116(1):114-122. [Medline]
- Darke S. Self-report among injecting drug users: a review. Drug Alcohol Depend 1998 Aug 1;51(3):253-63; discussion 267. [Medline]
- Neale J, Robertson M. Comparisons of self-report data and oral fluid testing in detecting drug use amongst new treatment clients. Drug Alcohol Depend 2003 Jul 20;71(1):57-64. [Medline]
- Napper LE, Fisher DG, Johnson ME, Wood MM. The reliability and validity of drug users' self reports of amphetamine use among primarily heroin and cocaine users. Addict Behav 2010 Apr;35(4):350-354 [FREE Full text] [CrossRef] [Medline]
- Ware NC, Wyatt MA, Bangsberg DR. Examining theoretic models of adherence for validity in resource-limited settings. A heuristic approach. J Acquir Immune Defic Syndr 2006 Dec 1;43 Suppl 1:S18-S22. [CrossRef] [Medline]
- mHealthAlliance. http://www.mhealthknowledge.org/sites/default/files/5_mHealth_5Year_Report.pdf. Five Years of Mobilizing for Health Impact; Key Achievements and Future Opportunities URL: http://www.mhealthknowledge.org/sites/default/files/5_mHealth_5Year_Report.pdf [accessed 2016-03-06] [WebCite Cache]
- Bedi G, Preston KL, Epstein DH, Heishman SJ, Marrone GF, Shaham Y, et al. Incubation of cue-induced cigarette craving during abstinence in human smokers. Biol Psychiatry 2011 Apr 1;69(7):708-711 [FREE Full text] [CrossRef] [Medline]
- Epstein DH, Marrone GF, Heishman SJ, Schmittner J, Preston KL. Tobacco, cocaine, and heroin: Craving and use during daily life. Addict Behav 2010 Apr;35(4):318-324 [FREE Full text] [CrossRef] [Medline]
- Shiffman S, Balabanis MH, Gwaltney CJ, Paty JA, Gnys M, Kassel JD, et al. Prediction of lapse from associations between smoking and situational antecedents assessed by ecological momentary assessment. Drug Alcohol Depend 2007 Dec 1;91(2-3):159-168 [FREE Full text] [CrossRef] [Medline]
- Minami H, McCarthy D, Jorenby D, Baker T. An Ecological Momentary Assessment analysis of relations among coping, affect and smoking during a quit attempt. Addiction 2011 Mar;106(3):641-650 [FREE Full text] [CrossRef] [Medline]
- O'Connell KA, Shiffman S, Decarlo LT. Does extinction of responses to cigarette cues occur during smoking cessation? Addiction 2011 Feb;106(2):410-417 [FREE Full text] [CrossRef] [Medline]
- Hedeker D, Mermelstein RJ, Berbaum ML, Campbell RT. Modeling mood variation associated with smoking: an application of a heterogeneous mixed-effects model for analysis of ecological momentary assessment (EMA) data. Addiction 2009 Feb;104(2):297-307 [FREE Full text] [CrossRef] [Medline]
- Shiffman S, Patten C, Gwaltney C, Paty J, Gnys M, Kassel J, et al. Natural history of nicotine withdrawal. Addiction 2006 Dec;101(12):1822-1832. [CrossRef] [Medline]
- Epstein DH, Preston KL. TGI Monday?: drug-dependent outpatients report lower stress and more happiness at work than elsewhere. Am J Addict 2012;21(3):189-198 [FREE Full text] [CrossRef] [Medline]
- Epstein DH, Preston KL. Daily life hour by hour, with and without cocaine: an ecological momentary assessment study. Psychopharmacology (Berl) 2010 Aug;211(2):223-232 [FREE Full text] [CrossRef] [Medline]
- Epstein DH, Willner-Reid J, Vahabzadeh M, Mezghanni M, Lin J, Preston KL. Real-time electronic diary reports of cue exposure and mood in the hours before cocaine and heroin craving and use. Arch Gen Psychiatry 2009 Jan;66(1):88-94 [FREE Full text] [CrossRef] [Medline]
- Preston KL, Epstein DH. Stress in the daily lives of cocaine and heroin users: relationship to mood, craving, relapse triggers, and cocaine use. Psychopharmacology (Berl) 2011 Nov;218(1):29-37 [FREE Full text] [CrossRef] [Medline]
- Preston KL, Vahabzadeh M, Schmittner J, Lin J, Gorelick DA, Epstein DH. Cocaine craving and use during daily life. Psychopharmacology (Berl) 2009 Dec;207(2):291-301 [FREE Full text] [CrossRef] [Medline]
- Digiusto E, Seres V, Bibby A, Batey R. Concordance between urinalysis results and self-reported drug use by applicants for methadone maintenance in Australia. Addict Behav 1996;21(3):319-329. [Medline]
- Ledgerwood DM, Goldberger BA, Risk NK, Lewis CE, Price RK. Comparison between self-report and hair analysis of illicit drug use in a community sample of middle-aged men. Addict Behav 2008 Sep;33(9):1131-1139 [FREE Full text] [CrossRef] [Medline]
- Wilcox CE, Bogenschutz MP, Nakazawa M, Woody G. Concordance between self-report and urine drug screen data in adolescent opioid dependent clinical trial participants. Addict Behav 2013 Oct;38(10):2568-2574 [FREE Full text] [CrossRef] [Medline]
- Kirk GD, Linas BS, Westergaard RP, Piggott D, Bollinger RC, Chang LW, et al. The exposure assessment in current time study: implementation, feasibility, and acceptability of real-time data collection in a community cohort of illicit drug users. AIDS Res Treat 2013;2013:594671 [FREE Full text] [CrossRef] [Medline]
- Linas BS, Latkin C, Westergaard RP, Chang LW, Bollinger RC, Genz A, et al. Capturing illicit drug use where and when it happens: an ecological momentary assessment of the social, physical and activity environment of using versus craving illicit drugs. Addiction 2015 Feb;110(2):315-325. [CrossRef] [Medline]
- Tumwebaze H, Tumwesigye E, Baeten JM, Kurth AE, Revall J, Murnane PM, et al. Household-based HIV counseling and testing as a platform for referral to HIV care and medical male circumcision in Uganda: a pilot evaluation. PLoS One 2012;7(12):e51620 [FREE Full text] [CrossRef] [Medline]
- PharmChem Inc. PharmChek Drugs of Abuse Sweat Patch Techical Questions &#38; Answers 2014 URL: http://www.pharmchem.com/default.asp?page=technical [accessed 2016-02-28] [WebCite Cache]
- Skinner HA. The drug abuse screening test. Addict Behav 1982;7(4):363-371. [Medline]
- Golub ET, Latka M, Hagan H, Havens JR, Hudson SM, Kapadia F, et al. Screening for depressive symptoms among HCV-infected injection drug users: examination of the utility of the CES-D and the Beck Depression Inventory. J Urban Health 2004 Jun;81(2):278-290 [FREE Full text] [Medline]
- King TS, Chinchilli VM, Wang K, Carrasco JL. A class of repeated measures concordance correlation coefficients. J Biopharm Stat 2007;17(4):653-672. [CrossRef] [Medline]
- Cohen J. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 1960 Apr 01;20(1):37-46. [CrossRef]
- Nelson KE, Galai N, Safaeian M, Strathdee SA, Celentano DD, Vlahov D. Temporal trends in the incidence of human immunodeficiency virus infection and risk behavior among injection drug users in Baltimore, Maryland, 1988-1998. Am J Epidemiol 2002 Oct 1;156(7):641-653 [FREE Full text] [Medline]
- Kidwell DA, Smith FP. Susceptibility of PharmChek drugs of abuse patch to environmental contamination. Forensic Sci Int 2001 Feb 15;116(2-3):89-106. [Medline]
- Chawarski MC, Fiellin DA, O'Connor PG, Bernard M, Schottenfeld RS. Utility of sweat patch testing for drug use monitoring in outpatient treatment for opiate dependence. J Subst Abuse Treat 2007 Dec;33(4):411-415 [FREE Full text] [CrossRef] [Medline]
- De GN, Fucci N. The current status of sweat testing for drugs of abuse: a review. Curr Med Chem 2013;20(4):545-561. [Medline]
- Tassiopoulos K, Bernstein J, Heeren T, Levenson S, Hingson R, Bernstein E. Hair testing and self-report of cocaine use by heroin users. Addiction 2004 May;99(5):590-597. [CrossRef] [Medline]
- Winhusen TM, Somoza EC, Singal B, Kim S, Horn PS, Rotrosen J. Measuring outcome in cocaine clinical trials: a comparison of sweat patches with urine toxicology and participant self-report. Addiction 2003 Mar;98(3):317-324. [Medline]
- Brady JE, Friedman SR, Cooper Hannah L F, Flom PL, Tempalski B, Gostnell K. Estimating the prevalence of injection drug users in the U.S. and in large U.S. metropolitan areas from 1992 to 2002. J Urban Health 2008 May;85(3):323-351 [FREE Full text] [CrossRef] [Medline]
- Islam MM, Topp L, Conigrave KM, van BI, Maher L, White A, et al. The reliability of sensitive information provided by injecting drug users in a clinical setting: clinician-administered versus audio computer-assisted self-interviewing (ACASI). AIDS Care 2012;24(12):1496-1503. [CrossRef] [Medline]
- Des Jarlais D C, Paone D, Milliken J, Turner CF, Miller H, Gribble J, et al. Audio-computer interviewing to measure risk behaviour for HIV among injecting drug users: a quasi-randomised trial. Lancet 1999 May 15;353(9165):1657-1661. [Medline]
- Macalino GE, Celentano DD, Latkin C, Strathdee SA, Vlahov D. Risk behaviors by audio computer-assisted self-interviews among HIV-seropositive and HIV-seronegative injection drug users. AIDS Educ Prev 2002 Oct;14(5):367-378. [Medline]
- Carey MP, Carey KB, Maisto SA, Gordon CM, Weinhardt LS. Assessing sexual risk behaviour with the Timeline Followback (TLFB) approach: continued development and psychometric evaluation with psychiatric outpatients. Int J STD AIDS 2001 Jun;12(6):365-375 [FREE Full text] [Medline]
- Graham CA, Catania JA, Brand R, Duong T, Canchola JA. Recalling sexual behavior: a methodological analysis of memory recall bias via interview using the diary as the gold standard. J Sex Res 2003 Nov;40(4):325-332. [CrossRef] [Medline]
- Morrison-Beedy D, Carey MP, Tu X. Accuracy of audio computer-assisted self-interviewing (ACASI) and self-administered questionnaires for the assessment of sexual behavior. AIDS Behav 2006 Sep;10(5):541-552 [FREE Full text] [CrossRef] [Medline]
- Rumptz MH, Tobias C, Rajabiun S, Bradford J, Cabral H, Young R, et al. Factors associated with engaging socially marginalized HIV-positive persons in primary care. AIDS Patient Care STDS 2007;21 Suppl 1:S30-S39. [CrossRef] [Medline]
- Tobias CR, Cunningham W, Cabral HD, Cunningham CO, Eldred L, Naar-King S, et al. Living with HIV but without medical care: barriers to engagement. AIDS Patient Care STDS 2007 Jun;21(6):426-434. [CrossRef] [Medline]
- Maisels L, Steinberg J, Tobias C. An investigation of why eligible patients do not receive HAART. AIDS Patient Care STDS 2001 Apr;15(4):185-191. [CrossRef] [Medline]
- Anand P, Springer SA, Copenhaver MM, Altice FL. Neurocognitive impairment and HIV risk factors: a reciprocal relationship. AIDS Behav 2010 Dec;14(6):1213-1226 [FREE Full text] [CrossRef] [Medline]
- Rosenthal R, Persinger GW, Fode KL. Experimenter bias, anxiety, and social desirability. Percept Mot Skills 1962 Aug;15:73-74. [CrossRef] [Medline]
- Epstein DH, Preston KL, Stewart J, Shaham Y. Toward a model of drug relapse: an assessment of the validity of the reinstatement procedure. Psychopharmacology (Berl) 2006 Nov;189(1):1-16 [FREE Full text] [CrossRef] [Medline]
|ACASI: audio-computer assisted self-interview|
|ALIVE: AIDS linked to the intravenous experience|
|CES-D: Center for Epidemiologic Studies-Depression Scale|
|CI: confidence interval|
|DAST: drug abuse screening test|
|EMA: ecological momentary assessment|
|EXACT: Exposure Assessment in Current Time|
|GEE: generalized estimating equations|
|IQR: interquartile range|
|mHealth: mobile health|
|OR: odds ratio|
|PDA: personal digital assistant|
Edited by G Eysenbach; submitted 27.03.15; peer-reviewed by P Sullivan, D Epstein; comments to author 15.08.15; revised version received 17.10.15; accepted 10.11.15; published 15.03.16
©Beth S. Linas, Andrew Genz, Ryan P. Westergaard, Larry W. Chang, Robert C. Bollinger, Carl Latkin, Gregory D. Kirk. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 15.03.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.