A prevailing challenge in dietary and eating research is the ability to accurately measure dietary intake. Historically, the assessment of dietary intake and eating behaviors uses self-reporting tools [1,2], such as food diaries, food frequency questionnaires, and 24-hour dietary recalls [3,4]. All dietary assessment self-report methods have some level of measurement error (difference between measured and true values) [5,6]. Dietary data collected via self-report methods may be misreported because of biases, such as recall or memory bias (when a respondent erroneously recalls their dietary intake) and social desirability bias (when a respondent desires to present oneself positively) [7-9]. Studies have also found that those with certain characteristics (eg, obese weight status and body image dissatisfaction) are more likely to underreport their energy intake [10,11].

The field of nutritional epidemiology has produced an abundance of studies that have examined the role of dietary intake (ie, what and how much is consumed) in human health and disease—specifically, macronutrients (eg, fats and carbohydrates), types of food, quality of food, dietary patterns, and more [12]. Decades of laboratory-based and observational research indicate that dietary intake is a critical component of chronic disease prevention [13]. However, the measurement of diet in free-living populations remains a significant challenge in the field. In addition, even if public health researchers can easily and accurately track free-living dietary intake, dietary intake patterns are notoriously difficult to change long-term [14].

Eating behaviors and patterns (ie, food choices and motives and feeding practices) and context (who is eating, when, where, with whom, etc) also play a significant role in the development of obesity and other chronic diseases, including type 2 diabetes and heart disease [15-20]. These findings indicate that the patterns and features of eating events may be key contexts that shape dietary intake, and thus could be more malleable features of eating behavior that could be intervened. However, the field lacks appropriate behavioral theories that provide a richer understanding of how eating behaviors vary across contexts and across time [21,22].

Emerging technologies offer a potential solution for the accurate assessment of dietary intake by addressing the limitations of self-reported dietary assessment methods. The incorporation of technologies into dietary assessment can improve the quality and validity of dietary data by passively measuring eating in naturalistic settings over long periods with minimal user interaction [23]. Two emerging technological advances in dietary assessment tools include the following:

EMA and wearable sensors are able to measure behavior near or just in time, thereby reducing or eliminating the recall bias that can affect retrospective self-report measures. In addition to improving the validity of data, these technologies offer the opportunity to measure eating behavior frequently and over long periods, allowing researchers to examine how it varies over multiple timescales (varies over the day, over the week, etc).

To address the limitations of traditional dietary assessment methods and theories, the Monitoring and Modeling Family Eating Dynamics (M2FED) study developed a sensor system that used smartphones as well as deployable and wearable sensors to collect synchronized real-time data on family eating behavior [28]. This study used the following: (1) wrist-worn smartwatches containing inertial sensors (accelerometer and gyroscope) to automatically detect arm movements and hand gestures associated with eating; (2) EMA via smartphone to confirm whether the eating occurred and to measure other contextual information, such as who was present during the eating event and the current mood of the respondent; and (3) Bluetooth proximity beacons to determine the approximate location of the smartwatches.

Rather than focusing on dietary intake (caloric intake, portion sizes, etc), this study took a novel approach by measuring eating behaviors (ie, food choices and motives and feeding practices) and context (who is eating, when, where, with whom, etc). Family eating dynamics have yet to be measured and modeled dynamically to better contextualize our understanding of social influence processes within family systems. This paper begins the first step toward producing new models that develop behavioral theory, and it may enable the identification of temporally specific processes and events within the family system that can be targeted for personalized, context-specific, real-time feedback.

The validity of using wearable sensors to automatically assess eating behavior and context has been tested in both laboratory and field settings [27,29-31], indicating that the performance of the wearable sensors decreases in naturalistic settings (compared with controlled laboratory settings). Studies have used a variety of sensors (eg, microphones, cameras, smartwatches, and electromyography electrodes) to measure various dietary outcomes, including bites, chewing, swallowing, and duration of eating occasions [27,29-33]. A review by Bell et al [27] indicates that there is still a strong reliance on retrospective self-report methods (eg, end-of-day food diaries) to determine ground-truth eating activity to evaluate wearable sensors in the field. Given the aforementioned limitations of retrospective self-report methods to accurately assess diet, the M2FED study used event-contingent EMA to determine ground-truth eating activity in families. The use of EMA offers unique methodological advantages, such as the following:

The validity of this method has been tested in a few in-field studies [34,35]; however, it has not yet been tested in a family-based study.

One disadvantage of using technologies for data collection is the potential for participant noncompliance. A recent systematic review and meta-analysis by Wen et al [36] found that compliance rates among EMA studies in youth samples were suboptimal; the weighted average compliance rate was 78.3%, falling under the recommended 80% compliance rate [24]. Many studies have explored EMA compliance for various behaviors in various populations [36-40], but the compliance rate for a family-based EMA study is underexplored. A recent EMA study involving mothers and their children found that mothers’ presence may enhance children’s compliance with EMA questionnaires [41], suggesting that family members and other social relations may be leveraged to increase compliance in future EMA studies.

Therefore, the overall purpose of this study is to report on participant compliance (feasibility) to the 2 distinct EMA protocols of the study (hourly time-triggered and eating event–triggered assessments) and on the performance (validity) of the wearable sensor in automatically detecting eating events in a family-based study. Specifically, the primary aims of this study include the following:

Following enrollment, 2 members of the research team visited the home of each recruited family 2 separate times.

A total of 74 participants from 20 families were enrolled in the M2FED study. In all, 18% (13/74) of participants dropped out of the study or were removed from the data set if their participation (as determined by the participation algorithm) was 0% (ie, they did not answer any EMAs and never wore the smartwatch; Figure 6).

In addition, the data from 4% (3/74) nonparent adult participants made up approximately 1.44% (61/4232) of the EMAs received, so these participants were removed from the analytic sample as well. The remaining 78% (58/74) of participants included in the analytic sample did not significantly differ from the enrolled sample (N=74) by age, gender, or parent role (P>.05; Table 2).

Of the 58 participants, 43% (n=25) were parents and 57% (n=33) were children. On average, children were aged 15.12 years (SD 3.97 years) and parents were aged 43.72 years (SD 6.71 years). There were 39% (13/33) female children and 68% (17/25) female parents. In all, 61% (20/33) of children and 68% (16/25) of parents identified as Hispanic or Latino (Table 3).

Of the 20 enrolled families, most (17/20, 85%) were 2-parent households, 15% (3/20) of the families had 1 child living at home, 75% (15/20) of the families had 2 children, 5% (1/20) of the families had 3 children, and 5% (1/20) of the families had 4 children (Table 4). On average, family deployments lasted 14.90 days (SD 3.13 days).

In total, 15,010 EMAs (14,348/15,010, 95.59% time-triggered and 662/15,010, 4.41% eating event–triggered) were sent by the M2FED system and received by study phones of the participants. After filtering the data through the participation algorithm, 27.78% (4171/15,010) EMAs remained in the data set: 88.95% (3710/4171) of which were time-triggered and 11.05% (461/4171) were eating event–triggered (Table 5).

On average, families received 209.0 EMAs (SD 89.4; range 86-391), and individuals received 71.9 EMAs (SD 34.3; range 8-176) each. Participants received, on average, 64.0 time-triggered EMAs (SD 31.3; range 8-147) and 8.0 eating event–triggered EMAs (SD 8.9; range 0-40) across the deployment. The daily average number of EMAs received per person was 5.2 (SD 2.7; range 0.6-11.7) for all EMAs, 4.7 (SD 2.4; range 0.3-10.2) for time-triggered EMAs, and 0.6 (SD 0.6; range 0-2.7) for eating event–triggered EMAs (Table 5). Of the 4171 total EMAs, 18.58% (775/4171) were received in the morning, 30.46% (1270/4171) in the afternoon, and 50.97% (2126/4171) in the evening. Of the 461 eating event–triggered EMAs, most, 45.8% (211/461), were sent in the evening (Figure 7). Children received 57.52% (2399/4171), fathers received 10.72% (447/4171), and mothers received 31.77% (1325/4171) of the total EMAs. Of the 461 eating event–triggered EMAs, 49.9% (n=230) were received by children, 7.4% (n=34) by fathers, and 42.7% (n=197) by mothers.

The overall compliance rate across the 20 deployments was 89.26% (3723/4171) for all EMAs, 89.7% (3328/3710) for time-triggered EMAs, and 85.7% (395/461) for eating event–triggered EMAs (Table 6). The average family-level compliance was 89.4% (SD 5.74%; range 75.7%-98.1%) for all EMAs, 89.8% (SD 5.84%; range 75.8%-98.7%) for time-triggered EMAs, and 85.9% (SD 14.3%; range 55.6%-100%) for eating event–triggered EMAs. At the individual-level, the average compliance for all EMAs was 89.6% (SD 9.5%; range 53.8%-100%), for time-triggered EMAs was 89.5% (SD 10.1%; range 50%-100%), and for eating event–triggered EMAs was 88% (SD 17.5%; range 28.6%-100%). The distributions of individual- (Figure 8A) and family-level compliance (Figure 8B) are shown in Figure 8.

Three separate logistic regression models were fitted with the following data sets: (1) all EMAs, (2) time-triggered EMAs, and (3) eating event–triggered EMAs.

Results from the first model indicate that time of day and whether other family members had also answered an EMA were significant predictors of compliance to all EMAs (Table 7). Participants were 37% less likely (odds ratio [OR] 0.63, 95% CI 0.46-0.86) to respond to an EMA in the afternoon and 39% less likely (OR 0.61, 95% CI 0.45-0.81) to respond to an EMA in the evening compared with the morning (reference group). Participants were 91% more likely (OR 1.91, CI 1.56-2.34) to respond to an EMA if another family member had responded to an EMA in the surrounding 30-minute time interval.

The results from the second model indicate that time of day and whether other family members had also answered an EMA were significant predictors of compliance to time-triggered EMAs (Table 7). Participants were 40% less likely (OR 0.60, 95% CI 0.42-0.85) to respond to a time-triggered EMA in the afternoon and 47% less likely (OR 0.53, 95% CI 0.38-0.74) to respond to a time-triggered EMA in the evening than in the morning (reference group). Participants were approximately 2 times as likely (OR 2.07, 95% CI 1.66-2.58) to respond to a time-triggered EMA if another family member had responded to any EMA in the surrounding 30-minute time interval.

Results from the third model indicate that weekend status and deployment day were significant predictors of compliance to eating event–triggered EMAs (Table 7). Participants were 2.4 times as likely (OR 2.40, 95% CI 1.25-4.91) to respond to an eating event–triggered EMA on the weekend, than on a weekday. Participants were 8% less likely (OR 0.92, 95% CI 0.86-0.97) to respond to an eating event–triggered EMA for every 1-day increase in deployment day.

At least one eating event was automatically detected during the deployment for 46 participants. This subsample (ie, the analytic sample for aim 2A) did not significantly differ from the enrolled sample (N=74) by age, gender, or parent role (P>.05; Table 2).

A total of 461 eating events were automatically detected using the smartwatch algorithm across these 46 participants. Participants responded to 85.7% (395/461) of the corresponding eating event–triggered EMAs. Participants confirmed that 76.5% (302/395) of the detected events were true eating events (ie, true positives) and 23.5% (93/395) were not true eating events (ie, false positives). For approximately one-third of these false positives, participants reported that they were using their phones at the time. The calculated precision measure, that is, the number of true positives divided by the sum of true positives and false negatives, was 0.77.

At least three eating event–triggered EMAs were received by 36 participants. This subsample (ie, the analytic sample for aim 2B) did not significantly differ from the enrolled sample (N=74) by age, gender, or parent role (P>.05; Table 2). For this subsample, the average individual-level proportion of correctly detected eating events (true positives / total number of detected eating events) was 78.5% (SD 19%; range 30%-100%). In all, 72% (26/36) of the analytic sample had at least one falsely detected eating event (false positive).

Neither age (years) nor height (inches) was significantly correlated with the proportion of correctly detected eating events (r_s=0.24, P=.17 and r_s=−0.12, P=.52, respectively). The average individual-level proportion of correctly detected eating events for women was 82.1% (SD 20.4%; range 30%-100%) and was 74% (SD 16.6%; range 50%-100%) for men. The difference between the 2 groups was not significant (W=112; P=.13). The average individual-level proportion of correctly detected eating events for children was 74.3% (SD 19.3%; range 30%-100%), for fathers was 76.1% (SD 21.5%; range 58.3%-100%), and for mothers was 86.5% (SD 16.8%; range 54.5%-100%). The differences among these 3 groups were not significant (Kruskal–Wallis χ²₂=2.998; P=.22).

After applying our customized participation algorithm, we found that both individual- and family-level compliance rates to the EMA protocols of the study were relatively high (both greater than the recommended 80%) [24]. Compliance was significantly higher in the mornings overall and higher on the weekends for eating event–triggered EMAs, which supported the informal feedback we received from participants that they were more likely to participate (ie, respond to EMAs and wear the smartwatch) when they did not need to go to work or school (typically the weekend days). We also saw that overall compliance decreased as the 2-week study went on, most likely attributable to participant fatigue.

One particularly interesting finding was that participants were significantly more likely to answer an EMA if another family member had answered an EMA in a similar time frame. A similar finding was reported by Dzubur et al [41], in which mother-child dyads were more likely to comply with prompts when they were together. Although the overarching aims of the M2FED study were to measure the social influence of family members on eating behavior, this finding also indicates that social influence came into play in other parts of the study as well. Drawing from the social psychology field, several social mechanisms could partially explain these findings. For instance, an expectation could have been set early on in particular families to answer the EMA prompts, thus establishing a social norm for EMA compliance [52,53]. Similarly, some individuals may have been inclined to answer EMA prompts to conform to the behavior of other family members around the same time [52,53], especially considering that family members received their time-triggered EMAs at approximately the same time as each other.

Studies have used EMA to measure various dietary outcomes, including frequency of food intake, intake of specific types of foods (eg, low glycemic index foods), and energy intake [25]. It has been suggested in a recent systematic review of mobile ecological momentary diet assessment methods that EMA has the potential to be a novel dietary assessment method, both on its own and as a supplement to other mHealth technologies [25]. The use of EMA to assess dietary intake and eating behavior provides some key advantages, namely, the reduction of participant burden and recall bias and the maximization of ecological validity [25]. Taken together with the findings from Dzubur et al [41] and Schembre et al [25], our findings suggest that EMA can be used to sufficiently supplement automatic dietary assessment (ADA) approaches and may be a particularly useful approach for leveraging social relations and maintaining compliance in dyad- and group-based EMA studies.

Various technologies have been used to passively measure eating activity in naturalistic settings over long periods with minimal user interaction. One of the most popular technologies for assessing eating activity in the field is the wrist-worn smartwatch or accelerometer [23,27]. The performance of automatic, wearable-based, in-field eating detection approaches to date has been reviewed by Bell et al [27]. The smartwatch used in the M2FED study performed on par with other in-field devices, although comparability is difficult owing to the wide and varying metrics used by other papers [27]. Although some wearable devices included in this review performed very well, the duration of the free-living deployment was 1 day (approximately 24 hours) or shorter for more than half of the studies, and another one-third were 1 week in length or shorter [27].

Overall, 3 studies had durations that lasted at least two weeks or longer [34,54,55], 66% (n=2) of which had sample sizes of only 1 participant each. Therefore, the M2FED study is one of the first studies to extensively test the feasibility of deploying an ADA approach for a considerable amount of time (2 weeks) and with a relatively large same size (>50 participants). Part of this success stems from the combined use of mobile devices (for EMA) and smartwatches, which were selected for the M2FED study to maximize long-term usability. Although other technologies have been able to perform better in the field, the usability of these technologies (electromyography electrodes, ear and neck sensors, wearable video cameras, etc) may be lower compared with wrist-worn devices because of the inconvenient location of sensor placement, the potential to interfere with the behavior of participants in real life [56], and the potential intrusiveness or discomfort caused by the sensor [57].

This study also demonstrates that EMA is a feasible tool for collecting ground-truth eating activity and thus evaluating the performance of wearable sensors in the field. Only 2 studies [34,35] included in the review by Bell et al [27] used a novel method for obtaining ground-truth eating activity in the wild, similar to the way EMA was used in the M2FED study. In a study by Ye et al [34], when an eating gesture was automatically detected via a wrist-worn sensor, participants were sent a short message on their smartwatch to confirm or reject in real time whether they were eating. Similarly, in a study by Gomes and Sousa [35], when drinking activity was detected via a wearable sensor, participants were sent an alert on their smartphone and could then confirm or reject whether they were drinking via EMA. Although EMA and similar self-report methods have their own limitations [23,58], they offer the ability to capture and validate ground-truth eating activity near the time of eating, thus improving research scalability and participant acceptability [25].

Another key feature of the M2FED study was the ability to capture intrapersonal (individual) and interpersonal (social) contexts with our combined event- and signal-contingent protocols. A systematic review noted that <7% of EMA studies assessing diet use a combined approach [59]. EMA is a powerful tool that can be used to validate automatically detected eating behavior in the field and to easily collect information about meaningful contexts; however, few studies have used this approach and still rely on paper–pen questionnaires to validate their findings [27].

The M2FED study design had notable limitations. First, our method of obtaining ground-truth eating was only deployed via eating event–triggered EMA after an eating event was detected by the smartwatch. Thus, we could only verify true positive and false positive eating events. The M2FED system was not designed to verify true negative or false negative eating events, which limited our ability to calculate common evaluation metrics (ie, accuracy and F₁-score) and compare our results to other in-field studies described in the literature. Future research can build upon our study by implementing a verification of true negative and false negative eating events, via time-triggered EMA or other methods, to gain a better understanding of the strengths and weaknesses of such an event detection algorithm.

Second, the false positive eating events were self-reported validation, which might be subject to social desirability in underreporting an eating event. This could potentially bias the validity of the results. Third, we encountered various difficulties with the deployed technologies, including smartwatches (ie, limited battery), mobile phones (ie, limited battery and app crashes), and the Wi-Fi router (ie, wireless connection dropped). Although these challenges were anticipated and were addressed in a timely manner on all occasions, some data were lost during the data collection process.

Finally, as the scope of this study only covered in-home eating behavior, we observed relatively few eating event–triggered EMAs per person across the 2-week study (approximately 8 per person). However, the range was very wide, indicating that some participants consumed more meals inside their homes compared with others. Reasons often provided informally by participants included eating all or most meals at school or work, working early or late, traveling for work, and participating in after-school extracurricular activities.

On the other hand, this study also possesses several strengths. First, we recruited a large and ethnically diverse sample of families from Los Angeles. It has been previously noted that the lack of diverse samples in eating-related mHealth and EMA studies is a major limitation of past research [60]. Second, as noted above, the M2FED study facilitated one of the longest in-field deployments found in the literature so far. Most ADA research has been conducted in the laboratory. By deploying in the field, we are able to better understand real-life eating behavior (vs eating behavior in a laboratory) and gain a better understanding of the challenges that arise when deploying wearable sensors outside of the laboratory. Third, as the deployment process was across a 2-year period, we were able to iteratively improve our automatic eating event detection algorithm and then use the newest version in the following deployments. Finally, this study produced momentary measures of theoretical constructs as well as momentary measures of eating behaviors. The theoretical work that we can now contribute would be to understand which constructs influence behavior, which behaviors influence various constructs, and which constructs play no role at all. We can also begin to understand the role of timing in these influences.

The mHealth field is converging toward the use of a combination of user-friendly devices to assess eating behavior in the wild (eg, mobile phones and wrist-worn devices) [27,31]. Implementing user-friendly technologies for in-field dietary assessment or eating behavior interventions offers at least two substantial advantages—people are generally familiar with them [31] and may be willing to use them for longer periods compared with more intrusive devices. Although early studies experimented with less familiar, often not off-the-shelf technologies (eg, piezoelectric strain gauge sensors), most recent studies have opted for accelerometers and gyroscopes that are embedded within a wrist-worn smartwatch [27]. Furthermore, the combination of a wrist-worn smartwatch to automatically detect eating and a mobile or wearable device to capture ground-truth eating has been featured in a few studies published in the past year [61-63]. This approach is becoming more common, and these types of devices offer advantages for the user (participant) and make the use of mHealth technologies more accessible to nonengineering behavioral researchers. However, a number of related challenges have emerged. Future research will need to address comparability between newer technology-assisted measures and more traditional self-report measures of eating [64] versus other similar technology-assisted measures [27].

These user-friendly technologies also allow for passive measurement or low-effort reporting of various contexts and environments with relative ease. For example, fine-grained real-time GPS data can be scraped from both mobile devices and smartwatches to determine an individual’s location and potentially assess the external influences on behavior [65,66]. Similarly, the social environment can be gleaned from wearable cameras [67], self-report EMA [68], or proximity Bluetooth sensors [69].

The ability to determine one’s context or environment is a necessary component of ecological momentary interventions [70] or just-in-time interventions [71]. These types of intervention designs aim to provide the right amount of support at the right time and in the right context to promote behavior change [71-73]. These types of designs are well-suited for and offer unique opportunities for family-based settings [74]. They offer the ability to intervene in children and adolescents and can be designed to target the behavior of multiple family members at once [74]. As family members share genetic, environmental, and behavioral risks, family units are especially important targets for intervention and prevention [75] and have the potential to halt the intergenerational transmission of obesity and other chronic diseases.

This paper demonstrates that EMA is a feasible tool to collect ground-truth eating activity and thus evaluate the performance of wearable sensors in the field. The combination of a wrist-worn smartwatch to automatically detect eating and a mobile or wearable device to capture ground-truth eating activity offers key advantages for the user (participant) and makes the use of mHealth technologies more accessible to nonengineering behavioral researchers.