We conducted both a laboratory-based and free-living study. In the laboratory-based study (study 1), individuals engaged in prescribed activities of various intensities over a 2-hour period wearing both an ActiGraph and 3 Android smartphones (ie, HTC MyTouch, Google [manufactured by HTC] Nexus One, and Motorola Cliq) all worn both on the hip and in the pocket. In the free-living study (study 2), individuals engaged in usual daily activities that were tracked over a 7-day period via an Android smartphone (Google Nexus One) and an ActiGraph.

There were a total of 15 participants (age: mean 55.5 years, SD 6.6, range 43-65; women: 56%, 8/15; education: mean 16.25 years, SD 1.6) who participated in study 1. A list of the activities in the laboratory study can be found in Figure 1. Table 1 reports Spearman rank order correlations between the ActiGraph counts and the 3 phones. Overall, results suggest strong correlations between the ActiGraph and the 3 phones and between the 3 phones (ρ=.90). Figure 1 is a visual representation of all activities with color-coded labeling to identify the classification of each activity as sedentary, light physical activity, or MVPA.

Table 2 reports results of the mixed model regression equations for each cut-point estimate. The betas and intercepts were the values used to calculate the phone-based cut-points listed when the ActiGraph cut-points (ie, <100 and >1952) were imputed into the equation. Figure 2 provides estimates of the mean differences between the predicted values from the phones and the actual ActiGraph counts across phones and across different modeling datasets (eg, the full dataset, to the dataset that only included hip data and excluded biking/standing). Overall, there was some instability in the overall estimate depending on the observations included/excluded from the models using the leave-one-out technique based on the large root mean standard errors but that these differences were not greatly impacted by the specific phone used or the filtering strategies. Overall, the models were improved if data were aggregated across all phones as opposed to using phone-specific estimates. For the remainder of the paper, we only refer to the aggregated estimates of cut-points across phones as these appeared most stable and we continue to report both the “full model” and “no bike and standing” models because there appeared to be improved model fit using this filter but no improved model fit when excluding measurements from the phones while worn in the pocket.

Figure 2 shows the mean difference between the predicted value of an ActiGraph count compared to the actual ActiGraph count (ie, predicted count – actual count). This mean difference across different models allows for an estimate of differences across models when using the leave-one-out strategy of model building. Higher mean differences suggest less stable regression models that are more influenced by individual observations in the model. The error bars represent the root mean standard error across all models run utilizing a leave-one-out procedure, which further provides insights on the interpretability of the models. High root mean standard errors suggest large variation in estimates during the leave-one-out procedure, but also provide an estimate of “meaningful” mean difference estimates across the various regression models. Results suggest that the mean differences observed did not differ by phone or data-filtering strategy used, but that there was large variation of impact across observations, thus justifying the use of the leave-one-out method for creating a more stable regression model.

Table 3 reports the percentage of time each device accurately classified an activity into its corresponding MET classification (ie, sedentary, light physical activity, or MVPA) [28]. Overall, results suggest that the phones correctly classified activities at nearly the same level as the ActiGraph. For example, when excluding behaviors from the model that are known to be poorly classified using cut-point estimates (ie, bicycling and standing), the ActiGraph correctly classified 91% (108/119) of all activities across all participants. In comparison, the phones correct classification levels ranged from 78% (73/93; MyTouch, Full Model) to 91% (85/93; Nexus One, no bike and standing model). Not surprisingly, cycling activities and standing exhibited the poorest estimated agreement (see Table 3). Interestingly, the phones actually did better at correctly classifying the light intensity activity of sweeping. Again, Figure 1 provides a visual summary of these classifications on a labeled scatterplot for the Nexus One.

There were 23 participants who had acceptable data for comparing the phone accelerometer to the ActiGraph (age: mean 57.0 years, SD 6.4, range 45-69; women: 74%, 17/23; body mass index: mean 29.5, SD 5.9, range: 20.8-40.9; white: 70%, 16/23; Asian/Asian American: 22%, 5/23; bachelor’s degree or higher: 79%, 18/23; married: 44%, 10/23; working full time: 64%, 14/23). On average, participants had approximately 5 days of valid wear time for both the ActiGraph and smartphone accelerometry data (days per participant: mean 4.8, SD 2.1; total days=107). Table 4 reports correlations between the estimated number of minutes engaged in sedentary, light physical activity, and moderate-to-vigorous levels of physical activity based on the Nexus One phone using the full model cut-points derived from study 1 compared to the ActiGraph (listed in Figure 1). Overall, results suggested moderate to strong correlations between the direct estimates for sedentary physical activity and MVPA (eg, raw count comparisons: ρ=.44, P<.001; sedentary: ρ=44, P<.001; MVPA: ρ=.67, P<.001) and weak correlations for light physical activity (ρ=.38, P<.001). These correlations were nearly the same as the no bike and standing correlations (eg, raw count comparisons: ρ=.35, P<.001; sedentary: ρ=.44, P<.001; light: ρ=.34, P<.001; MVPA: ρ=.68, P<.001).

Figures 3-5 report Bland-Altman plots comparing the ActiGraph and smartphones utilizing the full model cut-point estimates. Comparison of the plots suggest good absolute mean-level differences in sedentary and MVPA activity minutes/day estimates (sedentary: mean difference=–26.0 min/day, 95% CI –279.5 to 227.6; MVPA: mean difference=–1.3 min/day, 95% CI –38.4 to 35.8). Absolute mean-level differences for light physical activity were outside of the acceptable range (mean difference=–111.2 min/day, 95% CI –285.8 to –63.5). Although not shown, we did create Bland-Altman plots for the no bike and standing models and consistently found poorer absolute estimates suggesting that for absolute estimates in a free-living context, the full model cut-points listed in Table 2 (ie, <50.92 for sedentary and >305.36 for MVPA) were superior. Based on the lack of a gold standard in this study, the 95% confidence intervals are not as easily interpreted because the error likely comes from measurement issues with both devices. As such, they are provided more for broader context but should be interpreted with caution.

The purpose of this work was to determine the validity and reliability across different Android smartphones for tracking physical activity among midlife and older adults. Overall, results indicated (1) Android smartphone raw counts are strongly correlated to ActiGraph counts in a laboratory-based and free-living setting suggesting the accelerometers provide similar estimates regardless of the activity classification algorithm used, (2) Android smartphone raw counts were strongly correlated with one another suggesting that different Android phones reliably provide similar estimates before any algorithm, (3) placement in the pocket versus on the hip both in a laboratory-based and free-living context did not detrimentally impact estimates suggesting that phones can reliably be worn in either location, and (4) absolute classifications of activity intensity between the phone and ActiGraph were comparable both in-laboratory (based on similar correct classification percentages of known activities) and free-living context (based on absolute mean difference estimates via Bland-Altman plots) for sedentary and MVPA estimates but not light physical activity.

Results indicating that the phone accelerometers and ActiGraph are strongly correlated have important implications for not only cut-point-based activity classification strategies, but also any classification strategy utilizing an accelerometer. In particular, because the effective raw signals are providing similar information, it is quite plausible that similar algorithm strategies can be used between devices with similar effects, assuming some degree of calibration. This is important because it suggests other activity recognition strategies, such as machine learning/neural network analyses [10-13], can likely provide similar estimates from the raw accelerometer signals across phones. This also increases confidence that estimates between Android phones from apps that utilize the accelerometer for activity classification will likely be comparable across phones.

Results suggesting phone placement did not impact sedentary and MVPA estimates have important implications for usability. Specifically, we found an almost even split in preferences related to wearing the phone, with some desiring to wear it on the hip and others desiring to keep it in their pockets. A phone will likely be worn if it fits into a person’s daily life. As such, allowing individuals to wear the device in whatever way they so desire increases the likelihood of continuing to gather the data in the long-term (as we did in our MILES intervention trial [8]). Our results suggest that acceptable activity estimates can be acquired either on the hip or in the pocket. Further work should continue to explore ways to further improve “wearability” of a system. This interest is evident in the commercial sector with devices such as the Fitbit Flex, Jawbone UP, or Misfit Shine, which are all wrist-worn activity monitors.

Finally, our results suggesting that the phones gave similar absolute estimates of sedentary and MVPA activities to an ActiGraph in both in-laboratory and free-living studies has important implications for epidemiologic and intervention research. At present, there is far more data supporting the linkage between cut-point-derived estimates of physical activity with health outcomes compared to the other data analytic techniques [1,15,33]. Based on this, although it is likely the machine learning-based classification techniques will eventually provide better methods for classifying physical activity, at present, there is still value in focusing on validating the smartphone compared to the classical cut-point-derived estimates from the ActiGraph. Our results suggest that the phones can provide an acceptable alternative to an ActiGraph for classifying sedentary and MVPA, but not light physical activity.

The ActiGraph was utilized as our primary comparison metric. Although the ActiGraph is acceptable for field-based research, it is not a gold-standard activity monitoring device. This is particularly important to be mindful of when interpreting the Bland-Altman plots because the Bland-Altman plots are designed to support comparison of a new measure to a previous gold-standard metric. The Bland-Altman plots revealed good mean-level differences but poor confidence interval estimates between the smartphone and ActiGraph. Because the ActiGraph is not a gold-standard measure, it is quite plausible that the high confidence intervals can be attributed not only to variance from the smartphone but also the ActiGraph. As such, the confidence intervals from the Bland-Altman plots need to be interpreted with caution. Further, the ActiGraph cut-points utilized incorporated only the vertical axis whereas the phones used all 3 axes, thus establishing another potential source of error between the two.

A second limitation of our work is that we utilized a cut-point-based strategy for activity classification rather than newer strategies such as neural network analysis or machine learning techniques [10-13]. Although a cut-point strategy does not afford the level of precision that these newer techniques do, cut-point estimates are still pragmatic to use in both epidemiologic and intervention research. In particular, the national guidelines for physical activity do not focus on specific behaviors, but instead the broad classes of moderate and vigorous intensity physical activity that we can classify with cut-point algorithms [3]. As such, intensity classification has utility for physical activity recommendations because it more directly corresponds with national guidelines. In addition, there is also much more work linking cut-point-based estimates of physical activity to health outcomes compared with the more precise activity classification techniques [1,33]. As such, the regression equations generated provide a strategy for establishing a linkage from estimates derived from smartphones to those health outcome linkages made via previous work that found associations with ActiGraph estimates via cut-point algorithms and health outcomes.

Another limitation is the brands of the smartphones we studied. In 2009, HTC and Motorola were among the most important manufacturers of Android smartphones, but at this point Samsung has become the primary Android phone manufacturer. Unfortunately, our study cannot provide any insights on the quality of Samsung devices. Further, although the data were comparable across our phones, it is impossible to determine if the phone estimates would remain comparable in newer phones. That said, based on the plethora of different Android phones available, our data at least increases confidence that different manufacturers and running different firmware versions of Android may provide similar estimates of physical activity based on the accelerometry.

Strengths of this research were that data are available from both an in-laboratory and free-living context. Further, the samples used to create the estimates are in-line with other physical activity assessment/validation protocols. Finally, validation occurred via direct observation of activities in laboratory.

Overall, our results suggest that an Android smartphone appears to be an acceptable alternative for estimating sedentary and moderate-to-vigorous intensity physical activity to an ActiGraph accelerometer in both a laboratory-based and free-living context. This suggests that smartphones might be an effective mechanism, by themselves, for tracking physical activity. Future work should explore other potential analytic techniques (eg, machine learning) for further improving the classification.