The importance of sleep is paramount to health and performance. Insufficient sleep can impede physical, emotional, and mental well-being [1,2] and can lead to a multitude of health complications such as insulin resistance [3-5], high blood pressure [6], cardiovascular disease [7,8], a compromised immune or metabolic system [9,10], mood disorders (such as depression or anxiety) [11,12], and decreased cognitive function for memory and judgment [13-15].

There are many indicators of sleep quality, an important one being sleep efficiency. Sleep efficiency is a metric that takes into consideration the time spent asleep, the time it takes to fall asleep, and the time asleep but with disturbance (ie, awakenings). Poor sleep efficiency can lead to sleep deprivation, which is found to be a major health risk with links to diseases such as diabetes and obesity [16]. Also, sleep behavior has been found to impact adolescent health [17-19]. Recent systematic reviews have shown the relevance of physical activity to sleep, including sleep efficiency [20-22]. Although the relationship between physical activity and sleep is not yet fully understood, it is thought to be a strong and complex correlation contributing to multiple lifestyle diseases such as type 2 diabetes mellitus and obesity [20-22]. However, the underlying mechanism between physical activity and sleep is not yet fully understood.

There are many research tools to study the link between physical activity and sleep, including standardized questionnaires, actigraphy sensors, and polysomnography (PSG). All of these methods have different clinical indications (eg, diagnosis of different sleep disorders) [23]. For example, PSG is considered the “gold standard” for sleep medicine, as it involves the use of multiple sensors during sleep such as electroencephalogram, motion sensors, breathing sensors, SpO₂(oxygen saturation), and so forth. These sensors monitor and observe a patient overnight [24] and can be used for diagnosing different sleep disorders. The elaborate nature of PSG means that it is generally limited to one overnight observation. Even the portable solutions, which permit PSG assessment in the patient’s home, are complex to perform and not without limitations [25,26].

To better understand the impact of daily physical activity on sleep behavior, new tools are needed. Sleep researchers in the early 1990s developed a technique called actigraphy to study sleep interactions using wearable devices [27]. Although actigraphy traditionally uses wearable devices to evaluate the sleep period of a patient, it can also be used to observe physical activity. Actigraphy has become a widely used tool, as it has been found to be much more reliable than subjective or self-reported sleep diaries and behavior logs [28]. Patients wear the device for a period of time as they continue their daily routines. The technique has been especially influential for large cohort studies where PSG is not feasible [29]. Moreover, actigraphy allows for the continuous longitudinal monitoring of a patient. This is particularly impactful for the study of diseases such as chronic obstructive pulmonary disease where sleep disturbances can be a predictor of exacerbation of the disease [30]. Current approaches to the analysis of actigraphy data involve sleep experts performing a number of steps manually. This is a bottleneck, and hence there may be troves of actigraphy data left unanalyzed.

Furthermore, there are hundreds of consumer-grade physical activity and sleep tracking devices (eg, Fitbit) collecting motion data similarly to actigraphy devices. These consumer devices are being used by millions of people collecting huge amounts of data. Some devices even allow the collection of data for very long periods (eg, several months), as they require only an occasional need for battery recharge (ie, Garmin VivoFit). Recent studies have found that these consumer-grade devices can sometimes have similar precision to clinical-grade actigraphy sensors [31]. There are successful examples of the integration of physical activity wearable data into eHealth tailored applications [32], including smart-watch health applications [33] that can collect physical activity and sleep data directly from the watch.

As previously noted, physical activity and sleep are interrelated. In this study, we tested the feasibility of predicting poor or good sleep efficiency based on physical activity data from the awake periods from a wearable device (ie, actigraphy). We took a detour from classical methods and proposed deep learning approaches to modeling the relationship between sleep and physical activity. The importance of this research is two-fold.

First, since our approach can be used in cases where sleep sensor data is not available, our models can be used in the early detection of potential low sleep efficiency. This is a common problem with consumer-grade wearable devices, as users might not wear them during the night (battery recharging, sensors embedded in smart jewelry, and so forth).

Second, our study was focused on advanced deep learning methods. Traditional prediction models applied to activity raw accelerometer data (eg, logistic regression) suffer from at least 2 key limitations: (1) They are not robust enough to learn useful patterns from noisy raw accelerometer output. As a result, existing methods for classification and analysis of physical activity rely on extracting higher-level features that can be fed into prediction models [34]. This process often requires domain expertise and can be time consuming. (2) Traditional methods do not exploit task labels for feature construction, and thus can be limited in their ability to learn task-specific features. Deep learning has the advantage that it is robust to raw noisy data, and can learn, automatically, higher level abstract features by passing raw input signals through nonlinear hidden layers while also optimizing on the target prediction tasks. We leveraged this characteristic by building models using a range of deep learning methods on raw accelerometer data. This reduced the need for data preprocessing and feature space construction and simplified the overall workflow for clinical practice and sleep researchers.

The methodology of this study involved several steps: (1) the collection of wearable data with regards to physical activity and sleep patterns, (2) data processing and representation, (3) data modeling, and (4) performance evaluation. The following subsections explain each of these steps.

The deidentified data used in this study were collected by Weill Cornell Medical College-Qatar for a research study called Qatar’s Ultimate Education for Sleep in Teenagers. The aim of the study was to determine different sleep patterns in adolescents residing in Qatar and how these related to body weight status. The institutional review board (IRB) approval was initially obtained by the joint IRB for Hamad Medical Corporation and Weill Cornell Medical College. The selected cohort was chosen from adolescents attending 1 of the 2 high schools and registered in grades 7-11. The dataset used contained complete actigraphy data from a subset of 92 adolescents over 1 week. There were 322 total sleep instances: 102 from boys and 220 from girls.

After agreeing to participate in the study, the adolescent participants were provided with an ActiGraph GT3X+ device, placed on their nondominant wrist, and were instructed to wear it at all times for 7 consecutive days and nights. The ActiGraph GT3X+, shown in Figure 1, is a clinical-grade wearable device that samples a user’s activity at 30-100 Hz (we used 30 Hz in this study). The effectiveness of this device has been successfully validated against clinical polysomnography [35]. The participants were instructed not to remove the water-resistant device at any time. We used the manufacturer’s software (ActiLife version 6; available from ActiGraph, LLC) to export the data. Although the device is triaxial, our methods used the vertical axis only.

The subjective self-reported sleep diaries were collected, but they were not considered for the study, since actigraphy data provides a more objective measurement of physical activity and sleep patterns. Sadeh’s and American Academy of Sleep Medicine sleep definitions were used in this interpretation [27,36,37]. For calculating the sleep efficiency score, we considered each individual sleeping period as sleep.

As explained in more detail below, in this study we explored the use of deep learning methods to predict sleep quality based on actigraphy data. We compared the results of our deep learning models to those of logistic regression, a standard statistical method. In this section, we explain how each model was built. The models used in our study were as follows:

For logistic regression, we fed the input signals directly into the output layer for prediction. In contrast, for the deep learning models, we passed the inputs through one or more hidden layers before they were fed to the output layer for prediction. Each hidden unit in the hidden layers used a nonlinear activation function. In our study, we experimented with different hidden layer units using rectified linear unit as our activation function.

For the evaluation of the performance of the different models, we reported on several well-known metrics such as accuracy, precision, recall, F1-score, and area under the receiver operating characteristic (ROC) curve (AUC). These metrics are commonly used in data mining and clinical decision support systems.

As shown in Table 1 and Figure 6, the performance of the logistic regression in the metrics previously explained performed worse than the models based on deep learning. “Only the simple RNN performed worse than logistic regression in both F1-score (harmonic mean of precision and recall) and accuracy.

As shown in Table 1, the AUC of the logistic regression model was low. The AUC value for LR was 0.6463, which was close to 0.5 (equivalent to a random prediction). This showed the limitation of classical models in analyzing raw accelerometry.

In contrast, all the AUC values for the deep learning models were better with a range from 0.7143 to 0.9714, TB-LSTM being the best performer and RNN, the worst. Time-batched LSTM, CNN, and MLP performed the best with AUC scores showing an improvement over LR by 50%, 46%, and 46%, respectively.

Upon comparing the deep learning neural network models, we noticed that CNN yielded slight improvement over MLP in AUC (0.07% absolute), but more in F1 (3.57%) and accuracy (4.00%). These improvements could be attributed to the time- invariant convolution-pooling operations of the CNN model to pick key local patterns, which generalized well for small training data. The F1-score improved by 12% for time-batched LSTM, by 15% for CNN, and by 11% for MLP. The accuracy improved by 22% for time-batched LSTM, by 27% for CNN, and by 22% for MLP.

A comparison of the RNN models revealed that LSTM outperformed simple RNN by a wide margin; 19%, 10%, and 19% in AUC, F1, and accuracy, respectively. These gains over simple RNNs could be attributed to the specially designed gates of LSTMs that could capture long-range dependencies between physical activities in the sequences.

However, this is not surprising. Both simple and LSTM RNNs operate on sequences, where each time step comprises only one activity value. This often results in very long sequences. As mentioned earlier, in this setting RNNs cannot compose higher-level features effectively because of low-dimensional input at each time step, and also suffer from vanishing gradient problems due to lengthy sequences.

Our solution to surmount this problem was to use a time-batched input to LSTM. When we compared the results of our time-batched LSTM (TB-LSTM) with those of MLP and CNN, we found that TB-LSTM outperformed both MLP and CNN in AUC by 3%; in fact, it had the highest AUC score. It achieved better F1 score than MLP (1%), but worse than CNN (–2%). When we observed their precision and recall values, we found that TB-LSTM had a very high recall but lower precision, which meant that it tended to predict more goods than gold standard. For the same reason, its accuracy was also lower than that of CNN.

In our study, we focused on the prediction of poor versus good sleep efficiency. That is a simple, but important, problem, as sleep efficiency has been found to be a crucial sleep parameter with important health consequences [38,43,44]. Furthermore, we did not quantify in our prediction the overall sleep efficiency but simply the differentiation between two classes (poor versus good sleep efficiency). This classification is consequently not an indicator of sleep patterns, but the prediction of a sleep quality parameter that might indicate a potential sleep problem.

As in prediction or diagnostic problems, our results need to be discussed in terms of sensitivity and specificity (see Table 2). The deep learning methods of CNN and TB-LSTM were the best performers overall. Their sensitivity (0.97 and 1, respectively) showed that these models were able to detect nearly all the cases of “good-quality sleep,” meaning that in a tool for screening potential sleep problems these models will be able to detect easily people with normal sleep quality. Often high sensitivity comes at the price of low specificity (ie, failing to identify negative cases, or true negative error). This was the case of logistic regression, which had a high sensitivity but a specificity of 0.3, meaning that in such models many “poor sleeps” would have been wrongly classified as good sleep. This is very important, since misidentifying poor sleep cases can lead to underdiagnosis of problematic sleep.

The sensitivity (also known as recall) and specificity of each of the models are reported in Table 2. The high sensitivity values of each of the models indicate that deep learning has a strong capability of correctly identifying individuals with good sleep patterns from their preceding awake activity. The specificity is high for TB-LSTM, MLP, and CNN, indicating that these models were also able to successfully distinguish those with poor sleep patterns.

Previous algorithms, such as by Sadeh et al [27,37], do not focus on the prediction of sleep quality based on physical activity during awake time, but rather on prediction of sleep quality based on accelerometer data during the sleep periods. The objective of those studies was the validation of the actigraphy data compared with PSG. Consequently, the objectives and findings of those studies cannot be compared with our study.

There are several data analytics areas relevant to our work. To our knowledge, our research was the first one looking into the use of deep learning for the study of actigraphy data related to physical activity and sleep. Previous research on the use of deep learning for sleep science has been focused on PSG data [45,46]. In other application areas, deep learning has been used for human activity recognition [47,48] which is a similar technical problem. In a previous study, we combined human recognition of actigraphy data with other machine learning algorithms, but not deep learning [49].

There were some limitations in our study regarding the generalization of our results. Sleep behavior can be affected by cultural aspects and also change with age. Our study sample drew sleep data from adolescents, aged 10-17 years, living in Qatar. Future research will need to evaluate whether applications of deep learning for sleep research using actigraphy will yield similar results in different populations (eg, adults and people with chronic conditions).

In our study, the prediction was simplified to “good” and “poor” sleep quality with regards to sleep efficiency. This may be an oversimplification of complex sleep problems. To provide more precise predictions (eg, quantitative value of sleep quality), these techniques will need to be validated. A prediction, such as the one presented in this study, might be useful for the detection of people with unhealthy sleep patterns, but not to identify the causes of poor sleep efficiency.

There is also a limitation in the interpretation of deep learning. Deep learning models are “black boxes” and do not provide explanation of their sleep efficiency prediction. Other techniques such as logistic regression can provide insights on which features contribute to the prediction. However, this study showed that the performance of such models was much lower than that of deep learning. New techniques in deep learning are being researched to facilitate the interpretation of such models.

In our study, the models used the data from the individual’s awake time to predict of sleep quality. The prediction was made at the last moment before sleep using the full awake time activity. If these models were to be used to provide personalized feedback to individuals with sleep problems, they will need to be tested with fragments of the awake time, giving an individual time to alter their behavior. Since our data was fragmented into sleep periods and awake times specific to an individual, the models would be able to handle varying durations of awake time.

Our study showed the feasibility of deep learning in predicting sleep efficiency using wearable data from awake periods. This is of paramount importance because deep learning eliminated the need for data preprocessing and simplified the overall workflow in sleep data research. The feasibility of our approach can lead to new applications in sleep science and also to the development of more complex eHealth sleep applications for both professionals and patients. These models can also be integrated in the broader context of quantified self [55,56].