Questionnaire-based assessment is one of the most common methods of data collection in research settings. Therefore, the ADAM portal was designed around this need, from the process of questionnaire and assessment design to the remote delivery of questionnaires. Designing a questionnaire based on the study needs was made simpler for researchers with the inclusion of an easy-to-use, drag-and-drop interface and a library of frequently used question templates. The adaptive questionnaire feature in ADAM was designed to be used by study coordinators and investigators with no need to do programming, with a similar ease of use to that of proprietary solutions such as Qualtrics, but unlike most of these proprietary solutions, ADAM allowed integrations with an mHealth app. The ADAM portal also supported more advanced questionnaire features such as input-type validation, error messages, required field validation, and question branching. After a questionnaire was published in the ADAM portal, depending on the purpose, a public link or a participant-specific private link was generated for study use. Once the participant had answered and submitted this questionnaire via the given link, the ADAM portal automatically formatted the answers, placed time stamps, calculated scores, and triggered any necessary flags and status updates that could be immediately reviewed by the study coordinators. As the ADAM portal was designed to support multiphase study needs, questionnaires could be generated for each study phase while considering different restrictions for each phase, for example, adding certain filtering restrictions in the screening phase for certain criteria in the screening phase’s questionnaire. This questionnaire module can also be adapted to fit the need for remote participant consent delivery, as the study consent form usually shares the same structure and data requirements as an assessment questionnaire.

The ADAM portal also supported the management of other steps in the research study, which commonly included multiple phases or processes such as recruitment, enrollment, engagement, and retention. For the recruitment phase, the ADAM portal supported a multistep screening process. Each sequential screening step could be triggered by the answers or the calculated scores of the current step’s screening or assessment questionnaire. It could also support manual evaluation by study coordinators. Once screening decisions had been made, the system automatically moved the eligible participants to the next phase in the ADAM system. This real-time process reduced the study coordinators’ manual data review and data calculation workload. It also helped reduce the study cost by improving the overall recruitment process’s efficiency.

For the enrollment and engagement phases, the ADAM portal has a built-in account management system that can assign a unique study-specific ID for each participant for efficient progress tracking while protecting their identifiable information. Furthermore, these IDs can also be used as common IDs linking data coming in from outside sources such as the Fitbit and Nokia APIs. With the benefit of the study-specific ID, the ADAM portal can easily record participants’ digital footprints within the system and track the milestones and progress of all participants within the study. The portal also provides the study coordinator with access to aggregated data, such as that of important milestones or sorted by group, which is represented in a CONSORT (Consolidated Standards of Reporting Trials) flow diagram. Such a flow diagram is valuable for study management in 3 aspects: it allows the study coordinator to switch the recruitment effort to a specific group of people during recruitment in cases where a group has a less-than-optimal recruitment rate; it facilitates the adjustment of the speed of recruitment to help make sure the study can start and end on time; and it can monitor the screening process and rate of attrition, which may provide some hints for adjustments that may be needed in screening methods.

For the retention process, the ADAM portal provides a dashboard for showing study-related alerts or flags and participants’ performance according to the data gathered from wearable devices, the digital scale, and the study app. Using this dashboard, the study coordinators have a near real-time report on whether participants are following the study protocol strictly, in other words, adhering to the study; as a result, they can intervene or take alleviating actions at a point closer to when participants began to show a trend toward lower adherence. This feature also helps to estimate the milestones for participants’ assessments. Other features are study compensation and remuneration management, which were considered indispensable parts of the ADAM system. Study coordinators used the remuneration feature of the ADAM portal to record all payments made or planned at different time points in the study.

Data synchronization with wearable and IoT devices is one of the key features of the ADAM system. As more people rely on commercially available wearable and IoT devices to get their health data and track their fitness, a growing number of investigators have used this type of off-the-shelf wearable device as part of their studies. This trend indicates the need for a study management system that can integrate IoT and wearable data from multiple providers and vendors. Currently, the ADAM portal provides integration with Fitbit (wearable) and Nokia (IoT), since these companies have the largest market share, a secure authentication process, and stable open APIs. The steps to integrate data from wearables in ADAM from the users’ (study coordinators’) perspective are made simple. Study coordinators first need to create participant accounts in the ADAM portal and get participants’ permission and authorization to collect their data from outside APIs. Once ADAM has been granted authorization, data synchronization for the given participant will be scheduled daily. The ADAM portal also supports subscription-based data integration, where any new data that become available in the wearable will trigger synchronization or a data import into the ADAM database in a real-time fashion, although we did not implement this feature in the SMARTER study.

The ADAM system provided randomization by minimization, a more advanced randomization method, which was used to ensure the balance of prognostic factors between treatment groups and set equal treatment allocations [37].

ADAM adopts Firebase Analytics libraries to anonymously monitor app use and statistics. This part of the data mainly describe the use of the app within a period. The study team can obtain a real-time visualization on the ADAM main page. This information can provide an overview of the app use among all study participants and additional background information for the efficiency of message delivery.

An evaluation of the ADAM platform was conducted as part of a weight-loss clinical trial called the SMARTER trial [34]. This study was registered on ClinicalTrials.gov (NCT033677936) and approved by the Institutional Review Board of the University of Pittsburgh. The SMARTER trial was a randomized controlled trial that followed a single-site, 2-group design, with a 1-year study duration for both comparison and intervention groups [34]. In the SMARTER trial, participants were allocated equally to the 2 treatment conditions: SM only or SM with real-time tailored feedback messages (SM+feedback) [34]. The study provided all participants with a Fitbit Charge 2 and a Nokia Scale for physical activity and weight SM; participants used the Fitbit app for dietary SM. The SMARTER app generated 3 tailored messages per day responding to the SM data provided, for example, diet, physical activity, and weight. More details for the SMARTER trial design and screening criteria can be found in the study design paper [34].

The SMARTER trial’s recruitment began in August 2018, and the trial itself was completed in March 2021. This study was designed to include a 3-phase screening process. The first 2 screenings were based on web-based questionnaires, where the ADAM portal provided autoscreening services based on the answers submitted by the participants. As shown in Figure 2, there were 3 steps for generating a questionnaire: Figure 2A lists all the questionnaires and consent forms from the SMARTER trial that were available on the ADAM portal; Figure 2B shows the straightforward drag-and-drop interface to design or edit a questionnaire; and Figure 2C shows that the ADAM portal could autocreate a specific questionnaire link for each participant on the user list page in the ADAM portal. In the SMARTER trial, phase-1 and phase-2 questionnaires were the study’s basic screening surveys, and the ADAM portal provided automatic screening of the candidates based on preset criteria such as aged ≥18 years, BMI between 27-43 kg/m², smartphone user, general health history, lifestyle, and health and medical history. In phase 3, the ADAM portal still handled the display, submission, and scoring processes of the questionnaire but also gave the screening decisions back to the study coordinators. In this phase, participants were asked to complete a 5-day SM food diary using the Fitbit app. As illustrated in Figure 3, in our ADAM portal, the food diary data were automatically pulled from the Fitbit server for the study coordinators to review. Collecting data in one place relieved the pressure on study coordinators from switching back and forward between several different data management systems and tools.

As displayed in Figure 4A, all the SMARTER trial recruitment steps were visualized and cataloged in the ADAM system. By clicking on each recruitment step, study coordinators could review the current remaining number of participants in that phase and make decisions on moving them to the next phase of the study. Once participants were ready for the randomization step, they would be displayed on the randomization page, as shown in Figure 4E. The ADAM system provided a minimization randomization algorithm to support this randomization process. After randomization, participants’ Fitbit and Nokia accounts were linked to our ADAM system by study coordinators; all the Fitbit-related and Nokia-related data including physical activity, food diary, and self-weighing data could then be collected by the ADAM system daily for the following 1-year intervention period.

After randomization, participants were asked to do daily SM during the 1-year study and complete 3 assessments at baseline, 6 months, and 12 months. During the 1-year study, participants were asked to adhere to the daily SM protocol by wearing a Fitbit tracker to record their daily physical activities, report food intake using the Fitbit app, and weigh themselves using a smart scale. All data were synchronized to the ADAM system and were displayed in a raw format, as shown in Figure 3, as well as a study-tailored precalculated report format, as shown in Figure 4B and C. The clinical dashboard in Figure 4B acts as a valuable resource for study coordinators to track participants’ adherence to all SM protocols and to receive alerts if participants lose weight too rapidly (a precaution to prevent unwanted clinical effects) or have poor study adherence. Study coordinators used this page to mark withdrawals at any time during their study period. The ADAM portal then automatically recorded and marked the withdrawal date appropriately. The trial included 3 assessments that heavily used the ADAM portal as a method for appointment scheduling and web-based survey completion. Figure 4F represents an example of a 6-month assessment page, and Figure 4D is an all-in-one assessment calendar. Due to the COVID-19 pandemic, in-person assessments were no longer permitted as of March 2020. The study team made the decision to change all the assessments to be conducted remotely and to obtain the primary outcome of weight directly from participants’ smart scales in their homes instead of the usual scale at the research center. Due to the remote-ready adaptive design of the ADAM system, this COVID-19–triggered transition was able to be performed quickly and with almost no interruption to the study flow.

The ADAM portal was also used for participants’ compensation and reimbursement management. For example, once the payment was made, study coordinators recorded this on the Adam portal’s remuneration page. Another example of a useful feature provided by the ADAM portal for the trial is a real-time flowchart (Figure 5). The flowchart provided an overview of the entire study at any given time, which allowed for an adaptive recruitment strategy and effective adjustments to study resource allocation as needed. Furthermore, using the ADAM portal, the research staff coordinated activities with each other via a single platform remotely, since everything related to the study was accessible in real time and on the web.

The SMARTER app is an ADAM-integrated app that was designed for the SMARTER trial. Only the intervention group had access to the app. Figure 6 shows the SMARTER app use data based on reports provided by Google’s Firebase Analytics, which included aggregated data for the following aspects: number of daily active users, popular app versions, most common mobile devices among participants, and active users by city. This report worked as supporting information for the study team to track app use. As for the app’s content, Figure 7 shows sample messages tailored and delivered by the SMARTER app. The ADAM system tracked and logged these messages’ time stamps and status on whether the messages were delivered, opened, or missed—information for future data analysis that may provide auxiliary insight into the overall outcome of the trial.

The ADAM system incorporated 4 monitoring mechanisms to ensure the quality of collected data. First, as illustrated in Figure 4B on the clinical dashboard, ADAM can generate alerts. To guarantee data quality from various sources, the system sends automatic alerts for the following reasons: (1) if there are no food data recorded for 7 consecutive days; (2) if no weight data are recorded for 7 days or if there is a recorded weight change exceeding 5 lbs within a 7-day period; and (3) if there are no Fitbit tracker data received for 7 consecutive days. Second, the system performed automatic checks on API connections and promptly reported any issues to the study team. Third, to further ensure data integrity, study coordinators manually checked and reviewed selected records during the in-person assessments. Fourth, before data analysis, the research team also ran an algorithm to check for outliers in the collected data.

We introduced the architecture and implementation of a data collecting and clinical trial management system called ADAM as a solution to current barriers in clinical or behavioral studies involving wearables, IoT, and mHealth. The ADAM system’s adoption of the newest available interfaces in integration with multiple wearable devices makes it an attractive choice for behavioral and population-based health intervention studies. Using the ADAM system, the study team in the SMARTER trial no longer needed to switch between different systems or documents when working with study data from diverse sources and avoided most data missingness and data synchronization issues. ADAM’s study management functions supported recruitment, screening, and randomization and helped maintain retention with a dashboard and flagging. In addition, it provided an efficient and manageable environment for study tracking and adjustment as necessary. ADAM’s study flowchart was extensively used in the team’s weekly meetings and contributed widely to decision-making during the entire study period.

The system evaluations throughout the SMARTER trial indicated that the ADAM system was able to meet most of the data collection and study management demands. However, the ADAM system was not without limitations. The first limitation was not due to the ADAM system per se, but due to the limited availability of open APIs for third-party data collection sources also used in the study; therefore, the ADAM system had issues including all datasets for the study. For example, in the SMARTER trial, study participants needed to complete two 24-hour dietary recalls using the ASA24 system, a dietary assessment tool, at 3 assessment points. We could not directly synchronize the dietary recalls from the ASA24 system with the ADAM system due to the API limitations of the ASA24 platform. This limitation prevented study coordinators from receiving real-time data visualization and data analysis in the ADAM system for ASA24’s data. Therefore, all the data from ASA24 needed to be manually downloaded and uploaded to the ADAM system. Although the ADAM system can integrate more datasets and be an all-in-one data solution for the study, we still need to consider approaches that may have API restrictions or the unavailability of such APIs by different companies and organizations that provide additional data required by the study.

The second limitation to consider is the data missingness and data synchronization issues with Fitbit or Nokia devices. Fitbit’s open web API limits the number of API calls per hour for each participant or Fitbit account to 100. This stringent quota restricted the number of days that we could pull the data from the Fitbit server, as we needed to have at least a single call for each day of data. Furthermore, specifically for the SMARTER trial, we were restricted to pulling data only for the last 7 days. Since we were pulling data from several different areas of Fitbit’s API, such as physical activity data, diary data, and sleep data, it quickly consumed the quota for the API calls. At the same time, from time to time, some Fitbit and Nokia devices experienced issues sending data to the Fitbit server when there were internet or Bluetooth connection problems or participants failed to open the Fitbit or HealthMate app for an extended amount of time. Those situations caused a data transfer delay and ultimately led to data missingness and failed data synchronization in the ADAM system.

Furthermore, the version of ADAM used in the SMARTER trial did not have the capabilities to send emails, SMS text messages, or push notifications. Therefore, the interventionists and study coordinators needed to use different modalities of communication and tools to send emails or SMS text messages to the participants. The next iteration of ADAM will include these capabilities to streamline communications and monitoring within one system, ADAM. This would increase the efficiency and the general ease of using the system in reaching participants as needed.

Finally, the ADAM system is more suitable for digital intervention studies using consumer technologies. If the study is highly dependent on in-clinic assessments using offline tools and closed or proprietary systems, the ADAM system provides a limited benefit since the data may be hard or even impossible to transmit to the ADAM system. In addition, studies that target recruitment groups who are unfamiliar with mobile technology or who have limited access to the internet could experience barriers to the adoption of the ADAM system technology. According to the questionnaires, these groups represent a small portion of potential study participants, but this is an important limitation to consider in future studies that may target these groups. This limitation was not an issue in the SMARTER trial, as the study was designed to exclude participants who were not smartphone users.

To address these limitations, first, we are considering supporting additional, widely used wearable devices. Second, we plan to generalize the study management components and make them easier to set up and adapt to distinct types of study design. However, overall, the ADAM system provides a solution to digitalizing a research study by providing and integrating multiple methods for collecting study data and ensuring an efficient and compliant study process.