A retrospective analysis using deidentified data of the Retrofit weight-loss program was performed using a case series [34] approach that included the participants with known weight measurements at 6, 12, and 24 months. This study characterized the changes in participants’ body weight and body mass index (BMI) from the first weight measurement (start date) to different time points and the percentage of participants who reached a clinically significant weight loss of 5% at the corresponding time period. A secondary analysis was conducted focusing on participants with known weight at 12 months, to characterize the differences in various behaviors, age, and sex between participants who lost ≥10% of their starting weight and remaining participants. Western Institutional Review Board granted institutional review board exemption.

Clients in this study were paying customers of the Retrofit program who enrolled through the direct-to-consumer website (Retrofitme.com) or through an employer-sponsored program. Participants were defined as a client who provided at least one weight measurement (N=2720).

Inclusion criteria included participants who had a starting BMI of >25 kg/m², had signed up for the program between September 27, 2011, and December 31, 2014, and were at least 18 years of age. Exclusion criteria included a participant having no weight measurement available at 6, 12, or 24 months. A lack of available weight measurement was due to either a start date more recent than the reviewed data window (see Figure 1, inclusion criteria) or not providing a known weight measurement (see Figure 1, exclusion criteria). Decreasing numbers of participants at each data window was related to study design and not directly related to dropping out of the Retrofit program. The reported Retrofit programs are 12-month programs. However, participants could request to continue their program beyond 12 months. If a participant does not remain in an active program guided by an expert coach, the participant still has continued access to program devices, Wi-Fi scale and activity tracker, and private dashboard. According to previous study observations, self-weighing adherence decreases over time [35-39]; therefore, including only those participants with weight data in this study design, the number of excluded participants increased as the data windows progressed. For the purposes of this study, excluded participants will be defined as dropouts.

Initially, 2720 clients were considered as study participants who provided at least one weight measurement. On the basis of the inclusion and exclusion criteria defined in Figure 1, the final study sample sizes were determined at different time points. At 6 months, 1387 participants met the final inclusion criteria. At 12 months, 1075 participants met the final inclusion criteria. At 24 months, 338 participants met the final inclusion criteria. The sample was treated as an independent group at each data window, as a participant could have a known weight measurement at any one or more than one of the observed milestones. See Figure 1.

Retrofit offered 3 programs during the period of analysis: Expert 10 Weight Loss Program, Expert 15 Weight Loss Program, and Advisor Weight Loss Program. The programs were designed with a 6-month weight-loss phase and an additional 6-month weight maintenance phase. Weight maintenance after the 12 months was anticipated to be observed through continued application of the learned health behaviors.

The participant was initiated into the program, meeting with a personal program advisor who explained the components of the program, and tested familiarity with the provided technology and Web-based video communication capabilities. Client information was collected during the initial setup period.

The participant was provided with a Fitbit activity tracker, Wi-Fi–enabled scale, and access to a private dashboard (see Multimedia Appendix 2). The dashboard allowed each participant to keep a personal food and exercise log, review his or her personal data, and enabled communication between the participant and his or her expert coach through a Web-based electronic messaging feature. The private dashboard was accessed via the Retrofitme.com Web application, mobile website, or mobile phone app, which was available on Apple iOS and Android platforms.

Participants were provided with sessions and check-ins to use with an expert coach during their program. The initial session was scheduled for 60 minutes with follow-up sessions scheduled for 30 minutes. Scheduled check-ins were 15 minutes. Sessions were conducted via Web-based video chat or mobile phone (see Multimedia Appendix 3). All sessions included an educational component, allowing the participant to learn the Retrofit philosophy and weight-loss guiding principles associated with nutrition, mindset, exercise, and daily activity. Sessions and check-ins were used for collaboration between the expert coach and participant to evaluate current health-related behaviors, goal setting, and create individualized plans and strategies. The sessions and check-ins also provided accountability to previously agreed upon strategies. A minimum of 24 one-on-one coaching sessions that included sessions and check-ins, or only check-ins, were allotted to each participant’s program. One-on-one coaching session totals included 24 sessions for Expert 10, 36 sessions for Expert 15, and 12 sessions and 12 check-ins for Advisor.

Participants were encouraged to weigh in and wear their activity tracker daily. Wearing an activity tracker and setting a step goal has been associated with a decrease in BMI and increase in activity [40]. Step count goals were personalized to the participant’s baseline step count. Expert coaches recommended that participants increase step counts in increments of 500 to ultimately achieve their personal daily step goal at 6 months. Participants were encouraged to communicate daily with the expert coach via Web-based messages on the dashboard. Expert coaches were required to review a client’s food and exercise logs, step data, weight data, and progress toward goals a minimum of 1 time per week to provide feedback via Web-based messages. If a client initiated a coaching conversation, the expert coach was required to respond within 24 hours.

All 3 programs provided the participants with the same technology, access to a weight-loss expert, accountability, feedback, and the opportunity to communicate with a weight-loss expert via Web-based messages equally. The differences among programs were defined by number and type of one-on-one coaching sessions a participant was provided, and the Advisor program provided access to 1 expert coach, whereas Expert 10 and Expert 15 programs provided a team of experts in mindset, nutrition, and exercise. Weight-loss experts were employed professionals with a master’s or doctorate level college education in nutritional sciences, exercise physiology, health education, counseling, or psychology. Mindset experts had degrees in counseling, health education, social work, or psychology. Nutrition experts were registered and licensed dietitians, and exercise experts were exercise physiologists. Experts are assigned to a participant for the duration of their program; however, a new expert may have been assigned to a participant’s program related to employee turnover or as part of the quality assurance process. Expert coaches were trained on and utilized Retrofit’s weight-loss protocol.

Weight data were collected through use of the provided Wi-Fi connected scale (97.87% of recorded weights) or self-reported entry (2.13%). Self-reported entry was permissible if participants had difficulty setting up their Wi-Fi scale. Starting weight was determined at the first collected weight from the participant. The criteria for determining if a participant had a known weight measurement at different time points in the program are outlined in Table 1. A widening window approach was used at each time point to account for decreased self-weighing behavior. A reduction in the frequency of self-weighing behavior is observed in existing research [35-39]. Therefore, a wider range of time was used to determine a participant’s weight at 12 and 24 months. Similar widening window approaches have been used in previous commercial weight-loss studies [41].

When more than one measurement was collected within the identified range, the weight measurement on or closest to the specified day of measurement was used. Participants were encouraged to step on the scale daily.

Participant self-monitoring adherence was analyzed at 12 months through the use of the activity tracker and frequency of weigh-ins. Participant behaviors and engagement were observed through the number of coaching conversations in the form of an electronic message posted on the private dashboard between participant and weight-loss expert, number of days that an electronic message was logged, along with the length of each electronic message. Coaching conversations include both coach-initiated and participant-initiated Web-based messages. In addition, number of coaching sessions attended were also analyzed for participant engagement.

The primary outcome measurements were total weight lost in kilograms, percentage of weight lost, change in BMI, and percentage of participants who lost ≥5% and ≥10% of their starting weight. Program outcomes were analyzed at 6, 12, and 24 months, grouped by sex. At each milestone, data of participants with a known weight measurement were used to calculate outcomes.

Further analysis summarized self-monitoring behaviors and coaching conversations via Web-based messages between the participant and weight-loss expert at 12 months. The analysis grouped participants based on sex. Participants were also divided into groups that lost at least 10% of their starting weight at 12 months and those who had a weight loss of less than 10% at 12 months.

The summarized behaviors include total weigh-in measurements, total days of activity tracker use, daily step count average, total number of coaching conversations via Web-based messages, total count of days with a conversation, average conversation length, and number of coaching sessions attended.

Primary data analyses were performed using Python 2.7.11, which included NumPy 1.10.4, Pandas 0.17.1, and SciPy 0.17.0 analytic packages. For two-group comparisons, t tests of equal variance were conducted on continuous variables at baseline and subsequent time points. One-way analysis of variance (ANOVA) was used to determine mean differences for more than two-group comparisons. Subsequently, Tukey tests were conducted to determine mean differences. Chi-square analyses were performed to determine differences among categorical variables when appropriate. Outcome variable means are summarized with standard errors (SEs) and a 95% confidence interval is included in the populations summarized in Figure 2. Alpha was set at .05 for all statistical tests to determine statistical significance.

Of the participants with a known weight measurement at each time point, 41.9% (581/1387) at 6 months, 43.6% (469/1075) at 12 months, and 44.4% (150/338) at 24 months were male. There were no differences in age or starting BMI at baseline, between male and female participants, for each sample. Male participants had a higher starting weight at 6 months (P<.001), 12 months (P<.001), and 24 months (P<.001). Baseline summaries for age, starting weight, and starting BMI are outlined in Table 2.

The average weight loss at 6 months was −5.55% (SE 0.20) and −4.86% (SE 0.18) for males and females, respectively. Males at 12 months had an average weight loss of −6.28% (SE 0.28). Females at 12 months lost an average of −5.37% (SE 0.28). The average weight loss at 24 months was −5.03% (SE 0.61) and −3.15% (SE 0.62) for males and females, respectively. Weight loss was calculated by subtracting baseline weight from milestone weight. There was a significant difference in total weight lost in kilograms, percentage of weight lost, and BMI change at all of the observed milestones, when comparing males and females. A complete outline of these weight change outcomes, for all 3 observed durations is available in Table 3.

Figure 2 shows the percentage of the male and female participants who lost ≥5% of their starting weight. There was a significant difference between male and female participants at 6 months (P=.045) and 12 months (P=.02).

Participants who lost ≥10% of their starting weight were identified as high performers. Those participants who did not achieve that amount of weight loss were identified as the remaining participants.

At baseline, high-performing males had a statistically significant higher average starting weight of 112.7 (SD 22.11) kg, whereas the remaining male participants had an average starting weight of 107.8 (SD 21.84) kg (P=.045). High-performing females were older on average than the remaining females, where the average ages were 50.3 (SD 11.0) years and 46.9 (SD 10.7) years, respectively (P=.001). All other baseline characteristics were similar between the high-performing participants and the remaining participants. Baseline characteristics are outlined in Table 4.

Further analysis of participants’ sex, age, and average weight loss at 12 months was performed by dividing participants by sex, grouping them by 10-year age ranges, and conducting a one-way ANOVA. To address outlying age groupings, participants 20 years and younger or 80 years and older were not included in the assessment. For male participants, there was no statistically significant difference (P=.37) in relation to age and weight loss. However, for female participants, there was a significant difference of mean weight loss between the different groups (P=.002), see Figure 3. A subsequent Tukey test was performed, finding that the significant mean differences occurred between the 31- to 40-year age group and 51- to 60-year age group (P=.026) and between the 31- to 40-year age group and 61- to 70-year age group (P=.004).

For all self-monitoring behaviors, the high-performing participants had significantly higher adherence at 12 months. High-performing males weighed in 197.3 (SE 9.97) times, whereas remaining males weighed in 165.4 (SE 4.39) times (P=.001). High-performing females weighed in 222 (SE 8.47) times compared with 167 (SE 4.12) times for remaining females (P<.001). The total days of activity tracker use and average steps per day for high-performing females was 304.7 (SE 6.81) days and 8380.9 (SE 268.5) steps, respectively, whereas the remaining females had 266.6 (SE 98.1) days (P<.001) and 7059.7 (SE 2499.9) steps (P<.001). Males had 297.1 (SE 88.0) and 255.3 (SE 106.3) activity tracker days (P<.001) for high performers and remaining participants, respectively. On average, high-performing males had 9099.3 (SE 2954) average daily step counts, whereas the remaining males had 8251.4 (SE 2821) steps (P=.008). Self-monitoring measurements for males and females are outlined in Table 5.

Reviewing engagement between the participant and expert coach, high-performing female participants had significantly more total coaching conversations via Web-based messages with 341.4 (SE 15.7) compared with 301.1 (SE 8.88) for the remaining participants (P=.03). High-performing females also had more days with at least one coaching conversation, with 118 (SE 4.10) days compared with the remaining participants who had 108 (SE 2.17) days (P=.03). Similar trends were found when reviewing male participant coaching conversation totals; however, these were not significant. High performers had 260.2 (SE 14.02) coaching conversations and 100.9 (SE 3.60) days, compared with the remaining males with 236 (SE 7.87) coaching conversations (P=.14) and 95.9 (SE 2.21) days (P=.27). Interestingly, the coaching conversations were not longer for high performers. This was true for females where conversation length was 232.4 (SE 5.31) characters for high performers and 247.6 (SE 3.61) characters for remaining participants (P=.04). This also was found with males, as high performers’ conversation length was 239.9 (SE 6.93) characters compared with 262 (SE 5.13) characters (P=.03) for remaining participants. Coaching conversations via electronic message measurements can be found in Table 5.

Further engagement was reviewed in number of coaching sessions attended. High-performing male participants attended 18.7 (SE 0.62) sessions compared with remaining male participants attending 16.7 (SE 0.43) sessions (P=.008). High-performing female participants attended 19.7 (SE 0.57) sessions, whereas remaining female participants attended 17.7 (SE 0.34) sessions (P=.003). High-performing male and female participants attended a statistically significant number of coaching sessions over remaining participants. See Table 5 for number of coaching session measurements.

Participants on the Retrofit program lost an average of −5.21% (male −5.55%, female −4.86%) at 6 months, −5.83% (male −6.28%, female −5.37%) at 12 months, and −4.09% (male −5.03%, female −3.15%) at 24 months. Men consistently lost more weight than women at all the milestones. At 12 months, 56.3% (264/469) of males and 48.8% (296/606) of females had clinically significant weight loss, losing 5% of starting weight. High-performing male and female participants, who lost ≥10% of their starting weight, had higher adherence to all self-monitoring behaviors, whereas only high-performing female participants had a higher rate of engagement through coaching conversations. However, both male and female high performers attended a statistically significant number of one-on-one coaching sessions.

Although more females were included in the study population at each time point, no differences were observed in age or starting BMI at baseline; however, males had a higher starting weight at each time point. In addition, men were significantly more successful at 6, 12, and 24 months with more total weight lost in kg, greater percentage of weight loss, and change in BMI. Therefore, more male participants lost ≥5% than female participants. Enrollment, baseline data, and weight loss comparison between male and female participants are consistent with other weight-loss studies; however, a higher percentage of men were represented in the Retrofit study population than presented in the literature [6,9,15,17,19,20,22,23,30,33,40,42]. Women are shown to seek out weight-loss opportunities more than men, whereas men lose more weight regardless of age or baseline weight characteristics, likely due to biological differences between males and females [5,42,43]. However, regardless of total weight lost, losing a clinically significant amount of weight at ≥5% is of most importance to reduce comorbidities related to overweight and obesity [9-12,25,42,43].

Determining potential baseline indicators and behaviors of participants achieving 10% or greater weight loss was important to increase the reduction of overweight- and obesity-related health conditions [9-11,23]. A majority of the baseline characteristics for high-performing males and females were not significantly different compared with the remaining participants. However, high-performing males had a higher starting weight than remaining male participants and high-performing females were older on average than the remaining female participants. Older adults have been shown to be more successful in losing weight than younger adults owing to intervention adherence in the Look AHEAD trial [42,43]. Men, in general, and specifically older women are more motivated by health risks than cosmetic or social factors [44,45].

Of particular interest were the participant behaviors associated with high-performing participants at 12 months, which included the self-monitoring behaviors of weighing in, number of days wearing an activity tracker, and average number of steps per day, as well as engagement behaviors, including total number of coaching conversations, number of days with a coaching conversation, length of coaching conversations, and number of coaching sessions attended. Male and female high performers had a greater adherence rate to all self-monitoring behaviors and attended significantly more coaching sessions, which was consistent with the large amount of available research connecting program adherence to weight loss [20,25,33,44]. Self-monitoring behaviors, specifically when incorporated through technology, have consistently been shown to improve weight-loss outcomes [20,25,28,30,32,33].

High-performing female clients engaged more through coaching conversations than high-performing male clients on both total number of conversations and number of days with a conversation. This also was seen by Tate et al [33] in number of diary submissions being significantly associated with weight loss, although the study did not divide participants by sex. Participants, both male and female, who did not achieve 10% weight loss had longer conversations than high-performing participants. This observation identifies the hypothesis that frequency of messages, as opposed to length of messages, is a critical component of participant and coach asynchronous communication via Web-based messages. Multiple studies support the hypothesis that frequency of contact does improve weight-loss outcomes, specifically in achieving ≥5% weight loss [15,31,32].

This study has several strengths, including reporting of real-world weight-loss outcomes. Participants were actual clients of Retrofit and were not recruited or provided with any incentives to participate in the study. In addition, all clients who met the starting BMI, age, and weight inclusion criteria were included as participants. No client who had a lack of success on the program was removed or eliminated from the population. As an uncommon research practice noted by Gudzune et al [6], researchers conclude that this study adds value and brings a unique set of outcomes to weight-loss research. No previous commercial program has published all of its data in such a manner, providing a true picture of efficacy of the Retrofit program. In addition, outcomes were segmented by sex to identify specific baseline characteristics and behaviors for success between men and women; and lastly, age was used as an additional component to target baseline characteristics and behaviors to achieve 10% or greater weight loss in 12 months.

In addition to the identified strengths, the researchers also noted some weaknesses. According to the study design, cross-sectional samplings were performed at 6, 12, and 24 months to select participants with known weight, which provided a separate population for each weight-loss period. The case series approach of this study does not allow any causal inferences based on the critical observations. In addition, because of the real-world population, it is unknown if participants were integrating any other weight-loss practices outside of the Retrofit program components.

Because of a lack of real-world outcomes within the commercial weight-loss industry, Retrofit encourages all commercial weight-loss programs to publish similar data to show efficacy of programs. By reporting real-world outcomes in relation to targeted behaviors, commercial weight-loss programs can structure protocols and client strategies to enhance long-term weight-loss success. Clearly defining the necessary behaviors for long-term weight-loss success and the efficacy of each commercial weight-loss program will solidify our ability as an industry and country to combat the obesity crisis, including obesity-related diseases such as heart disease and diabetes.

Recommended future research includes studying a population over time for a causal effect of weight loss, as well as comparing the population with a control group and targeting specific characteristics and behaviors for high-performing clients such as what engagement factors matter for male participants, why older female participants are more successful, and why men with a higher starting weight are more successful. In addition, it is recommended to compare male and female high performers as a single population with the remaining participants for further insight into baseline characteristics and behaviors for success, and observing self-monitoring and health behavior adherence beyond 12 months. Finally, evaluate the impact of Retrofit’s weight-loss program on short-term and long-term employer health care spending.

In conclusion, participants on the Retrofit program lost an average of −5.21% at 6 months, −5.83% at 12 months, and −4.09% at 24 months. Men, on average, lost more weight than women. High-performing participants, or participants who lost ≥10% of starting weight at 12 months, had a greater adherence to the self-monitoring behaviors of weighing in, number of days wearing an activity tracker, and average number of steps per day. Female high performers had a higher engagement in coaching conversations through conversation days and total number of conversations with their expert coaches. However, both high-performing male and female participants attended significantly more one-on-one coaching sessions.