The COVID-19 pandemic represented an exceptional threat to human health worldwide. In that context, the collective adoption of protective behaviors (eg, hand hygiene, physical distancing, mask wearing) became an essential measure to prevent the transmission of the virus, especially during the pandemic outbreak when there was no vaccine available [1,2]. Consequently, the development of effective behavior change techniques aiming at supporting the uptake and adoption of such behaviors emerged as a public health priority [3]. However, during the course of the outbreak, there was limited or no contextualized knowledge about how to effectively promote protective behaviors in the general population [4,5].

During the early stages of the pandemic, we addressed the call for applying principles of behavior change to reduce the transmission of COVID-19 [2,3] and devised a Multiphase Optimization Strategy (MOST) [6] to develop, optimize, and evaluate a smartphone-based behavior change technique to promote hand hygiene during the COVID-19 pandemic [7,8]. We opted for a digital intervention to avoid in-person interaction, which is crucial for preventing virus transmission. Additionally, digital interventions can be tailored to individual needs and have the potential to reach a large number of users. The MOST framework was deemed highly suitable for rigorously contextualizing the intervention to the pandemic through optimization. We targeted correct hand hygiene (ie, hand washing or sanitizing) at key times because it represents an effective strategy in preventing the transmission of respiratory illnesses, including COVID-19 [9,10], and was therefore included in public health guidelines [11].

Following the MOST framework [12], the project included 3 phases: preparation, optimization, and evaluation. In the preparation phase [7], we developed a set of intervention modules and defined the optimization criteria. The development of intervention modules was guided by the Theoretical Domain Framework [13]. First, we identified behavioral determinants of hand hygiene through a literature review, and through 2 focus group discussions with the target population [7]. Then, we mapped behavior change techniques [14] to the behavioral determinants. Specifically, three intervention modules were developed targeting (1) motivation, (2) habit formation, and (3) social influence. The identified modules were paired and sequenced, resulting in 9 different intervention conditions. Each intervention condition was hosted in the developed smartphone app Soapp. During the subsequent optimization phase, we devised a mixed method double-blind parallel randomized trial between March 2021 and August 2021, which lasted 34 days. The trial aimed at identifying the intervention condition that met our optimization criteria: superiority regarding change in hand hygiene pre-post intervention, and regarding user satisfaction, usability, and engagement [8]. The results of the trial showed that hand hygiene improved while using the Soapp app. However, no superiority was found for any of the versions of the app. The qualitative results indicated that participants preferred the motivation and habit formation modules over the social influence ones. A further emerging theme pointed out the need for better distribution of intervention content over time. Thus, in the optimized version of the Soapp app, named Soapp+, we selected the motivation and habit formation modules and delivered them in a parallel fashion rather than sequentially.

Soapp+ comprises behavior change techniques aimed at targeting key motivational and volitional behavioral determinants including intention, attitudes toward hand hygiene, risk perception, outcome expectancies, self-efficacy, action planning, coping planning, action control, and habit. These determinants represent fundamental psychological constructs in prominent behavior change theories [15], addressing both reflective and automatic processes. Reflective processes are deliberate and conscious, involving setting intended goals, planning, and monitoring progress [16,17]. In contrast, automatic processes, such as habit, operate unconsciously, enabling individuals to perform behaviors without deliberate thought once learned through repetition. These automatic processes help sustain behavior change by reducing the cognitive load associated with decision-making and promoting consistency in behavior performance [18,19].

The current paper focuses on the evaluation phase of the MOST in which we tested the efficacy of Soapp+ (intervention group) by comparison to an active control group in a randomized controlled trial (RCT). As presented in the study protocol [7], we tested the following preregistered hypotheses

As the secondary aim, we examined between-group differences in flu-like infection symptoms and occurrences of COVID-19 (secondary outcomes). The corresponding results are reported in Multimedia Appendix 1.

The study design for the evaluation phase was a double-blind RCT (1:1 ratio) comparing the optimized app Soapp+ with an active control group using a simplified version of the app that focused on providing information on hand hygiene. The trial was carried out from March 9, 2022, to April 18, 2023, a few days before the World Health Organization declared the end of the global health emergency [20].

The target population was German-speaking adults from the Swiss population interested in using an app to improve hand hygiene. Inclusion criteria were (1) being at least 18 years of age, (2) owning a smartphone with mobile access to the internet, (3) being proficient in the German language, and (4) having signed an informed consent form to participate in the study. Additionally, smartphone proficiency was an implicit eligibility criterion, as participants needed to perform various tasks on their smartphones prior to the study’s commencement (eg, downloading the app and registering). As per protocol [7], the initial target sample size was 205 participants. The sample size was calculated a priori for an independent sample 1-tailed t test (β=.80; α=.05; Cohen d=0.35). The target sample size was raised to 245 participants to account for a 20% dropout rate.

Participants were recruited via social media (eg, Facebook, Instagram), mailing lists, and leaflets with the support of a market research company. We aimed at recruiting a broad range of people from the German-speaking adult Swiss general population in terms of sex, age, and socioeconomic status. The content of the recruitment materials targeted individuals interested in forming new hand-hygiene habits and learning how to correctly wash or disinfect their hands to counter the spread of the virus. Interested people were redirected to the study page on RedCap (Vanderbilt University) [29] where they could go through the study information, fill out an eligibility and consent survey, and sign the e-consent form. Afterward, participants received a registration code via email and were guided to download fthe Soapp+ app from the Apple App Store or Google Play Store and register it. The day after the registration, participants received the baseline questionnaire (T1) and were then randomized to an intervention or active control group. A simple randomization was implemented in Qualtrics, which preserved the allocation concealment. Additionally, the researchers involved in the study were blinded to intervention assignment because the participant identifier was pseudoanonymized before randomization (Figure S1 in Multimedia Appendix 1). During the intervention phase, participants completed the hand hygiene diary on days 2, 8, 16, 24, and 32. At the end of the intervention phase (day 34), participants received a second questionnaire (T2). At 6 months (182 days) after T1, participants received the last follow-up questionnaire. Questionnaires and diaries were integrated into Qualtrics services using Soapp+’s application programming interface, and the participants’ data were stored on Qualtrics. The recruitment for the evaluation trial began on March 9, 2022, and ended on October 20, 2022. Follow-up data were collected between September 5, 2022, and April 18, 2023.

The content of the intervention was delivered to participants via their personal smartphone through the study app Soapp+. The intervention group received the optimized Soapp+ app while the active control group was exposed to a simplified version of Soapp+. Table 1 shows the content delivered. The exact wording of the intervention messages and the intervention timeline can be found in Table S3 and Figure S2 in Multimedia Appendix 1.

The active control group received information on how to perform correct hand hygiene (both in written and video formats), the key times to perform hand hygiene, and a statement of the goal of the app (basic module). This content was accessible throughout the intervention phase. Based on the results of the optimization phase, we also added a list of fun facts that were delivered to the participants via push notifications throughout the intervention phase. Finally, quizzes on hand hygiene aimed at ensuring that the active control group interacted with the app throughout the intervention phase.

Like the control group, the intervention group received the basic module. They additionally received the content targeting motivation and habit. Based on the results of the optimization phase (ie, request for better-distributed content or tasks), this content was delivered in parallel instead of sequentially as done during the optimization trial.

The trial received ethical approval from the Swiss Ethics Committee of the canton of Bern (ID 2021-00164). The reporting of the trial is in line with the CONSORT (Consolidated Standards of Reporting Trials) guidelines (Multimedia Appendix 2) [35]. Informed consent was obtained from all participants, who were also informed of their right to opt out at any stage of the study. To protect privacy and confidentiality, data were deidentified through pseudonymization. As compensation, participants had the opportunity to win one of three iPhone 12s as an incentive for their participation. Additionally, after completing T2, participants were offered a small gift (ie, a bar of hand soap and a thank you card) in order to prevent dropout, which was sent to their homes.

We stopped the trial 13 months after the start of the study due to the end of the project timeline. A total of 193 participants were recruited and randomized into one of the 2 experimental conditions. Among these, 12 participants did not complete any of the 6 hand hygiene diary days, while the other 35 participants did not fill out the first diary at T1. Further, 3 participants completed the first diary but did not encounter any of the key situations to perform hand hygiene during that day. These participants (n=50) were excluded from the analysis because the main outcome (ie, hand hygiene) at T1 was missing. Out of the 193 participants who were randomized, 146 (75.6%) filled out the hand hygiene diary at T1, 91 of these (47.2% of the randomized participants) completed the hand hygiene diary at T2, and 109 (56.5% of the randomized participants) completed the hand hygiene diary at the 6-month follow-up. Figure 1 shows the participants’ flow through randomization, T1 diary assessment, T3 diary assessment, and follow-up for each intervention group.

Sociodemographics and hand hygiene behavior at baseline are reported in Table 2 (see Table S1 in Multimedia Appendix 1 for a comparison with the sociodemographic characteristics of the Swiss population). The figures refer to the 146 participants who completed the first diary at T1. Participants’ mean age was 41 (SD 17) years, 69.2% (n=101) were women, 69.9% (n=102) had high school qualifications, 45.9% (n=67) were employed, and 24% (n=35) were living alone. Descriptive statistics in hand hygiene behavior (mean 3.05, SD 0.78; median 3.25, IQR 2.63-3.60; skewness=–1.24) suggested that hand hygiene behavior was already high at baseline and characterized by a moderate left-tailed distribution. No significant differences in social desirability were found between the intervention and the control group (M₁=4.72; M₂=4.79; F_1,189=0.44; P=.51).

Dropout analysis was performed to investigate baseline differences between participants who reached the end of the intervention phase and those who dropped out at any point during the intervention. We analyzed all 193 randomized participants and the ones who did not complete the T2 panel assessment were categorized as dropouts (n=120). The results suggested that participants who dropped out were significantly younger than retainers (M₁=39; M₂=46; F_{1, 191}=7.06; P=.009). Dropouts and retainers did not differ regarding sex (male vs female; χ²₁=1.72; P=.19), hand hygiene (F_1,190=0.12; P=.73), intention to increase hand hygiene behavior (F_1,191=2.36; P=.13), or intervention group allocation (χ²₁=2.04; P=.15).

Hand hygiene behavior at T1, T2, and follow-up is summarized in Table 3. The whole sample exhibited a nonsignificant increase in hand hygiene behavior from T1 (median 3.25, IQR 2.63-3.60) to T2 (median 3.42, IQR 2.62-3.88; W=8904; P=.06, r=0.11). This was followed by a significant decrease between T2 and the 6-month follow-up (median 3.25, IQR 2.50-3.65; W=11,655; P=.04; r=0.12). When looking at changes in hand hygiene within each experimental group, the data shows that hand hygiene between T1 and T2 remained almost stable for the active control group (W=2050; P=.77; r=0.03), while the intervention group showed a significant increase in hand hygiene after the intervention phase (W=2386; P=.02; r=0.19). At follow-up, both the active control group (W=2313; P=.35; r=0.08) and the intervention group (W=2924; P=.68; r=0.03) exhibited a nonsignificant decrease in hand hygiene compared to T1.

The between-group differences in hand hygiene behavior between T1 and T2 and T1 and follow-up are reported in Table 4. Nonparametric analysis with the Wilcoxon rank sum test showed that the change in hand hygiene behavior between T1 and T2 (W=2034; P<.04; effect size r=0.17) and between T1 and follow-up (W=2005; P<.03; effect size r=0.18) significantly differed between the groups, indicating better hand hygiene in the intervention group.

Behavioral determinants values at T1, T2, and follow-up are summarized in Table 3, while results from the paired Wilcoxon tests between T1 and T2 and T1 and follow-up are reported in Table S5 in Multimedia Appendix 1. Shapiro-Wilk normality test showed that almost all the behavioral determinants were not normally distributed at all assessment points, with the exception of outcome expectancies at T2 and follow-up (Table 3). The variables that increased the most in the whole sample between T1 and T2 were coping planning (W=1474; P<.001) and action control (W=611; P<.001), followed by action planning (W=1338; P=.006) and self-efficacy (W=2115; P=.01). The change in behavioral determinants between T1 and follow-up was less pronounced and sometimes negative (ie, intention, self-efficacy, attitudes, outcome expectancies, and risk perception). The between-group differences in the targeted behavioral determinants between T1 and T2 and T1 and follow-up are reported in Table 4. Results showed that the change in coping planning between T1 and T2 (W=3840; P=.03; effect size r=0.16) significantly differed between the intervention groups.

All the hypotheses were also tested without applying any missing value imputation algorithm. Results are available in Tables S6 and S7 in Multimedia Appendix 1 and confirm most of the findings. Additionally, Wilcoxon tests without missing values imputation showed between-group differences in coping planning between T1 and T2, and in risk perception between T1 and follow-up in favor of the intervention group.

As part of a MOST to develop, optimize, and test a smartphone-based hand hygiene intervention during the COVID-19 pandemic, this RCT aimed at evaluating the efficacy of the Soapp+ app intervention against an active control group. Results showed that hand hygiene can be increased short and long term using the Soapp+ app intervention compared to active controls toward the end of the COVID-19 pandemic. Specifically, we observed small but stable effect sizes (r=0.17 between pre-post intervention and r=0.18 between preintervention and follow-up) for an increase in hand hygiene among the intervention group compared to the control group. The results of this study contribute to the notion that simultaneously targeting both reflective and automatic processes is an effective strategy for promoting and sustaining behavior change, even in challenging contexts such as a pandemic.

Discussing the clinical relevance of the effect size is challenging due to the limited evidence available regarding the necessary target effect size for a hand hygiene intervention to prevent the acquisition or transmission of COVID-19 [37]. However, we can note that the effect sizes we observed were smaller than the average effect size typically reported in previous interventions, primarily RCTs, aimed at promoting hand hygiene to mitigate the transmission of respiratory viruses (Cohen d=0.62) [4]. One plausible reason for this difference is that the study’s target population consists of individuals who are already inclined toward hand hygiene behavior. This is evident from the fact that the frequency of hand hygiene in key situations was already relatively high in our sample at baseline, with median values of 3.25 and 3.23 out of 4 for the intervention and active control group, respectively, where a value of 3 indicates that hand hygiene was performed “often.” In spite of such a ceiling effect, participants in the intervention group still managed to increase the frequency of correct hand hygiene compared to those in the active control group.

Additionally, even though between-group differences in hand hygiene were small, we saw a trend that they resulted in fewer self-reported flu-like symptoms at postintervention and follow-up for the intervention group (Table S8 in Multimedia Appendix 1). Although this difference was not statistically significant due to the small sample size and claims about the clinical significance of the intervention cannot be made, it does indicate a tendency toward an intervention effect on flu-like symptoms. Finally, it should be noted that digital interventions like Soapp+ offer the advantage of scalability and the potential to reach a broader segment of the population. This is especially relevant in a pandemic context, where the diffusion of the virus’s transmission grows exponentially, and even small changes in individual behavior, enacted on a larger scale, have the potential to significantly contribute to countering the pandemic’s spread at the community level.

Regarding our second hypothesis, results evidenced significant group differences pre-post intervention in coping planning in favor of the intervention group (Table 4). The fact that coping planning was the only behavioral determinant associated with group differences pre-post intervention aligns well with the intervention content targeting coping planning skills (ie, problem-solving) delivered toward the end of the intervention and close to the postintervention assessment (Multimedia Appendix 1). A possible explanation is that coping planning was a determinant with a low baseline value. Therefore, the intervention content might have played a significant role on this variable as there were more margins for improvement compared to the other targeted determinants.

To the best of our knowledge (based on a search on ClinicalTrials.gov with the following key terms: condition or disease “COVID 19,” age “Adult (18-64),” study type “Interventional,” outcome measure “Hand hygiene,” this study is one of the few registered RCTs testing the effectiveness of a behavior change technique targeting hand hygiene in the general population during the COVID-19 pandemic. Additionally, this RCT study is the third of a series where we have used the MOST approach for intervention development. This feature of our project stands as a significant strength, given that the MOST places strong emphasis on a rigorous, evidence-based approach to developing, refining, and evaluating interventions. Moreover, the mixed but overall encouraging evidence for the efficacy of Soapp+ points to the potential value of this app as an effective tool to address the next pandemic. This aspect is of primary relevance, as before the pandemic, there was only limited or no contextualized knowledge about how to effectively promote hand hygiene during an outbreak [3,4]. In case of future epidemics, Soapp+ is a contextualized intervention that is readily available to be tested at a larger scale.

This study is not without limitations. First, the achieved sample size (N=143) was smaller than planned. Therefore, the probability of detecting a true effect of the intervention was below standard (ie, .80) and some small effect sizes might have been missed. The decision to stop the recruitment even though the target sample size was not reached was due to the project timeline [7], but also considering that the pandemic situation was developing toward better scenarios to the point where the World Health Organization declared the end of the global health emergency [20]. Based on the found effect size, future efficacy trials should be powered to detect small effects.

A second limitation is related to the dropouts between randomization and the first diary assessment. Indeed, the first diary was scheduled for the day after the randomization, and some participants who had been randomized (n=47) did not complete it. Other participants dropped out through the course of the study without filling out the assessment at T2 and follow-up. A possible explanation for the attrition might be due to the longitudinal study design with multiple diaries and other daily tasks that might have generated some fatigue. To address potential differences between dropouts and retainers, we conducted a dropout analysis, indicating a higher likelihood of younger participants dropping out. Thus, when extending these results to young adults, caution is advised.

Another limitation is the high prevalence of women in the trial (69%), resulting in a sex ratio that does not mirror the target population. This ratio aligns with the sample characteristics of the optimization trial [8] and other COVID-19 hand hygiene research [38]. This imbalance can be explained in light of empirical evidence suggesting that women were more worried about the COVID-19 pandemic and keener to adopt hand hygiene behavior [39]. Finally, the primary outcome of the study was self-reported, which can be biased by social desirability and recall issues [21]. The latter were minimized using the electronic hand hygiene diary. Nevertheless, social desirability might still have played a role.

A key practical implication is that the Soapp+ app, a behavior change technique contextualized to a pandemic, is now available and can be readily offered to the general public at the emergence of a future infectious disease outbreak (eg, via public health institutions). However, since the app’s efficacy has not yet been robustly demonstrated due to sample size, we recommend conducting a larger-scale RCT in a future outbreak. It is also important to note that testing behavior change techniques like Soapp+ during a pandemic remains challenging due to the constantly evolving context, including shifts in the pandemic trajectory, public health policies, government regulations, and medical advancements such as vaccine availability. These multifaceted and evolving factors likely contributed to the observed temporal variations in the adoption of hand hygiene behaviors, with implications for the evaluation of behavior change techniques [33,38]. For this reason, we believe that further testing of Soapp+ could extend beyond pandemic contexts and be conducted in more feasible settings, provided that the behavior change techniques specifically referring to the COVID-19 pandemic (eg, the ones targeting risk perception and outcome expectancies; Table S3 in Multimedia Appendix 1) are adapted to the specific characteristics of the new context. Upon confirmation of the current positive effect of Soapp+ on hand hygiene, this further testing beyond the pandemic context would also pave the way for a broader application of Soapp+, particularly in settings where there is a demand for hand hygiene interventions (eg, health care settings, during regular flu seasons, and to prevent the transmission of other viruses).

A methodological lesson learned regards the suitability of the MOST as a reference framework for intervention development in a pandemic context. The sequential decision-making process unique to the MOST may be too rigid for dealing with a continuously changing context. Indeed, the contextual conditions under which the intervention is developed may differ from those in which the intervention is later optimized and evaluated. Therefore, we foresee a validity risk when using the results from one phase to inform the implementation of the upcoming phase. Additionally, the pandemic context has amplified the already known challenges in completing the full MOST cycle—preparation, optimization, and evaluation—within the project timeline [40], as evidenced by the slow recruitment pace and the high dropout rates we saw in this study.

One strategy to enhance the feasibility of RCTs in future pandemics is to join forces and contribute to large-scale multicenter trials [41]. This approach was successfully adopted by observational studies investigating protective behaviors during the COVID-19 pandemic [38,42] but was less commonly used in behavior change techniques for protective behaviors, which saw fewer collaborative efforts [43]. Another strategy is represented by adaptive clinical trials [44], whose adaptive design aims to decrease the time needed to complete a trial and increase the likelihood of detecting intervention effects. Specifically, they allow for interim analysis, which is used to enhance the efficiency of the trials by adapting the original study design. Examples include response-adaptive randomization (ie, decreasing the allocation ratio to less promising arms), sample size reassessment (ie, redefining the target sample size), or seamless trials (ie, discontinuing the most unpromising arms of the trial).

This study described the evaluation phase of Soapp+, a smartphone app to promote hand hygiene in the context of the COVID-19 pandemic. We leveraged digital technology and the MOST approach to address the call raised by public health experts for developing evidence-based behavior change techniques that are built to be used in a pandemic context [3]. Our findings provided tentative support for the efficacy of Soapp+, and for targeting reflective and automatic behavior change processes more broadly, in promoting hand hygiene during an ongoing pandemic. Beyond our individual results, the trial has provided important learnings for conducting rigorous trials of contextualized behavior change techniques during a pandemic, which can be further developed in the next outbreak.