During the entire study period, 10,002,512 fever-related health data were collected. The number of total records decreased by 18.5%, whereas the number of children whose data were submitted decreased by 44.2% from the development period to the validation period (Table 1).

During the development period, 149,329 people used the service at least once, and 115,674 (77.5%) of them used it for more than 2 separate days during the 1-year development period. From the 149,329 users, fever-related information was recorded from 209,270 children, and 49.2% (103,083) of them were female. This was 7.7% of the total population in that age group in South Korea [22]. They submitted 41,578 diagnoses, including 2965 influenza reports. The minimum and maximum values of F (t) were 0 and 0.495, respectively, and the range of P (t) was from 0.0037 to 0.0862 in the same period. The influenza activity during the study period is visualized geographically on a web page titled Fever Coach Flu Visualization [23]. The video record of this visualization can be found in Multimedia Appendix 1.

During the validation period, fever-related information of 116,804 children was recorded, and 26,848 diagnoses, including 5800 influenza reports, were collected. F (t) and P (t) ranged between 0 to 0.495 and 0.0023 to 0.0928, respectively. We classified all diagnosis records into 3 categories: influenza, pharyngitis/common cold, and herpangina/stomatitis. The change of diagnosis and app user activity over time is shown in Figure 2.

The prediction model derived from the linear regression between ILI visits [P (t)] for children aged 1 year through 6 years and influenza diagnosis proportion submitted to Fever Coach [F (t)] came out with its slope (β₁) as 0.1316, intercept (β₀) as 0.0063, and R² as 0.930 (Figure 3). P (t) is the reference value from KCDC. Spearman ρ was 0.895 (P<.001) and RMSE was 0.0083 between P (t) and predicted value in the validation period (the maximum value of P (t) was 0.0928 and minimum value was 0.0023). Figure 4 shows a graphical plot of P (t) and predicted values over the study period as well as the error of the prediction model. From the all-age model that was developed between F (t) and proportion of ILI visits in all age ranges, the 𝜌 was 0.924 (P<.001) and the RMSE was 0.0135. The predicted value exceeded the epidemic threshold on November 21, 2017, by this model. KCDC announced the influenza epidemic alert on December 1, 2017.

The Fever Coach app successfully collected data from 7.7% of the target population over a 1-year period, which is remarkably higher than participation rates of previously reported Web- or mobile-based participatory surveillance systems that reported participation rates between 0.02% [24] and 0.13% [25]. The influenza activity monitoring model based on voluntary data submitted through the app showed a close association with the large-scale government-reported influenza activity. The participatory data are updated daily, rather than weekly, and the prediction model based on these data led to more sensitive detection of an influenza epidemic compared with the epidemic alert from the KCDC.

The result of this study sheds light on the potential of mobile apps in collecting data for public health surveillance and highlights the importance of user benefits for voluntary data submission. Previous participatory systems for influenza surveillance relied on the social responsibility of individuals for recruiting and retention of participants, which resulted in low participation rate and biased representation of target populations.

There are several other Web- or mobile phone-based survey platforms for participatory surveillance of influenza and other infectious diseases [17,26,24,27]. Some only collect information regarding symptoms [24] and health-related behavior, whereas others also ask outcomes of medical services [17]. These platforms rely on volunteers by advertising the benefits of self-reported data in disease surveillance and do not incentivize individual participants. Therefore, recruitment and retention of users have been identified as the main challenges for them [24]. They used social media, national television, institutional networks, professional societies, dissemination events, and word of mouth to recruit participants. Using this strategy, Flu Near You (FNY) in the United States recruited 61,000 participants during its second influenza season (2012-2013), which was about 24 per 100,000 population of age 13 years and older. Influenzanet, which was established in 5 European countries in 2009, reported 36,192 participants in 10 countries in Europe during the 2015-2016 influenza season, with a participation rate of 13 individuals per 100,000 population.

In South Korea, online communities are very active and Koreans seek information and communicate with their peers about studying, career, hobbies, and parenting through such communities as well as blogs [28]. Fever Coach used these communities to promote the app by providing free coffee coupons for randomly selected users who downloaded the app and shared their experience and by providing free online health consultation. After reaching 100,000 downloads in May 2016, there have not been active marketing activities, but parents who used the app have voluntarily spread the word on the internet and in their local communities. In February 2017, Fever Coach had been downloaded 300,000 times and over 200,000 users submitted their data at least once. Using this strategy, the participation rate was about 7700 per 100,000 population aged 1 year to 6 years, one of the high-risk groups for influenza infection. The Fever Coach app was primarily designed to enable parents to take care of their kids when they have any kind of fever, but it was able to collect data that can be used specifically for influenza surveillance. We believe that by providing immediate benefit and targeting the vulnerable population, recruiting and retention of participatory surveillance can be much more effective than by relying on volunteers. This strategy can be applied to target other high-risk groups such as pregnant women, elderly people, or immunocompromised individuals.

As users of Fever Coach submit data on behalf of their children, the only information we collected about the users themselves was email address and device location. However, as the service is designed for caregivers of children younger than 5 years, we can assume that the users’ age range is approximately between late 20s and early 40s, which is younger than the user age ranges reported by other systems. As the app is only accessible via a smartphone, every user is assumed to be a smartphone user. The rate of smartphone ownership in South Korea is reported to be as high as 89.5% [29,30], and the smartphone usage rate is above 99% in the age group between 20 and 50 years [29]. Given the very high rate of smartphone penetration, smartphone users represent the general population of this age group in South Korea. Although there are no data about the gender of the users, it is expected that more females than males use the app based on the higher dependency on mothers for childcare in South Korea. This aligns with other studies that reported female participants being the majority. In a study that analyzed data from FNY, participants reporting for household members showed higher participation rate. This finding can partially explain the successful user recruitment and retention of Fever Coach, which is mostly used by parents for their children. As the service itself is targeted for children, the data collected from the app can only represent the pediatric population. However, our model from these data was able to predict influenza activity in the total population as well as in the pediatric population. This was because influenza activity in children is very closely related to that in adolescents and adults, but it may not be applicable for monitoring other diseases. Another noteworthy finding was the close prediction of influenza activity using the prediction model developed from the previous year despite several changes in the user population. Compared with the development period, the number of users decreased and the median number of data submission per child increased. This finding suggests that retention of active users can make up for the loss of inactive users for participatory surveillance.

There are many limitations of this study that need to be addressed. First, most of the data submitted to Fever Coach came from children with febrile illness, and users generally do not use the app when their children are well. Therefore, increased app usage can either indicate an outbreak of an infectious disease or a surge of interest in the app itself following national media coverage. To overcome this limitation, we only used diagnosis reports to estimate ILI visits, contrary to other participatory surveillance systems that ask symptoms of users and use the data for syndromic surveillance. Although this strategy led to a good approximation of influenza activity, it introduces limited generalization. In most regions of South Korea, with physician density of 2.3 per 1000 population [31], primary care is considered affordable and readily accessible. However, in areas where health care is not as accessible, use of physician-made diagnosis may result in a bias toward a small group of people who have better access to health care than the general population. To overcome this second limitation, an additional study to develop more complex prediction models that include symptom and demographic data is in progress.

The Fever Coach app successfully collected data from 7.73% (207,699/2,686,580) of the target population by providing care instruction for febrile children. These data were used to develop a model that accurately estimated influenza activity measured by the central government agency using reports from sentinel facilities in the national surveillance network.