This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Population cohort studies are useful to study infectious diseases episodes not attended by health care services, but conventional paper diaries and questionnaires to capture cases are prone to noncompliance and recall bias. Use of smart technology in this setting may improve case finding.
The objective of our study was to validate an interactive mobile app for monitoring occurrence of acute infectious diseases episodes in individuals, independent of health care seeking, using acute otitis media (AOM) symptom episodes in infants as a case study. We were interested in determining participant compliance and app performance in detecting and ascertaining (parent-reported) AOM symptom episodes with this novel tool compared with traditional methods used for monitoring study participants.
We tested the InfectieApp research app to detect AOM symptom episodes. In 2013, we followed 155 children aged 0 to 3 years for 4 months. Parents recorded the presence of AOM symptoms in a paper diary for 4 consecutive months and completed additional disease questionnaires when AOM symptoms were present. In 2015 in a similar cohort of 69 children, parents used an AOM diary and questionnaire app instead.
During conventional and app-based recording, 93.13% (17,244/18,516) and 94.56% (7438/7866) of symptom diaries were returned, respectively, and at least one symptom was recorded for 32.50% (n=5606) and 43.99% (n=3272) of diary days (
The use of the study’s smart diary app improved AOM case finding and disease questionnaire completeness. For common infectious diseases that often remain undetected by health care services, use of this technology can substantially improve the accurateness of disease burden estimates.
A key issue in many prospective infectious disease epidemiological studies, both interventional and observational, is completeness of detecting disease events of interest among study participants. This is particularly true for events that cannot be comprehensively and reliably detected through health care-based research, such as self-limiting respiratory or gastrointestinal infections. For instance, in the Netherlands, it is estimated that health care encounters are not involved for up to 50% of acute otitis media (AOM) episodes in young children. AOM is therefore notoriously underdetected when health care contacts alone are relied on [
Research on mobile apps for monitoring or promoting patient health is expanding rapidly, forming an entire new area within epidemiology. Most research focuses on evaluations of mHealth apps as an intervention in which apps are used to improve health [
We performed a proof-of-concept validation study of an interactive mobile app for monitoring occurrence of acute infectious diseases episodes in individuals, independent of health care seeking, using AOM symptom episodes in infants as a case study. Our aim was to determine participant compliance and app performance in detecting and ascertaining (parent-reported) AOM symptom episodes with this novel tool compared with traditional methods used for monitoring study participants.
We compared use of traditional survey methods (paper diary sheets) with use of a smartphone diary app for the purpose of prospectively detecting AOM symptom episodes and measuring their disease burden in infants. We made comparisons by applying the different methods consecutively over 2 periods (in 2013 and 2015) of a nested AOM study within a larger ongoing birth cohort study.
Both the first (2013) and second (2015) AOM study period were (partly) nested within the Wheezing and Illnesses Study Leidsche Rijn (WHISTLER) birth cohort that recruited healthy, term neonates between 2001 and 2012 to study perinatal and infant risk factors for wheezing illness. The study design and rationale of WHISTLER are described in detail elsewhere [
In January 2013, we invited by mail 300 participating WHISTLER parents (randomly selected out of 594 parents) with children aged 0 to 3 years for additional participation in our nested AOM study. According to the WHISTLER traditional method, parents reported respiratory symptoms and answered the monthly questionnaire using the paper diary sheets. For children older than 1 year, we requested parents to restart recording during 4 consecutive months (February to May 2013), as symptoms were recorded only in the first year of life according to the WHISTLER protocol. In addition to the daily recording, parents were asked to contact the study team within 24 hours by email, text message, or telephone call when they recorded a combination of symptoms suggestive of an AOM symptom episode. For this, parents received detailed instructions upon enrollment explaining which (combination of) symptoms was suggestive of an AOM symptom episode and should prompt notification. Researchers verified these symptoms over the telephone. Subsequently, we asked parents to complete paper versions of a validated AOM severity score (AOM-SOS) during 7 consecutive days and an additional disease questionnaire on day 7 [
We invited 404 parents of 0- to 3-year-old children by mail to participate in the (nested) AOM study. As WHISTLER completed recruitment in January 2013, no infants under 1 year of age were participating in WHISTLER in January 2015. Thus, we invited 91 WHISTLER age-eligible (ie, <3 years) participants and an additional 313 non-WHISTLER participants aged between 3 months and 2 years who also lived in the Leidsche Rijn district. Instead of using the paper diary sheet, all participating parents were now instructed to use a mobile device diary app, the InfectieApp, which we developed for this study, during 4 consecutive months (February to May 2015).
During each study period, we used identical criteria for occurrence of an AOM symptom episode: a combination of fever and either otalgia or otorrhea on the same day. This combination of symptoms had to be actively reported by the parents in the 2013 pilot study, while this combination was automatically recognized and reported using the InfectieApp in the 2015 pilot study.
Each period of the AOM study received separate approval by the medical ethics committee of the University Medical Centre Utrecht. Written informed consent was given by all participating parents.
Paper diary sheet used in the 2013 study period. WHISTLER: Wheezing and Illnesses Study Leidsche Rijn.
The InfectieApp software app was custom made in 2014 by University Medical Centre Utrecht in collaboration with VitalHealth Solutions, a company specializing in eHealth, based in Uddel, the Netherlands, and was compatible with iOS, Android, and Windows Mobile operating systems. The InfectieApp was developed for use by study participants to self-report 3 types of data: (1) symptom diary data (
The onset and ending of an AOM symptom episode was detected based on diary entries, using built-in algorithms: fever together with either earache or otorrhea occurring on the same day marked the onset of an episode. An episode ended when fever was not recorded for 7 consecutive days. Detection of a new-onset AOM episode triggered additional app content: participants received an app message explaining that AOM symptoms were detected and that additional questionnaires would follow in the coming days. For ascertainment of the AOM symptom episodes, the AOM-SOS scoring list and disease questionnaire appeared in the app questionnaire menu. The message also contained a link to a Dutch independent, professional, patient website on AOM where parents could read general AOM medical information [
To encourage participant compliance with diary recording, daily reminders at 8 PM appeared as push notifications on the smartphone. A diary that had not been filled in remained accessible to the participant up to 7 days after the diary date. We could also decide to contact the household by telephone.
The InfectieApp was password protected at first log-in; thereafter, the participant could use a 4-digit code to enter the app. When the app was used offline, data were stored encrypted in the InfectieApp. The recorded data were sent to the server via a secure connection (hypertext transfer protocol secure, HTTPS), meaning that the data were sent encrypted to the server. When the app was online, the recorded data were sent immediately to the server, providing us with the opportunity to monitor the participants in real time both for the occurrence of AOM symptom episodes and for compliance with the questionnaires.
Screenshots of the diary app (InfectieApp) used in the 2015 study period. (A) symptom diary; (B) home screen.
A security access layer determined which actions a user could perform, meaning that, depending on the rights assigned to the user, the user could or could not perform certain actions. This server was hosted by VitalHealth Solutions. The research team could access the study data stored on the server using a Web interface. Each researcher had access to the decrypted data using their personal username and passwords.
For the primary outcome, we compared the proportion of AOM symptom episodes ascertained by complete disease questionnaires between both study periods. Next, we compared the number of diaries completed and the number of monthly questionnaires filled in. For the 2015 study period, we compared the difference in number of diaries and monthly questionnaires completed between the WHISTLER and non-WHISTLER participants.
In the 2015 study period, we could assess the number of diaries completed more reliably, because a confirmation of absence of symptoms was required (symptom checkbox “None of the below”;
We estimated the incidence rate of AOM symptom episodes as the number of AOM symptom episodes per 1000 child-years and compared the 2 study periods by using OpenEpi (Open Source Epidemiologic Statistics for Public Health, version 3.01 [
The supplementary analysis included an assessment of characteristics of the ascertained AOM symptom episodes in both periods. We compared AOM-SOSs, as well as other characteristics, including health care use, medication, and parental work absenteeism derived from the disease questionnaire.
Comparisons were made using chi-square and independent-sample
Of the 300 invited WHISTLER participants, 155 (51.7%) participated during the first study period (2013) and returned at least one paper diary sheet. For the second study period (2015), 69 (17.1%) of 404 invited parents participated and completed at least one monthly app questionnaire. Of these, 36 (52%) were former WHISTLER participants (
Flowcharts of the study population. WHISTLER: Wheezing and Illnesses Study Leidsche Rijn.
Baseline characteristics of the 2013 and 2015 study populations.
Characteristics | Study | ||||||
2013 paper diary (n=155) | 2015 diary app (n=69) | ||||||
Age (years), mean (SD) | 1.50 (0.72) | 1.76 (0.78) | .02 | ||||
Male sex, n (%) | 69 (45) | 32 (46) | .88 | ||||
Participant with at least 1 sibling, n (%) | 93 (60) | 29 (54) | .43 | ||||
Daycare visit during study monthsa, n (%) | 138 (89) | 55 (80) | .09 | ||||
Former WHISTLERb study participant, n (%) | 155 (100) | 36 (52) | <.001 | ||||
≥1 AOMc symptom episode, n (%) | 24 (16) | 13 (19) | .56 | ||||
Age (years), mean (SD) | 36.2 (3.95) | 35.7 (3.75) | .38 | ||||
High level of educationd, n (%) | 131 (89) | 60 (92) | .47 |
aMinimum of 1 month of daycare during study period.
bWHISTLER: Wheezing and Illnesses Study Leidsche Rijn.
cAOM: acute otitis media.
dDefined as 1 or both parents having a high vocational or university degree.
In 29 diary sheets of 24 different children, the criteria for an AOM symptom episode (fever in combination with otalgia or otorrhea) were met in the 2013 study period, and 18 (62%) of these episodes were actively reported to the study team by the parents. For 17 (59%) episodes, an AOM-SOS scale was completed for 7 days by the parents, and 15 (52%) disease questionnaires were completed. In the 2015 study period, the app automatically detected 18 AOM symptom episodes in 13 different children. For all (100%) of these episodes, the parents completed the questionnaire about health care use and family impact, and 7 days of the AOM-SOS scale. Ascertainment of AOM symptom episodes (
The 2013 study period contained a total of 18,516 observation days. Data were received for 17,244 days (93.13%). In 2015, data were recorded by the app for 7438 of the 7866 observation days (94.56%,
During the 2013 study period, of the 617 monthly questionnaires that could have been completed, 575 (93.2%) were returned to the study team. In the 2015 study period, 299 of 329 (90.9%) monthly questionnaires were completed (
During the 2015 study period, for the former WHISTLER participants, data were retrieved for 4090 of the 4271 observation days (95.8%), while for the non-WHISTLER participants, data were retrieved for 3348 of the 3595 observation days (93.1%) (
The incidence of AOM symptom episodes was 605 per 1000 child-years in 2013 and 835 per 1000 child-years in 2015 (
Acute otitis media (AOM) incidence and participant compliance with study procedures.
Questionnaire results | Study period | |||
2013 | 2015 | |||
AOM symptom episodes, n | 29 | 18 | .003 | |
AOM incidence/1000 child-years | 605 | 835 | .29 | |
AOM-SOSa questionnaires completed, n (%) | 17 (59) | 18 (100) | .003 | |
Disease questionnaire completed, n (%) | 15 (52) | 18 (100) | .001 | |
Total days for which data received, n/N (%) | 17,244/18,516 (93) | 7438/7866 (95) | <.001 | |
Total days with ≥1 symptom reported in diary, n (%) | 5605 (33) | 3272 (44) | <.001 | |
Monthly questionnaires completed, n/N (%) | 575/617 (93) | 299/329 (91) | .20 |
aAOM-SOS: acute otitis media severity score.
bThe degree of compliance was compared between all participants of the 2013 and 2015 study periods (n=155 vs n=69).
Characteristics of parent-reported acute otitis media symptom episodes.
Characteristics | Study period | |
2013 | 2015 | |
Episodes with otalgia, n/N (%) | 26/29 (90) | 17/18 (94) |
Episodes with otorrhea, n/N (%) | 11/29 (38) | 6/18 (33) |
Number of days with fever, median (range) | 3.0 (1-11) | 2.0 (1-5) |
Episodes for which parents stayed home, n/N (%) | 6/15 (40) | 9/18 (50) |
Episodes when parents worried regularly to a lot, n/N (%) | 7/15 (47) | 9/18 (50) |
Episodes for which antibiotics were prescribed, n/N (%) | 5/15 (33) | 6/18 (33) |
General practitioner visits, n/N (%) | 9/15 (60) | 8/18 (44) |
Highest AOM-SOSa, mean (SD) | 8.6 (3.0) | 9.9 (3.4) |
aAOM-SOS: acute otitis media severity score. Highest possible AOM-SOS is 14 for each day. This score consists of 7 discrete items: tugging of ears, crying, irritability, difficulty in sleeping, diminished activity, diminished appetite, and fever. Parents were asked to rate these symptoms daily during 7 days following symptom onset in comparison with the child’s usual state, as “none,” “a little,” or “a lot,” with corresponding scores of 0, 1, and 2. Higher scores indicated more severe symptoms. For this study the AOM-SOS scale was translated into Dutch [
This study evaluated the use of a symptom diary app to detect the occurrence of parent-reported AOM in comparison with conventional (paper) survey methods accompanied by written instructions. The results of this study showed that improved case finding and completeness of disease burden information can be achieved by using an interactive app. Moreover, participants stayed well engaged with app procedures, resulting in 95% completeness of diary data.
Epidemiological research on common infectious diseases often struggles with underdetection of disease events [
The field of mobile apps health research (mHealth) is rapidly expanding [
The increasing number of smartphone users, the fact that most smartphone owners have their smartphone on or near them most of the day, and the computer features of smartphones make them attractive tools for health care and research [
One possible threat when using apps for epidemiological research is the possibility of introducing selection bias because of the requirement of smartphone ownership. In the Netherlands the use of smartphones is widespread and steadily increasing, especially in the age group of young parents, where it now reaches almost 90% [
In conclusion, our results indicate that intensive follow-up of study participants by means of an interactive app has the potential to improve the data quality of infectious diseases occurrence in populations, especially for health events that are difficult to capture by health care-based research. Our findings could have important implications for design and execution of research, both observational and interventional, involving population disease burden quantification, especially when conducted in populations with a high percentage of smartphone users, because digitization will continue and, over time, paper questionnaires may become less accepted.
acute otitis media
severity score
The authors thank all parents and children who participated in the WHISTLER study and in our AOM studies, and all coworkers who took part in the measurements and data management.
The development of the mobile app was supported by an Innovation Grant provided by Pfizer.
None declared.