In June 2021, we systematically searched 2 major app stores: the Apple App Store and Android Google Play Store. In the 2 app stores, search terms included “COVID-19,” “Coronavirus,” “COVID-19 and Symptoms,” “Coronavirus and Symptoms,” “COVID-19 and Symptom Monitoring,” and “Coronavirus and Symptom Monitoring” to identify relevant apps. Following subsequent searches of “COVID-19” and “Coronavirus,” the authors found that the results were the same. The terms “COVID-19,” “Coronavirus,” “COVID-19 and Symptoms,” and “COVID-19 and Symptom Monitoring” were then collapsed to streamline the search.

All apps underwent three rounds of screening. During the first round, the title and screenshots of the apps were used to exclude those that were exclusively for contact tracing (without symptom tracking functionality), were not available in English, required an institution-specific login, were clinical guidelines for clinicians, were games, or provided general or other COVID-19–related information. During the second round of review, screenshots and descriptions of the apps were reviewed to exclude apps that were too general, only had a singular function (eg, only track and trace), or were not relevant to tracking COVID-19 symptoms. During the third round of review, the final apps were downloaded, and apps requiring institutional credentialing were excluded.

All apps were evaluated using the Mobile Application Rating Scale (MARS) [13,14], IQVIA for Healthcare Informatics [15] functionality scoring system, and specific COVID-19 symptoms according to the CDC [1,3,11] and WHO COVID-19 health topic guidelines [12]. Both the MARS and IQVIA functionality scores were used for this review. The MARS functionality score focuses on performance, ease of use, navigation, and gestural design of the app [13]. The 7 IQVIA functionality scores focus on the scope of functions, including informing, instructing, recording, displaying, guiding, reminding, and communicating information [14].

The MARS [13] is a widely used, multidimensional tool to evaluate the quality of mobile health apps and was developed based on semantic analysis and a combination of relevant literature. The MARS was used to rate app quality and includes four sections: classification, quality, satisfaction, and a modifiable app-specific section. The classification section provides descriptive information about the apps. The objective app quality section includes 19 items divided into four scales: engagement, functionality, aesthetics, and information quality. MARS items are scored using a 5-point Likert scale (1=inadequate, 2=poor, 3=acceptable, 4=good, and 5=excellent) [13]. The final MARS scores include a total mean score for the engagement, functionality, aesthetics, and information subscales. In addition, the MARS also includes a subjective quality score and an app-specific subscale that assesses perceived effect on the user’s knowledge, attitudes, and intentions to change, as well as the likelihood of changing the identified targeted behaviors. The functionality domain measures whether an app is easy to learn, encourages seamless navigation, supports a logical flow, and examines the overall app gestural design. The engagement score includes evaluation of whether the app is fun, interesting, customizable, and interactive (eg, sends alerts, messages, reminders, and feedback, and enables sharing), and is well targeted to the intended audience.

The IQVIA functionality score (formerly termed the IMS functionally score) is based on 7 functionality criteria and 4 functional subcategories as described in detail in the IQVIA Institute for Healthcare Informatics report [14]. IQVIA is not an acronym; the name represents a merger between two companies, IMS Health and Quintiles. The apps were evaluated on the 7 functionality criteria of inform, instruct, record, display, guide, remind or alert, and communicate. The record functionality criteria were further scored on 4 subcategories: collect (ability to enter or store data on individual phone), share (ability to transmit data), evaluate (ability to evaluate health data by caregiver or health care entity), and intervene (ability to alert in response to collected data or propose behavioral intervention).

The specific symptoms tracked in each app were evaluated against acute and long-term symptoms of COVID-19 as reported by the CDC [1] and the WHO [12] as of June 2021. Acute COVID-19 symptoms included fever, dry cough, tiredness, aches and pains, sore throat, diarrhea, conjunctivitis, headache, loss of taste or smell, skin rash or discoloration of fingers or toes, difficulty breathing or shortness of breath, chest pain or pressure, and loss of speech or movement. Long-COVID-19 symptoms, defined as more than 4 weeks from the initial infection [3], included tiredness or fatigue, difficulty thinking or concentrating (“brain fog”), headache, loss of smell or taste, dizziness while standing, heart palpitations, chest pain, difficulty breathing or shortness of breath, cough, joint or muscle pain, depression or anxiety, fever, and symptoms that worsen after physical or mental activities. Symptoms that are consistent with both short-term and long-term COVID-19 (eg, cough), were evaluated based on how data entry was described in the app and the current version of the app.

We created a Google data extraction form consisting of questions from (1) the MARS questionnaire [13], (2) IQVIA functionality guidelines [14], and (3) the CDC [1] and WHO [12] COVID-19 symptoms. A total of 5 reviewers independently evaluated the 4 randomly selected apps using the data extraction form to assess interrater reliability, which was acceptable (0.75-0.83). Domains with low agreement between reviewers (<0.7) were discussed until consensus was reached, apps were rereviewed, and interrater reliability was recalculated. The remaining apps were independently evaluated by 2 reviewers. Mean MARS scores for each domain, MARS total scores, and IQVIA functionality scores are reported.

The average overall MARS rating for the 20 apps was 3.89 with the functionality domain having the highest score (4.20) and the engagement score having the lowest average score (3.52). The top 4 rated apps overall were state-specific apps developed and deployed for public use in New York, Pennsylvania, Rhode Island, and New Jersey (Table 2), between July and October 2020.

Overall, the “instruct” function was the most common among all apps (n=19, 95%) (Figure 2). In many of the apps, the instruct function aided in directing users on next steps based on the symptoms that were recorded. The “Beebe COVID-19” screening app was the only app that had the ability to communicate between patients and health care providers. There were no apps that had all 11 functionalities. Another domain that the majority of the apps had was the collect function (n=18, 90%). These apps collected patients’ daily COVID-19 symptoms and collected some demographic information (n=18, 90%) including: age, name, date of birth, ethnicity, and geographical location.

The acute COVID-19 symptoms tracked in each app (Table 3), and long-term symptoms (Table 4) were evaluated against the recommendations provided by the CDC [1] and the WHO [12]. The majority of apps focused on acute COVID-19 symptoms (n=18) with the most common acute symptoms including fever and dry cough (n=18, 90%), aches and pains (n=17, 85%), difficulty breathing (n=17, 85%), tiredness, sore throat, headache, loss of taste or smell (n=16, 80%), and diarrhea (n=15, 75%).

The aim of this review was to evaluate COVID-19 mHealth apps that support tracking acute and long-term symptoms of COVID-19. The majority of the apps reviewed focused on acute symptoms, including fever and dry cough, aches and pains, difficulty breathing, tiredness, sore throat, headache, loss of taste or smell, and diarrhea. Few apps measured long-term symptoms of COVID-19. Overall, the apps scored high on quality, and the most common functionality was providing the user with instructions on what to do in response to symptoms.

Previous reviews of mHealth apps for COVID-19 were published by Alanzi [16] and Davalbhakta [17]. The review published by Alanzi [16] reviewed apps from 7 countries (Saudi Arabia, India, Singapore, Australia, Italy, United Kingdom, and the United States). These apps were reviewed on functionality (eg, purpose, services offered, and networking technologies) and effectiveness (eg, learnability, communication strategies, and design). There were no overlapping apps between the review published by Alanzi [16] and this review. The review by Davalbhakta and colleagues [17] was consistent methodologically by using the MARS to review the apps, in addition to adding more functionalities specific to COVID-19 (eg, individual tracking, contact tracing, and health care worker training). The review by Davalbhakta [17] and this review shared 6 common apps (Apollo, PatientSphere, Apple COVID-19, Bc COVID-19 Support, Check COVID, and Canada COVID-19). Other reviews did not evaluate short- and long-term symptom monitoring according to the CDC [1] or WHO COVID-19 health topic guidelines [12].

The highest quality apps were developed by individual states within the United States to support acute COVID-19 symptom tracking. Overall, the state-specific apps had the most up-to-date and accurate information, which could be attributed to collaborative efforts between local, county, or state departments of health [18-23]. The state-specific apps all listed most of the common symptoms and provided information about local COVID-19 case counts and developments in research. Strengths of the state-specific apps included long-term storage of the symptom evaluations, high-quality aesthetics, comprehensive symptom lists, and evidence-based information that was aligned with CDC guidelines [1]. While the contact tracing functions were likely most useful to in-state residents, the symptom tracking functionalities were available for anyone regardless of their home state.

At the time of app development and deployment, less was known about long-term symptoms of COVID-19 [24]. Only 2 (10%) apps specifically tracked long-term symptoms of COVID-19 (Table 4), though the symptoms of long COVID-19 are increasingly concerning [25-27]. As such, mHealth apps that track long COVID-19 symptoms could play a significant role in helping to manage them more efficiently and gather additional data about how this disease is affecting patients over a long period of time [25-27]. The authors suggest that app developers should follow updated technical and clinical guidelines from the CDC and the WHO to ensure consistency and efficacy of long-term symptom monitoring. Future research should focus on expanding these apps to support patients with long-term symptoms [3] and providing educational materials about the disease.

The largest gap in app functionality was in the communication feature, which could be further developed to allow patients to communicate their symptoms to chosen family members for supportive care or their own health care professionals. These algorithms are useful for providing basic information about what to do in response to the symptoms and quarantine guidance. However, many patients, especially those with complex care needs, require additional support and communication with their health team to manage long-term symptoms of COVID-19. There are specific legal and regulatory issues around data sharing that are likely the reasons why many apps do not actively support more data-sharing features. Additional areas of future development include expansion into pediatric populations and long-term symptom tracking integrated into mHealth apps. Lastly, these apps should be evaluated for usability and inclusivity characteristics, including non-English languages.

A limitation of this review is that it provides a snapshot of the app landscape at a specific point in time when the app stores have continued to rapidly evolve. Many of the apps are also going on and off the market based on the introduction of novel variants. To mitigate the limitation of subjectivity of reviewing apps, we applied a rigorous multistep methodology including using the MARS [13,28,29] and IQVIA for Healthcare Informatics [15] functionality scoring system and mapping all the apps to the CDC and WHO COVID-19 health topic guidelines for symptom monitoring.

In general, mHealth apps for tracking COVID-19 symptoms offer a potential solution to help people identify and track virus symptoms. Overall, the mHealth apps had high quality, considering the expedited needs during the COVID-19 pandemic. Future apps should also incorporate monitoring long-term symptoms of COVID-19 and include a communication feature that could allow patients to communicate their symptoms to specific caregivers or their own health care team.