This paper is in the following e-collection/theme issue:

Theme Issue: Apps for COVID-19 (#Apps4Covid) (133) JMIR Theme Issue: COVID-19 Special Issue (2388) Digital Contact Tracing, Digital Proximity Tracing, Precision Public Health (138) Hygiene and Infection Prevention (297) Infectious Diseases (non-STD/STI) (1478) Outbreak and Pandemic Preparedness and Management (1673)

Published on in Vol 8, No 10 (2020): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/23148, first published .
Performance of Digital Contact Tracing Tools for COVID-19 Response in Singapore: Cross-Sectional Study

Performance of Digital Contact Tracing Tools for COVID-19 Response in Singapore: Cross-Sectional Study

Performance of Digital Contact Tracing Tools for COVID-19 Response in Singapore: Cross-Sectional Study

Authors of this article:

Zhilian Huang1 Author Orcid Image ;   Huiling Guo1 Author Orcid Image ;   Yee-Mun Lee2, 3 Author Orcid Image ;   Eu Chin Ho4 Author Orcid Image ;   Hou Ang5 Author Orcid Image ;   Angela Chow1, 3 Author Orcid Image
Article Authors Cited by (57) Tweetations (10) Metrics

    Original Paper

    1Department of Clinical Epidemiology, Office of Clinical Epidemiology, Analytics, and Knowledge, Tan Tock Seng Hospital, Singapore, Singapore

    2Department of Urology, Tan Tock Seng Hospital, Singapore, Singapore

    3Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore

    4Department of Ear, Nose, and Throat, Tan Tock Seng Hospital, Singapore, Singapore

    5Department of Emergency Medicine, Tan Tock Seng Hospital, Singapore, Singapore

    Corresponding Author:

    Angela Chow, MBBS, MPH, MS, PhD

    Department of Clinical Epidemiology

    Office of Clinical Epidemiology, Analytics, and Knowledge

    Tan Tock Seng Hospital

    11 Jalan Tan Tock Seng

    Singapore, 308433

    Singapore

    Phone: 65 63577477

    Email: angela_chow@ttsh.com.sg


    Background: Effective contact tracing is labor intensive and time sensitive during the COVID-19 pandemic, but also essential in the absence of effective treatment and vaccines. Singapore launched the first Bluetooth-based contact tracing app—TraceTogether—in March 2020 to augment Singapore’s contact tracing capabilities.

    Objective: This study aims to compare the performance of the contact tracing app—TraceTogether—with that of a wearable tag-based real-time locating system (RTLS) and to validate them against the electronic medical records at the National Centre for Infectious Diseases (NCID), the national referral center for COVID-19 screening.

    Methods: All patients and physicians in the NCID screening center were issued RTLS tags (CADI Scientific) for contact tracing. In total, 18 physicians were deployed to the NCID screening center from May 10 to May 20, 2020. The physicians activated the TraceTogether app (version 1.6; GovTech) on their smartphones during shifts and urged their patients to use the app. We compared patient contacts identified by TraceTogether and those identified by RTLS tags within the NCID vicinity during physicians’ 10-day posting. We also validated both digital contact tracing tools by verifying the physician-patient contacts with the electronic medical records of 156 patients who attended the NCID screening center over a 24-hour time frame within the study period.

    Results: RTLS tags had a high sensitivity of 95.3% for detecting patient contacts identified either by the system or TraceTogether while TraceTogether had an overall sensitivity of 6.5% and performed significantly better on Android phones than iPhones (Android: 9.7%, iPhone: 2.7%; P<.001). When validated against the electronic medical records, RTLS tags had a sensitivity of 96.9% and specificity of 83.1%, while TraceTogether only detected 2 patient contacts with physicians who did not attend to them.

    Conclusions: TraceTogether had a much lower sensitivity than RTLS tags for identifying patient contacts in a clinical setting. Although the tag-based RTLS performed well for contact tracing in a clinical setting, its implementation in the community would be more challenging than TraceTogether. Given the uncertainty of the adoption and capabilities of contact tracing apps, policy makers should be cautioned against overreliance on such apps for contact tracing. Nonetheless, leveraging technology to augment conventional manual contact tracing is a necessary move for returning some normalcy to life during the long haul of the COVID-19 pandemic.

    JMIR Mhealth Uhealth 2020;8(10):e23148

    doi:10.2196/23148

    Keywords

    infectious disease; real-time locating systems; electronic medical records; COVID-19; contact tracing; public health 


    The fight against COVID-19 is expected to be a long haul due to its high transmissibility, despite its low virulence rates [1]. There have been predictions of potential second waves of SARS-CoV-2 infections, and several countries have experienced repeated waves of outbreaks after seemingly successful efforts of containing COVID-19 transmissions [2,3]. In the absence of effective treatment or vaccination, approaches to preventing transmissions have included social distancing, use of face masks, hygiene measures (eg, disinfection), case isolation, contact tracing, and quarantine [1]. Effective contact tracing is essential for preventing transmissions and subsequent generations of SARS-CoV-2 infections [4].

    Contact tracing is the process of identifying and subsequently acquiring information from persons potentially exposed to infectious diseases [4]. Conventional contact tracing methods include a mixture of patient interviews and contact verifications to map the social and work encounters of an infected individual. These methods enable the contact tracer to obtain a comprehensive list of persons potentially exposed to infectious individuals [5,6]. Additional measures, such as patient and visitor registration system downloads, staff roster and electronic medical record (EMR) reviews, and direct observations via closed-circuit television have been employed for contact tracing in health care settings [7]. These processes are labor intensive, yet time sensitive and crucial for stemming the spread of the infectious disease at the emergence of an outbreak [4,8,9].

    During the COVID-19 outbreak, contact tracing can be challenging in large communities with a limited pool of contact tracers [6,10]. Technology can potentially overcome this barrier by simplifying the process of data collection. Several health care institutions have benefitted from the use of a real-time locating system (RTLS) for contact tracing. Hellmich et al [11] were able to identify twice as many potential contacts of confirmed pertussis cases in an emergency department with RTLS compared with EMR review. Ho et al [12] also found that the use of location-based RTLSs to complement EMR review increased the sensitivity of detecting health care workers’ contact with COVID-19 inpatients from 47.2% to 77.8%. However, despite the positive outcomes of using RTLSs for contact tracing [11-13], such technologies have largely been confined to health care institutions due to the high costs of infrastructure setup [14,15].

    Contact tracing apps can provide a more feasible solution to the barrier of scale presented by large communities with high smartphone penetration rates. The ubiquity and relatively low developmental cost of smartphone apps allow for large-scale deployment in time-sensitive situations. Therefore, many countries have jumped on the bandwagon of contact tracing apps [16-19] to augment contact tracing capabilities in response to the COVID-19 pandemic. Likewise, Singapore launched its first Bluetooth-based contact tracing app—TraceTogether—in March 2020 [16,18]. The app is available on both the Google and Apple app stores, and approximately one-sixth of Singapore residents downloaded the app during the peak of the COVID-19 outbreak in April 2020 [20].

    Despite the potential of contact tracing apps in enhancing contact tracing efforts during the COVID-19 pandemic, this potential can only be realized when 60% of the population use the app [6]. The efficacy of large-scale technology adoption depends on the law and enforcement measures of the country, public trust in personal data protection, and users’ perceived utility of adopting the technology, along with other factors [21-23]. Prior to large-scale technology adoption, the validity of novel contact tracing methods should be assessed to optimize the time and resources used in tackling the COVID-19 pandemic. The effectiveness of digital contact tracing tools has yet to be established due to the paucity of published data in real-life outbreak settings [24]. Hence, we compared the performance of the TraceTogether app with that of a wearable tag-based RTLS and validated both against the EMRs at the National Centre for Infectious Diseases (NCID), the national referral center that managed the majority of Singapore’s hospitalized COVID-19–positive patients in May 2020.


    Setting

    This study was conducted in the COVID-19 screening center of the NCID in Singapore. The NCID was the designated hospital for managing suspected and confirmed COVID-19 cases during the COVID-19 outbreak in Singapore. Physicians from the co-located Tan Tock Seng Hospital were deployed to the NCID screening center on 10-day rotating shifts to manage the center during the outbreak. Singapore entered a 2-month partial lockdown phase from April 7 through June 1, 2020 due to a surge in COVID-19 cases in the community and foreign worker dormitories.

    The study was conducted over a 10-day physician-posting period from May 10 through May 20, 2020. At the time of the study, the NCID was managing more than 70% of Singapore’s COVID-19–positive cases. The majority of patients who attended at the screening center during the study period were residents of foreign worker dormitories.

    Study Design

    We employed a cross-sectional study design to validate the TraceTogether app and the NCID’s wearable RTLS wrist tag against EMRs. All patients and physicians in the NCID screening center were issued temporary RTLS tags for contact tracing (Figure 1). Wearing the RTLS tags was mandatory for entry into the NCID screening center (Figure 1). Physicians deployed to the NCID screening center from May 10 to May 19, 2020 were instructed to install the TraceTogether app (version 1.6), activate their smartphone’s Bluetooth function during their shifts, and urge their patients to download and activate the TraceTogether app when they medically attended to their patients. Pictorial instructions to download and activate the TraceTogether app were available in English, Tamil, Bengali, and Mandarin at every screening station (Figure 2).

    ‎
    Figure 1. Mandatory real-time locating system (RTLS) tags worn by patients and staff within the vicinity of the COVID-19 screening centre for contact tracing purposes.
    View this figure
    ‎
    Figure 2. Pictorial instructions to download and activate version 1.6 of the TraceTogether app provided at the National Centre for Infectious Diseases COVID-19 screening centre, Singapore from May 10 through May 20, 2020.
    View this figure

    Each patient was assigned to a designated screening station (arranged 2 meters apart from other stations), at which the patient was medically attended. Patients could download the app while awaiting attendance by their physicians using the complimentary Wi-Fi at the screening center. Attending physicians also encouraged patients to download and activate the TraceTogether app.

    Tag-Based RTLS

    The RTLS was fitted into the NCID building since the building’s official opening in September 2019. RTLS location exciters and wireless access points were fitted throughout the NCID building to detect signals from RTLS tags (CADI Scientific). The tag receives a signal whenever it passes a location exciter and sends a radio-frequency signal to the access points to determine the exact location of the RTLS tag. Concurrently, RTLS tags are enabled with tag-tag radio-frequency identification technology to determine close contacts of <=2 meters between staff and patients. Although the location-based radio-frequency identification system in the NCID has been validated by Ho et al [12], the performance of tag-based technology has yet to be assessed.

    TraceTogether App

    The TraceTogether app was developed by a government-linked company—GovTech—in response to the COVID-19 outbreak. The free-to-download app worked by exchanging Bluetooth signals with other nearby app users and storing the encrypted data locally in the smartphone. Users were requested to upload the data captured on their smartphones should they be confirmed with COVID-19 infection to facilitate contact tracing. Data stored on the phone were automatically deleted after three weeks [17]. Version 1.6 of the app only stored the user’s mobile phone number as identity data and collected the phone number, proximity, and duration of users’ contacts. Both the mobile phone number and anonymized random user ID (generated from account creation) were stored in the secure GovTech server.

    EMR System

    All staff providing clinical care to patients at the NCID were issued with rights to make entries in patients’ EMRs. Therefore, every patient screened at the NCID screening center had their clinical encounters recorded by the NCID EMR system. Hence, the EMR system was considered the gold standard for validating patient-physician contacts in this study.

    Participants

    Data validation was only possible when TraceTogether and RTLS data from physicians were available. Therefore, only physicians who uploaded their TraceTogether data and had working RTLS tags were included as participants in this study. By the end of the 10-day posting period, 18 physicians with working RTLS tags had uploaded the data captured from the TraceTogether app from their smartphones to the cloud system.

    Data Analyses

    We compared patient contacts identified by TraceTogether with those detected by the tag-based RTLS within the NCID screening center from 8:30 AM on May 10, 2020 through 8:30 AM on May 20, 2020 (Table 1). The mobile phone numbers of patients identified by the 18 participating physicians’ RTLS tags were compared with the mobile phone numbers captured by the physicians’ TraceTogether app. These mobile phone numbers were identified as those from the NCID patient registration (ie, from patients who attended the NCID during the study period).

    We also reviewed the EMRs of 156 patients who attended the NCID screening center from 8:30 AM on May 12, 2020 to 8:30 AM on May 13, 2020 to determine if the eight participating physicians who were on shift during that time were among the attending physician(s) of each patient. Only records that fell within each physician’s shift were considered. The mobile phone numbers of these 156 patients were compared with the mobile phone numbers of patients identified by the eight physicians’ RTLS tags and TraceTogether app, respectively, to determine whether the respective technologies had detected them. We then computed the sensitivity, specificity, positive and negative predictive values, and likelihood ratios of the two digital technologies, using EMR review as the gold standard (Table 2).

    Table 1. Comparison of the number of patient contacts identified by a tag-based real-time locating system within the vicinity of the NCID and the TraceTogether app on iPhones and Android phones.
    Phone type and physician numberRTLSaTraceTogetherbRTLS+TraceTogetherRTLS or TraceTogetherSensitivity of RTLSc (%)Sensitivity of TraceTogetherd (%)
    iPhones

    		16500651000

    		25200521000

    		320021000

    		47000701000

    		559416493.87.8

    		6930127525

    		71400141000

    		81820209010

    		95900591000

    		101400141000

    		Total3629137297.62.7e
    Android phones

    		1183128698.83.5

    		126100611000

    		131101121008.3

    		1478528594.18.2

    		151200121000

    		16801269887.818.4

    		1743204595.64.4

    		18421035581.823.6

    		Total410301445493.49.7e
    iPhone+Android Phone

    		Total772391582695.36.5

    aRTLS: real-time locating system.

    bTraceTogether-only data includes contacts that were concurrently detected by the TraceTogether app and the NCID location-based RTLS to ensure that the encounters were within NCID’s vicinity.

    cRTLS sensitivity = (number of contacts detected by RTLS + (RTLS+TraceTogether))/(number of contacts detected RTLS or TraceTogether).

    dTraceTogether sensitivity = (number of contacts detected by TraceTogether + (RTLS+TraceTogether))/(number of contacts detected by RTLS or TraceTogether).

    eP<.001 when the sensitivity of TraceTogether was compared between Android phones and iPhones.

    Table 2. Performance of the tag-based real-time locating system and TraceTogether app validated against electronic medical records.a
    Digital contact tracing toolSensitivitybSpecificitycPositive predictive valueNegative predictive valueLR+dLR−e
    Tag-based real-time locating system96.9% (31/32)83.1% (103/124)59.6% (31/52)99.0% (103/104)5.730.04
    TraceTogether app0.0% (0/32)98.4% (122/124)0.0% (0/2)79.2% (122/154)0.001.02
    Either digital contact tracing tool96.9% (31/32)81.5% (101/124)57.4% (31/54)99.0% (101/102)5.240.04

    aPhysicians 1, 4, 5, 8, 9, 11, 12, and 18 from Table 1 were included in the Table 2 analyses.

    bDenominator for sensitivity = number of patients attended by eight physicians who uploaded their TraceTogether data and had a work shift between 8:30 AM on May 12, 2020 and 8:30 AM on May 13, 2020.

    cDenominator for specificity = number of patients attended by other physicians in the same work shift between 8:30 on May 12, 2020 and 8:30 AM on May 13, 2020.

    dLR+: Positive Likelihood Ratio; LR+ = sensitivity/(1 − specificity).

    eLR−: Negative Likelihood Ratio; LR− = (1 − sensitivity)/specificity.

    Ethical Considerations

    This study was approved by the National Healthcare Group Domain Specific Review Board in Singapore.


    Table 1 shows the comparative performance of the RTLS and TraceTogether app in identifying patient contacts of individual physicians. The RTLS had a high sensitivity of 95.3% in detecting all patient contacts identified either by the RTLS system or TraceTogether app, while TraceTogether had an overall sensitivity of 6.5%. Version 1.6 of the app performed significantly better on Android phones than iPhones (Android: 9.7%, iPhone: 2.7%, P<.001).

    Table 2 shows the performance of the tag-based RTLS and TraceTogether app validated against EMRs. RTLS tags had high sensitivity (96.9%) and specificity (83.1%), as the tags could detect patient contacts other than those between patients and the participating physicians who medically attended to them. TraceTogether detected only 2 patient contacts with physicians who did not attend to them. Hence, the app had a sensitivity of 0% and specificity of 98.4% in a clinical setting. The sensitivity of identifying patient contacts increased to 96.9% when both digital contact tracing tools were used.

    The positive predictive value and negative predictive value of the RTLS were 59.6% and 99.0%, respectively, while those of TraceTogether were 0% and 79.2%, respectively. Positive and negative predictive values are influenced by the prevalence of the disease, which in this case, was the proportion of patients in contact with physicians. A higher prevalence likely leads to higher positive predictive values. The RTLS’s moderately high positive likelihood ratio of 5.73 and high negative likelihood ratio of 0.04 suggest that the RTLS is capable of ruling in and ruling out patient contacts.


    Effective and timely contact tracing is essential in slowing the spread of COVID-19 in the community. We compared the performance of a contact tracing app—TraceTogether—and NCID RTLS tags in identifying patient-physician contacts and validated both digital contact tracing tools against the EMRs at the NCID COVID-19 screening center. To our knowledge, this is the first study to assess the validity of a contact tracing app in a real-life outbreak setting (ie, the COVID-19 pandemic).

    TraceTogether had a much lower sensitivity than the tag-based RTLS in identifying patient contacts in a clinical setting. High sensitivity is preferred for digital contact tracing tools to rule out the possibility of failing to detect close contacts with COVID-19 cases. The low sensitivity of TraceTogether could be attributed to the small proportion of patients who activated the app and turned on their Bluetooth, despite physicians’ prompts and the reminder posters at the screening center. In contrast, all patients had RTLS tags attached to their wrists at registration. Hence, we were able to more accurately assess the validity of the RTLS tags in this study.

    Although the tag-based RTLS performed well for contact tracing in a clinical setting, the high setup cost would render it less feasible for a community-wide scale-up. Distributing RTLS tags and enforcing their use in the community would be much more challenging than doing so in a clinical setting. Previous RTLS studies have been mainly confined to a defined setting [11,13,25,26]. On the other hand, Bluetooth technology is low cost, available on personal digital devices, and interoperable among Bluetooth-enabled devices [27]. This flexibility is essential for facilitating widespread adoption of contact tracing tools, as users would have the convenience of selecting their preferred form of the contact tracing tool.

    A critical mass of app adoption must be achieved to increase the sensitivity and positive predictive value of TraceTogether [28]. Most people have probably not heard of the app or found it useful to download the app during the study period, as movement was restricted due to lockdown measures. The concept of contact tracing apps only actualized on a large scale after it was realized that the fight against COVID-19 was going to be a long haul. Privacy concerns regarding data storage and location tracking were likely the biggest barrier against such contact tracing apps [20,23,28]. Despite the hype surrounding contact tracing apps, health systems worldwide have not revolutionized contact tracing efforts for COVID-19. Partial regulatory enforcements, effective communication of app utility, and a good understanding of the barriers and facilitators of contact tracing app adoption are crucial for app adoption to reach a critical mass.

    Given the uncertainty of the adoption and capabilities of contact tracing apps [22,29], contact tracers and policy makers should be cautioned against the overreliance on such apps for contact tracing. An infectious disease workgroup estimated that manual contact tracing would reduce COVID-19 transmissions by 61% compared with the 44% reduction provided by app-based tracing if 53% of the population uses the app [30]. Even if a contact tracing app achieved widespread adoption and high sensitivity in detecting contacts, manual verifications would still be required to ascertain the contact before actions can be undertaken to quarantine potential exposures. In reality, a mixture of contact tracing methods is required to optimize the performance of contact tracing [23].

    There were limitations to this study. First, we were unable to enforce the usage of TraceTogether among all physicians at the screening center for a comprehensive review on its performance. We could only consider the patient contacts of physicians who uploaded their TraceTogether data to the cloud system. This limitation in a clinical setting reflects the challenges of wide-spread adoption of app-based contact tracing tools in the community. However, despite the difficulties in enforcing wide-spread adoption, app-based contact tracing can complement conventional contact tracing by speeding up the process. Second, the lackluster performance of TraceTogether version 1.6 on the iPhone may have decreased the performance of the app. Many patient contacts could have been missed, since half of the physicians in this study used an iPhone and the majority of the dormitory workers used an Android phone.

    The performance of TraceTogether is expected to improve with app upgrades and increased use over time. We have provided feedback to the developers of TraceTogether, and the app has been upgraded to improve its performance on the iPhone. Since TraceTogether is Singapore’s national contact tracing app, many firms have encouraged its use [11] among employees who had to return to work after lockdown. App use has been made mandatory among dormitory workers [31] who have the highest risk of COVID-19 transmissions, and Bluetooth tokens have been distributed to populations susceptible to SARS-CoV-2 infection, such as older adults [32]. Factors that influence people’s willingness to adopt TraceTogether are being assessed to achieve higher adoption rates so that the digital contact tracing tool can be effective in the community.

    In conclusion, technological and app adoption barriers must be overcome for digital contact tracing tools to be effective for contact tracing during the COVID-19 pandemic. Although the RTLS performed well for contact tracing in a clinical setting, its implementation will be confined to a defined setting. The sensitivity of contact tracing apps needs to be improved for app-based contact tracing to be viable in the community. Leveraging technology to complement conventional manual contact tracing is a necessary move for returning some normalcy to life after exiting lockdowns. The capabilities and utility of digital contact tracing tools are expected to grow over the long haul of the COVID-19 pandemic as the technology matures.

    Acknowledgments

    We wish to acknowledge the support from our collaborators from GovTech. We also wish to thank the participating physicians for activating their TraceTogether app and reminding their patients to use the app during their COVID-19 duties. Their participation made the study possible.

    Authors' Contributions

    ZH conceived the study, analyzed and interpreted the data, and drafted the manuscript with input from all authors. GH assisted with interpreting study data and provided input for the manuscript. YML and ECH provided support for study planning and data collection and provided input for the manuscript. HA provided support for study planning and provided input for the manuscript. AC conceived the study, provided overall direction and planning for the study, analyzed and interpreted the data, and critically revised the manuscript. All authors reviewed and approved the final version of the manuscript prior to submission.

    Conflicts of Interest

    None declared.

    1. Ferretti L, Wymant C, Kendall M, Zhao L, Nurtay A, Abeler-Dörner L, et al. Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 2020 May 08;368(6491) [FREE Full text] [CrossRef] [Medline]
    2. Xu S, Li Y. Beware of the second wave of COVID-19. The Lancet 2020 Apr;395(10233):1321-1322. [CrossRef]
    3. Wise J. Covid-19: Risk of second wave is very real, say researchers. BMJ 2020 Jun 09;369:m2294. [CrossRef] [Medline]
    4. World Health Organization. Contact tracing in the context of COVID-19. COVID-19: Surveilllance, case investigation and epidemiological protocols Internet.   URL: https://www.who.int/publications/i/item/contact-tracing-in-the-context-of-covid-19 [accessed 2020-10-19]
    5. World Health Organization. Contact tracing. World Health Organization. 2017.   URL: https://www.who.int/features/qa/contact-tracing/en/ [accessed 2020-03-17]
    6. Baraniuk C. Covid-19 contact tracing: a briefing. BMJ 2020 May 13;369:m1859. [CrossRef] [Medline]
    7. Chow A, Htun HL, Kyaw WM, Ang H, Tan G, Tan HN, et al. Atypical COVID-19: Preventing transmission from unexpected cases. Infect Control Hosp Epidemiol 2020 Aug 13:1-3. [CrossRef] [Medline]
    8. Saurabh S, Prateek S. Role of contact tracing in containing the 2014 Ebola outbreak: a review. Afr Health Sci 2017 Mar;17(1):225-236 [FREE Full text] [CrossRef] [Medline]
    9. Kretzschmar M, Rozhnova G, Bootsma M, van Boven M, van de Wijgert JHHM, Bonten M. Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study. The Lancet Public Health 2020 Aug;5(8):e452-e459 [FREE Full text] [CrossRef]
    10. Hellewell J, Abbott S, Gimma A, Bosse NI, Jarvis CI, Russell TW, et al. Feasibility of controlling COVID-19 outbreaks by isolation of cases and contacts. Lancet Glob Health 2020 Apr;8(4):e488-e496. [CrossRef]
    11. Hellmich TR, Clements CM, El-Sherif N, Pasupathy KS, Nestler DM, Boggust A, et al. Contact tracing with a real-time location system: A case study of increasing relative effectiveness in an emergency department. Am J Infect Control 2017 Dec 01;45(12):1308-1311 [FREE Full text] [CrossRef] [Medline]
    12. Ho HJ, Zhang ZX, Huang Z, Aung AH, Lim W, Chow A. Use of a Real-Time Locating System for Contact Tracing of Health Care Workers During the COVID-19 Pandemic at an Infectious Disease Center in Singapore: Validation Study. J Med Internet Res 2020 May 26;22(5):e19437 [FREE Full text] [CrossRef] [Medline]
    13. Chang Y, Syed-Abdul S, Tsai C, Li Y. A novel method for inferring RFID tag reader recordings into clinical events. Int J Med Inform 2011 Dec;80(12):872-880. [CrossRef] [Medline]
    14. Yazici HJ. An exploratory analysis of hospital perspectives on real time information requirements and perceived benefits of RFID technology for future adoption. Int J Inf Manage 2014 Oct;34(5):603-621 [FREE Full text] [CrossRef]
    15. Vilamovska AM, Hatziandreu E, Schindler HR, van Oranje-Nassau C, de Vries H, Krapels J. Study on the requirements and options for RFID application in healthcare: Identifying areas for Radio Frequency Identification deployment in health care delivery: A review of relevant literature. RAND Technical Report. 2009.   URL: https://www.rand.org/pubs/technical_reports/TR608.html [accessed 2020-10-20]
    16. O'Neill PH, Ryan-Mosley T, Johnson B. A flood of coronavirus apps are tracking us. Now it's time to keep track of them. MIT Technology Review. 2020 May 07.   URL: https://www.technologyreview.com/2020/05/07/1000961/launching-mittr-covid-tracing-tracker/ [accessed 2020-10-19]
    17. Brack S, Reichert L, Scheuermann B. Decentralized Contact Tracing Using a DHT and Blind Signatures.   URL: https://eprint.iacr.org/2020/398.pdf [accessed 2020-05-01]
    18. Dubov A, Shoptawb S. The Value and Ethics of Using Technology to Contain the COVID-19 Epidemic. Am J Bioeth 2020 Jul;20(7):W7-W11. [CrossRef] [Medline]
    19. Cohen IG, Gostin LO, Weitzner DJ. Digital Smartphone Tracking for COVID-19: Public Health and Civil Liberties in Tension. JAMA 2020 Jun 16;323(23):2371-2372. [CrossRef] [Medline]
    20. Mathews M, Tan A, Suhaini S. Attitudes towards the use of surveillance technologies in the fight against COVID-19. National University of Singapore, Lee Kuan Yew School of Public Policy, Institute of Policy Studies.   URL: https://tinyurl.com/y3ruk5zy [accessed 2020-10-19]
    21. Li J, Guo X. Global Deployment Mappings and Challenges of Contact-tracing Apps for COVID-19. SSRN. Preprint posted online on May 26, 2020. [CrossRef]
    22. Osman M, Fenton NE, McLachlan S, Lucas P, Dube K, Hitman GA, et al. The thorny problems of Covid-19 Contact Tracing Apps: The need for a holistic approach. Journal of Behavioral Economics for Policy 2020;4(COVID-19 Special Issue):57-61 [FREE Full text]
    23. Cohen IG, Gostin LO, Weitzner DJ. Digital Smartphone Tracking for COVID-19: Public Health and Civil Liberties in Tension. JAMA 2020 Jun 16;323(23):2371-2372. [CrossRef] [Medline]
    24. Anglemyer A, Moore THM, Parker L, Chambers T, Grady A, Chiu K, et al. Digital contact tracing technologies in epidemics: a rapid review. Cochrane Database Syst Rev 2020 Aug 18(8). [CrossRef]
    25. Lowery-North DW, Hertzberg VS, Elon L, Cotsonis G, Hilton SA, Vaughns CF, et al. Measuring social contacts in the emergency department. PLoS One 2013 Aug 21;8(8):e70854 [FREE Full text] [CrossRef] [Medline]
    26. Ebrahimzadeh F, Nabovati E, Hasibian M, Eslami S. Evaluation of the Effects of Radio-Frequency Identification Technology on Patient Tracking in Hospitals: A Systematic Review. J Patient Saf 2017 Dec 15. [CrossRef] [Medline]
    27. Digi-Key's North American Editors. Comparison of Proprietary RF and Bluetooth. 2018 Feb 28.   URL: https://www.digikey.sg/en/articles/comparison-of-rf-and-bluetooth [accessed 2020-10-19]
    28. Cho H, Ippolito D, Yu YW. Contact Tracing Mobile Apps for COVID-19: Privacy Considerations and Related Trade-offs. arXiv. Preprint posted online on March 30, 2020 [FREE Full text]
    29. Vaughan A. The problems with contact-tracing apps. New Sci 2020 Apr 25;246(3279):9 [FREE Full text] [CrossRef] [Medline]
    30. Kucharski AJ, Klepac P, Conlan A, Kissler SM, Tang M, Fry H, CMMID COVID-19 Working Group. Effectiveness of isolation, testing, contact tracing and physical distancing on reducing transmission of SARS-CoV-2 in different settings. medRxiv. Preprint posted on April 29, 2020. [CrossRef]
    31. FAQs on using TraceTogether for contact tracing. Ministry of Manpower.   URL: https://tinyurl.com/y5t3yz88 [accessed 2020-10-19]
    32. Seniors to receive their first batch of TraceTogether Tokens from 28 June 2020. gov.sg A Singapore Government Agency Website. 2020 Jul 08.   URL: https://tinyurl.com/yydkufpw [accessed 2020-10-19]

    ‎
    EMR: electronic medical record
    NCID: National Centre for Infectious Diseases
    RTLS: real-time locating system

    Edited by G Eysenbach; submitted 03.08.20; peer-reviewed by J Li, A Johnson; comments to author 27.08.20; revised version received 09.09.20; accepted 30.09.20; published 29.10.20

    Copyright

    ©Zhilian Huang, Huiling Guo, Yee-Mun Lee, Eu Chin Ho, Hou Ang, Angela Chow. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 29.10.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.