Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, December 24 through Wednesday, December 26 inclusive. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 01.02.19 in Vol 7, No 2 (2019): February

Preprints (earlier versions) of this paper are available at, first published Jul 28, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Change in Patient Comfort Using Mobile Phones Following the Use of an App to Monitor Tuberculosis Treatment Adherence: Longitudinal Study

    1San Diego State University, San Diego, CA, United States

    2University of California San Diego, School of Medicine, La Jolla, CA, United States

    Corresponding Author:

    Richard S Garfein, MPH, PhD

    University of California San Diego

    School of Medicine

    9500 Gilman Drive, MC-0507

    La Jolla, CA, 92093-0507

    United States

    Phone: 1 1 858 822 3018

    Fax:1 858 534 7566



    Background: As mHealth apps proliferate, it is necessary for patients to feel capable and comfortable using devices that run them. However, limited research is available on changes in comfort level before and after the use of an mHealth app.

    Objective: The objective of this study was to determine whether patients with tuberculosis who used an mHealth app called Video Directly Observed Therapy (VDOT) to monitor their antituberculosis treatment became more comfortable using mobile phones after the intervention and to identify factors associated with change in comfort.

    Methods: We analyzed data from a longitudinal study assessing the feasibility and acceptability of the VDOT app among patients receiving antituberculosis treatment from public health departments in San Diego, San Francisco, and New York City. Comfort levels on six domains of mobile phone use (making phone calls, taking pictures, recording videos, text messaging, internet and email use on the phone) were measured on a 10-point scale (1=very uncomfortable; 10=very comfortable) at the start and end of treatment using VDOT via telephone interviews. The main outcomes were change in comfort level on each domain (recoded as binary measures) and an overall change score (sum of individual measures). Linear and logistic regression analyses were performed to assess whether sociodemographics, risk factors, and VDOT perceptions were associated with change of comfort measures.

    Results: Among 120 participants with complete data, mean age was 39.8 years (SD 14.8, range 18-87 years), 46.7% (56/120) were female, and 76.7% (92/120) were foreign born. The combined comfort level at baseline was high overall (mean 48.8, SD 14.2, interquartile range 43.0-60.0) and the mean comfort score increased by 1.92 points at follow-up (P=.07). Statistically significant increases in comfort on individual domains included taking pictures (P=.02) and recording videos (P=.002). Females were more likely to have increased comfort in using the internet on the phone compared to males (odds ratio [OR] 3.03, 95% CI 1.08-8.52, P=.04). Participants who worked less hours per week were more likely to have increased comfort recording videos although this did not meet statistical significance (OR 1.03, 95% CI 1.00-1.05, P=.06).

    Conclusions: Findings suggest that, despite a high level of comfort using mobile phones at baseline, experience using the VDOT app was associated with increased comfort using mobile phone features. Additional research involving participants with lower baseline mobile phone experience is needed. An implication of these findings is that as patients begin to use mHealth apps for one health condition, they could acquire skills and confidence to more quickly adapt to using mHealth apps for other conditions.

    JMIR Mhealth Uhealth 2019;7(2):e11638




    Mobile phone apps play an increasingly important role in health care as evidence supporting these interventions accumulates [1]. These mHealth (mobile health) interventions have been applied to monitor postoperative care [2,3], diabetes self-management [4,5], smoking cessation [6,7], care and prevention of HIV and other sexually transmitted diseases [8], and more. As these interventions begin to spread throughout health care, there has been a growing interest in the response to these apps, including topics such as acceptability and feasibility [2], satisfaction by patients and providers [5], and efficiency [4]. However, there is limited research measuring users’ comfort in operating mobile phone functions and whether comfort increases as a result of engagement with these apps.

    The World Health Organization and Centers of Disease Control recommend the use of directly observed therapy (DOT) for monitoring treatment of both tuberculosis and sometimes latent tuberculosis infection. DOT is a case management system developed for monitoring antituberculosis treatment [9]. Patients with tuberculosis have to travel to a health care facility or a trained DOT worker travels to the patient’s home to observe them taking their medication. DOT improves adherence because it requires close supervision, but it has some barriers as well [10]. One example of a barrier to DOT is that it requires both a patient’s and DOT worker’s presence; both schedules may interfere with time and place of when the medication is taken. These and other barriers prompted the development of an mHealth intervention called Video Directly Observed Therapy (VDOT). VDOT was developed as an alternative to DOT to help reduce the barriers of DOT and increase adherence [11]. The VDOT process allows patients to record themselves taking the medication and send the date- and time-stamped video to a DOT worker who watches the video and documents each dose taken. These recorded videos allow patients the flexibility of choosing the best time and place to take their medication. Patients receive daily medication reminders by short message service (SMS) text message, and may use the phone to text, email, or call their provider throughout treatment.

    A study was conducted from August 2013 through July 2014 to assess the feasibility and acceptability of VDOT for monitoring tuberculosis treatment adherence in three US health departments (San Diego, San Francisco, and New York); patient comfort using mobile phones was also measured during baseline and follow-up interviews [11]. In this paper, we assess the change in comfort using mobile phone features and apps and look at factors associated with change. Understanding patient’s comfort and use of mHealth apps will assist in the design and development of other health-based mobile phone apps.


    Participant Recruitment

    This analysis used data from a three-city study of VDOT among patients with active tuberculosis. Patients 18 years or older undergoing treatment for active tuberculosis in San Diego, San Francisco, and New York City Health Departments were invited to participate in the study by health department staff. Research staff conducted phone interviews before patients started to use the VDOT app (baseline) and after their treatment or study participation ended (follow-up). The interviews included questions on sociodemographics, risk behaviors, VDOT perceptions, and comfort using mobile phone apps and phone features. Participants were provided phones with the VDOT app installed and were told that they could use the phones for anything related to their tuberculosis treatment: making calls, text messaging, emailing, taking pictures, and accessing the internet. The study was approved by the University of California San Diego Human Research Protection Program. All participants provided written informed consent.


    The main outcome for this analysis was change in comfort using mobile phone features between the baseline and follow-up interviews. Comfort with phone features was stratified into six categories: making phone calls, taking pictures, recording videos, text messaging, internet browsing, and using email. Participants were asked, “On a scale of 1 to 10, how comfortable are you using a cell phone to (category)? 1=very uncomfortable and 10=very comfortable.” This scale was developed by the authors and was first used in our study; therefore, we assessed internal reliability using robust alpha coefficients [12]. The estimated alpha was .93 (95% CI .87-.96), which showed excellent reliability. These questions were asked during both the baseline and follow-up interviews. Change in comfort was calculated by subtracting the baseline scores from the follow-up scores. “Not applicable” and “refuse to answer” were potential response options and were treated as missing for this analysis. Overall change in comfort was calculated by summing the six individual technology comfort scores (range 1-10) to produce a cumulative score for baseline and follow-up (range 6-60).

    Data Analysis

    The overall change in technology comfort and the change in six separate technology comfort variables were summarized by descriptive statistics (mean, median, quartiles, standard deviation) and assessed by Wilcoxon signed rank test. Univariate tests (Spearman correlation coefficient, Wilcoxon rank sum test, and Kruskal-Wallis test) and simple linear regression analysis were performed to assess the bivariate association between overall comfort change score and each potential covariate (sociodemographics, site, risk behaviors, VDOT perceptions, and baseline comfort score). For individual item comfort change measures, we analyzed them as binary outcomes (improved vs not improved) since normal assumption of residuals for a linear regression model was not satisfied; a change score greater than zero was defined as an improvement. Univariate tests (Wilcoxon rank sum test and Fisher exact test) and simple logistic regression analysis were performed to assess bivariate associations between each individual item change and baseline covariates. Results from the univariate test and regression analysis were consistent; therefore, estimated coefficients, odds ratios (ORs), and P values from regression analyses were reported. Variables that were significant with P<.10 in the simple linear/logistic regression were considered for inclusion in the multivariable regression analysis. Backward elimination was used to remove nonsignificant variables from the model and the variable with the largest P value was removed from the model at each step, until only variables with P<.10 remained in the final model. A normal probability plot was used to assess the normality assumption of linear model residuals and influential observations were assessed by residual and Cook’s distance. Sensitivity analyses were performed by excluding the influential observations from the analysis and comparing it to the original model. Since the results remained robust, we report regression analysis results from the original model. A P<.05 was interpreted as statistically significant. All data analyses were performed using R (version 3.3.0) and SAS 14 software (Cary, NC, USA).


    Participant Characteristics

    Overall, 151 participants were enrolled in the parent study, of which 126 completed the follow-up interview. Since participants could refuse to answer any question in the interviews, this analysis includes only those who provided responses to at least three comfort questions at both visits, leaving 120 participants for analysis. The mean age was 39.8 years (SD 14.8, range 18-87), 46.7% (56/120) were female, 47.5% (57/120) were Asian, 26.7% (32/120) were Hispanic, and 12.5% (15/120) were African American. Overall, 76.7% (92/120) were foreign born, 55.4% (62/112) reported an annual income of more than US $10,000/year, 79.0% (94/119) had health insurance, and 47.0% (55/117) had education beyond high school. Behavior characteristics showed that 27.7% (33/119) consumed alcohol at least once a month and 40.0% (48/120) had ever smoked cigarettes. Before starting VDOT, 72.5% (87/120) of participants owned a mobile phone and 80.8% (97/120) of participants preferred communicating with their health care provider through text, phone call, or email (text message: 11.7%, 14/120; phone call: 63.3%, 76/120; 14/120; email: 5.8%, 7/120). The mean duration of daily VDOT use was 5.46 months (SD 3.25, interquartile range [IQR] 3.09-7.47).

    Overall, mean baseline comfort scores for the six individual domains of mobile phone use were high (range 7.43-9.02; Table 1). The average comfort scores were lowest for email use on phone and highest for making phone calls at both baseline and follow-up. The average change in comfort score was highest for recording videos (mean 0.79, SD 3.04) and lowest for making phone calls (mean 0.11, SD 2.03) and email uses (mean 0.11, SD 2.72); however, the changes were only statistically significant for taking pictures (P=.02) and recording video (P=.002). Although not statistically significant, an increase was also observed for the combined overall comfort score (P=.07).

    Table 1. Self-reported comfort score using mobile phone features before and after using an mHealth app to monitor antituberculosis treatment adherence among patients in San Diego, San Francisco, and New York City (N=120).
    View this table

    Combined Overall Comfort

    From simple linear regression analysis (Table 2), we found that older age was associated with an increase in overall comfort score change (P<.001). Those without a mobile phone at baseline (P=.01) and those who had lower comfort at baseline (P<.001) were more likely to have increased comfort at follow-up. Using multivariable linear regression, we found only the baseline overall score stayed significant with a higher overall baseline score (ie, more comfortable at baseline) associated with a smaller increase at follow-up (beta=–0.45, P<.001).

    Comfort in Six Individual Comfort Domains

    To identify factors independently associated with change in comfort level for each of the six individual phone functions, we performed simple (Table 2) and multivariable (Table 3) logistic regression analyses. Variables significantly associated with comfort changes on individual items in the simple logistic regression analysis included age, country of birth, education, hours worked per week, previous mobile phone ownership, ever recording VDOT videos while away from home, and baseline comfort score (all P’s<.05).

    Variables significantly associated with comfort change for each individual item in simple logistic regression were assessed in multivariable logistic regression models. Baseline comfort levels were the strongest predictors in the individual item change measures (all P’ s<.001). After controlling for baseline scores, only female participants (OR 3.03, 95% CI 1.08-8.52, P=.04) were more likely to have increased comfort in internet use. Participants with fewer hours worked per week (OR 1.03, 95% CI 1.00-1.05, P=.06) were also more likely to have increased comfort in recording video although this did not meet statistical significance.

    Table 2. Simple linear/logistic regression analysisa of change in comfort using mobile phone functions among Video Directly Observed Therapy (VDOT) study participants (N=120).
    View this table
    Table 3. Multivariable regression modela for associations with change in comfort using mobile phone functions among VDOT participants.
    View this table


    The main function of the VDOT app was to record and send videos of each medication ingestion event, and we found a significantly positive change in comfort with using a mobile phone to take videos and photos. We also observed a nonstatistically significant increase in comfort using all other mobile phone functions following the use of the VDOT app for monitoring tuberculosis treatment adherence. Except for baseline comfort, sex, and hours worked, we did not find baseline sociodemographics to affect comfort change. This indicates that the use of VDOT has the potential to increase patient skills with various mobile phone functions across a range of patient characteristics, which should increase their willingness and ability to utilize future mHealth apps.

    Other mHealth studies considered the usability and receptiveness of app-based tools using different forms of Likert scales [2,7] or qualitative measures from patient responses [5,6] to find important associations related to app quality and user opinions. These studies assessed a variety of health interventions and most had positive results when investigating topics such as usability [3] and acceptability [2,7]. Unlike previous studies, this study quantified comfort with using mobile phone functions and examined correlates of those changes. These measures of change in comfort help to elucidate how patients perceive the use of an mHealth app and how its use can impact comfort using mobile phones. Finding that comfort increased after using an mHealth app suggests that introducing patients to one mHealth app can increase the likelihood that they will be willing and able to engage with future mHealth interventions. The factors that contribute to comfort using mobile phone features have important relevance to whether patients have a good experience using the different tools and benefit from the intended purpose of the app.

    Some study limitations must be considered. All participants in this study were undergoing treatment for tuberculosis; thus, this sample is not representative of all patient populations. Tuberculosis in the United States most commonly occurs among immigrants from high tuberculosis-burden countries; persons who are homeless, inject illegal drugs, or are HIV infected; and people who live or work in hospitals, homeless shelters, correctional facilities, nursing homes, and residential homes for those with HIV [13]. The study took place in three major metropolitan cities, so the results might not reflect those of patients in rural areas. Also, the modest sample size might have limited our ability to detect statistically significant associations. Since most participants owned mobile phones and had high comfort levels at baseline, they had little room to increase comfort, which also limited our ability to detect changes. Future studies involving patients with less mobile phone experience at baseline should be considered. Lastly, the data were all self-reported, which may have introduced information bias.

    This study has several strengths. The longitudinal study design supports a causal relationship between the observed change in comfort and use of the VDOT app. Baseline and follow-up questionnaires encompassed a wide range of variables including sociodemographic, behavioral, and treatment perceptions, which allowed us to assess multiple potential correlates of change in comfort. The study assessed comfort using six different functions of a mobile phone rather than mobile phone use in general, which allowed for a more comprehensive investigation of the effect of using an mHealth app on change in comfort using mobile phones. Lastly, this is the first study to measure change in comfort using an mHealth app, which helps to address gaps in existing literature.

    This study shows that after using a mobile phone for one mHealth intervention, patients became more comfortable using the device, which suggests that they will more quickly embrace other mHealth interventions (eg, smoking cessation or diabetes management). With expanded use of mHealth apps like VDOT, it is important for patients to be comfortable with using mobile phone functions. Measuring the comfort level that patients have while using mHealth apps can help to determine if these apps are improving patient skills using technology. This will help prepare patients to adopt the use of other forms of mHealth, as well as accessing health information through their mobile phone.


    The authors would like to thank the study participants and tuberculosis control program staff who contributed their time and information for this study. Funding for the parent study was provided by a gift from the Verizon Foundation to the University of California San Diego. RSG and LL are also funded by the National Institutes of Health (U01-AI116392).

    Authors' Contributions

    DD contributed to analysis of data, interpretation of results, and drafted the manuscript. RSG conceptualized the research questions, advised on study implementation, and contributed to manuscript preparation and final review. JCM assisted in data collection, data quality assurance, and manuscript preparation. KC contributed to the questionnaire design, data collection, and topic conceptualization. LL led the study implementation, designed data analysis plan, supervised data analysis, interpretation of results, manuscript writing, and final review. All authors critically reviewed and approved the manuscript.

    Conflicts of Interest

    RSG and KC are cofounders of SureAdhere Mobile Technology, Inc—a VDOT service provider. No funding, software, or other resources were provided by SureAdhere for the study. To mitigate potential conflicts of interest, interpretation and reporting of the study findings were approved by coauthors who are unaffiliated with SureAdhere. The terms of this arrangement have been reviewed and approved by the University of California, San Diego, in accordance with its conflict of interest policies.


    1. Whittaker R. Issues in mHealth: findings from key informant interviews. J Med Internet Res 2012 Oct;14(5):e129 [FREE Full text] [CrossRef] [Medline]
    2. Semple JL, Sharpe S, Murnaghan ML, Theodoropoulos J, Metcalfe KA. Using a mobile app for monitoring post-operative quality of recovery of patients at home: a feasibility study. JMIR Mhealth Uhealth 2015 Feb;3(1):e18 [FREE Full text] [CrossRef] [Medline]
    3. Scott AR, Alore EA, Naik AD, Berger DH, Suliburk JW. Mixed-methods analysis of factors impacting use of a postoperative mHealth app. JMIR Mhealth Uhealth 2017 Feb 08;5(2):e11. [CrossRef] [Medline]
    4. Wu Y, Yao X, Vespasiani G, Nicolucci A, Dong Y, Kwong J, et al. Mobile app-based interventions to support diabetes self-management: a systematic review of randomized controlled trials to identify functions associated with glycemic efficacy. JMIR Mhealth Uhealth 2017 Mar 14;5(3):e35 [FREE Full text] [CrossRef] [Medline]
    5. Holtz BE, Murray KM, Hershey DD, Dunneback JK, Cotten SR, Holmstrom AJ, et al. Developing a patient-centered mHealth app: a tool for adolescents with type 1 diabetes and their parents. JMIR Mhealth Uhealth 2017 Apr 19;5(4):e53 [FREE Full text] [CrossRef] [Medline]
    6. McClure JB, Hartzler AL, Catz SL. Design considerations for smoking cessation apps: feedback from nicotine dependence treatment providers and smokers. JMIR Mhealth Uhealth 2016 Feb 12;4(1):e17 [FREE Full text] [CrossRef] [Medline]
    7. Finkelstein J, Cha EM. Using a mobile app to promote smoking cessation in hospitalized patients. JMIR Mhealth Uhealth 2016 May 06;4(2):e59 [FREE Full text] [CrossRef] [Medline]
    8. Muessig KE, Pike EC, Legrand S, Hightow-Weidman LB. Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review. J Med Internet Res 2013 Jan 04;15(1):e1 [FREE Full text] [CrossRef] [Medline]
    9. Centers for Disease Control and Prevention, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Division of Tuberculosis Elimination. Core Curriculum on Tuberculosis: What the Clinicians Should Know. 2015.   URL: [accessed 2018-11-27] [WebCite Cache]
    10. Hopewell PC, Pai M, Maher D, Uplekar M, Raviglione MC. International standards for tuberculosis care. Lancet Infect Dis 2006 Nov;6(11):710-725. [CrossRef] [Medline]
    11. Garfein RS, Collins K, Muñoz F, Moser K, Cerecer-Callu P, Raab F, et al. Feasibility of tuberculosis treatment monitoring by video directly observed therapy: a binational pilot study. Int J Tuberc Lung Dis 2015 Sep;19(9):1057-1064 [FREE Full text] [CrossRef] [Medline]
    12. Zhang Z, Yuan K. Robust coefficients alpha and omega and confidence intervals with outlying observations and missing data: methods and software. Educ Psychol Meas 2016 Jun;76(3):387-411 [FREE Full text] [CrossRef] [Medline]
    13. Centers for Disease Control and Prevention. TB Risk Factors. 2016 Mar 18.   URL: [accessed 2018-06-19] [WebCite Cache]


    DOT: directly observed therapy
    OR: odds ratio
    VDOT: Video Directly Observed Therapy

    Edited by G Eysenbach; submitted 28.07.18; peer-reviewed by S Li, K Murray, P Di Carlo, V Dhikav; comments to author 30.08.18; revised version received 17.10.18; accepted 23.10.18; published 01.02.19

    ©Diana Do, Richard S Garfein, Jazmine Cuevas-Mota, Kelly Collins, Lin Liu. Originally published in JMIR Mhealth and Uhealth (, 01.02.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.