This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.
Tuberculosis (TB) is a highly infectious disease. Negative perceptions and insufficient knowledge have made its eradication difficult. Recently, mobile health care interventions, such as an anti-TB chatbot developed by the research team, have emerged in support of TB eradication programs. However, before the anti-TB chatbot is deployed, it is important to understand the factors that predict its acceptance by the population.
This study aims to explore the acceptance of an anti-TB chatbot that provides information about the disease and its treatment to people vulnerable to TB in South Korea. Thus, we are investigating the factors that predict technology acceptance through qualitative research based on the interviews of patients with TB and homeless facility personnel. We are then verifying the extended Technology Acceptance Model (TAM) and predicting the factors associated with the acceptance of the chatbot.
In study 1, we conducted interviews with potential chatbot users to extract the factors that predict user acceptance and constructed a conceptual framework based on the TAM. In total, 16 interviews with patients with TB and one focus group interview with 10 experts on TB were conducted. In study 2, we conducted surveys of potential chatbot users to validate the extended TAM. Survey participants were recruited among late-stage patients in TB facilities and members of web-based communities sharing TB information. A total of 123 responses were collected.
The results indicate that perceived ease of use and social influence were significantly predictive of perceived usefulness (
This study can be used to inform future design of anti-TB chatbots and highlight the importance of services and the environment that empower people to use the technology.
Tuberculosis (TB) is a highly infectious disease and one of the top 10 causes of death worldwide, claiming approximately 4000 lives a day [
In recent years, mobile health (mHealth) has rapidly emerged as a vehicle for delivering better health services at a lower cost, regardless of time and place [
Chatbots are a conversational agent, a software program that interacts with natural language, and have emerged as a new form of mHealth service [
However, despite the expansion of mHealth solutions for TB control and the potential of chatbots, little research has been conducted on applying these tools to the management of TB. To the best of our knowledge, few studies have attempted to develop chatbots and virtual agents to support information accessibility for patients with TB [
In 2019, we developed an anti-TB chatbot that provides information about the disease, its treatment, and TB hospitals and facilities. It targets people vulnerable to TB, as well as those affected by it.
Providing information on tuberculosis
Functions
Overview of tuberculosis
Diagnosis of tuberculosis
Tuberculosis treatment
Information on drugs to treat tuberculosis
Side effects of tuberculosis drugs
Screening for contact and latent tuberculosis infection
Providing information on hospitals and facilities
Functions
Institutions for tuberculosis screening and treatment
Tuberculosis treatment support project
Tuberculosis treatment support facility
Information on welfare and administration related to tuberculosis
Information on welfare facilities related to tuberculosis
The chatbot was built on an open-source platform and operates within an instant messenger app called Kakao Talk. An advantage of using this platform is that the medium through which users interact with the chatbot, that is, the messenger app, is widely used in South Korea, with over 72% of the total population or roughly 36.6 million people using it [
The knowledge base was obtained from the information provided by the Korea Disease Control and Prevention Agency. We acquired the content with permission and then reorganized it in a dialog format. In addition to the text information, multimedia content was actively adopted, considering the tendency of low health literacy level of the poor and older people [
We gave the chatbot the personality of a doctor. A chatbot with identity cues, such as a name, profile, and language style, is perceived as more empathetic, friendly, and personal [
The chatbot provides graphic and text information on the disease, its treatment, and neighboring TB facilities. Users navigate the content by scrolling the page vertically and horizontally. They communicate with the chatbot by selecting menus at the bottom of the screen, pushing buttons, or typing texts (
Antituberculosis chatbot user interface.
Davis et al [
The TAM is widely used in technology acceptance research; however, it can predict only approximately 40% of the overall explanatory power [
Among the studies that have validated the TAM, some extended the model to address different contexts and populations, including the acceptance and continuous use of chatbots. For example, Huang and Chueh [
In the absence of studies that explain the acceptance of chatbots in the context of TB control, we aim to explore the benefits and concerns regarding accepting an anti-TB chatbot as perceived by potential users, to provide an extended TAM that can better predict the acceptance of anti-TB chatbots. Thus, we present studies 1 and 2. Study 1 aims to identify the factors that predict the acceptance of anti-TB chatbots through interviews with patients with TB and homeless facility personnel. On the basis of the interview results, we derived an operational definition of the questionnaire items and identified the factors for the extended TAM. Study 2 aims to verify the proposed theoretical model and identify the factors predicting the acceptance of an anti-TB chatbot.
To collect data for study 1, we conducted interviews with potential users of our anti-TB chatbot. Interviewees were recruited by posting a notice at a municipal TB hospital. The participants were selected using convenience sampling among people who have or had TB. People who could neither understand nor respond to the questionnaire provided in Korean were excluded. In total, 16 patients with TB received a gift worth US $50. We also conducted a focus group interview with 10 experts on TB from the academia, hospitals, shelters, support facilities, and housing providers for homeless people who have worked for patients with TB and thus have sufficient knowledge about them and are willing to use the chatbot or introduce it to them. Participant information is presented in
Participant information of study 1 (N=26; site: Seoul; year: 2020).
Demographics | Values, n (%) | |||
|
||||
Gender | ||||
|
Male | 16 (100) | ||
|
Female | 0 (0) | ||
Age (years) |
|
|||
|
30s | 2 (13) | ||
|
40s | 3 (19) | ||
|
50s | 3 (19) | ||
|
60s | 6 (38) | ||
|
70s | 2 (13) | ||
Experience of smartphone use | ||||
|
Yes | 9 (56) | ||
|
No | 7 (44) | ||
Experience of chatbot use | ||||
|
Yes | 0 (0) | ||
|
No | 16 (100) | ||
|
||||
|
Academia | 1 (10) | ||
|
Hospital | 1 (10) | ||
|
Shelters | 2 (20) | ||
|
Support facilities | 5 (50) | ||
|
Housing provider | 1 (10) |
aTB: tuberculosis.
Data collection followed the protocols of the American Psychological Association (APA) ethical principles and code of conduct [
We analyzed the collected data using thematic analysis in ATLAS.ti, a qualitative data analysis and research tool. Three researchers designed the coding frame to analyze the interviewees' attitude or intent to accept the anti-TB chatbot. We classified the results into a set of subthemes, which were clustered into the main themes. These main themes were assigned as TAM factors.
Study 2 aimed to evaluate factors that predict the acceptance of the anti-TB chatbot. According to Davis et al [
Study 1 demonstrated that social influence and facilitating conditions were relevant to the acceptance of the chatbot by patients with TB. In previous studies that modeled technology acceptance, social influence such as subjective norm, voluntariness, and image is known as an external construct of perceived usefulness [
Hypothetical model of study 2.
We conducted both offline and web-based surveys, considering that older adults and other vulnerable groups have limited access to the internet. In the offline survey, we recruited participants at TB facilities that were mainly used by patients in the late stage of TB treatment, who can take medication on their own after discharge from the hospital. The research team visited the facility and instructed and provided assistance to those who expressed their willingness to participate. In the web-based survey, participants were recruited from web-based communities that share information on TB. The web-based survey was distributed among the potential users of the anti-TB chatbot, and their responses were collected via Google Forms. All participants received a monetary reward worth US $5.
As in study 1, the data collection process in study 2 was guided by the protocols of the APA ethical principles and code of conduct [
The questionnaire was developed based on the theoretical framework of the TAM and the findings from study 1. It consisted of 32 items inquiring about demographic and attitudinal data—participants were asked general questions on demography and experience with chatbots and specific questions regarding their attitude toward the anti-TB chatbot. The attitudinal components were measured using a 7-point Likert-type scale, where the choice of answers ranged from
A total of 127 cases were collected in March 2020. After the screening, 4 cases were excluded: there were missing values in 3 cases, and a straight line was found in 1 case. We used the partial least squares structural equation modeling (PLS-SEM) approach to statistically analyze and process the collected data using SmartPLS 3.0, a dedicated structural equation program with a strong verification power, even for small sample sizes. First, we used the PLS algorithm to evaluate the measurement model. This was followed by bootstrapping and blindfolding techniques for evaluation and hypothesis testing of the structural models.
The interviewees noted that the usefulness of the chatbot was associated with the characteristics of the information content, the chatbot's ability to communicate in a similar manner as a peer, and enhanced access to information (
Perceived usefulness of the antituberculosis chatbot among potential users (n=the number of times a theme was mentioned by the interviewees of study 1).
Regarding the perceived ease of use of the anti-TB chatbot, the interviewees mentioned the following themes: legibility, comprehension, error prevention and efficiency, and learnability (
Perceived ease of use of the antituberculosis chatbot among potential users (n=the number of times a theme was mentioned by the interviewees of study 1; positive and neutral comments in normal and negative comments in italics).
Facilitating conditions associated with the interviewees’ acceptance of the anti-TB chatbot were classified as internal and external resources (
Facilitating conditions of the antituberculosis chatbot (n=the number of times a theme was mentioned by the interviewees of study 1; positive and neutral comments in normal and negative comments in italics).
The social influence on the use of the anti-TB chatbot was governed by recommendations from professionals treating TB and the context of use. Interviewees mentioned that recommendations from hospitals would facilitate their adoption of the chatbot (
Social influence of the antituberculosis chatbot in potential users. It should be noted that n=number of times a theme was mentioned by the interviewees of study 1.
Participants’ ages ranged from 22 to 85 years, with almost equal participation by men and women. Most respondents did not have any history of TB, and approximately half had no experience using chatbots. Out of 123 participants, 120 (97.5%) participants had already used the messenger app (
Participant demographics of study 2 (N=123; site: Seoul; year: 2020).
Demographics | Values, n (%) | ||
|
|||
|
Female | 60 (48.7) | |
|
Male | 63 (51.2) | |
|
|||
|
22 to 29 | 26 (21.1) | |
|
30 to 39 | 33 (26.8) | |
|
40 to 49 | 34 (27.6) | |
|
50 to 59 | 9 (7.3) | |
|
60 to 85 | 21 (17.1) | |
|
|||
|
Yes | 16 (13) | |
|
No | 107 (86.9) | |
|
|||
|
Yes | 120 (97.5) | |
|
No | 3 (2.5) | |
|
|||
|
Yes | 61 (49.5) | |
|
No | 62 (50.4) |
The measurement models of study 2 using PLS-SEM were evaluated for internal reliability, convergent validity, and discriminant validity. The internal reliability was assessed using Cronbach α and composite reliability, in which a value greater than .70 for each indicates acceptable internal consistency [
Reliability and convergent validity of the measurement model in study 2.
Construct and items | Factor loadings | Cronbach α | Composite reliability coefficient | Average variance extracted | ||||
|
|
.854 | 0.902 | 0.697 | ||||
|
PU 1 | 0.893 |
|
|
|
|||
|
PU 2 | 0.859 |
|
|
|
|||
|
PU 3 | 0.860 |
|
|
|
|||
|
PU 4 | 0.783 |
|
|
|
|||
|
.927 | 0.948 | 0.822 | |||||
|
PEOU 1 | 0.870 |
|
|
|
|||
|
PEOU 2 | 0.892 |
|
|
|
|||
|
PEOU 3 | 0.941 |
|
|
|
|||
|
PEOU 4 | 0.920 |
|
|
|
|||
|
|
.858 | 0.904 | 0.702 | ||||
|
SI 1 | 0.801 |
|
|
|
|||
|
SI 2 | 0.852 |
|
|
|
|||
|
SI 3 | 0.887 |
|
|
|
|||
|
SI 4 | 0.808 |
|
|
|
|||
|
|
.798 | 0.868 | 0.625 | ||||
|
PR 1 | 0.860 |
|
|
|
|||
|
PR 2 | 0.852 |
|
|
|
|||
|
PR 3 | 0.635 |
|
|
|
|||
|
PR 4 | 0.847 |
|
|
|
|||
|
|
.899 | 0.930 | 0.768 | ||||
|
ATC 1 | 0.834 |
|
|
|
|||
|
ATC 2 | 0.893 |
|
|
|
|||
|
ATC 3 | 0.899 |
|
|
|
|||
|
ATC 4 | 0.878 |
|
|
|
|||
|
|
.932 | 0.951 | 0.831 | ||||
|
BI 1 | 0.851 |
|
|
|
|||
|
BI 2 | 0.931 |
|
|
|
|||
|
BI 3 | 0.924 |
|
|
|
|||
|
BI 4 | 0.937 |
|
|
|
aPU: perceived usefulness.
bPEOU: perceived ease of use.
cSI: social influence.
dPR: facilitating conditions.
eATC: attitude to chatbot.
fBI: behavioral intention.
Discriminant validity was assessed using the square root of the AVE in the cross-loading matrix. To establish a satisfactory discriminant validity of the model, the square root of the AVE for a given construct should be greater than its correlation with other constructs [
Discriminant validity of the measurement model in study 2.
Constructs | Perceived usefulness | Perceived ease of use | Social influence | Facilitating conditions | Attitude to chatbot | Behavioral intention |
Perceived usefulness | 0.835 | 0.512 | 0.81 | 0.519 | 0.714 | 0.588 |
Perceived ease of use | 0.512 | 0.906 | 0.422 | 0.707 | 0.325 | 0.410 |
Social influence | 0.81 | 0.422 | 0.838 | 0.508 | 0.743 | 0.664 |
Facilitating conditions | 0.519 | 0.707 | 0.508 | 0.791 | 0.421 | 0.494 |
Attitude to chatbot | 0.714 | 0.325 | 0.743 | 0.421 | 0.876 | 0.713 |
Behavioral intention | 0.588 | 0.410 | 0.664 | 0.494 | 0.713 | 0.911 |
The results of the structural model for the TAM are shown in
Path analysis results for study 2.
Results of the structural model in study 2.
Endogenous variable and exogenous variable | β | ||||
|
|
||||
|
Perceived ease of use | .15 | 2.062 | .04 | |
|
Social influence | .746 | 12.023 | <.001 | |
|
|
||||
|
Perceived usefulness | .720 | 11.314 | <.001 | |
|
Perceived ease of use | −.012 | 0.151 | .88 | |
|
|
||||
|
Facilitating conditions | .235 | 2.242 | .03 | |
|
Attitude to chatbot | .614 | 7.438 | <.001 |
We also performed a PLS multigroup analysis (PLS-MGA) by dividing the participants into 2 groups based on their history of TB. There were 107 participants with a history of TB and 16 with no history of TB. The results indicated that perceived usefulness was positively predicted by social influence in both groups (
Results of the multigroup analysis.
Path | TBa history group | Non–TB history group | Difference | TB history group | Non–TB history group | |
|
β | β | β |
|
||
PUb→ATCc | .662 | .733 | −.071 | .002 | <.001 | .72 |
PEOUd→ATC | .186 | −.046 | .233 | .41 | .60 | .34 |
PEOU→PU | −.118 | .194 | −.313 | .56 | .008 | .13 |
ATC→BIe | .113 | .66 | −.547 | .66 | <.001 | .01 |
SIf→PU | .906 | .726 | .18 | <.001 | <.001 | .34 |
FCg→BI | .826 | .175 | .651 | .002 | .07 | .02 |
aTB: tuberculosis.
bPU: perceived usefulness.
cATC: attitude to chatbot.
dPEOU: perceived ease of use.
eBI: behavioral intention.
fSI: social influence.
gFC: facilitating conditions.
This study aimed to propose a chatbot that provides information for the prevention and treatment of TB and identify factors that predict the acceptance of the chatbot. We conducted interviews with 16 patients with TB and 10 experts in TB and identified the factors that predict the acceptance of the anti-TB chatbot in study 1. From the results, we found social influence and facilitating conditions as additional factors in the extended TAM model. In study 2, we proposed an extended TAM model capable of predicting the acceptance of the anti-TB chatbot and evaluated it. We found that social influence was a strong predictor of perceived usefulness, regardless of history of TB. Study 1 suggests that social influence can arise from both health care experts and peers. Regarding users' behavioral intention, the predictive factor varied in the participants’ history of TB. Overall, our findings were consistent with those of other researchers [
Our study confirmed that people needed information about the disease, as well as TB hospitals and support facilities. It also suggested that the reliability of the information provided by the chatbot is crucial to perceived usefulness and eventually the acceptance of the chatbot. Although this may sound rather obvious, existing mHealth apps that provide information on TB have been found to contain errors such as spelling and grammatical mistakes, outdated information, and wrong and potentially harmful content, according to a recent study that investigated 29 e-learning and information apps on TB [
Perceived usefulness was significantly predictive of people’s attitude toward the anti-TB chatbot if they have experienced TB. When people seek information about TB, stigmatization and its consequences (eg, social isolation and reduced economic opportunities) can be barriers to active information seeking and timely access to necessary services [
Study 2 confirmed that perceived ease of use was predictive of perceived usefulness but not predictive of the attitude toward technology. The latter result has been observed in studies where participants were proficient in using the technology (eg, responses of experienced mobile phone users to a mobile app or a chatbot) [
In study 1, we observed social influence acting on the interviewees when a staff member in the hospital or TB treatment facility recommended the use of the anti-TB chatbot or when a peer introduced them. Thus, social influence can have a positive impact on the perceived usefulness of the chatbot. In a study that investigated the acceptance of conversational agents for disease diagnosis, social influence was identified as a factor influencing users’ intention to adopt or use a chatbot [
Social influence can be derived from the authority and credibility of the service provider (ie, hospital) and those who have (expert or user) knowledge about the disease and technology. Among the different types of social influence was the peer pressure from other people who use a smartphone and a chatbot. For example, a patient with TB whom we met in study 1 was among several patients who did not have a smartphone and were eager to learn to use the smartphone and the anti-TB chatbot (
Facilitating condition is strongly associated with the acceptance of chatbots by patients with TB and thus should be considered when designing an anti-TB chatbot. TB occurs more commonly in older adults and low-income groups. It is also these groups who find it most challenging to access and use mHealth solutions. They are often reluctant to accept new solutions, such as chatbots, due to a lack of internal resources (eg, information on and capabilities to use mHealth solutions) according to study 1. However, this lack can be compensated by the provision of external support, such as a peer who can help them learn how to use a chatbot or smartphone tutorials. Existing chatbot-related studies tend to focus on the efficiency and usefulness of these technologies [
Trans-sectoral efforts have been made to disseminate smartphones among homeless people as a strategy to reinforce self-sufficiency and mitigate poverty. Organizations, such as the Community Technology Alliance, Seoul Municipality, and Underheard in New York, have implemented smartphone giveaway projects in which donated smartphones were delivered to homeless people and used to find accommodation, economic opportunities, and fulfill other basic needs [
This study has several limitations. First, our hypotheses were evaluated using correlation methods; therefore, the derived model did not explain causal relationships among the identified constructs. Second, this study was conducted using a convenience sample, which limits the generalizability of our findings. Thus, future studies should conduct a more comprehensive inspection of how these individual differences are associated with the acceptance of technology using representative and larger samples [
Despite the expansion of mHealth solutions for TB control and the potential of chatbots to save costs and reduce the risk of stigma associated with the diagnosis and treatment of TB, few studies have sought to investigate the determinants of their adoption. In this context, we conducted 2 studies to develop an extended TAM that incorporates additional variables obtained from an empirical study with patients with TB and explain the intention to use a chatbot for TB control. The results showed that the intention to use the anti-TB chatbot was predicted by attitude toward the chatbot and facilitating conditions. Attitude toward the chatbot was positively predicted by its perceived usefulness but was not significantly predicted by perceived ease of use. The results also suggested that the perceived usefulness of the anti-TB chatbot was positively predicted by perceived ease of use and social influence. The importance of this study is to identify the underlying factors associated with the intention to use an anti-TB chatbot. These findings can be used to inform future design of anti-TB chatbots. For future work, it will be necessary to integrate the proposed model with other theories and factors that can help explain greater acceptance.
Definition for the constructs used in study 2.
average variance extracted
direct observed treatment
mobile health
partial least squares multigroup analysis
partial least squares structural equation modeling
Technology Acceptance Model
tuberculosis
video-observed treatment
This work was supported by the Ministry of Science and Information and Communication Technology on the “Digital Social Innovation Project” of National Information Society Agency and Yonsei University Research Grant of 2021. The authors thank the Misohealing Centre and Seoul Seobuk Hospital for their cooperation.
None declared.