This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Despite being an important cardiovascular risk factor, hypertension has low control levels worldwide. Computerized clinical decision support systems (CDSSs) might be effective in reducing blood pressure with a potential impact in reducing cardiovascular risk.
The goal of the research was to evaluate the feasibility, usability, and utility of a CDSS, TeleHAS (tele–
The TeleHAS app consists of a platform integrating clinical and laboratory data on a particular patient, from which it performs cardiovascular risk calculation and provides evidence-based recommendations derived from Brazilian and international guidelines for the management of hypertension and cardiovascular risk. Ten family physicians from different primary care units in the city of Montes Claros, Brazil, were randomly selected to use the CDSS for the care of hypertensive patients for 6 months. After 3 and 6 months, the feasibility, usability, and utility of the CDSS in the routine care of the health team was evaluated through a standardized questionnaire and semistructured interviews.
Throughout the study, clinicians registered 535 patients with hypertension, at an average of 1.24 consultations per patient. Women accounted for 80% (8/10) of participant doctors, median age was 31.5 years (interquartile range 27 to 59 years). As for feasibility, 100% of medical users claimed it was possible to use the app in the primary care setting, and for 80% (8/10) of them it was easy to incorporate its use into the daily routine and home visits. Nevertheless, 70% (7/10) of physicians claimed that the time taken to fill out the CDSS causes significant delays in service. Clinicians evaluated TeleHAS as good (8/10, 80% of users), with easy completion and friendly interface (10/10, 100%) and the potential to improve patients’ treatment (10/10, 100%). A total of 90% (9/10) of physicians had access to new knowledge about cardiovascular risk and hypertension through the app recommendations and found it useful to promote prevention and optimize treatment.
In this study, a CDSS developed to assist the management of patients with hypertension was feasible in the context of a primary health care setting in a middle-income country, with good user satisfaction and the potential to improve adherence to evidence-based practices.
Hypertension is a major modifiable cardiovascular risk factor responsible for substantial morbidity and mortality worldwide. It causes around 9.4 million deaths every year [
In the last three decades, clinical practice guidelines addressing hypertension management invariably recommended that cardiovascular risk assessment must be a core feature of hypertension care [
To face these challenges, the health systems in LMIC struggle to improve their quality and overcome underfunding and lack of resources [
Most of the literature that evaluated CDSSs in the management of hypertension focused on outcomes, with mixed results [
This study was conducted in four steps, according to the Medical Research Council framework [
We assessed epidemiological studies and systematic reviews on hypertension management as well as feasibility studies, randomized controlled trials, and systematic reviews on CDSSs to evaluate which gaps were already known. Additionally, the topic was discussed in meetings with Brazilian health system stakeholders and primary care physicians to identify other issues.
Meetings and internal workshops were conducted to discuss the topic with primary care physicians. Primary care physicians, internal medicine specialists, cardiologists, and endocrinologists discussed the gaps with experts in information technology to identify solutions and components of an intervention.
The CDSS, named TeleHAS (tele–
The CDSS consists of a structured clinical evaluation assessing identification, medical history, physical examination, current medications, and laboratory and image test results (
After patient registration, any subsequent consultation was recorded in the database in a file under the patient name. Data were recorded in TeleHAS and transmitted to a telehealth care central whenever internet connection was available. The CDSS was updated if problems were identified. A Web-based data panel was developed to help researchers access the database.
The CDSS was electronically tested to verify that recommendation results matched the prespecified decision tree. Manual insertion of data by a physician was later performed to verify the recommendation response suitability. After adjustments, TeleHAS was submitted for analysis to an expert panel comprising 2 cardiologists and 3 primary care physicians, known as technical reference, for a period of 7 days. Structured questionnaires and semistructured interviews were used to assess strengths, inconsistencies, and satisfaction with the device use. The participants were asked to classify their general impression of the CDSS as very appropriate, appropriate, indifferent, or inappropriate. It was then readjusted with the necessary changes according to the criticisms and suggestions from the expert panel.
Identification: patient name, date of birth, mother’s name, and sex
Comorbidities: may be used to select other diseases or risk factors presented by the patient
Physical exam: blood pressure, waist circumference, weight, and height
Laboratory studies: lipid profile, biochemical panel, echocardiography, and electrocardiography data
Medications: several blood pressure medications categorized by class
TeleHAS open screen and main menu.
TeleHAS previous diseases and alerts menu.
The field study was conducted in Montes Claros, the largest city in the north of the Minas Gerais state (population 361,000), 270 miles north of the state’s capital. It has a human development index of 0.77 [
All 66 primary care physicians from Montes Claros were invited to attend a lecture on hypertension diagnosis and management. Of the 63 attendees, 51 agreed to join the study and 10 were randomly selected. The selected doctors used TeleHAS in routine care of hypertensive patients to build up their impressions about the CDSS for 6 months. Physicians planned and individualized the frequency of blood pressure measurements for each patient.
A nurse and an information technology technician held biweekly visits to address issues and resolve difficulties in the CDSS use and allow data transfer to the TeleHAS network server through a 3G connection available only on the nurse’s tablet due to its high cost in Brazil.
Paper-based questionnaires evaluating feasibility, usability, and utility of the CDSS were developed and applied with semistructured interviews at the end of 3 and 6 months (
Study design.
Categorical variables were presented as frequencies and proportions. As the sample size of physicians was small, continuous variables were expressed as median and interquartile range. Data analyses were performed using SPSS Statistics for Windows version 21.0 (IBM Corp).
The study was approved by the ethics committee of the Universidade Federal de Minas Gerais, Minas Gerais, Brazil. Informed, written consent was obtained from all participants contributing data to the study.
All specialists considered the initial menu, identification, comorbidities, medicine, and complementary exam screens as adequate (
Specialists perceived TeleHAS as an intervention that could change clinical practice.
The CDSS helps in the process of decision making for treatment of patients with hypertension, both in pharmacologic and nonpharmacologic therapy.
I believe it can be very helpful to control high blood pressure.
There was a unanimous impression that the device has the potential to deliver updated information to health care providers and education promotion is a major feature of its use. It was highlighted that the device has the potential to promote implementation of evidence-based recommendations.
Great potential to promote best scientific evidence implementation for the individual patient.
When asked about the feasibility of TeleHAS in the clinical setting, all participants agreed it was feasible.
TeleHAS has the potential to improve clinical practice quality, and it’s applicable to the routine of primary care services.
One primary care physician considered the physical exam screen inadequate. He stated that he would like to enter data about other conditions not associated with hypertension or cardiovascular risk such as glycemic control and data from skin, respiratory, and abdomen semiology. He suggested that a free-text field could be added to address this demand.
Participant characteristics are described in
During the 6 months of field study, participant physicians registered 535 patients in the TeleHAS database and performed 632 consultations.
Quantitative analysis of impressions for feasibility, usability, and utility is described in
Although doctors were motivated to use TeleHAS, finding the time to enter data was a major concern. Physicians believed it caused significant delay in the daily routine and developed alternative forms of use to deal with this challenge. Some used the CDSS in educational groups and on days specifically scheduled to attend patients with hypertension. Others used it daily when attending a patient with hypertension. Due to the scarce time available, some of them attended patients without the CDSS and filled it out later, saving remarks for the next appointment. In the case of one clinician, this choice was due to fear of patient opinion.
...I was concerned that they might find I was distracted or writing something else, like sending messages during the consultation.
Work duplication was also identified as a problem, and clinicians demanded a printed handout to deliver to patients and attach to the patient record at the end of the consultation.
Expert panel perception of TeleHAS for each screen (n=5).
Screen | Rating | ||
Very adequate, n (%) | Adequate, n (%) | Inadequate, n (%) | |
Initial menu | 3 (60) | 2(40) | 0 (0) |
Identification | 3 (60) | 2 (40) | 0 (0) |
Comorbidities | 1 (20) | 4 (80) | 0 (0) |
Physical exam | 0 (0) | 4 (80) | 1 (20) |
Medicine | 2 (40) | 3 (60) | 0 (0) |
Complementary exams | 1 (20) | 4 (80) | 0 (0) |
Characteristics of primary care physicians in the study (n=10).
Variable | Value, n (%) | |
Sex, female | 8 (80) | |
<5 | 7 (70) | |
5-10 | 1 (10) | |
>10 | 2 (20) | |
Angiology | 1 (10) | |
Family medicine | 2 (20) | |
Geriatrics | 1 (10) | |
Occupational medicine | 1 (10) | |
Pediatrics | 1 (10) | |
Nonea | 4 (40) | |
Inadequate | 2 (20) | |
Satisfactory | 4 (40) | |
Good | 3 (30) | |
Excellent | 1 (10) | |
Use of any form of technology before TeleHAS, yes | 9 (90) | |
Computer available in the workplace for routine use, yes | 0 (0) | |
Internet access in the workplace, yes | 5 (50) | |
Daily | 9 (90) | |
Monthly | 1 (10) | |
Completed continuing education on management of hypertension or cardiovascular risk in the last year, yes | 6 (60) | |
Guidelines | 3 (30) | |
Books | 2 (20) | |
Articles | 1 (10) | |
Congress | 1 (10) |
aResidency is not a prerequisite for doctors who work in primary care in Brazil.
Feasibility score on a 5-point scale by item (n=10).
Item | Score mean |
The CDSSa can be used in the primary care setting. | 4.5 |
It can be used in home visits. | 4.2 |
It is easy to incorporate in work routine. | 4.0 |
Internet connection is not essential for the use of the CDSS. | 3.6 |
The CDSS does not cause significant delays in daily routine. | 3.0 |
aCDSS: clinical decision support system.
Usability score on a 5-point scale by item (n=10).
Item | Score mean |
My overall evaluation of the CDSSa is good. | 4.5 |
The CDSS screens are easy to understand. | 4.5 |
The definitions of comorbidities are clear and unambiguous. | 4.1 |
The CDSS fields are easy to complete. | 4.3 |
The CDSS is intuitive and requires no previous training to use. | 3.4 |
The CDSS is stable, and no errors occur during use. | 3.0 |
aCDSS: clinical decision support system.
Utility score on a 5-point scale by item (n=10).
Item | Score mean |
I believe that the CDSSa might improve the treatment of hypertensive patients. | 4.5 |
Reading the recommendations of the CDSS, I had access to new information on hypertension and cardiovascular risk. | 4.1 |
According to my previous knowledge, I believe the recommendations generated by the CDSS are appropriate. | 4.1 |
The CDSS was useful to calculate the cardiovascular risk of hypertensive patients. | 3.8 |
The CDSS was useful to promote cardiovascular disease prevention actions among my patients. | 4.3 |
The CDSS helped me treat my patients. | 4.6 |
I used the recommendations to modify the behavior of my patients. | 4.0 |
I would recommend the CDSS to my colleagues. | 4.7 |
aCDSS: clinical decision support system.
With regard to usability, during the interviews, all physicians classified the identification screen as practical and simple to use. Clinicians also reported the comorbidities screen as simple and intelligible.
Although TeleHAS was designed to be intuitive, some physicians believed training was essential for its proper use, even though clinicians did explore its features and learned the hidden tasks it was designed to do (eg, the majority found out that date of birth could be inserted by the keyboard or by a cursor displayed after continuously pressing the data field).
Most clinicians stated that other chronic conditions should be addressed in the CDSS and requested the option to enter glucose levels, diabetes mellitus complications such as diabetic foot, and other classes of medication for associated conditions such as methimazole, insulin, metformin, isosorbide and cilostazol.
Some clinicians observed errors during CDSS use. The most discussed one was that the software would sometimes stop working. This was due to a fault in the Android system that would stop functioning if the tablet enter key was pressed twice. No primary health care center had Wi-Fi connection available. System actualizations and data transfer to the TeleHAS database server occurred whenever Wi-Fi connection was available, at the physician’s home or during the visit by the nurse and information technology technician. Eventually, due to the absence of 3G coverage or inadequate connection speed, the nurse needed to set up an appointment with the physician in the city to proceed with data transfer.
With regard to utility, physicians reported that TeleHAS eventually changed clinical practice. Some of them changed their blood pressure measurement techniques by reading the description available in the CDSS. Others started to measure blood pressure in 3 positions to complete all the fields available in TeleHAS.
For the physical exam screen, clinicians suggested a change in the sequence available in the CDSS, as they normally make the first measure with the patient seated.
Sometimes I entered the measure in the wrong field, because I start with the patient seated. Then I had to enter the measures again, after other measures.
It was perceivable that clinicians worked with different levels of systematization. Only three of them calculated cardiovascular risk systematically using the 10-year global risk assessment score chart. Physicians perceived TeleHAS as helpful in promoting cardiovascular risk assessment and prevention actions.
Some clinicians perceived the CDSS fulfillment and recommendations delivered as repetitive.
After a while, you get tired of reading the same things. You don’t have much time available [to deal with the repetitive content].
However, some considered the repetition a CDSS strength.
Sometimes we remember to guide the patient only about medication, and we forget to talk about nonpharmacologic treatment. The alerts are repetitive, but it helps us to remember.
I used to forget to ask about the salt in their diet. With the CDSS, I remembered to ask about it with every patient.
The accuracy of the CDSS cardiovascular risk calculation was considered unsatisfactory by 30% (3/10) of participants. In the interviews, this issue was often discussed. The clinicians reported that the cardiovascular risk calculator sometimes presented spurious results and made them confused about the correct estimated risk. This error was found to occur when the date of birth was not entered or entered incorrectly.
This field study designed to evaluate the feasibility and usability of a CDSS to assist hypertension and cardiovascular risk management found that it is feasible to implement such technology in a primary care setting in a middle-income country with good satisfaction by health professionals. Equally important, the study elicited strengths and faults of the CDSS and helped identify facilities and barriers for TeleHAS implementation.
In Brazil, primary care physicians are commonly young and inexperienced doctors, frequently recently graduated [
This feasibility study showed several barriers for the implementation of TeleHAS, including health professional issues, telecommunication structure, and health system infrastructure. Physician resistance to adopting information technologies is one of the main barriers for implementation, often due to poor satisfaction with physical characteristics of digital technologies, difficulties with their use, and low effectiveness of devices [
With regard to the telecommunication structure, problems with internet connection, poor broadband coverage, and high cost of 3G limited data transfer to the TeleHAS database server. Problems in telecommunication structures that limit dissemination of mHealth services in LMIC are similar to those seen in LMIC medical care, like services limited in scope, unevenly distributed across geographic areas, and of variable quality when available [
Barriers to CDSS use were also identified. Low computerization of the primary health centers was a major limitation for the CDSS implementation. There was a perceptible demand for communication technologies in primary health system and much of the demand presented by physicians was related to the need to register and organize patient records and clinical data. This is in agreement with CDSS usability tests performed previously. In a study to assess the usability of a CDSS for the management of diabetes, the absence of electronic medical records was also pointed out as a major barrier for system definitive implementation in primary care [
Another major barrier for TeleHAS use in primary care was health professionals’ excessive workload. The work duplication generated by the need to register data in the tablet and in the patient records limited the use of the CDSS. In a meta-analysis of 311 unique studies on CDSSs, including 148 randomized controlled trials, several features have shown to be predictors of improved health care process measures, including integration with charting or order entry system with no need for additional clinician data entry [
Task shifting to other health professionals is proposed as a solution to improve quality of care in chronic diseases. CDSSs designed to integrate the work of health professionals have been developed and may be the way for TeleHAS to diminish data entry requirements by clinicians. In a study performed in India, physicians and community health workers used a CDSS designed to address hypertension and assess cardiovascular risk. It was observed that such technology might aid to standardize the care and empower other health workers to help in cardiovascular risk assessment and patient care [
This study was designed to test and improve a CDSS and used qualitative assessments rather than clinical outcomes. However, its findings are very important, as usability testing is an essential step before software implementation in clinical practice or testing the impact in clinical outcomes. Poor usability may lead to various undesirable effects, ranging from user dissatisfaction and failed implementations to endangered patient safety [
In this study, a CDSS developed to assist in decision making in hypertension and cardiovascular risk assessment was feasible in the context of the primary care setting, with good user satisfaction and possible positive impact on the implementation of guidelines, recommendations, and best available evidence. The study provided the opportunity to strengthen TeleHAS to face the barriers identified in its implementation. The CDSS is currently being tested in a large-scale clinical study to access clinical end points and the possibility to scale up its use for the primary care setting within the country and in other LMIC.
Example of TeleHAS decision tree.
clinical decision support system
Conselho Nacional de Desenvolvimento Científico e Tecnológico
Fundação de Amparo à Pesquisa de Minas Gerais
Financiadora de Estudos e Projetos
low- and middle-income countries
mobile health
tele–hipertensão arterial sistêmica (arterial hypertension system)
This study was supported by the Brazilian research agencies
None declared.