Mobile Access to ClinicalConnect: A User Feedback Survey on Usability, Productivity, and Quality

Introduction

Mobile devices are rapidly becoming a part of everyday life as both communication and information tools. Recent studies indicate that this is true in the case of health care providers as well. Health care providers often use their mobile devices to access educational material, pharmaceutical compendiums, practice guidelines, clinical pathways, and electronic medical records [1].

Electronic health information exchange (HIE), in which patients’ clinical data is efficiently shared between care delivery settings, is expected to produce a number of quality improvements and cost savings [2]. ClinicalConnect is a federated clinical viewer that provides health information exchange between health services and care providers of South West Ontario [3]. It is considered to be one of the largest federated HIE models in North America. A mobile version of ClinicalConnect was developed and deployed in June 2012.

Health care providers often fail to realize the intended effects of their eHealth systems due to inadequate usability [4]. Despite the widely held belief that the computerization of health information systems contributes to improved quality of patient care and care management, studies have found medical staff attitudes toward computerization to be negative [5]. In order to discern whether the ClinicalConnect mobile version was meeting user requirements, an evaluation survey was conducted in November 2013. A range of survey tools was used for the assessment with a focus on usability, and the perceived impact of the mobile version on the quality of patient care and productivity of health care providers.

Methods

Tools

The System Usability Scale (SUS), a nonproprietary validated survey tool [6], was used to assess the usability of the ClinicalConnect mobile interface. SUS includes 10 statements presented on a 5-point Likert scale, which results in an overall score from 0 to 100 that indicates the perceived usability of the interface.

Canada Health Infoway published a Benefits Evaluation Indicators Technical Report in 2006 that was subsequently updated in 2012 to provide guidance for benefits evaluation planning related to information and communications technology (ICT) in health care [7]. The framework proposes the indicators that could have an impact on productivity and quality of health care. Based on these indicators, productivity and quality questionnaires using a 5-point Likert scale [8] were designed. Productivity and quality questionnaires had 8 (Table 1) and 10 questions (Table 2), respectively.

The SUS, productivity, and quality survey tools were hosted and administered using SurveyMonkey [9].

Participants

Five hundred and four health care providers who expressed interest in participating in periodic evaluations while signing up for the HIE service were sent an e-mail with a link to the survey. A second reminder e-mail was sent if no response was received from a provider. One hundred and ten providers responded to the survey, though some surveys were incomplete. Seventy-seven responses were obtained for SUS. On productivity and quality scores, we received 74 and 71 responses, respectively.

Analysis

Results of the SUS questionnaire were recoded and normalized, and the mean SUS score and the standard deviation were then recorded. The Likert-type questions for productivity and quality were independently tabulated. Since all questions within each category measure a single concept, the values were combined into a composite score by calculating the mean and standard deviation. Scores for negatively worded questions (fifth question in the productivity matrix and second and third questions in the quality matrix) were normalized prior to the calculation of the composite score.

Results

Eighty-five (77%) of the respondents were physicians. The mean SUS score was 67 (SD 14.4) with a percentile score of 46.9 [10]. This means that the mobile version of ClinicalConnect can be considered more usable than 46.9% of all products evaluated with the SUS instrument.

Table 1 depicts the responses to the productivity-related questions and Table 2 summarizes the quality questions. The mean productivity score was 3.37 (SD 1.06) and the mean quality score was 3.62 (SD 0.99), with 3 being neutral on the 5-point Likert scale.

Discussion

HIE is the process of sharing electronic health information between different providers and organizations. As in most other health information systems, mobile devices are increasingly becoming popular as an HIE platform [11].

The SUS consists of 10 alternating positively and negatively worded statements scored on a 5-point Likert scale. The mean SUS score for 2324 surveys about usability conducted over a 10-year period was 70.14 with a median score of 75 [6].

Inadequate usability is a major cause of failure of eHealth systems [4], especially of mobile platforms. Our data show that health service providers perceive the mobile interface of ClinicalConnect as useful, but are neutral about the ease of use. This pattern has been noticed before in other mHealth interventions [12]. The ease of use can be affected by factors beyond mobile user interface such as ergonomic and social aspects [13].

HIE is vital for improving efficiency and quality of health care. It has been demonstrated that perceived usefulness is a stronger predictor of the use of an eHealth technology than the perceived ease of use [14]. Most of our respondents agreed that ClinicalConnect on mobile devices had a positive impact on their productivity and quality. Anywhere, anytime access to patient information on mobile devices was perceived as an important factor in faster consultations, referrals, and handoffs, and in improved communication between health care providers. However, information access on mobile devices was not considered faster than the desktop counterpart, and there was no consensus on whether mobile devices allow more face-to-face time with patients. This corroborates previous studies that showed that physicians consider improvements in the quality domain as the overarching benefit of HIE [15].

Though HIE programs have demonstrated clinical value in some situations such as emergency departments [16], a consistent empirical proof of value is lacking [17]. In a study, emergency physicians reported workflow disruptions from HIE use [18]. Our study suggests that mobile access to integrated health records is perceived as beneficial, especially to the quality of patient care. However, our sample size may be insufficient and not representative enough of the health care roles for us to statistically draw a precise conclusion.

It is important for HIE system implementations to be integrated into the care practice improvement process [4]. The study shows that bringing data from disparate health care systems to the point of care via mobile HIE systems has a perceived potential for improvements in patient care.