This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
ClinicalConnect, a federated clinical viewer for South West Ontario, Canada, launched a mobile interface in June 2012.
The aim of the study was to assess usability of the mobile interface and the perceived impact on productivity of health care providers and quality of healthcare delivery.
A survey was conducted using the System Usability Scale (SUS) and questionnaires designed to measure productivity and quality based on Canada Health Infoway's Benefits Evaluation framework.
The mean SUS score was 67 based on 77 responses. The mean scores for productivity and quality were 3.37 (N=74) and 3.62 (N=71), respectively, on a 5-point Likert scale where 3 was neutral.
Users perceived the mobile interface of ClinicalConnect as useful but were neutral about the ease of use.
Mobile devices are rapidly becoming a part of everyday life as both communication and information tools. Recent studies indicate that this is true in the case of health care providers as well. Health care providers often use their mobile devices to access educational material, pharmaceutical compendiums, practice guidelines, clinical pathways, and electronic medical records [
Electronic health information exchange (HIE), in which patients’ clinical data is efficiently shared between care delivery settings, is expected to produce a number of quality improvements and cost savings [
Health care providers often fail to realize the intended effects of their eHealth systems due to inadequate usability [
The System Usability Scale (SUS), a nonproprietary validated survey tool [
Canada Health Infoway published a Benefits Evaluation Indicators Technical Report in 2006 that was subsequently updated in 2012 to provide guidance for benefits evaluation planning related to information and communications technology (ICT) in health care [
The SUS, productivity, and quality survey tools were hosted and administered using SurveyMonkey [
Five hundred and four health care providers who expressed interest in participating in periodic evaluations while signing up for the HIE service were sent an e-mail with a link to the survey. A second reminder e-mail was sent if no response was received from a provider. One hundred and ten providers responded to the survey, though some surveys were incomplete. Seventy-seven responses were obtained for SUS. On productivity and quality scores, we received 74 and 71 responses, respectively.
Results of the SUS questionnaire were recoded and normalized, and the mean SUS score and the standard deviation were then recorded. The Likert-type questions for productivity and quality were independently tabulated. Since all questions within each category measure a single concept, the values were combined into a composite score by calculating the mean and standard deviation. Scores for negatively worded questions (fifth question in the productivity matrix and second and third questions in the quality matrix) were normalized prior to the calculation of the composite score.
Eighty-five (77%) of the respondents were physicians. The mean SUS score was 67 (SD 14.4) with a percentile score of 46.9 [
Productivity questions and summary of responses (N=74).
Answer options | Strongly agree | Agree | Neither agree nor disagree | Disagree | Strongly disagree |
|
n (%) | n (%) | n (%) | n (%) | n (%) |
Accessing patient/client test results and information on my mobile device has decreased time spent tracking down or waiting for these reports | 15(20) | 27(36) | 17(23) | 13(18) | 2(3) |
Using a mobile device allows me to spend more face-to-face time with my patient/client | 5(7) | 19(26) | 31(42) | 18(24) | 1(1) |
I have more portability as I can now access ClinicalConnect with a mobile device | 19(26) | 47(64) | 4(5) | 3(4) | 1(1) |
The “New Results” feature draws my attention to results quickly | 10(14) | 34(46) | 22(30) | 6(8) | 2(3) |
It is difficult to view patient information on my mobile device screen | 6(8) | 21(28) | 22(30) | 21(28) | 4(5) |
I can access information faster on my mobile device than on a desktop computer | 3(4) | 18(24) | 20(27) | 17(23) | 16(22) |
Treatment decisions are made faster now that I can access patient/client information on my mobile device | 8(11) | 20(27) | 32(43) | 14(19) | 0(0) |
Access to patient information on my mobile device allows for better communication between health care providers | 14(19) | 31(42) | 22(30) | 7(9) | 0(0) |
Quality questions and summary of responses (N=71).
Answer options | Strongly agree | agree | Neither agree nor disagree | Disagree | Strongly disagree |
|
n (%) | n (%) | n (%) | n (%) | n (%) |
Education activities are enhanced when the patient/client can view their results on my mobile device with me | 7(10) | 16(23) | 33(46) | 14(20) | 1(1) |
There is high risk that a mobile device can be the source of a nosocomial infection | 3(4) | 11(15) | 24(34) | 24(34) | 9(13) |
Current results are not easy to access on a mobile device | 5(7) | 16(23) | 16(23) | 25(35) | 9(13) |
A mobile device allows faster access to vital patient information facilitating quicker consultation, diagnostic tests, and interventions | 14(20) | 29(41) | 17(24) | 11(15) | 0(0) |
Access to patient/client information anywhere, anytime enhances consultations, referrals, and handoffs | 17(24) | 38(54) | 10(14) | 5(7) | 1(1) |
I am more confident that my mobile device is cleaner than the desktop computer that is used by multiple people | 11(15) | 24(34) | 23(32) | 8(11) | 5(7) |
I can better prioritize my actions to follow-up on test results with the “New Results” feature | 6(8) | 29(41) | 31(44) | 4(6) | 1(1) |
I feel more confident in care decisions because I have the information I need at my fingertips | 15(21) | 30(42) | 22(31) | 3(4) | 1(1) |
I am less likely to order a duplicate test because I have easier access to current results | 15(21) | 36(51) | 16(23) | 3(4) | 1(1) |
HCPs should have access to their patient’s/client’s health information no matter where they are | 33(46) | 29(41) | 7(10) | 2(3) | 0(0) |
HIE is the process of sharing electronic health information between different providers and organizations. As in most other health information systems, mobile devices are increasingly becoming popular as an HIE platform [
The SUS consists of 10 alternating positively and negatively worded statements scored on a 5-point Likert scale. The mean SUS score for 2324 surveys about usability conducted over a 10-year period was 70.14 with a median score of 75 [
Inadequate usability is a major cause of failure of eHealth systems [
HIE is vital for improving efficiency and quality of health care. It has been demonstrated that perceived usefulness is a stronger predictor of the use of an eHealth technology than the perceived ease of use [
Though HIE programs have demonstrated clinical value in some situations such as emergency departments [
It is important for HIE system implementations to be integrated into the care practice improvement process [
health information exchange
Hamilton Niagara Haldimand Brant
information and communications technology
local health integration network
System Usability Scale
Funding for this ClinicalConnect mobile project and Benefits Evaluation as well as funding for the publication costs was provided by Canada Health Infoway Inc, a non-profit corporation funded by the Government of Canada. The funders had no role in study design, data collection and analysis, or decision to publish the manuscript. In-kind services support was provided by the Hamilton Niagara Haldimand Brant (HNHB) eHealth Office and associated health care providers. The authors would like to acknowledge the support of Dale Anderson, John Haywood, Debra Hutchinson, Rob Lloyd, and Kate Black.
ClinicalConnect is a trademark of the public hospital Hamilton Health Sciences. ClinicalConnect is jointly funded by public hospitals and agencies as well as eHealth Ontario.
The study was conducted by the HNHB eHealth Office/Hamilton Health Sciences, the regional solution provider deploying ClinicalConnect regionally, and the service delivery partner for the HNHB local health integration network (LHIN).