This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Any system applied to the control of parenteral nutrition (PN) ought to prove that the process meets the established requirements and include a repository of records to allow evaluation of the information about PN processes at any time.
The goal of the research was to evaluate the mobile health (mHealth) app and validate its effectiveness in monitoring the management of the PN process.
We studied the evaluation and validation of the general process of PN using an mHealth app. The units of analysis were the PN bags prepared and administered at the Son Espases University Hospital, Palma, Spain, from June 1 to September 6, 2016. For the evaluation of the app, we used the Poststudy System Usability Questionnaire and subsequent analysis with the Cronbach alpha coefficient. Validation was performed by checking the compliance of control for all operations on each of the stages (validation and transcription of the prescription, preparation, conservation, and administration) and by monitoring the operative control points and critical control points.
The results obtained from 387 bags were analyzed, with 30 interruptions of administration. The fulfillment of stages was 100%, including noncritical nonconformities in the storage control. The average deviation in the weight of the bags was less than 5%, and the infusion time did not present deviations greater than 1 hour.
The developed app successfully passed the evaluation and validation tests and was implemented to perform the monitoring procedures for the overall PN process. A new mobile solution to manage the quality and traceability of sensitive medicines such as blood-derivative drugs and hazardous drugs derived from this project is currently being deployed.
The use of information and communication technologies (ICTs) in the health field opens a long list of possibilities aimed at achieving benefits at many levels in the health environment. A wide spectrum of research trends emerges. The use of ICTs plays a paramount role in the development of innovative clinical nutrition projects, especially in collaborative environments that go far beyond Web apps. A new area emerges, eHealth, in which great advances, new benefits, and possibilities arise when applying ICT to the health care domain, especially with the use of mobile apps, mHealth.
Systems supported by telematic apps enable registries that allow linking and evaluation of information on parenteral nutrition (PN) processes at any time [
The good practices guide for the preparation of medicines in hospital pharmacy services establishes that all procedures related to the preparation of medicines, including PN, should be approved and reviewed. This guide establishes the need to evaluate and verify all procedures related to the preparation of medicines under the premise of “applying the principles of risk management for quality and quality by design” [
The Spanish Society of Hospital Pharmacy (Sociedad Española de Farmacia Hospitalaria), in its standards of practice for specialized nutritional support established by its Nutrition Working Group [
The objective of this study was to evaluate the mHealth app and validate its effectiveness in monitoring the management of the PN process.
A follow-up study was planned to evaluate a previously developed mHealth app and validate that it covers the real needs of the general process of PN.
The units were the PN bags prepared by the pharmacy service of the Son Espases University Hospital (HUSE) of Palma, Spain.
For the management of the stages of validation and transcription of the prescription, HospiWin 2000 version 8 (Baxter SL) software was used. This program provided the label that was attached to each PN bag. In addition, the entities to be monitored (PN bags) were identified with a unique code (a QR label was attached).
The tracking of the PN bags was carried out from June 1 to September 6, 2016.
The flow diagram, including all the stages of traceability and management of the PN, was generated and is published elsewhere [
Using a methodology based on hazard analysis and taking advantage of semantic technologies, a holistic service platform for traceability and process control was designed.
Upon the reading of a QR tag attached to an element to be monitored (in this case a PN bag), a data exchange occurs between the mobile app and the analysis server (steps 1 and 2). This server accesses its semantic knowledge base to infer the potential control operations associated with the PN bag. Using this data and other contextual information (such as the user type, location, or time), a personalized list of ranked operations is sent to the mobile app (steps 2 and 3). The user selects the most convenient operation (step 3), probably the first one on the list, and the app generates a form to capture the values required for the task under consideration (step 4). At this point, the human user completes the form and captures photos when required, and the mobile app acquires any other required pieces of data, such as GPS location or time. These data are sent to the server to update the knowledge base (step 5). Finally, analysts and auditors can use a Web dashboard to visualize the data and trace the operations. A more detailed description of the platform is published elsewhere [
The verification of this app was previously published [
General behavior of the software platform.
Each one of the requests performed by this app was linked to a certain control operation in a particular stage. In case an operation was invoked more than once, by PN bag, the system only validated the first request, storing the remaining ones as “null requests.”
An assessment of the utility and ease of use was performed. For this purpose, the Poststudy System Usability Questionnaire (PSSUQ) [
Validation was performed by checking the compliance of all the operations in each of the stages and corroborating that they were adequately monitored, particularly the operative control points (OCPs) and, more importantly, the CCPs. In order to validate the process, the results obtained from 387 PN bags at HUSE were analyzed.
In the preparation stage for the gravimetric control, according to HUSE protocol for PN bags with a volume greater than 100 mL, the real weight should not exceed a margin of ±5% of its theoretical weight.
For control of the storage temperature of the PN bags according to the procedure for the preservation of thermolabile medications at HUSE [
In the administration stage, in the absence of preestablished limits for time and speed infusion, recommendations for each specific nutrition were observed according to the patient's clinical and social situation and the device used for its administration.
The stages of the process were prescription, preparation, preservation, and administration. All control operations at each stage of the process were recorded in a dichotomous way (yes/no), according to whether they were finished or not.
The operations characterized as CCPs were gravimetric control, temperature of the refrigerator, and volume and time of infusion (administration). These operations were recorded in a continuous and quantitative manner.
The operations characterized as OCPs were microbiological control (which is recorded a posteriori), final check of the PN (including the checking of physical parameters such as turbidity, precipitate, strange color, etc), and existence of the filter in the infusion pump (according to the PN type). These operations were recorded by means of dichotomous observations (yes/no). In all cases, it was possible to capture an image to prove compliance with a certain operation.
The parameters monitored through the mobile device (phone or tablet) were sent to the Web server. This server processed these data and stored them in the knowledge base in the form of facts. The information model used was designed using Semantic Web–based techniques that allowed representation of concepts and relationships and offered advanced query services and analysis capabilities [
To access the information managed by the system, the Web service offered a dashboard accessible through a common Web browser. In this dashboard, analysts and auditors were able to study and evaluate the history of records generated in any phase of the process, filter them based on various aspects of the context (date, responsible user, etc), download the information in a standardized format (comma-separated values), and visualize usage statistics through easily interpretable graphics.
A descriptive analysis was performed on the control operations assigned to each stage. The quantitative variables were characterized by their sample mean, standard deviation, median, mode, maximum, minimum, and interquartile interval (IQI). To avoid problems caused by outliers in the mean, the robust average was also calculated. The qualitative variables were characterized by their absolute and relative frequencies. Figures and tables were used to represent the most relevant qualitative variables.
For analysis of the internal consistency of the answers obtained from the PSSUQ (study of the reliability of the measurement scale), a Cronbach alpha coefficient was used. To evaluate the significance in the difference of means for independent samples, a Student
Quality control of the data was performed through double tables and active search for errors. When errors were found, they were corrected by consulting the original source. SPSS Statistics version 22.0 (IBM Corp) was used for the analysis.
This work is part of the PI13/00464 project and was approved by the project evaluation body of the Miguel Hernández University with registration number 2013.95.E.OEP.
During the implementation of the mHealth system for the traceability and management of PN at HUSE, 50 professionals took part and a total of 387 PN bags were managed.
The usability test was performed at HUSE, with a total of 21 hospital professionals collaborating (42% of the staff was invited to participate).
Results according to the dimensions of the Poststudy System Usability Questionnaire [
Usability dimensions | Description | Mean (SD) | Mediana | Cronbach alphab |
OVERALL | Overall satisfaction | 5.1 (1.9) | 6 | .98 |
SYSUSE | System usefulness | 5.2 (1.8) | 6 | .94 |
INFOQUAL | Information quality | 5.8 (1.9) | 6 | .96 |
INTERQUAL | Interface quality | 4.8 (2.0) | 5 | .98 |
aMaximum value: 7.
bMaximum value: 1.
Control operations for each stage of the parenteral nutrition management system at Son Espases University Hospital.
Stage | System control operation | Adult (n=353), n | Pediatric (n=34), n |
Validation of the prescription | Validate prescription | 353 | 34 |
Transcription of the prescription | Transcribe prescription | 353 | 34 |
Preparation | Initial check of trays/materials Control of PNa preparation Final check of trays/materials Gravimetric control (CCPb) Adequacy of physical parameters (OCPc) Final check of PN (OCP) Microbiological record of PN (OCP) |
353d | 34d |
Conservation | Temperature control of the refrigerator (CCP) | 353 | 34 |
Interruption/suspension of administration | Cause of interruption/suspension of the administratione | 26 | 4 |
Administration | Start time Infusion time (CCP) Volume to be administered (CCP) Identification of the pump Existence of the filter (OCP) |
327 | 30 |
aPN: parenteral nutrition.
bCCP: critical control point.
cOCP: operative control point.
dThe records regarding the microbiological analysis of the PN and, where appropriate, the counter sample were generated a posteriori once the results were known.
eSuspension: complications due to underlying disease or complications related to PN (mechanical, infectious, metabolic). Interruption: improvement (oral/enteral), transfer to another center or exitus.
The overall result of the responses to the questionnaire presented the following values: mean 96.0 (SD 6.6), median 102, minimum 29, maximum 133, and IQI 72.5 to 118.5 (maximum score 133 = 19 items ×7 maximum score). The statistics obtained according to the different usability dimensions and study of the reliability of the measurement scale using the Cronbach alpha coefficient can be seen in
In the validation study, 387 PN bags were managed; 91.2% (353/387) of the bags were from adult patients and 8.8% (34/387) from pediatric patients, with a total of 3860 control operations (see
All operations related to OCP were recorded and audited in the knowledge base, including the type of operation, date, and person who carried out the operation. Compliance with these controls was 100%.
The statistics obtained from the control of the operations labeled as CCPs for PN bags intended for adults and children can be checked in
As shown on
In the administration stage, it was found that for infusion time (measured in hours), the deviation from the mean was 0.3 hours for adults and 0 hours for children (see
Statistics regarding the control operations identified as critical control points in the management system and the traceability control of parenteral nutrition.
Patient, stage, and control operation | Mean (SD) | Robust mean | Median | Max | Min | ||
Gravimetric control (g) | 2182.0 (11.3) | 2195.0 | 2285 | 2885 | 1730 | ||
Infusion time (h) | 22.2 (0.3) | 21.7 | 24 | 48 | 11 | ||
Volume to be administered (mL) | 1868.8 (10.8) | 1886.5 | 1998 | 2028.4 | 1200 | ||
Gravimetric control (g) | 937.9 (46.8) | 936.0 | 955 | 1270 | 640 | ||
Infusion time (h) | 24.0 (0) | 24.0 | 24 | 24 | 24 | ||
Volume to be administered (mL) | 736.7 (58.4) | 703.5 | 700 | 2028 | 300 |
Label on a bag of parenteral nutrition, including data from the gravimetric control.
The results obtained in the period under consideration from daily recording of the temperature by 2 probes in the refrigeration room (Kardex Remstar, BioCold Environmental Inc) can be seen in
A nonconformity was detected from June 24 to July 7 that did not influence the validity of the PN bags since although the operative limit was exceeded (8ºC), the critical limit was not reached (12ºC). Likewise, there was a 10-day period when the temperature was lower than 2ºC, although in no case was it below 1.1ºC, so the temperature never fell below the critical limit (0.1ºC). In both situations, measures described in the established protocol had to be adopted [
The statistics for data collected from the temperature sensors in the refrigeration room were:
Maximum temperature (in ºC):
Probe 1 (P1): mean 7.2 (SD 0.1) (CI 95% 6.9-7.5); robust average 7.1; median 6.9; maximum 11.8; minimum 5.5; and IQI (6.1-7.5)
Probe 2 (P2): mean 7.0 (SD 0.1) (CI 95% 6.7-7.3); robust average 6.9; median 6.7; maximum 11.0; minimum 5.2; and IQI (5.9-7.2)
Comparison of means between P1 and P2 was
Minimum temperature (in ºC):
Probe 1 (P1): mean 4.1 (SD 0.1) (CI 95% 3.8-4.4); robust average 4.1; median 3.8; maximum 7.4; minimum 1.4; and IQI (3.3-4.8)
Probe 2 (P2): mean 3.8 (SD 0.2) (CI 95% 3.5-4.2); robust average 3.7; median 3.4; maximum 7.7; minimum 1.1, and IQI (3.0-4.6)
Comparison of means between P1 and P2 was
The geolocation function provided by the app made it possible to know the location, day, and hour of administration of every PN bag (see
Maximum and minimum temperatures measured by the 2 probes in the refrigeration chamber from June 1, 2016 to September 6, 2016.
Collected data about the parenteral nutrition administration, including location, date, and time of administration. PN: parenteral nutrition.
The evaluation and validation performed verified the ability of the mHealth app to register all designed processes and record nonconformities that occurred. Therefore, it was found that holistic monitoring of the traceability and management system of the PN could be implemented.
The results of the evaluation of the mHealth app through a survey revealed that, in general, users assessed its usability positively. The 4 dimensions measured (general satisfaction, app utility, information quality, and interface quality), on average, tended to the highest values of the Likert scale used. This means that the professionals considered the mHealth app useful in their work, and they found it easy to manage even without technological experience. The calculated Cronbach alpha (with values close to 1) allowed us to be sure about the consistency of the evaluation test (ie, the absence of errors in the measurements made). This result showed that the positive perception of the app is common to all the users who took part in the process.
The validation of the system allowed us to observe the monitoring and registration of all stages of the PN system. As already stated in the scientific literature, compliance with protocols minimizes unjustified clinical variability, which directly affects the reduction of risks associated with PN [
The characterization and quantification of CCPs was essential to know to what extent the process designed met the quality standards (scope or degree of compliance with the standards) [
All bags prepared were of volumes greater than 100 mL, so the margin of error of actual weight compared to theoretical weight should not exceed 5%. These results are in line with previous studies on gravimetric control in the PN process [
In the administration stage, characterized mainly by the speed or rate of infusion of PN (volume to be infused per time), there are no specific regulations. Usually, only recommendations based on the clinical situation of the patient and device used for administration are established.
A margin of ±1 hour of the scheduled infusion time is a sensible recommendation, and the reasons for larger deviations should be evaluated [
Other data on administration, such as location, date, and time, made it possible to collect very important data on control of the processes and holistic traceability of the PN. A large number of errors associated with the use of PN occur at the administration stage. Therefore, every stage of the process must be properly followed and managed. It is appropriate to establish procedures, especially in home care, that provide alerts about unwanted deviations [
The use of a geographic information system made it possible to be certain about the disposition of the PN and get additional information about administration of the PN. In home care, the app would allow us to be aware of the actual arrival date of the PN at the patient’s home [
But mobile apps not only help the adherence to treatment; the traceability also contributes to minimizing security issues related to custody of the data and controlling the integrity of the medicine itself [
In any case, in a context of progressive penetration of mobile apps for the management and traceability of medicines, it is desirable for health professionals to contribute by increasing health literacy (necessary to adequately recognize risks) and proper decision-making to take advantage of the opportunities inherent in Medicine 2.0 [
Furthermore, as has been demonstrated in previous projects, the use of mobile apps designed to guide the medication-related processes significantly reduces, compared to conventional preparation methods, the medicine preparation time, administration time, and rate of medication errors [
From the point of view of the authors, it is clear that the use of mobile apps for the management of medication, including parenteral nutrition, is a hot research topic with a bright future [
To control the volume being administered, it is difficult to establish a margin. The PN has a particular volume. Therefore, the potential hazard is linked to infusions done in less time than established, which would generally be due to a failure in the infusion pump [
In its recommendations [
From the results obtained, we have concluded that the developed app successfully passed the evaluation and validation tests and, more importantly, properly completed the procedures for monitoring the overall PN process. In addition, the repository of records allowed us to see the deviations at all times. Therefore, a system for the management and traceability of PN that quantifies the main control points at a low cost has been successfully implemented and verified at HUSE in Palma, Spain.
A new mobile solution to manage the quality and traceability of sensitive medicines such as blood-derivative drugs and hazardous drugs derived from this project is currently being deployed.
critical control point
Son Espases University Hospital
information and communication technology
interquartile interval
operative control point
parenteral nutrition
Poststudy System Usability Questionnaire
This work has been partially funded by the Instituto de Salud Carlos III (Spain) under project PI13/00464 and by the European Regional Development Fund “A way of making Europe.”
None declared.