Published on in Vol 7, No 7 (2019): July

Preprints (earlier versions) of this paper are available at, first published .
Electronic Patient-Reported Outcome Measures in Radiation Oncology: Initial Experience After Workflow Implementation

Electronic Patient-Reported Outcome Measures in Radiation Oncology: Initial Experience After Workflow Implementation

Electronic Patient-Reported Outcome Measures in Radiation Oncology: Initial Experience After Workflow Implementation

Original Paper

1University Hospital Tübingen, Department of Radiation Oncology, Tübingen, Germany

2University Hospital Tübingen, Section for Information Technology, Tübingen, Germany

3University Hospital Tübingen, Internal Medicine I, Tübingen, Germany

4German Cancer Research Center (DKFZ) Heidelberg and German Consortium for Translational Cancer Research (DKTK), Partner Site Tübingen, Tübingen, Germany

Corresponding Author:

Cihan Gani, MD

University Hospital Tübingen

Department of Radiation Oncology

Hoppe-Seyler-Str 3

Tübingen, 72076


Phone: 49 70712986142


Background: Mobile health (mHealth) technologies are increasingly used in various medical fields. However, the potential of mHealth to improve patient care in radiotherapy by acquiring electronic patient reported outcome measures (ePROMs) during treatment has been poorly studied so far.

Objective: The aim of this study was to develop and implement a novel Web app (PROMetheus) for patients undergoing radiotherapy. Herein, we have reported our experience with a focus on feasibility, patient acceptance, and a correlation of ePROMs with the clinical course of the patients.

Methods: In the period between January and June 2018, 21 patients used PROMetheus to score side effects, symptoms, and quality of life–related parameters during and after their treatment. Items of the Patient Reported Outcome version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) were chosen based on the primary site of disease, 27 items for head and neck tumors, 21 items for thoracic tumors, and 24 items for pelvic tumors.

Results: In total, 17 out of the 21 patients (81%) regularly submitted ePROMs and more than 2500 data points were acquired. An average of 5.2, 3.5, and 3.3 min was required to complete the head and neck, thorax, and pelvis questionnaires, respectively. ePROMS were able to detect the occurrence of both expected and unexpected side effects during the treatment.
In addition, a gradual increase in the severity of side effects over the course the treatment and their remission afterward could be observed with ePROMs. In total, 9 out of the 17 patients (53%), mostly those with head and neck and thoracic cancers, reported PRO-CTCAE grade III or IV fatigue with severe impairments of activities of daily life.

Conclusions: This study shows the successful implementation of an ePROM system and a high patient acceptance. ePROMs have a great potential to improve patient care in radiotherapy by providing a comprehensive documentation of symptoms and side effects, especially of ones that are otherwise underreported.

JMIR Mhealth Uhealth 2019;7(7):e12345



Mobile Health and Patient Reported Outcomes

Mobile health (mHealth) is a rapidly growing field and has, according to the World Health Organization, the potential to transform the face of health service delivery across the globe [1]. In oncology, there has been great interest to use mHealth technologies for the acquisition of patient reported outcome measures (PROMs), which are then termed electronic PROMs (ePROMs) [2-5]. A variety of studies have shown significant benefits of PROMs in terms of improved communication, patient well-being, detection of unrecognized problems, and also, most strikingly, long-term survival [6-9]. ePROMs bring the additional advantage of the immediate availability of PROMs, avoidance of data entry errors, or the possibility of triggering notifications. It has been shown that the data acquired do not differ between a traditional paper- and pencil-based assessment and an acquisition via ePROMs [10]. Recently, this was again validated by Matthies et al, who reported highly significant correlations between a paper-based version of the Functional Assessment of Cancer Therapy—Breast questionnaire and an electronic version designed for breast cancer patients [11]. For patients undergoing radiotherapy, data regarding the use of ePROMs are sparse despite surveys showing a considerable interest to use mobile technologies in clinical practice, both on the caregiver and patient side [12,13].

Electronic Patient Reported Outcome Measures in Radiotherapy

In radiotherapy, where the majority of treatments are performed in an outpatient setting, ePROMs might be a useful tool to monitor acute toxicities during therapy and shortly afterward, as well as late toxicities. Furthermore, signs of disease recurrence or progression might be detected at an earlier stage. We developed a Web-based application, PROMetheus, that allows patients to submit ePROMs over the internet to the treating team. We hypothesized that ePROMs will be well accepted by our patients and provide a complete and comprehensive documentation of side effects and quality of life (QoL)–related parameters during radiotherapy. In this study, we report our initial experience with a focus on feasibility, patient acceptance, and a correlation of ePROMs with the clinical course of the patients.

Participants and Recruitment Process

In the period between January 2018 and June 2018, 21 patients used PROMetheus to score symptoms and QoL-related parameters. Patients who had provided an email address in their demographic data were approached by a physician before treatment or latest during the first week of treatment and asked if they were interested to use PROMetheus. All the patients who were offered to use PROMetheus provided consent. If requested, the first scoring was completed under supervision. After this, the patients were instructed to complete a Web-based questionnaire whenever desired but at least once weekly. A reminder to complete the questionnaire was sent once a week via email. In general, patients were approached by the treating physician whenever Patient Reported Outcomes-Common Terminology Criteria for Adverse Event (PRO-CTCAE) grade IV toxicity was reported on the Web or when a 2-point increase from grade I to grade III was observed. ePROMs were reviewed daily by the principal investigator of the study (CG). We had defined that no medical intervention or treatment would be initiated solely based on ePROMs without a confirmatory interaction of the treating physician with the patient. Irrespective of the ePROM submission, patients had weekly consultations with the treating physician to assess toxicity according to our institutional standard. If a patient was admitted for inpatient treatment, the patient was asked to continue the ePROM submission and email reminders were continued. Free Wi-Fi access was offered to all inpatients in the study. This study was approved by the institutional review board of the medical faculty in Tuebingen, Germany (approval number: 421/2018B02).

Technology Platform Development

PROMetheus was developed as a progressive Web application using HTML5, CSS3, and JavaScript and is accessible through browsers on all internet-compatible devices such as smartphones, tablets, or computers. Patients log in with an alphanumeric pseudonym and a password and are then immediately forwarded to the first item. Figure 1 depicts a screenshot of PROMetheus. PROMetheus is built on the Integrated Mobile Health Research Platform (IMeRa), which facilitates the submission of data from the patient to the hospital network over the internet. IMeRa provides a Secure Sockets Layer–encrypted Web service for Web applications. Via this Web service, the patients’ answers are transmitted in the Mobile Data Repository for Research of the IMeRa platform (based on an Oracle Relational Database Management System [RDBMS]). The IMeRa platform also provides the physician with a Web-based data browser where patient data can be selected, displayed, and exported for further statistical evaluations.

Figure 1. Screenshot of PROMetheus during usage (functions of the buttons: top left=sign out, top right=help, bottom left=back, bottom center=upload all items answered so far, and bottom right=next). The question displayed assesses the severity of shortness of breath with a 5-tier scale from not at all to very.
View this figure

Structure of the Questionnaires

The questionnaires fed into PROMetheus were based on the certified German translation of the PRO-CTCAE developed by the National Cancer Institute [14]. In short, the questions assess severity, frequency, and impact on activities of daily life (ADL) on a 5-tier scale from none to very severe or from never to almost always based on the average of the last 7 days. Questions referring to the severity or the impact on ADLs of an item were automatically skipped if the given symptom was not present. We defined 3 different sets of questions for the included treatment categories: head and neck (up to 27 questions), thoracic (up to 21 questions), and abdominal (up to 24 questions) tumors, respectively (Multimedia Appendices 1-3). For head and neck cancer patients, we individually developed 3 items that assessed pain and swallowing. Patients had the opportunity to discontinue and submit the completed items at any point; however, it was not possible to skip questions.

Data Analysis

Regarding patient acceptance, we defined ePROMs as accepted by a patient if the ePROMs were submitted at 5 time points at least, which approximately corresponds to weekly submissions. For descriptive data, median values and interquartile (25th to 75th quantile) ranges (IQR) are reported. For statistical analysis SPSS version 25 (IBM) and Microsoft Excel were used.

Patient and Treatment Characteristics

Patient characteristics are shown in Table 1. All the patients (n=21) completed radiotherapy with a median treatment dose of 60 Gy (IQR 50.4 Gy to 64 Gy). Patients received either radiotherapy alone (5 patients) or radiochemotherapy (16 patients). In total, 12 patients received the concomitant systemic treatment in a preplanned inpatient setting. In most cases, this inpatient treatment took place during the first and fifth week of treatment, whereas treatment was conducted in an outpatient setting on the remaining days. No admissions for inpatient treatment occurred because of toxicity.

Table 1. Patient and treatment characteristics.
Total number of patients, N (%)21 (100)
Sex, n (%)

Male14 (67)

Female7 (33)
Age (years), median (IQRa)59.4 (51.5-66.5)
Treatment, n (%)

Radiotherapy5 (24)

Radiochemotherapy16 (76)
Radiotherapy dose (Gray), median (IQR)60 (50.4-64)
Primary site of cancer, n (%)

Pelvic10 (48)

Thoracic5 (24)

Head and neck4 (19)

Upper gastrointestinal2 (10)
Stageb, n (%)

I1 (5)

II5 (24)

III13 (62)

IV1 (5)

aIQR: interquartile range.

bOne patient with a tumor of an unknown primary site was excluded.

Feasibility and Patient Acceptance

ePROM acquisition with PROMetheus was feasible. None of the patients reported any technical issues that prevented the submission of ePROMs or problems in understanding the usage of PROMetheus. In terms of patient acceptance, 17 out of 21 patients (81%) regularly submitted ePROM data during treatment, with a median of 6 (IQR 4 to 8) submissions (Figure 2).

The patients required a median of 5.45 min (IQR 4.65 to 5.91 min), 2.39 min (IQR 2.38 to 4.78 min), and 2.95 min (IQR 2.72 to 3.36 min) for the completion of the head and neck, thorax, and abdominal questionnaires, respectively. By including the submissions after the end of treatment, we were able to collect more than 2500 data points in these 21 patients. In all the submitted questionnaires, all items had been completed.

Figure 2. Time points with electronic patient-reported outcome measure (ePROM) submissions of the 21 patients in our cohort. d0 (green) indicates the day of the first treatment, time points filled in red indicate the last day of treatment. Red bars with a dash indicate ePROM submission on the last day of treatment. d: day.
View this figure

Clinical Examples of Electronic Patient-Reported Outcome Measures Assessed During and After Treatment

Figure 3 shows an example of a 75-year-old patient who was treated for a locally advanced tumor at the base of the tongue (cT2 cN2b cM0) using definitive radiotherapy with 70 Gy and concomitant chemotherapy with weekly Cisplatin (40 mg/m²). A percutaneous endoscopic gastrostomy (PEG) tube was placed 2 days before the initiation of the treatment. Treatment toxicity and impairment of ADL continuously increased with treatment time. Taste changes and lack of appetite were the PROMs to reach grade III or higher. Three weeks after the end of treatment, all high-grade toxicities had resolved.

ePROMs from 2 other patients with rectal cancer and gastric lymphoma are shown in Multimedia Appendices 4 and 5. The first patient (male, aged 50 years) was treated using preoperative radiochemotherapy (50.4 Gy) with concomitant 5-fluorouracil during the first and fifth week for locally advanced rectal cancer. The only PRO-CTCAE grade IV toxicity during the treatment was the severity of vomiting reported on the third day of radiochemotherapy. Further investigation revealed a norovirus as the cause of this toxicity. The other patient was a 55-year-old female diagnosed with mucosa-associated lymphoma tissue (MALT) lymphoma of the stomach. Treatment using radiotherapy with a total dose of 39.6 Gy was planned, and prophylactic intravenous antiemetics were offered. On the basis of the patient’s request, only oral antiemetics were prescribed and ePROMs were assessed during the treatment. No nausea or vomiting of PRO-CTCAE grade III or IV was scored during the treatment. Instead, fatigue and sleeping problems were the highest scored items. The patient completed the treatment without any intravenous treatment.

Figure 3. Example of the course of electronic patient reported outcome measures in a 75-year-old patient who underwent definitive radiochemotherapy for locally advanced head and neck cancer. Day 45 constitutes the last assessment during therapy; d76 was approximately 3 weeks after the end of treatment. d: day; ADL: activities of daily life.
View this figure

Fatigue as the Most Frequently Reported Single Item

With regard to fatigue, 9 out of the 17 patients (53%) who regularly submitted ePROMs reported PRO-CTCAE grade III or IV fatigue at the end of the treatment; in all cases, impairment of ADLs was scored as PRO-CTCAE grade III or IV as well. All the patients except one with head and neck or thoracic cancer had PRO-CTCAE grade III or IV fatigue, whereas only 1 patient with a pelvic primary had scored high-grade fatigue.

Principal Findings, Strengths, and Limitations

This study is one of the first to report ePROMs for the assessment of treatment toxicity and QoL aspects in patients undergoing radiotherapy. We observed high acceptance with over 80% of the patients regularly submitting complete ePROM datasets. We believe that this was facilitated by the very simple setup of PROMetheus and the weekly reminders for the completion of the questionnaires, which confirms the results of a patient survey conducted by El Shafie et al who reported a great interest in patients to use mobile technologies during radiotherapy and thereafter [12]. The potential benefits of ePROMs are manifold. First, concerns regarding a potential underreporting of toxicities in clinical trials have repeatedly been expressed [15]. Usually, toxicity during treatment is evaluated by a physician at regular intervals and entered into the patient’s chart, either already graded according to a standard scoring system or as descriptive text. In case of the latter, this information needs to be translated into a grading system (potentially by a third person) if a graded score is required, for instance, for entry into a research database. All these individual steps are prone to errors or loss of crucial information [16,17]. With ePROMs, this information is immediately available as digital and parametric data. Furthermore, patients can go through the items at home or in the waiting area, without the stress associated with a doctor’s visit. It has been suggested in many studies that self-reporting increases patient contentedness and potentially enables physicians to recognize adverse events at an earlier time point [18,19]. Clearly, a limitation of this study is the sample size, which does not permit detailed subgroup analyses. The rationale for the sample size was to evaluate the feasibility and patient acceptance at an early time point before moving forward to large-scale randomized trials. Even though the patient cohort in this study is small, it represents patients with various tumor entities and age groups.

Further Benefits of Electronic Patient-Reported Outcome Measures

Self-reporting may also improve communication between physicians and patients, as investigated by Velikova et al who showed in a randomized trial that regular assessment of QoL improved patient-physician communication even resulting in better emotional functioning and overall QoL [7]. We made the experience that offering a Web-based PROM solution lowered the patients’ threshold to contact the treating physician, for instance, via email when symptoms occurred that were not assessed by the predefined questionnaires. For example, 1 patient with nonsmall cell lung cancer and receiving definitive radiochemotherapy developed severe lower back pain only a few weeks after the end of treatment. As no PRO-CTCAE item in our questionnaires assessed pain outside the treated area, the patient submitted this symptom via email, which prompted a face to face assessment of pain characteristics, physical examination, and, subsequently, a computed tomography within a few days, confirming the suspected metastatic disease and the swift initiation of systemic treatment. It appears likely that the early detection of disease progression, as in our exemplary case, played a crucial role in a recently published study of ePROMs in the care of cancer patients undergoing palliative chemotherapy. In this randomized trial of 766 patients, in which the sole intervention was the inclusion of a Web-based ePROM platform in the experimental arm, a 5-month benefit in overall survival was seen [8].

We consider ePROMs as a very useful complement to face-to-face patient-physician interaction and not a replacement. First, according to a recent survey, a considerable number of patients would refuse ePROM usage because of the wish for personal contact with the treating physician [13]. Second, and most importantly, self-reported high-grade toxicities and symptoms scored with ePROMs need clinical validation before any diagnostic or therapeutic intervention is initiated. This is well reflected by the mentioned case of a rectal cancer patient whose severe diarrhea and vomiting during the first week of treatment turned out to have a viral origin rather than being a treatment-related side effect (Multimedia Appendix 4). As we approached patients who had provided an email address during the registration process, our cohort has to be considered a positive selection in terms of computer experience. This is an important aspect, as Basch et al showed a correlation between Web usage before enrollment and log-in times during data acquisition [16]. On the contrary, the benefits of ePROM usage in terms of survival and frequency of emergency room visits have been shown to be independent of computer experience [19]. It is likely that the technological progress and the increasing incorporation of the internet into our daily life will soon limit the patient cohort that is either not able to or not willing to use ePROMs.

Content Validity of the Selected Items

Content validity is an important aspect in the context of PROMs. In an interview-based study among radiotherapy patients—published before we designed our questionnaires—Sandler et al found that all except 5 frequently reported toxicities were covered by PRO-CTCAE items. In total, 3 of these items were seen in patients with head and neck cancers (mucus production, oral pain, and pain when swallowing) [20]. Indeed, when we compiled the items for the 3 treatment sites, we saw the need to add additional items that assessed PEG usage and pain medication intake.

The Challenge of Big Data

A major challenge faced when implementing ePROMs in clinical practice is the huge amount of data, which needs to be reviewed. Notification systems that inform the treating team of either uncommon symptoms or critical severity of toxicity before meeting the patient need to be implemented. Defining these thresholds for all items already constitutes a challenge of its own and warrants further research [21-23].

Electronic Patient-Reported Outcome Measures and Cancer-Related Fatigue

It is well known that cancer-related fatigue (CRF) is an underreported symptom in patients undergoing treatment [24]. Conventional symptom assessment by clinicians has been shown to diagnose subjective symptoms, such as fatigue, less frequently than self-reporting using PROM items [25]. Indeed, almost all patients who received treatment of the head and neck or thoracic tumors self-reported a severe level of fatigue. Even more important is that all these patients considered their fatigue to be impairing their activities to a high degree. Several groups were able to show that exercise may have an impact on the severity of CRF [26]. A meta-analysis by Tomlinson et al reported a positive effect of exercise on fatigue as well as depression and sleep disturbance. At our department, we are currently evaluating the potential of commercially available activity trackers to tackle CRF and to implement ePROMs as a tool to monitor CRF [27,28].

Optimization of Normal Tissue Complication Probability Models Through Electronic Patient-Reported Outcome Measures

The large amount of data that accumulate when PROMs are implanted into the clinical workflow can have a much broader use than the sole monitoring of PROMs. Weighing the likelihood of acute and long-term toxicities against tumor control probabilities is the daily routine in radiotherapy. Normal Tissue Complication Probability models have been established to estimate the likelihood of a specific toxicity [29]. The majority of these models are based on clinician-assessed endpoints with the associated limitations discussed earlier. One can envision that PROM data could be used to further refine these models and improve their ability to predict long-term toxicities, similar to an approach presented by Miften et al [30]. Here, the selection of the adequate items is crucial, and a recent review has shown that only a minority of PROM-based models is accurate or can be generalized on external validation [31].

In conclusion, our results show that the implementation of an ePROM system for the assessment of treatment side effects and QoL during radiotherapy is feasible and well accepted by the patients. We, therefore, consider ePROMs as a very useful tool to complement face-to-face patient-physician interaction. However, randomized trials will have to prove whether a measurable benefit through ePROMs in terms of QoL, side effects, or even survival can be achieved [32]. Furthermore, the development of strategies to handle the large amounts of data is among the major challenges that need to be addressed in the future.


The IMeRa platform, which facilitated this project, is funded by the State Ministry of Baden-Wuerttemberg for Sciences, Research, and Arts funding line escience Baden-Württemberg, Virtual Research Environment. CG is partly supported by the Clinician Scientist Program of the Medical Faculty, Eberhard Karls University of Tübingen (funding number: 363-0-0). The authors would like to acknowledge the support provided by Deutsche Forschungsgemeinschaft and Open Access Publishing Fund of the University of Tübingen.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Questionnaire for patients with head and neck cancer.

PDF File (Adobe PDF File), 785KB

Multimedia Appendix 2

Questionnaire for patients with thoracic tumors.

PDF File (Adobe PDF File), 784KB

Multimedia Appendix 3

Questionnaire for patients with abdominal or pelvic cancer.

PDF File (Adobe PDF File), 786KB

Multimedia Appendix 4

Electronic patient-reported outcome measures of a 50-year-old male patient who received preoperative radiochemotherapy for locally advanced rectal cancer are depicted. The only grade IV score was the severity of vomiting at a very early stage of treatment. After a medical history, this turned out to be caused by a noroviral infection.

PDF File (Adobe PDF File), 38KB

Multimedia Appendix 5

Electronic patient-reported outcome measures of a 55-year-old female patient who underwent radiotherapy of the stomach for mucosa-associated lymphoma tissue lymphoma. Although nausea or vomiting were not an issue until the end of the treatment, fatigue and sleeping problems were the predominant symptoms during the treatment.

PDF File (Adobe PDF File), 36KB

  1. Pieterse AH, Stiggelbout AM, Baas-Thijssen MC, van de Velde CJ, Marijnen CA. Benefit from preoperative radiotherapy in rectal cancer treatment: disease-free patients' and oncologists' preferences. Br J Cancer 2007 Sep 17;97(6):717-724 [FREE Full text] [CrossRef] [Medline]
  2. Ishaque S, Karnon J, Chen G, Nair R, Salter AB. A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Qual Life Res 2019 Mar;28(3):567-592. [CrossRef] [Medline]
  3. Papadakos JK, Charow RC, Papadakos CJ, Moody LJ, Giuliani ME. Evaluating cancer patient-reported outcome measures: readability and implications for clinical use. Cancer 2019 Apr 15;125(8):1350-1356. [CrossRef] [Medline]
  4. Dobbs TD, Samarendra H, Hughes S, Hutchings HA, Whitaker I. Patient-reported outcome measures for facial skin cancer: a systematic review and evaluation of the quality of their measurement properties. Br J Dermatol 2019 May;180(5):1018-1029. [CrossRef] [Medline]
  5. Kim HJ, Kim SM, Shin H, Jang JS, Kim YI, Han DH. A mobile game for patients with breast cancer for chemotherapy self-management and quality-of-life improvement: randomized controlled trial. J Med Internet Res 2018 Dec 29;20(10):e273 [FREE Full text] [CrossRef] [Medline]
  6. Pakhomov SV, Jacobsen SJ, Chute CG, Roger VL. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 2008 Aug;14(8):530-539 [FREE Full text] [Medline]
  7. Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol 2004 Feb 15;22(4):714-724. [CrossRef] [Medline]
  8. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. J Am Med Assoc 2017 Dec 11;318(2):197-198 [FREE Full text] [CrossRef] [Medline]
  9. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 2013 Jun 11;13:211 [FREE Full text] [CrossRef] [Medline]
  10. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11(2):322-333 [FREE Full text] [CrossRef] [Medline]
  11. Matthies LM, Taran FA, Keilmann L, Schneeweiss A, Simoes E, Hartkopf AD, et al. An electronic patient-reported outcome tool for the FACT-B (functional assessment of cancer therapy-breast) questionnaire for measuring the health-related quality of life in patients with breast cancer: reliability study. J Med Internet Res 2019 Jan 22;21(1):e10004 [FREE Full text] [CrossRef] [Medline]
  12. El Shafie RA, Weber D, Bougatf N, Sprave T, Oetzel D, Huber PE, et al. Supportive care in radiotherapy based on a mobile app: prospective multicenter survey. JMIR Mhealth Uhealth 2018 Aug 30;6(8):e10916 [FREE Full text] [CrossRef] [Medline]
  13. Kessel KA, Vogel MM, Kessel C, Bier H, Biedermann T, Friess H, et al. Mobile health in oncology: a patient survey about app-assisted cancer care. JMIR Mhealth Uhealth 2017 Jun 14;5(6):e81 [FREE Full text] [CrossRef] [Medline]
  14. Healthcare Delivery Research Program. 2017. PRO-CTCAE Instrument & Form Builder   URL: [accessed 2019-06-19]
  15. Trotti A, Bentzen SM. The need for adverse effects reporting standards in oncology clinical trials. J Clin Oncol 2004 Jan 1;22(1):19-22. [CrossRef] [Medline]
  16. Basch E, Artz D, Dulko D, Scher K, Sabbatini P, Hensley M, et al. Patient online self-reporting of toxicity symptoms during chemotherapy. J Clin Oncol 2005 May 20;23(15):3552-3561. [CrossRef] [Medline]
  17. Gwede CK, Johnson DJ, Daniels SS, Trotti A. Assessment of toxicity in cooperative oncology clinical trials: the long and short of it. J Oncol Manag 2002;11(2):15-21. [Medline]
  18. Moradian S, Krzyzanowska MK, Maguire R, Morita PP, Kukreti V, Avery J, et al. Usability evaluation of a mobile phone-based system for remote monitoring and management of chemotherapy-related side effects in cancer patients: mixed-methods study. JMIR Cancer 2018 Dec 21;4(2):e10932 [FREE Full text] [CrossRef] [Medline]
  19. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016 Feb 20;34(6):557-565 [FREE Full text] [CrossRef] [Medline]
  20. Sandler KA, Mitchell SA, Basch E, Raldow AC, Steinberg ML, Sharif J, et al. Content validity of anatomic site-specific Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) item sets for assessment of acute symptomatic toxicities in radiation oncology. Int J Radiat Oncol Biol Phys 2018 Dec 1;102(1):44-52. [CrossRef] [Medline]
  21. Sloan JA, Cella D, Hays RD. Clinical significance of patient-reported questionnaire data: another step toward consensus. J Clin Epidemiol 2005 Dec;58(12):1217-1219. [CrossRef] [Medline]
  22. Sloan J, Symonds T, Vargas-Chanes D, Fridley B. Practical guidelines for assessing the clinical significance of health-related quality of life changes within clinical trials. Ther Innov Regul Sci 2003;37(1):23-31. [CrossRef]
  23. Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003 May;41(5):582-592. [CrossRef] [Medline]
  24. Vogelzang NJ, Breitbart W, Cella D, Curt GA, Groopman JE, Horning SJ, et al. Patient, caregiver, and oncologist perceptions of cancer-related fatigue: results of a tripart assessment survey. The fatigue coalition. Semin Hematol 1997 Jul;34(3 Suppl 2):4-12. [CrossRef] [Medline]
  25. Basch E, Iasonos A, McDonough T, Barz A, Culkin A, Kris MG, et al. Patient versus clinician symptom reporting using the national cancer institute common terminology criteria for adverse events: results of a questionnaire-based study. Lancet Oncol 2006 Nov;7(11):903-909. [CrossRef] [Medline]
  26. Kangas M, Bovbjerg DH, Montgomery GH. Cancer-related fatigue: a systematic and meta-analytic review of non-pharmacological therapies for cancer patients. Psychol Bull 2008 Sep;134(5):700-741. [CrossRef] [Medline]
  27. Gani C. 2018. Fitness Trackers During and After Oncological Treatments   URL: [accessed 2018-09-27] [WebCite Cache]
  28. Tomlinson D, Diorio C, Beyene J, Sung L. Effect of exercise on cancer-related fatigue: a meta-analysis. Am J Phys Med Rehabil 2014 Aug;93(8):675-686. [CrossRef] [Medline]
  29. Marks LB, Yorke ED, Jackson A, Ten HR, Constine LS, Eisbruch A, et al. Use of normal tissue complication probability models in the clinic. Int J Radiat Oncol Biol Phys 2010 Mar 1;76(3 Suppl):S10-S19 [FREE Full text] [CrossRef] [Medline]
  30. Miften M, Gayou O, Parda DS, Prosnitz R, Marks LB. Using quality of life information to rationally incorporate normal tissue effects into treatment plan evaluation and scoring. In: Rubin P, Constine LS, Marks LB, Okunieff P, editors. Cured I - Lent Late Effects Of Cancer Treatment On Normal Tissues, First Edition. Berlin: Springer; 2007:83-90.
  31. O'Callaghan ME, Raymond E, Campbell JM, Vincent AD, Beckmann K, Roder D, et al. Patient-reported outcomes after radiation therapy in men with prostate cancer: a systematic review of prognostic tool accuracy and validity. Int J Radiat Oncol Biol Phys 2017 Dec 1;98(2):318-337. [CrossRef] [Medline]
  32. Rincon E, Monteiro-Guerra F, Rivera-Romero O, Dorronzoro-Zubiete E, Sanchez-Bocanegra CL, Gabarron E. Mobile phone apps for quality of life and well-being assessment in breast and prostate cancer patients: systematic review. JMIR Mhealth Uhealth 2017 Dec 4;5(12):e187 [FREE Full text] [CrossRef] [Medline]

ADL: activities of daily life
CRF: cancer-related fatigue
ePROM: electronic patient-reported outcome measure
IMeRa: Integrated Mobile Health Research Platform
IQR: interquartile range
mHealth: mobile health
PEG: percutaneous endoscopic gastrostomy
PRO-CTCAE: Patient Reported Outcomes-Common Terminology Criteria for Adverse Events
QoL: quality of life

Edited by G Eysenbach; submitted 27.09.18; peer-reviewed by R El Shafie, P Johnson, S Rush; comments to author 10.01.19; revised version received 31.01.19; accepted 31.03.19; published 24.07.19


©Franziska Hauth, Verena Bizu, Rehan App, Heinrich Lautenbacher, Alina Tenev, Michael Bitzer, Nisar Peter Malek, Daniel Zips, Cihan Gani. Originally published in JMIR Mhealth and Uhealth (, 24.07.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.