A descriptive qualitative study was employed. The first author recruited women from the intervention group in an RCT examining the effectiveness of the Breast Cancer e-Support program. Women were eligible for the main study if they had commenced chemotherapy at the study sites after diagnosis of breast cancer, were able to access the internet via a mobile phone, and were able to read and write Mandarin. The study was approved by the institutional review board of Xiamen University affiliated Zhong Shan Hospital (zhu20151023), Central South University affiliated Hunan Cancer Hospital (zhu20151026) in China, and the University of Newcastle in Australia (H-2015-0448). Written consent forms were obtained from all participants.

A website was developed, with the introduction of the Breast Cancer e-Support program and the two-dimensional code for women to scan (Figure 1) [22]. Once women downloaded the app, the home page displayed a button for the user to submit an application that the first author (JZ) either approved or declined through the app background thread to ensure only the intervention group had access to the Breast Cancer e-Support program with a unique username and self-set password. This application process prevented contamination between the two groups and protected the privacy of participants in the Breast Cancer e-Support program.

The development process of the Breast Cancer e-Support program applied user-centric design and assessed the perceived ease of using the app [15]. This program supported women for 12 weeks covering four cycles of chemotherapy (1st-4th cycle of chemotherapy), with each cycle lasting 3 weeks. The Breast Cancer e-Support program had four components: Learning forum, Discussion forum, Ask-the-Expert forum, and Personal Story Forum [16]. The Learning forum provided knowledge and symptom management strategies related to breast cancer and chemotherapy. The Discussion forum offered an online platform for women to chat with one another. The Ask-the-Expert forum served as a communication channel for women to ask questions and receive advice directly from experts within 24 hours. The Personal Story forum presented five encouraging stories of other women who had successfully overcome obstacles during chemotherapy. On the basis of women’s topics and questions, the information in the learning forum was updated every 2 weeks to address women’s concerns. The Discussion forum and Ask-the Expert forum were moderated by the first author (JZ) who is an HCP. The moderator facilitated the online discussions and sent a reminder message to the doctors with incoming questions. Eight doctors from the study sites joined this program and responded to the questions in the Ask-the Expert forum. Technical assistance was available for the workday. The details of the Breast Cancer e-Support program can be found in the study protocol published in BMC Cancer [16].

For the process evaluation, we applied purposive sampling based on scores in the primary outcome of self-efficacy at 3 months. The Stanford Inventory of Cancer Patient Adjustment was used to assess women’s self-efficacy to manage cancer-related problems [23]. The Stanford Inventory of Cancer Patient Adjustment contained 38 items rated from 0 (not at all confident) to 10 (absolutely confident). Higher total scores represent higher self-efficacy. We selectively approached women with higher and lower (ratio 1:1) scores than the mean score of self-efficacy. We achieved data saturation at the 10th participant. No additional data were yielded with three additional interviews, giving a total sample of 13 women [24]. Of the 13 interviewed women, 7 participants were higher and 6 participants lower than the mean score of self-efficacy.

Interviews were conducted from November 2016 to February 2017. As these two participating hospitals are in two different provinces in China, we undertook face-to-face interviews for women in Zhong Shan Hospital and telephone interviews for women in Hunan Cancer Hospital to achieve a sufficient sample size. The first author (JZ), who was a female PhD student in nursing and who had established a good relationship with women during moderating the online discussion, invited women by telephone to participate, giving a clear explanation of the aims of the interview. Women were informed that the interview would take 10 to 20 min and that the interview would be audiorecorded. A total of 18 women were approached. Five women declined to be audiorecorded and thus excluded. A total of 5 women from Zhong Shan Hospital were interviewed one-to-one in a meeting room in the hospital. Field notes were taken during the interviews. A total of 8 women from Hunan Cancer Hospital were interviewed by telephone. The duration of the interviews ranged from 7 to 27 min.

A semistructured interview guide (Table 1) with five questions was prepared for the interview. Two academic experts in psychoeducation and cancer care provided comments on clarity and content of the interview guide.

Inductive content analysis was used to analyze the transcripts, allowing the categories to flow from the data [25]. Qualitative content analysis was conducted in the original language, following Graneheim and Lundman’s [26] steps for analysis procedures to achieve trustworthiness. Audiotapes were transcribed verbatim into written Chinese by the first author immediately after the interview. The first (JZ) and third authors (DG) performed the initial coding independently. Transcripts and field notes were repeatedly read through to obtain a broad picture of the whole. Transcribed data were coded and analyzed. The various codes were compared regarding the differences and similarities, and interpretations of the codes were sorted into subcategories. Finally the underlying meanings of the subcategories were grouped into categories. The research team had regular meetings to examine and revise the tentative subcategories and categories. Any coding discrepancies were discussed to ensure consensus. Two experts in qualitative research reviewed the process of data collection and analysis. Participant’s quotes were translated into English. English translation of the results and direct quotes were checked by the last author who is bilingual (English and Chinese).

For all 114 participants in the RCT, the mean of self-efficacy score at 3 months was 212.1 (SD 58.1; range 74-368). The mean age was 47.2 years. The majority of women were married (96.5%, 110/114), had received elementary or middle school (57%, 65/114) education, and were currently unemployed (74.6%, 85/114). Only 14 women (12.3%, 14/114) lived in a household with above average level of monthly family income (≥US $739) [27]. The majority of women were diagnosed with breast cancer stage II (43.0%, 49/114), followed by stage III (36.7%, 42/114). A total of 97 women (85.1%, 97/114) had undergone a mastectomy, and 5 (4.4%, 5/114) had chosen breast conserving surgery. The majority of women received eight cycles (51.8%, 59/114) and six cycles (21.1%, 24/114) of chemotherapy.

In total, 13 women in the intervention group participated in the interview, with mean score of self-efficacy at 3 months as 218.58 (SD 68.1; range 108-349). These 13 women had comparable demographic or clinical characteristics as all participants in the RCT, except for education level (χ²_0.05,4=10.8, P=.01). The interviewed women had significantly higher education level than all participants in the RCT. The demographic or clinical variables of the 13 participants are summarized in Table 2.

From the content analysis, four categories emerged: (1) benefits of participation, (2) challenges to engagement, (3) suggested improvement, and (4) future direction. All the categories and subcategories are summarized in Textbox 1.

In this study, we evaluated participants’ perception of an app-based program that they had access to during four cycles of chemotherapy. This process evaluation supported the quantitative results of the RCT that the Breast Cancer e-Support program was useful and feasible [19]. This study helped in understanding what challenges exist with the engagement of the Breast Cancer e-Support program. Despite the challenges, the participants provided constructive suggestions for improvement related to the design, content, and operation of this program. The participants recommended the Breast Cancer e-Support program be incorporated as routine care to support women during chemotherapy.

Women in this study reported that the Breast Cancer e-Support program was helpful in enhancing knowledge, improving confidence level, and promoting emotional well-being. They valued the tailored advice from experts. These findings could be explained by the theoretical framework of this program: Bandura’s self-efficacy theory [17] and the social exchange theory [18]. Self-efficacy, an individual’s perceived ability to perform a particular task in a given situation, is an important concept that influences women’ ability to manage their disease and chemotherapy [17]. Four factors such as direct mastery experiences, vicarious experiences, verbal persuasion, and perception of their physiological state from Bandura’s self-efficacy theory [17] were addressed in the Breast Cancer e-Support program by offering knowledge and skills, providing hopeful stories, and encouragement from peers and HCPs. In addition, supported by social exchange theory, the Breast Cancer e-Support program provided a channel to connect women with structural support and a variety of functional support, which are identified as the essential components of social support [18].

Many women stated that the Breast Cancer e-Support program was easy to use and convenient. Compared with traditional face-to-face interventions, connecting with the Breast Cancer e-Support program meant that women were able to access information and social support at anytime and anywhere [28]. Taking advantage of the app’s technical capabilities, this program provided women the opportunity to learn tailored information, watch encouraging stories, and chat with peers and HCPs [16]. Even though the majority of women enjoyed the convenience and multifunctional ability of the Breast Cancer e-Support program, one woman identified the app instability as a challenge to the engagement of this program. It is hard to determine whether the app instability was because of the internet connection or the app itself. But still, there is a need to improve the app reliability and to offer sufficient technical support service to participants [29].

This study revealed that some participants felt too sick or in a low mood to use the Breast Cancer e-Support program during chemotherapy. Consistent with our study, the association between poor health status and infrequent computer use was reported [29,30]. Furthermore, Chinese women might fail to disclose their negative emotions when confronting a life-threatening disease [31], which may have prevented participants in this study from sharing their feelings in the Breast Cancer e-Support program. In traditional Chinese culture, family members are obliged to take care of the patients [32]. Family-centered approach should therefore be adopted in the future app-based studies, so that a family member can also use the app in need when women are experiencing severe physical and psychosocial symptoms.

In our study, stigma related to breast cancer was reported to prevent the utilization of this program. Culturally, Chinese women are not willing to talk openly about breast cancer because of the perceived social rejection and low self-esteem associated with being ill and the disfigurement of breast surgery [31]. Women’s concerns on stigma may also hinder their emotional expression and support seeking behaviors in this program, thus reducing their engagement. Culturally sensitive education and peer support were reported to help reduce stigma and increased a sense of belonging for Chinese women with breast cancer [33]. Integration of culturally tailored support regarding stigma into the Breast Cancer e-Support program may alter women’s perception on stigma, thus improving their engagement.

During process evaluation, women also gave constructive suggestions for improvement in design, content, and operation of the Breast Cancer e-Support program. Design improvement involved adding message reminders to prompt instant communication with peers and shorten the response time from the experts, as well as adding a search engine function to help quickly locate information. Content improvement included supplementing more interesting and practical knowledge (eg, more pictures, cartoons, and food choices), plain language used, and short and concise videos to convey information. Our study indicates the demand for health service that optimizes on instant communication and improves efficiency on information delivery [34]. Operational improvement referred to knowledge being updated more often and questions being responded to more quickly. HCPs’ involvement, such as knowledge updating and timely feedback, is crucial to reinforce women’s app engagement [34]. In recent years, user-centered design has gained recognition in app development [35], and user-friendliness has been incorporated in the app assessment [36,37]. Thus, these valuable comments from users’ experiences identified in our study should be addressed in future app designs and operation to encourage app engagement.

In our study, half of the interviewed women emphasized the importance of diet guidelines during chemotherapy. They believed that food choice was very practical, and this program should add more diet details. Diet is imbued with Chinese culture [38], and the need for more specific diet information was considered important. Traditional Chinese medicine is evident regarding diet and food properties among Chinese women with breast cancer undergoing chemotherapy [39]. Specific food recommendations that are culturally sensitive should be addressed for future app-based programs targeting different sociocultural groups.

Many participants recommended the incorporation of this program as routine care during chemotherapy. Some suggested the Breast Cancer e-Support program should be generalized to other family members or other cancer patients. These recommendations indicated that women regarded this program useful and beneficial to other patients. With the increasing growth of mobile phone users [12], it is possible for all patients and their family caregivers to have such an app service available. Moreover, benefiting from the rapid advances in mobile technologies [7], future app-based intervention may convey information via multimedia for illiterate and old patients.

This study has limitations. This study was limited to those who were able to access the internet via a mobile phone and who were willing to be interviewed. This inclusion criteria may have narrowed the participant cohort to those who were technologically comfortable with mobile phone use and who had a high level of motivation to share their experiences. Moreover, there might be bias of the findings as the interviewed women were better educated than all participants in the RCT. Our results may not be representative of the views of all women with breast cancer in general. Two women in the intervention group who never accessed the program were not interviewed (one lost contact, one refused). Therefore, we were unable to explore further the reasons for nonuse of the app-based program. The strength of the study is that this process evaluation gathered rich qualitative data of participants’ views on the benefits, challenges, improvement suggestions, and recommendation for an innovative intervention.

The study provides evidence on the benefits of the Breast Cancer e-Support program and contributes to an increased understanding of some challenges of why women could not engage in this program as much as expected. This study also discussed some improvement suggestions and future direction of the app-based program from participants’ view. Such knowledge is crucial, and future app-based interventions could address participants’ improvement suggestions on design and operation, apply family-centered approaches, and provide more support on stigma associated with the condition to encourage the app engagement and provide better support.

Our study also has global significance because apps are being increasingly recognized as supplementary interventions when the feasibility of traditional face-to-face interventions are challenged. This app has the potential to be used by Chinese women in Australia and also be translated to other languages to help the culturally and linguistically diverse groups to promote their health outcomes. The knowledge generated from the study can be used to develop guidelines for future health care app development.