Digital health innovations have gained support as a means to improve health service delivery while strengthening health systems [1,2]. Mobile technologies (mobile health, mHealth) for maternal health in low-resource settings can play a role in addressing information, skills, and resource needs at various points in the continuum from prenatal to postnatal care [1,3,4].

The majority of digital health innovations for maternal health involve use of short messaging services, voice calls, point-of-care diagnostics, and health information management systems [1,4]. Other less explored areas recently gaining attention include its use for clinical decision support and remote monitoring. This is particularly important in the context of poor road networks, remote geographical locations, weak referral chains, and alarming workforce shortages. Diagnostic and decision-support systems are a group of digital health innovations that aim to address challenges of timely and effective health care, using evidence-based principles [2]. Despite evidence of their importance for task shifting and promoting adherence to clinical practice guidelines, attempts to embed them into large-scale service structures are yet to be attained [4].

Evidence on mHealth has been critiqued for being a black box with little knowledge from pilot projects to inform prototype development and scale-up [5]. The dominant discourse is that low technological skills alongside infrastructural barriers are at the root of poor mHealth uptake in low- and middle-income countries (LMICs). An alternative and less explored explanation is that factors unique to the ecosystem of health service delivery need to be accounted for, motivating calls for knowledge on mHealth that is grounded in theoretical understanding [6-8]. A recent theory-based analysis on what works or not for mHealth in maternal health service delivery has shown that LMICs are still at the adoption-utilization stage [9]. In their review, Chib et al also highlight a knowledge gap on mechanisms for mHealth adoption and the role of theoretical explanations in addressing these gaps [8].

This study aims to identify causal factors underlying the variation in mHealth usage during the adoption and utilization phases of an intervention and understand how to overcome or leverage these in subsequent implementation cycles. Findings will contribute to the body of evidence on contextual and domain-specific applications of similar innovations in other low-resource settings.

In 2016, a consortium of 7 organizations representing a south-north public-private partnership embarked on a project to prove the viability of a modular integrated diagnostic kit tagged the Bliss4Midwives (B4M) device (unpublished data [10]). The B4M device supports instant informed diagnosis during antenatal care (ANC) by enabling noninvasive point-of-care screening for preeclampsia, gestational diabetes, and anemia—3 main screening components of ANC. The components of the device include a noninvasive hemoglobin reader with infrared sensors mounted on a finger clip, a self-inflating blood pressure cuff, and an automated urinary dipstick reader for measuring urinary protein and glucose. In the absence of B4M, target beneficiaries in remote areas would otherwise have to travel to other health facilities to conduct these tests, delaying timely detection and management of high-risk complications [11]. B4M was introduced in 7 health facilities in the upper east region and northern region of Ghana. Additional details on the device, project setting, viability, and beneficiary experiences have been reported elsewhere (unpublished data [10];[11]).

Although both users and beneficiaries valued B4M, results from the pilot showed wide variations in usage behavior and duration of use across project sites (unpublished data [10]). Beyond establishing viability of the intervention, application of a theory-based approach requires assessing why and how exactly it works [12]. In line with the long-term goals of the consortium, evaluation findings will inform the design and implementation of an adapted B4M prototype.

A total of 6 prototype devices were deployed in 7 predominantly rural locations—4 facilities in the upper east region and 3 in the northern region. A total of 25 maternal health workers were trained to operate B4M. As the device was withdrawn from 1 facility in the second month of the intervention, the evaluation focused on 6 of the 7 health facilities: facilities A to D in the upper east region and facilities E and F in the northern region. Facility A is the ANC unit of a district hospital and the first-level referral point for facilities B, C, and D, which are health centers. Facility E is an independent public health unit of a district hospital, whereas F is a health center. With the exception of facilities B and C, which shared a single B4M device on a rotating schedule, the other facilities had stable access to 1 device each.

We employed a multiple case study design, defining a case as 1 B4M health facility [13]. Informed by knowledge of the project, ANC volume per facility and trend analysis on adoption (first 2 months) and utilization (continued or prolonged use over time) of the device over a 10-month period (unpublished data [10]), health facilities were classified as low (average number of screenings <15 per month), moderate (average number of screenings ≥16 and ≤40 per month), or high (average number of screenings ≥41 and ≤75 per month) adoption and utilization (Table 1). Cases were subsequently grouped into 3 usage clusters: low, moderate, and high, whereby the term usage is a composite term describing adoption and utilization (Table 2). No health facility fell under the high usage cluster, which was recognized as the ideal state. The evaluation sought to understand usage variation between low and moderate usage clusters and reflect on how a high usage state may be attained in implementing an improved prototype.

Realist evaluation is a theory-based approach for opening the black box on complex interventions [14,15]. It has shown promise in unraveling explanations for complex interventions in health, international development, and technological innovation [16-18]. It involves an iterative process beginning and ending with program theories, systematically moving from the specific to the abstract, described as “climbing the ladder of abstraction” [19,16]. Realist methodology is suited for evaluating B4M because it is method neutral and can aid an in-depth understanding of the explanatory processes for program outcomes as well as in the identification of implicit and explicit mechanisms underlying them.

Due to its theoretical underpinning and applicability in real-life settings, realist methodology was applied to assess differences between low and moderate B4M usage clusters. This involved developing and subsequently testing initial program theories using qualitative and quantitative data. Identified causal explanations underlying variation in mHealth usage between clusters were framed in configurations that showed the interrelationship between the Intervention, implementation Context, participating Actors, explanatory Mechanisms, and Outcomes. Simply put, ICAMO configurations. Using this analytical heuristic, 2 main layers of context may be differentiated: the broad external environment in which interventions are situated (C₁) and the health system or health facility setting in which mobile technology is introduced (C₂). Where mechanisms broadly refer to the reasoning and responses to the B4M intervention underlying observed outcomes, main mechanisms (M) were differentiated from subexplanatory mechanisms (m).

The initial program theories of the B4M intervention, which describe how the intervention was expected to work, were developed using a 2-pronged approach:

Using quantitative and qualitative methods, the 3 candidate initial program theories were empirically tested. Data collection activities are presented in Multimedia Appendix 1 and summarily involved:

All interviews and meetings were conducted in English, audio recorded, and transcribed verbatim.

For the data on usability, negative statements were reverse coded, and raw scores were exported to SPSS. Nonparametric t test was used to compare perceived usefulness and perceived ease of use of B4M between clusters. Interview transcripts as well as observation and field notes were analyzed using realist thinking, applying an interpretive lens to build a casual web of explanations from multiple strands of evidence [22]. Using abductive inference, we started from the main outcomes of interest (adoption and utilization) and worked backward to trace plausible underlying explanations. We queried the data for mechanisms of perceived usefulness, perceived ease of use and empowerment (self-efficacy and confidence) for adoption, and the mechanism of recognition for utilization, while being open to new configurations.

A cumulative stepwise approach applying inductive and deductive reasoning was employed. First, aided by an Excel spreadsheet, we entered information on each health facility that spoke to elements of the ICAMO configuration into rows and columns, including supporting quotes. Furthermore, previous analysis has shown that over time, the intervention itself can become a new contextual layer within the study setting [9]. Nevertheless, we chose to differentiate the intervention (I) from the existing contextual factors (C₁ or C₂) to clarify the resources and support that are specifically introduced by B4M. As our data were closer to the project itself than to the broader environmental context (C₁), we did not have sufficient strands of evidence on this level. Next, the realist thinking of “if C, then O, because M, for A” was applied to develop ICAMO configurations for each cluster. This involved grouping similar patterns and corroborating or voiding strands of preliminary evidence. Although most evidence strands manifested to varying degrees in each facility, when these were not sufficient to explain usage behavior, they were discarded from the configuration. Theory testing and refining were incremental; data from the low usage cluster were first assessed and then compared with data from the moderate usage cluster. Finally, a cross-case comparison between clusters was used to develop refined program theories.

Study approval was granted by the Navrongo Health Research Centre Institutional Review Board (approval ID: NHRCIRB18) and the EMGO+ Scientific Committee of the Amsterdam Public Health Institute (reference number: WC2017-026). Before all interviews, written consent was secured using informed consent forms.

Respondents’ characteristics and usability scores are presented in Multimedia Appendix 4. Acknowledging individual variations, the perceived usefulness and perceived ease of B4M use were relatively high in all facilities (range 39.0-58.0). The t test showed no statistically significant differences between the 2 usage clusters (Table 3).

Next, we present the refined program theories under each outcome of interest in narratives of ICAMO configurations. Intervention features are marked “(I)” factors related to the health system context as “(C₂)” evaluation outcome “(O₁)” represents adoption and “(O₂)” utilization, whereas “(O_a)” represents additional outcomes. Mechanisms are identified “(M₁)” or “(M₂)” following the outcomes they are linked to, with related explanatory mechanisms further marked “(m₁)” or “(m₂).” Actor or user characteristics are marked “(A).” Explanations are included in the narrative, noting differences between cases and usage clusters. An overview of the realist analysis is depicted in Figure 2.

Adoption of B4M was characterized by an initial upward climb in both clusters. Differences, however, stemmed from experienced technical failures (I), complete or partial presence of an alternative point-of-care device or onsite laboratory (C₂), and dispositions of individual users (A). In health facilities with limited capacity to perform basic ANC screening tests (C₂), trained midwives and community health workers (I) were enthusiastic (m₁) to adopt B4M (O₁). This was due to its novelty (M₁) as a noninvasive automated device (I) and in anticipation of service delivery benefits, which they considered important (m₁) for providing focused ANC:

In facilities A, E, and F, long-standing relationships with local project partners (C₂) played a role in their selection as project sites (I). Their adoption response was transactional (M₁), triggered by a sense of obligation (m₁) to the project partner and by pride from being selected (m₁). Where alternative screening options were not functionally reliable (C₂), were not trusted (C₂); as was the case in facility F, required a longer turnaround time (C₂); as in facility A, or when screening was a paid service (C₂), users were motivated (M₁) to adopt B4M (O₁). This is because they considered it to be a necessary alternative (m₁), a trustworthy expert (m₁), a time-efficient resource (m₁), and a cost-effective substitute (m₁):

Irrespective of age, baseline technological skills, years of clinical experience, and the additional time use of B4M required, users with high technological self-efficacy (A) or willing to acquire technological skills (A) adopted the device despite an initial (1-2 weeks) difficult learning curve. Workers with low technological self-efficacy (A) avoided B4M (M₁) out of fear (m₁) of damaging it and low effort expectancy, although they reported sufficient training (I) and encouragement from peers and the project (C₂). To fill training attrition gaps due to user disinterest (A) and staff turnover (C₂), additional training (I) was organized for other health workers, thereby expanding the pool of trainees (O_a) who were mostly younger, lower cadre personnel (ie, community health nurses and nursing assistants who are trained professionals):

In response to contextual and program factors, postadoption utilization of B4M was explained by the dominance of either suppressive or supportive mechanisms triggered at facility and individual levels. Supportive mechanisms identified in the moderate usage cluster were less prominent or lacking in the low usage cluster.

The theoretical bases of knowledge on adoption and postadoption have been largely developed in the field of management information systems with a focus on higher-income countries [23,24]. As digital innovation systems continue to expand in LMICs, the implications of these theories in low-resource settings such as Ghana are making their way into the research agenda [8,25,26]. To our knowledge, this is the first study that applies a realist lens to elicit theory-based explanations on mHealth for maternal health services in LMICs. Our analysis confirmed some components of the initial program theories, voided others, and unveiled additional elements previously unaccounted for. Below, we reflect on key findings and their relevance to the science of mHealth implementation.

In facilities with limited diagnostic capacity, motivated workers adopted B4M for its novelty and benefits, in contexts of existing collaborations and authoritative or participative supervision styles. Although technology novelty triggered supportive adoption mechanisms, we found that the actual utilization of the device was the most important phase of the usage cycle [23]. Above-average usability scores from most health facilities did not fully explain variation between usage clusters, confirming the disconnect between usability and actual use [27]. Fear, enthusiasm, and high expectations for service delivery, especially in the absence of alternatives, better explained low to moderate adoption of B4M. With increased experience of use, we found that the initial emotive adoption response was replaced by rational behavior in the utilization phase: perceived usefulness being overshadowed by experienced contextual difficulties. Saccol and Reinhard describe this contrast between the perceived magic of technology and the disappointment of its limitations in the real world, which dampens users’ initial enthusiasm [28]. Although the program designers’ expectation was that all or most facilities would operate under the high usage cluster, that is, high adoption followed by high utilization, the identified supportive or suppressive mechanisms within and between cases shed light on why no health facility fell under this ideal state.

Realignment of mHealth to workflow and beneficiary expectations of ANC was identified as a crucial adaptive mechanism for its utilization. In addition to intrinsic motivation and a sense of accountability in users, utilization was influenced by mechanisms triggered in third-party actors. Negative reactions from pregnant women, bystander effect in peers, and low support or ownership from supervisors and program managers caused low utilization. mHealth adoption has been described as a social process [29], which may explain the strong third-party effect, although its influence has been specifically linked to contexts of mandatory technology use [30]. Despite perceived usefulness and user motivation, utilization mechanisms were suppressed by intervention-related (technical challenges, device rotation, lack of performance feedback, and refresher training), contextual (staff turnover, high workload, competing priorities, and low job satisfaction), and individual (low technological self-efficacy and knowledge) factors. Champion users displayed moderate but inconsistent adoption-utilization behavior, by taking ownership of the device, defying usage barriers. This adaptive behavior of users as a distinguishing factor in usage behavior is in line with other studies [27].

Contrary to the expectation that usage behavior was related to age, we found that internal and external motivations and technological self-efficacy were stronger explanatory factors. However, these are linked to age as a predictor of technology usage. Previous research confirms that older users have lower technological self-efficacy and are intimidated by the steep learning curve, especially when they have low baseline technological skills and inadequate learning support [31-33]. Although we found that empowerment was triggered in the utilization phase, adoption behavior has been shown to predict utilization response [34]. This mechanism might, therefore, manifest in both phases.

Beyond initial training, introducing technology requires careful planning and adaptation in low-resource settings where not many users experience job satisfaction or have adequate technological training as part of their professional competencies [35]. Admittedly, most factors related to the intervention context and actors such as service delivery demands, workforce shortages, and staff turnover are beyond the control of implementation teams. Nevertheless, these will have to be constantly negotiated especially in the utilization phase, with a responsive implementation strategy that supports workflow alignment and integration, which are crucial to the success of mHealth [35,36]. A preintervention situation analysis that takes our findings into account would go a long way in ensuring that future interventions are holistic and context-specific. A practical starting point to this could involve incorporating ICAMO elements into applicable implementation research frameworks such as the Consolidated Framework for Implementation Research, which incorporates multilevel factors and is adaptable through program cycles [37].

The temporal nature of pilot projects imposes a false sense of reality. Although users may briefly accommodate the innovation, they will be less invested in making long-term commitments requiring individual and organizational realignment, for short-term gains. In addition to being user-centered and accounting for the context, it is imperative that multiple stakeholder perspectives are leveraged during innovation design [6,7,35]. High-ranking supervisors might seem distant from the usage process but could compel or foster accountability and usage. They can also support adaptive strategies to integrate technologies into routine practice, especially in contexts of hierarchical supervision [30,38].

Selective training of a few workers unintentionally limits collective ownership and accountability for usage behavior. All health workers involved in maternal health service delivery at each site should be trained on device use, with regular monitoring and supervision, and periodical refresher training to help sustain or improve technological self-efficacy and dexterity, consequently preventing frustration and utilization decline [39]. Closer supervision and attention will be necessary in users with lower baseline technological skills and self-efficacy. Although it may cause tension and resistance from higher cadre users or peers, workers who fit the typology of champion users should be identified and encouraged to serve as opinion leaders within their health facilities. This would improve collective ownership, minimizing the bystander effect and optimizing social pressure [39,40].

The value of preexisting collaborations between the local partner organizations and health facilities and other administrative bodies remains crucial to gain access and influence, motivate, and encourage users. However, sustainability of transactional responses as a favor to program managers is doubtful. Financial incentives as a mechanism for behavior change have elicited mixed reports [41]. Indirect incentives such as encouragement, recognition, and support, which were highly desired and valued by B4M users, can, however, be promoted. To leverage the competitive mechanism and give users regular performance feedback, respectively, the design features of B4M prototype II could include gamification and dashboard analytics.

At the time of data collection, the device was in limited use in 3 of the 6 sites, with some respondents reporting not using the device for up to 5 months. This introduced recollection bias, in addition to socially desirable answers. Furthermore, not all user experiences were captured because a small number of trained users were unavailable for interviews. By triangulating data from multiple sources and interviewing at least two users per site, we attempted to compensate for these. The data validation workshop and dissemination meeting also informed group consensus on our findings. A realist approach is best applied throughout the life cycle of a project, from design to evaluation and reporting [22]. The nature of B4M as a short-term pilot, in addition to other constraints, restricted this possibility. Nevertheless, by developing and testing 3 initial program theories, the refined theories as a result of our analysis are sufficient for the next phase of prototype development.

This study shed light on optimal conditions necessary for B4M to thrive in a complex social and organizational setting. Evidence on the growth and potential of mHealth in improving service delivery, especially in a critical domain such as maternal health, may have overshadowed important individual, health system, and implementation factors that preclude its alignment in specific contexts and by certain user types. Beyond usability and viability studies, advocates of innovative technologies for maternal care need to consider how contextual factors, such as existing collaborations and supervision styles, trigger supportive mechanisms that influence program outcomes. This knowledge can be used to design and implement mHealth in similar settings. In addition to informing scale-up of the B4M prototype, our results and approach highlight the need for interventions that are guided by research methods that account for complexity.