This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Limited evidence exists on the value for money of mHealth information programs in low resource settings.
This study sought to model the incremental cost-effectiveness of gradually scaling up text messaging services to pregnant women throughout Gauteng province, South Africa from 2012 to 2017.
Data collection occurred as part of a retrospective study in 6 health centers in Gauteng province. Stage-based short message service (SMS) text messages on maternal health were sent to pregnant women twice per week during pregnancy and continued until the infant’s first birthday. Program costs, incremental costs to users, and the health system costs for these women were measured along with changes in the utilization of antenatal care visits and childhood immunizations and compared with those from a control group of pregnant women who received no SMS text messages. Incremental changes in utilization were entered into the Lives Saved Tool and used to forecast lives saved and disability adjusted life years (DALYs) averted by scaling up program activities over 5 years to reach 60% of pregnant women across Gauteng province. Uncertainty was characterized using one-way and probabilistic uncertainty analyses.
Five-year program costs were estimated to be US $1.2 million, 17% of which were incurred by costs on program development and 31% on SMS text message delivery costs. Costs to users were US $1.66 to attend clinic-based services, nearly 90% of which was attributed to wages lost. Costs to the health system included provider time costs to register users (US $0.08) and to provide antenatal care (US $4.36) and postnatal care (US $3.08) services. Incremental costs per DALY averted from a societal perspective ranged from US $1985 in the first year of implementation to US $200 in the 5th year. At a willingness-to-pay threshold of US $2000, the project had a 40% probability of being cost-effective in year 1 versus 100% in all years thereafter.
Study findings suggest that delivering SMS text messages on maternal health information to pregnant and postpartum women may be a cost-effective strategy for bolstering antenatal care and childhood immunizations, even at very small margins of coverage increases. Primary data obtained prospectively as part of more rigorous study designs are needed to validate modeled results.
The use of mobile and wireless technology for health (mHealth) [
In Zanzibar and Malawi, maternal SMS text messaging initiatives have demonstrated a significant effect on the utilization of health services and health outcomes. In Zanzibar, the Wired Mothers Program provided unidirectional SMS text messaging and direct two-way communication using a free call voucher system to provide reminders for ANC visits; gestational age-specific reproductive, maternal, newborn, and child health (RMNCH) information; and an emergency medical response system. Program activities were associated with an increase in ≥4 ANC visits (OR 2.39, 95% CI 1.03-5.55) [
Beyond these programs, the Mobile Alliance for Maternal Action (MAMA) program has provided stage-based maternal health information using SMS text messaging in Bangladesh, India, Nigeria, and South Africa and has supported content to projects in 54 countries globally. With the exception of formative findings from activities in Bangladesh [
The emergence of data suggesting that stage-based SMS text messages on maternal health information may yield improvements in utilization across the continuum of care, from pregnancy, delivery to postpartum, is promising [
To complement efforts to determine the effectiveness of MAMA in South Africa, we modeled the incremental cost-effectiveness of gradually scaling up SMS text messaging services to pregnant women throughout Gauteng province, South Africa from 2012 to 2017. This model-based analysis is anticipated to provide an early estimate of the cost-effectiveness of MAMA and inform future efforts to prospectively monitor costs and consequences of maternal SMS text messaging programs in low- and middle-income countries.
In South Africa, 1 in 24 children die before their 5th birthday; 25% due to undernutrition, 25% in the first 28 days of life, and >50% outside of the formal health sector [
The MAMA program was initiated in 2012 in South Africa to bolster the utilization of RMNCH services among pregnant and postpartum women by sending registered users stage-based SMS text messages twice per week during pregnancy and up to the infant’s first birthday [
Data on costs and effects were collected during exit interviews with women attending ANC services in 6 clinics in Johannesburg as part of a retrospective case-control study performed from October 2014 to June 2015. Among 608 eligible women, 356 appeared for requested face-to-face interviews. Of these women, 181 had been allocated to the intervention group and 175 to the comparison group (
Primary outcome measures assessed as part of the retrospective case-control study included attendance rates for ANC visits 1 through 4 and immunization rates at birth, 6, 10, 14 weeks, and 9 months after birth. We have also presented estimates of the proportion of children fully immunized and those who received comprehensive care (defined as at least 4 ANC visits [ANC4+] and full immunizations). Despite data limitations, findings from the retrospective case-control study suggested that women exposed to SMS text messages were more likely to attend at least the recommended 4 ANC visits (OR 3.21, 95% CI 1.73-5.98) and complete comprehensive care, (OR 3.2, 95% CI 1.63-6.31) than women not exposed [
For this model-based analysis, economic costs were estimated from a societal perspective inclusive of program, health systems, and user costs for a 5-year analytic time horizon (2012-2017). Program costs were defined as the costs required to develop, start-up, and support ongoing implementation. These were captured using an ingredients approach based on program activities, drawing from financial records and informant interviews with project implementing partners (Wits RHI, Cell-Life, and Praekelt Foundation), and through observations of health care workers providing routine ANC and PNC services within facilities. Costs were further categorized into capital (costs with a life expectancy of >1 year) and recurrent costs, with the former annualized over the lifetime of the project or life span of the item as appropriate and discounted at 3%. Development and start-up phase costs were viewed as one-time activities and similarly annualized over the lifetime of the project.
Incremental costs to the health system sought to capture costs associated with registration and increases in utilization. These included provider time costs to register patients into MAMA, as well as to provide routine clinical services during pregnancy and postpartum, including immunizations. These costs were estimated based on informant interviews, with provider salaries drawn from PayScale.com, an online salary, benefits, and compensation information company, and verified by the human resources department of Wits RHI.
Costs to users included all out-of-pocket payments incurred for care or treatment-seeking, including direct costs associated with medical care (consultation fees, medicine/commodity costs) as well as costs for transport/treatment-seeking and indirect costs due to wages lost resulting from time spent seeking care or away from income-generating activities. These were measured through patient interviews in intervention and comparison arms and generalized to the sample population as rollout occurred.
Costs were adjusted to 2015 USD dollars using consumer price indices [
Observed and forecasted enrollment trends over 5 years: July 2012-June 2017. Total users denote women registered to receive MAMA messaging while comprehensive care refers to the subpopulation that attended all recommended antenatal care 1 to 4 visits and had children that received the fully package of immunizations.
Forecasted 5-year costs in 2015 US $ for gradual rollout in Gauteng province, South Africa.
Category | Total program cost, July 2012 to June 2013 | Total program cost, July 2013 to June 2014 | Forecasted, July 2014 to June 2015 | Forecasted, July 2015 to June 2016 | Forecasted, July 2016 to June 2017 | Total cost over 5 years (% of total cost) | ||
Development | 37,353.42 | 38,474.03 | 39,628.25 | 39,628.25 | 42,041.61 | 197,125.55 (17) | ||
Start-up | 17,765.76 | 18,298.74 | 18,847.70 | 18,847.70 | 19,995.52 | 93,755.41 (8) | ||
Training | 149.73 | — | 73.86 | — | 80.51 | 304.10 (0) | ||
Personnel | 18,375.80 | 18,798.00 | 19,630.41 | 20,514.67 | 21,398.94 | 98,717.82 (8) | ||
Buildings | 5974.80 | 5644.53 | 5894.48 | 6160.00 | 6425.52 | 30,099.33 (3) | ||
Transport | 3223.16 | 3044.99 | 3179.83 | 3323.06 | 3466.30 | 16,237.34 (1) | ||
Communication | 537.19 | 507.50 | 529.97 | 553.84 | 577.72 | 2706.22 (0) | ||
Subtotal implementation support | 83,379.87 | 84,767.78 | 86,081.31 | 89,027.53 | 93,986.12 | 437,242.60 (37) | ||
Start-up/development | 129.89 | 129.89 | 137.03 | 144.08 | 158.74 | 699.63 (0) | ||
Content maintenance | 10,478.75 | 12,876.98 | 9167.21 | 9442.22 | 9725.49 | 51,690.64 (4) | ||
Technology maintenance | 8279.94 | 36,864.04 | 18,509.92 | 19,333.07 | 20,156.87 | 103,143.84 (9) | ||
Project management/personnel | 25,697.90 | 30,970.64 | 15,588.69 | 16,625.15 | 17,709.78 | 106,592.15 (9) | ||
Monitoring and evaluation | 1842.36 | 1961.43 | 108.72 | 111.99 | 115.34 | 4139.84 (0) | ||
Building/overhead | 10,073.21 | 13,088.45 | 12,225.42 | 12,765.21 | 13,305.66 | 61,457.94 (5) | ||
Travel | 11,353.19 | 8606.91 | 3100.02 | 3237.30 | 3374.71 | 29,672.13 (3) | ||
SMS text message delivery | 5384.11 | 9857.96 | 23,233.68 | 75,421.98 | 246,909.46 | 360,807.20 (31) | ||
SMS text message translation | 1736.80 | 1855.87 | 1855.87 | 1855.87 | 1855.87 | 9160.28 (1) | ||
Printing | 4726.26 | — | — | — | — | 4726.26 (0) | ||
Subtotal technology | 79,702.40 | 116,212.16 | 83,926.55 | 138,936.88 | 313,311.93 | 732,089.92 (63) | ||
Total | 163,082.27 | 200,979.94 | 170,007.86 | 227,964.40 | 407,298.04 | 1,169,332.51 (100) |
5-year trends in the total program cost per registered user and per case of comprehensive care (CC) received among MAMA users over 60 months.
Estimates of service utilization were used to forecast the costs to the health system associated with registering pregnant women to MAMA as well as treating additional cases (
Mean costs to users for attending ANC and PNC were drawn from structured interviews and included food, wages lost, child care, and transport costs (
Using the South Africa’s Gross National Income per capita for 2015 of US$6080 as the threshold, program activities have a 100% probability of being cost-effective. At lower willingness-to-pay thresholds, the probability of MAMA being cost-effective increases over time as the number of users increases along with anticipated health effects.
To compliment probabilistic sensitivity analyses, we also conducted one-way sensitivity analyses to identify key drivers of the incremental cost per DALY averted (
Cost effectiveness plane of years 1-5 of MAMA implementation vs. Status quo in Gauteng, South Africa. Individual dots represent the incremental costs and incremental disability adjusted life years (DALYs) averted for each of 1,000 simulations conducted by year of implementation.
Incremental cost effectiveness acceptability curve of years 1-5 of MAMA implementation vs status quo in Gauteng, South Africa. Using the South Africa’s gross national income (GNI) per capita for 2015 of US $6,080 as the threshold, program activities have a 100% probability of being cost effective. At lower willingness pay thresholds, the probability of MAMA being cost effective increases over time as the number of users increases along with anticipated health effects. DALY: Disability adjusted life years.
One way sensitivity analysis of key drivers of incremental cost per disability adjusted life year (DALY) averted for year 5 of program implementation (all costs in US $). ANC: antenatal care; PNC: postnatal care; SMS: short message service.
Study findings modeling the incremental cost-effectiveness of exposure to SMS text messages during delivery and postpartum on care-seeking and childhood immunizations suggest that the cost per DALY averted ranges from US $1985 in year 1 when only 1% of pregnant women are registered to US $200 in year 5 when 60% of pregnant women are included. Societal costs to implement MAMA in the 5th year of implementation were estimated to be US $3.6 million dollars, 59% of which represent costs borne by users to seek care for ANC and PNC, 30% costs to the health system, and 11% program costs. When considered against a status quo comparator, the incremental annual cost to implement MAMA at 60% coverage is US $1.03 million.
To estimate the health effects of SMS text messaging exposure, we drew from sample data on changes in the utilization of services among MAMA users and nearby non-MAMA-using mother-infant pairs for ANC and childhood immunizations up to 9 months. Less than 100 complete records were available in each study arm. When coupled with the existing high rates of service utilization, it meant that we were not powered to measure changes in the utilizations rates for individual vaccines. However, we were powered to measure observed changes in the utilization for ANC and comprehensive care (defined as ANC 1-4 and full immunizations). Based on available data, 1%-4% increases in immunization rates were observed by vaccine type along with a 14% increase in all 4 ANC visits. The latter finding is consistent with changes in coverage observed in Zanzibar as part of the Wired Mothers Program [
In the 2 years since the MAMA program ended, the National Department of Health (NDOH) has developed and rapidly scaled a maternal SMS text messaging program called “MomConnect,” which is based on MAMA but with less specific messages on the prevention of mother-to-child transmission of HIV. Like MAMA, MomConnect registers pregnancies and links expectant mothers to gestation age-specific pregnancy information, while also providing access to a help desk for reporting compliments or complaints on service delivery. Since its inception, MomConnect has grown to become one of the largest mHealth initiatives globally, registering >1 million pregnant women in >95% (3300) of health facilities in South Africa to receive SMS text messages on maternal health information [
Elsewhere, data on the value for money of digital health programs are slowly emerging. However, to our knowledge, this is the first study to provide evidence on the value for money of maternal SMS text messaging programs. To date, a dozen peer-reviewed papers comprise the body of evidence on the value for money of mHealth solutions, including CEAs (6 studies) [
From the outset, we sought to base this analysis on primary data and focus only on the 2-year analytic time horizon of the program. However, primary data collection efforts were hampered by challenges in patient recruitment and data completeness. While we sought to obtain data on immunizations from the paper-based booklets provided by NDOH to mothers and completed by health workers at the time of service delivery, in practice, significant gaps in the completeness and quality of record keeping meant that a large proportion of interviewed participants were excluded from the final analyses. In shifting to a model-based analysis, we sought not only to more rigorously capture uncertainty but also consider the implications of service delivery at scale. Except for variable costs associated with SMS text message delivery, much of the technological costs associated with the MAMA program activities were fixed, irrespective of the scale of implementation. That said, our findings clearly suggest that greater value for money is attained with increasing scale. We hope that further analyses drawing from primary data of maternal SMS text messaging at scale through MomConnect and other initiatives will confirm this.
This is a first of its kind study to provide an evidence-informed model of the value for money of maternal SMS text messaging programs. Study findings suggest that SMS text messages to pregnant and postpartum women are cost-effective, according to the GNI per capita thresholds for South Africa. Cost-effectiveness improves with scale. Further efforts are needed to determine the value for money of maternal SMS text messaging under more robust study designs and in differing settings where technological (network coverage and access to mobile phones), epidemiological, and health systems profiles may differ.
antenatal care
comprehensive care
disability adjusted life year
gross national income
incremental cost-effectiveness ratio
Lives Saved Tool
Mobile Alliance for Maternal Action
National Department of Health
postnatal care
quality adjusted life year
reproductive, maternal, newborn, and child health
skilled birth attendance
short message service
The data underpinning this analysis were gathered through interviews with providers and mothers registered to receive MAMA messages in Gauteng. The authors are grateful to these individuals for their generosity, time, and insights. We additionally thank Dr Garrett Mehl and Dr Patty Mechael for their guidance and support to the project.
MAMA activities in South Africa were funded by USAID through a grant provided to Wits RHI. Time costs for JHSPH study authors were provided by the World Health Organization’s Innovation Working Group Catalytic mHealth Grants Program.
JC is the principal investigator. JC conceived the idea for this research with inputs from ALeFevre, and JE. ALeFevre, MCE, and JC designed the data collection instruments for patient interviews. JC oversaw the data collection and quantitative data entry with support from IB. DR, AN, and JC collected data on program costs. ALeFevre conducted the probabilistic sensitivity analysis with inputs from YJ. IB, DM, and JE provided critical inputs to the quantitative analyses and interpretation of results. ALeFevre conducted the analyses and wrote the first draft of the manuscript with inputs from JC, MCE and all other authors. All authors approved the final draft.
None declared
Eligibility and enrollment flow of study participants included.
Parameters for probabilistic sensitivity analysis drawing from Year 5 costs (All currency in US $).
Year 1 Program costs in US $ for gradual rollout in Gauteng province, South Africa.
Year 2 Program costs in US $ for gradual rollout in Gauteng province, South Africa.
Year 3 Program costs in US $ for gradual rollout in Gauteng province, South Africa.
Year 4 Program costs in US $ for gradual rollout in Gauteng province, South Africa.