Behavioral, social, and emotional difficulties that begin early in life have long-term learning, academic, and relational consequences [1-3]. Chronic pediatric mental health problems are among the top five disabilities affecting children in the United States and now more prevalent than childhood physical disabilities [4,5]. Importantly, many of these social, emotional, and behavioral problems begin in the preschool years [2].

Prevention and intervention in the preschool years is critical before the behavior problems become fixed and disabling [6,7]. Pediatric primary care clinicians are often the first professionals that families solicit regarding parenting concerns or behavioral problems in their children [8,9]. Although a significant need exists in pediatric primary care for programs that teach effective parenting and prevent behavior problems these programs are not readily accessible to clinicians and parents. The purpose of this study was to examine the use and efficacy of a tablet-based parent training program (the ez Parent program) in a sample of parents recruited from a pediatric primary care setting.

Parent training (PT) - a set of systematic programs for teaching parents child management skills - is widely used to promote positive parenting and reduce behavioral risk in young children [10,11]. Early prevention efforts focused on PT are important because parent behavior is a modifiable risk factor for early child behavior problems and improving positive and skilled parenting is a powerful predictor of positive child outcomes [10,12,13]. Though PT programs have been shown to be effective, there are two important limitations to these programs commonly delivered in face-to-face group or individual settings. These limitations include low parent participation rates, particularly among low-income parents, and implementation challenges in existing primary health care systems.

Parent attendance rates for those who enroll in PT typically range 35% to 50% of sessions and up to one-third who sign up attend no sessions [14,15]. Primary logistic barriers to enrollment and attendance in group-based PT include lack of time, childcare, schedule conflicts, and competing demands [14,16]. For low-income families, these barriers may be magnified by concerns with transportation, childcare, and neighborhood safety; as well as demands from work, family and friends, high stress, and poor health.

In addition to parent participant barriers, there are practice barriers. Although pediatric primary care is an ideal venue for programs for improving parenting and preventing behavior problems, there are limited treatment and prevention options that fit into the primary care setting [17]. Indeed, PT programs are traditionally delivered in face-to-face formats (either individually or in group-based settings) and require trained facilitators and weekly parent attendance. Implementing multiple face-to-face sessions of PT is challenging in a busy primary care setting. For instance, a typical well-child visit lasts 8 to 18 minutes during which time clinicians are expected to complete activities including a physical exam, assessment of developmental milestones met, assessment of the child’s nutritional status, and deliver any treatments and immunizations. Providing in depth parenting guidance to families struggling with children’s emotional or behavioral problems may be unrealistic [18,19]. Further, clinicians’ time for pediatric visits related to behavioral problems is reimbursed at a significantly lower rate than general medical visits [19]. The use of Internet-based methods to deliver PT is a feasible and potentially cost-effective approach to address the challenges of delivering PT in primary care settings. Internet delivery addresses issues related to parent participation by increasing accessibility, availability, and parent-controlled access.

The ez Parent program is a delivery adaptation of the evidence-based, group delivered Chicago Parent Program (CPP). The program is a prevention intervention designed to promote parenting competence and prevent child behavior problems in children 2- to 5-years old [20,21]. Like the group-based program, the six modules of the ez Parent program teach parents evidence-based strategies for encouraging good behavior and decreasing misbehavior in children. See Figure 1 for a list of module content. The ez Parent Program was originally named the electronicCPP and changed to the ez Parent Program in 2016. For a full description of the development of the program see Breitenstein et al [20].

The ez Parent program is a self-administered program that is downloaded onto an Android tablet computer as an app. It is one of the first adaptations of an evidence-based group delivered program to incorporate tablet-based technology. The program is set up so that users move sequentially through the modules (ie, parents must complete the first module prior to the second module being unlocked). Once a module is unlocked, parents can access the module and return to any portion of that module at a later time. We estimate that initial completion of one module would take approximately 1 hour [20].

Like other PT programs, rather than focusing primarily on education or increasing knowledge, the ez Parent program assists parents in acquiring a range of evidence-based parenting skills [13,20]. There are several unique aspects of the ez Parent program developed to promote parent engagement and support learning. These are (1) video vignettes of parents interacting with their children, which support vicarious learning and present examples of the various parenting strategies, (2) knowledge questions, which assess parent understanding of the module content and provide added information when needed, (3) interactive “game” activities, which are intended to be fun and keep parents stimulated and engaged, and (4) module practice assignments, which provide opportunities for parents to practice what they are learning with their children [20].

Consistent with the group-based CPP, the ez Parent program is designed to be culturally and contextually relevant for low-income, ethnically diverse parents of young children [21]. During the development of the tablet program, we worked with low income and ethnic minority parents to assure the content was relevant and that the app was easy to use, attractive, and interesting [20]. The use of mobile technology allowed us to design the program to be interactive and accessible, a particularly salient feature for African American and Hispanic parents who are among the most active users of the mobile Internet [22,23]. Therefore, parent training via an app on mobile devices may be an ideal method to increase the reach and accessibility of parent training programs, particularly among low-income African-American and Hispanic families.

The purpose of this study was to establish the feasibility and efficacy of the ez Parent program. To examine feasibility, we examined parent satisfaction with the program and the percent of modules completed relative to percent of face to face sessions attended in prior studies of the group-based CPP. To test efficacy, we examined the effect of the ez Parent program against an attention control condition for improving parenting and child behavior in a sample of low-income ethnic minority parents of young children recruited from a primary care setting. We hypothesized that the percent of ez Parent modules completed would be comparable to or greater than the percent of group-based CPP session completed; compared with an attention control group, parents in the ez Parent intervention group would report using more positive discipline strategies, greater parenting self-efficacy, and decreased parenting stress and child behavior problems; and effect sizes for improvement in parent and child behaviors would be comparable to or greater than the group-based CPP.

The study used a two-group randomized control trial (RCT) design with repeated measures follow-up. Subjects were randomly assigned to either the intervention (ez Parent group) or attention control condition (health promotion group). Data collection in both conditions was at baseline and at 12 and 24 weeks post baseline. This study was reviewed and approved by the University’s institutional review board.

Parents in the ez Parent group received the six module ez Parent program to be completed over a 12-week period (allowing approximately 2 weeks per module). During the same time period, parents in the health promotion control group received information on various topics (ie, nutrition, exercise, finance, safety, medical information, and entertainment) via a website that included links and portable document format handouts for each topic. These materials were the same health promotion handouts and websites distributed at the primary care pediatric recruitment site. The health promotion condition was designed to control for attention related to technology use over the same period of time.

Parents were eligible to participate in the study if they were the parent or legal guardian of a child between the ages of 2 to 5 years, the child was receiving or eligible to receive Medicaid insurance (eg, All Kids Assist or Share in Illinois), and the child received medical care at the pediatric primary care recruiting site. Only one parent per family participated in the study. If a parent had more than one child between the ages of 2 and 5 years, the parent selected one child to report on their behavior over the length of the study. Currently, the ez Parent program is only available in English; therefore, parents who were unable to speak and read English were excluded.

Parents were recruited between October 2013 and June 2014 from a large, urban, primary care pediatric clinic located on the near west side of Chicago. According to the clinic, most of the families they serve are African American (50%) or Latino (30%) and over 65% of families have low incomes or receive Medicaid. We advertised broadly at the primary care site using project flyers that contained information about the purpose of the study, inclusion criteria, expectations for participation, and contact information for parents to directly contact the study staff. An interest form was included in the flyer; parents completed the form indicating their willingness to be contacted by the study staff. If a parent was eligible and interested in participating, then a 2-hour meeting was scheduled in our research offices to complete the participant consent, baseline data collection, and randomization to condition.

Figure 2 depicts the participant flow in the trial. Two hundred eighty-seven parents were assessed for eligibility. Of the 118 parents eligible for the study, 70.3% (83/118) of parents consented to participate, completed baseline assessments, and were randomized to either the ez Parent or health promotion control condition. Of the 83 parents randomized, 79 parents completed follow-up assessments (95.2% retention rate).

The sample was predominantly comprised of single, ethnic minority (African-American or Hispanic) mothers of young children earning annual incomes of less than $20,000 (see Table 1). There were no significant demographic differences between the intervention and control groups.

Parents earn a “module badge” at the end of each module and after they review the practice assignment. After completion of the module, parents are alerted with a pop-up in the program congratulating them on earning the module badge. Parents can view their earned badges in the “my badges” section of the program. The ez Parent digital platform provides time stamps of when parents earn the module badge. Intervention dose was defined by parent receipt of the module badge. For this analysis, we included badges that were earned prior to the 12 week post baseline data collection appointment corresponding with the intervention completion period (eg, 6 modules × 2 weeks per module).

Satisfaction was measured using an end of program survey administered 6 months after baseline. Parents rated how helpful they found the ez Parent program (very helpful, a little helpful, or not at all helpful), their overall satisfaction with the program (very dissatisfied, dissatisfied, satisfied, or very satisfied), and whether they would recommend the ez Parent program to other parents (not recommend, recommend, or highly recommend). Parents in the control group rated their perception of helpfulness (very helpful, a little helpful, or not at all helpful) of the information provided in the health promotion website (eg, nutrition, exercise, finances, safety, medical, and entertainment) and whether they would recommend the website to other parents (not recommend, recommend, or highly recommend).

At their first study appointment, parents were enrolled in the study by completing the informed consent process and responding to all demographic and survey instruments. Consents and survey data were collected on an Internet-based data collection app developed for this study. The goal of using Internet-based data collection was to increase the efficiency and ease of data collection and entry. Prior to recruitment, study staff tested the data collection app for ease of use and accuracy of data collection. Once established, the data collection app was launched. Total time for completion of the surveys at baseline was 35 to 40 minutes. Randomization to group assignment was made using stratified random sampling based on time sequential cohorts. The randomization table was built into the data collection app and group assignment was made after all baseline data were acquired. Parents were assured that the study assignment was random and had no connection with their responses to the survey questions. After study assignment was revealed, all parents in the control and intervention groups were given Android tablets to use for the duration of the study.

Several strategies were implemented to ensure intervention fidelity (eg, strategies to monitor and enhance the reliability and validity of the intervention) [30]. After study assignment was revealed, the study staff provided a structured training (using written and verbal information) to all participants on how to use the tablet. Additional training was then provided specific to assigned condition. Following training, parents were prompted to verbalize their understanding and conducted return demonstrations to assess ability to use the tablet and the health promotion website (control) or ez Parent app (intervention).

Parents in the ez Parent condition received automated text message reminders to complete the ez Parent program if they had not completed the module content. Automated text messages also included positive encouragement regarding completion of the program and of practice assignments. These messages were consistent across all intervention parents.

Parent participants were included in the analysis nondependent on their adherence or nonadherence to the ez Parent program. Parents (n=4) who were lost to follow up after enrollment were not included because we were unable to collect any follow up data for outcome analysis (see Figure 2). Parent completion of the ez Parent modules was calculated as the percentage of the six modules completed by each of the intervention group parents (n=40). For comparison to the group-based CPP, intervention attendance was calculated as the percentage of sessions 1 through 10 of the CPP group attended by intervention group parents (n=267). This analysis is based on content completion not on the amount of time spent in either the group-based or tablet-based program. Content across the ez Parent modules and the CPP group sessions is comparable (see Textboxes 1 and 2). The ez Parent and group based rates of participation were compared using 95% confidence intervals (CI). Satisfaction data were summarized using item frequencies.

Chi square tests were used to compare baseline data between parents in the ez Parent and health promotion control group and determine if there were significant differences. Parent and child data were analyzed using a repeated measures analysis of variance (RM-ANOVA) with simple contrasts to determine if there were significant intervention effects on the outcome measures from baseline to 12 weeks post baseline (T1 to T2) and from baseline to 24 weeks post intervention (T1 to T3).

Effect sizes for between group comparisons were calculated for all outcome variables using Cohen’s d [31]. For comparison to the group-based CPP, we calculated effect sizes from data published by Breitenstein and colleagues [15] (effect size comparisons for the PSI-SF are not included as this measure was not part of the group-based evaluation). Effect sizes were calculated T1 to T2 and for T1 to T3 using the following equation:

( ͞x _t1 − ͞x _t2 ) − ( ͞x _c2 − ͞x _c1 ) /SD _pooled

where ͞x _t2 = treatment group mean at time 2; ͞x _t1 = treatment group mean at time 1; ͞x _c2 = control group mean at time 2; ͞x _c1 = control group mean at time 1; and SD _pooled = the pooled SDs of all reported groups (eg, treatment and control) [31].

Table 2 presents the percent of the sample completing each module (ie, earned a module badge) during the first 3 months of the study and the percent of parents in the archival comparison sample attending the face to face CPP group-based sessions. All parents in the ez Parent sample earned the module 1 completion badge. Across all six modules, the average ez Parent module completion by parent was 85.4% (34/40) of modules (95% CI = 74.5%-96.4%). In contrast, average parent attendance for the face-to-face CPP groups was 50.6% (135/267) of sessions (95% CI = 44.6%-56.6%), and 24% (64/267) of the archival CPP sample failed to attend any sessions [15]. The ez Parent dose rate was higher than the upper CI for the group CPP dose, suggesting that parent participation rates of the ez Parent and group based CPP were significantly different (P<.05).

Of the parents in the ez Parent group, 87.5% (35/40) reported the ez Parent program being very helpful and 12.5% (5/40) a little helpful. In the control group parent reports of helpfulness of the topics on the health promotion website (control group) ranged from 46.2% (18/39) to 74.4% (29/39) very helpful; 20.5% (8/39) to 28.2% (11/39) a little helpful; and 2.6% (1/39) to 5.1% (2/39) not at all helpful. The nutrition topic in the website was the highest ranked as very helpful. Of parents in the ez Parent condition, 42.5% (17/40) reported that it was a little bit hard to regularly use the ez Parent program, 2.5% (1/40) very hard, and 52.5% (21/40) not at all hard. Satisfaction with the ez Parent program was high, with 80.0% (32/40) of parents reporting they were very satisfied and 20.0% (8/40) reporting they were satisfied with the program.

Parents in the ez Parent group were more likely to recommend the ez Parent Program to other parents than parents recommending the health promotion site (Kendall’s Tau-b = 0.25; P<0.05). Overall, 82.5% (33/40) of the ez Parent group reported they would highly recommend, 15.0% (6/40) would recommend, and 2.5% (1/40) didn’t know if they would recommend the program to another parent. In contrast, 59.0% (23/39) of parents in the health promotion control group reported they would highly recommend, 35.9% (14/39) would recommend, 2.6% (1/39) would not recommend, and 2.6% (1/39) didn’t know if they would recommend the website to another parent.

Table 3 presents means and SDs for each variable by condition at each data collection time point. At baseline, there were no significant differences between the ez Parent and health promotion control groups for parent behavior (PQ), self-efficacy (TCQ), and stress (PSI-SF), or child behavior (ECBI). At baseline, 6.3% (5/79) of the sample scored above the PSI cut off score (>109) [28]; 25.3% (20/79) scored above the ECBI problem scale cut off score (>14); and 19% (15/79) of the sample scored above the ECBI intensity scale cut-off score (>130) [29].

From T1 to T3, there was a significant difference between conditions for parenting warmth on the PQ (F _1,77 = 4.82, P<.05). There were no significant differences between the intervention and control conditions on parents’ reports of their follow through on discipline, use of corporal punishment, parenting stress, parenting self-efficacy, or child behavior problems.

In Table 4, we present effect size estimates for the ez Parent program and comparison effect sizes for the group-based CPP [15]. A Cohen’s d of .2 is considered a small effect, .5 a medium effect, and .8 or larger a large effect [31]. Effect sizes from T1 to T3 were in the small range for parent warmth (PQ), parent corporal punishment (PQ), follow through (PQ), parenting stress (PSI-SF), and child behavior problems based on the ECBI intensity scale. With the exception of parenting self-efficacy (TCQ) and corporal punishment (PQ), ez Parent effect sizes were comparable or greater than those of the group-based CPP.

In this study we evaluated the feasibility, acceptability, and preliminary efficacy of the tablet-based adaptation of the CPP, the ez Parent program, in a sample of low-income, ethnic minority parents of 2- to 5-year-old children seen in primary care. The CPP is an evidence-based parenting skills program originally designed to be delivered in a face-to-face parent group format [21]. However, like many preventive interventions targeting low-income families that require face-to-face involvement, participation rates have typically been low [14,16]. Moreover, pediatric primary care providers, who serve a large population of families receiving Medicaid, are often unable to devote the time needed to provide parenting skills training and support. These problems diminish the impact and reach of evidence-based parenting programs. Given the challenges of using face-to-face formats for parenting skills training, we were particularly interested in examining the amount of intervention (ie, dose) parents would receive using a self-administered digital format for acquiring parenting skills training compared with a face-to-face group format.

Overall, the results indicate that the ez Parent program is feasible and acceptable in a low-income, ethnic minority population of parents. Compared with a control group of parents using a health promotion website, ez Parent program parents were more likely to report that they would recommend the program to another parent. Compared with a comparable archival sample of parents enrolled in the face to face group-based CPP, parents enrolled in ez Parent received a higher dose of parenting content. Specifically, parents in the ez Parent group completed over 85.0% (34/40) of the modules compared with parents in the face-to-face format who attended on average only 50.6% (135/267) of the parent group sessions. Moreover, all of the ez Parent participants completed at least one module whereas 24.0% (64/267) of parents enrolled in the face-to-face group format, never attended a single session. These findings are consistent with a recent review of digitally delivered parenting interventions reporting that 41.7% to 99.2% of participants completed their digitally delivered parent training interventions [32]. Furthermore, these data demonstrate that a digital format for delivering parenting skills training to a predominantly low-income ethnic minority population of parents is feasible and acceptable, and may greatly extend the reach for helping more parents and young children in primary care.

Although module completion rates in this study were promising, there is more work to be done to understand patterns of parent use of the ez Parent program. A benefit of using technology is the ability to track all aspects of parent usage of the app. Digital tracking data can provide important information regarding parent behavior in interacting with the intervention, specifically, how and when parents use the ez Parent program and what content they access. For example, information regarding the number of visits parents make to components of the program, time stamps of visits to each portion of the program, and the frequency which parents return to completed content. In future work, we will conduct detailed analysis of this usage data. Indeed, research of Internet-delivered interventions is beginning to emerge related to patterns of program use and intervention outcome [33,34]. A fine tune analysis of usage metrics will expand our knowledge related to dose and program effectiveness.

Overall, we found the effects of the ez Parent program on parenting and child behavior problems were in the small range and many were comparable to the effect sizes obtained from archival samples using the CPP in the face-to-face format. Although significant improvements were found for parent use of warmth in their discipline, no improvements were found for parent use of corporal punishment, following through on discipline, parenting stress, and child behavior problems relative to the control group. There are three possible reasons for these results. First, the power calculation for this study was based on estimated effect sizes for intervention dose rather than parent and child outcomes. Much larger sample sizes would be needed to detect significant differences based on small effects, which are typical for prevention studies [35,36]. Therefore, we may not have adequate power to detect differences between the intervention and control parents in this analysis.

Second, although we recruited from a population of families with social risk factors related to parent and child dysfunction (eg, low income), the parents in our study reported low levels of parenting stress and relatively few child behavior problems at baseline. Less than one-quarter of the parents reported child behavior problems exceeding the cut point for clinically significant problems and less than 6.3% (5/79) reported clinically significant levels of parenting stress. Therefore, floor effects in this relatively healthy population may have made it difficult to detect improvements over time. Greater effects may be more detectable in a referred population of parents and young children.

Third, the lack of effects may be due to the limited amount of time allotted for assessing parent and child behavior change. It is possible that more time for parents to absorb the new information, practice the new skills, and observe changes in themselves and their children may yield greater improvements in discipline strategies, parenting self-efficacy, and child behavior.

Although our findings indicate feasibility and acceptability of the ez Parent program and comparable effect sizes to face-to-face delivery for parents recruited from primary care we recruited from one location from a relatively healthy population. It is possible that parents involved in this study were highly motivated to participate and learn from the intervention. Future studies of the ez Parent program will employ a larger sample size from multiple primary care sites and longer term follow-up to establish the efficacy of the ez Parent program in modifying parent and child behaviors. In addition, to lay the ground work for full scale implementation in primary care sites, implementation, and cost effectiveness of the ez Parent program in primary care will be determined.

Using an archival sample for a comparison group of delivery methods presents methodological limitations because of the lack of experimental control across the two groups. Although not a purpose of the original evaluation of the ez Parent program, we felt it was important to demonstrate an initial comparison of dose and outcome to the group-based CPP. Our analysis suggests they may be comparable but a true test would require a randomized design comparing group-based CPP to ez Parent. Indeed, a RCT structured as an equivalence or noninferiority trial with an adequately powered sample size would allow a head to head comparison of the two formats, directly compare intervention outcomes and dose, and allow us to understand which parents are likely to benefit from each delivery format.

Another potential limitation is the use of parent report to evaluate the intervention. A more robust measure of intervention efficacy could include multi-informant and multimethod data. However, previous findings from RCTs of the group-based CPP have found significant findings across three sources of data: parent self-report, teacher report, and parent-child observation [15,37]. In these studies, the parent self-report findings have consistently agreed with other methods of measuring parent and child outcomes.

It is important to understand how parents use the information and program. In this analysis we assessed one metric of program usage – module completion. Although module completion provides an overall picture of parent dose it does not provide us with specific information related to patterns of use, time spent on the program, and other important data of ez Parent use. In addition, we assessed program use from baseline to the 12-week follow-up even though parents had access to the program throughout the 24-week study period. Therefore, as previously noted, an analysis of usage data is critical in understanding patterns using the ez Parent intervention and parent’s ongoing use of information after the initial module completion. This information will help understand the change mechanisms of the intervention. Further, demonstrating the processes and patterns related to using tablet-based behavior change interventions is important to the field of eHealth behavioral interventions.