RefCheck Maintenance Notice

On Monday, December 3, 2018, from 16:00-18:00 EST, RefCheck will be undergoing maintenance. RefCheck is the process where, during copyediting, all references are extracted from the manuscript file, parsed, matched against various databases (eg, PubMed and CrossRef), and automatically corrected. For more information on RefCheck, please visit our Knowledge Base.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 20.04.16 in Vol 4, No 2 (2016): Apr-Jun

This paper is in the following e-collection/theme issue:

    Original Paper

    Parent Use and Efficacy of a Self-Administered, Tablet-Based Parent Training Intervention: A Randomized Controlled Trial

    1Rush University, College of Nursing, Chicago, IL, United States

    2Johns Hopkins University, School of Nursing, Baltimore, MD, United States

    *these authors contributed equally

    Corresponding Author:

    Susan M Breitenstein, RN, PhD

    Rush University

    College of Nursing

    600 S. Paulina Street

    Chicago, IL,

    United States

    Phone: 1 312 942 6259

    Fax:1 312 942 6226

    Email:


    ABSTRACT

    Background: Parent training programs are traditionally delivered in face-to-face formats and require trained facilitators and weekly parent attendance. Implementing face-to-face sessions is challenging in busy primary care settings and many barriers exist for parents to attend these sessions. Tablet-based delivery of parent training offers an alternative to face-to-face delivery to make parent training programs easier to deliver in primary care settings and more convenient and accessible to parents. We adapted the group-based Chicago Parent Program (CPP) to be delivered as a self-administered, tablet-based program called the ezParent program.

    Objective: The purpose of this study was to (1) assess the feasibility of the ezParent program by examining parent satisfaction with the program and the percent of modules completed, (2) test the efficacy of the ezParent program by examining the effects compared with a control condition for improving parenting and child behavior in a sample of low-income ethnic minority parents of young children recruited from a primary care setting, and (3) compare program completion and efficacy with prior studies of the group-based CPP.

    Methods: The study used a two-group randomized controlled trial (RCT) design with repeated measures follow up. Subjects (n=79) were randomly assigned to an intervention or attention control condition. Data collection was at baseline and 12 and 24 weeks post baseline. Parents were recruited from a large, urban, primary care pediatric clinic. ezParent module completion was calculated as the percentage of the six modules completed by the intervention group parents. Attendance in the group-based CPP was calculated as the percentage of attendance at sessions 1 through 10. Satisfaction data were summarized using item frequencies. Parent and child data were analyzed using a repeated measures analysis of variance (RM-ANOVA) with simple contrasts to determine if there were significant intervention effects on the outcome measures. Effect sizes for between group comparisons were calculated for all outcome variables and compared with CPP group based archival data.

    Results: ezParent module completion rate was 85.4% (34.2/40; 95% confidence interval [CI] = 78.4%-93.7%) and was significantly greater (P<.05) than face-to-face CPP group attendance (135.2/267, 50.6%) attendance of sessions; 95% CI = 46.8%-55.6%). ezParent participants reported the program as very helpful (35/40, 88.0%) and they would highly recommend the program (33/40, 82.1%) to another parent. ezParent participants showed greater improvements in parenting warmth (F1,77 = 4.82, P<.05) from time 1 to 3. No other significant differences were found. Cohen’s d effect sizes for intervention group improvements in parenting warmth, use of corporal punishment, follow through, parenting stress, and intensity of child behavior problems were comparable or greater than those of the group-based CPP.

    Conclusions: Data from this study indicate the feasibility and acceptability of the ezParent program in a low-income, ethnic minority population of parents and comparable effect sizes with face-to-face delivery for parents.

    JMIR mHealth uHealth 2016;4(2):e36

    doi:10.2196/mhealth.5202

    KEYWORDS



    Introduction

    Background

    Behavioral, social, and emotional difficulties that begin early in life have long-term learning, academic, and relational consequences [1-3]. Chronic pediatric mental health problems are among the top five disabilities affecting children in the United States and now more prevalent than childhood physical disabilities [4,5]. Importantly, many of these social, emotional, and behavioral problems begin in the preschool years [2].

    Prevention and intervention in the preschool years is critical before the behavior problems become fixed and disabling [6,7]. Pediatric primary care clinicians are often the first professionals that families solicit regarding parenting concerns or behavioral problems in their children [8,9]. Although a significant need exists in pediatric primary care for programs that teach effective parenting and prevent behavior problems these programs are not readily accessible to clinicians and parents. The purpose of this study was to examine the use and efficacy of a tablet-based parent training program (the ezParent program) in a sample of parents recruited from a pediatric primary care setting.

    Parent training (PT) - a set of systematic programs for teaching parents child management skills - is widely used to promote positive parenting and reduce behavioral risk in young children [10,11]. Early prevention efforts focused on PT are important because parent behavior is a modifiable risk factor for early child behavior problems and improving positive and skilled parenting is a powerful predictor of positive child outcomes [10,12,13]. Though PT programs have been shown to be effective, there are two important limitations to these programs commonly delivered in face-to-face group or individual settings. These limitations include low parent participation rates, particularly among low-income parents, and implementation challenges in existing primary health care systems.

    Parent attendance rates for those who enroll in PT typically range 35% to 50% of sessions and up to one-third who sign up attend no sessions [14,15]. Primary logistic barriers to enrollment and attendance in group-based PT include lack of time, childcare, schedule conflicts, and competing demands [14,16]. For low-income families, these barriers may be magnified by concerns with transportation, childcare, and neighborhood safety; as well as demands from work, family and friends, high stress, and poor health.

    In addition to parent participant barriers, there are practice barriers. Although pediatric primary care is an ideal venue for programs for improving parenting and preventing behavior problems, there are limited treatment and prevention options that fit into the primary care setting [17]. Indeed, PT programs are traditionally delivered in face-to-face formats (either individually or in group-based settings) and require trained facilitators and weekly parent attendance. Implementing multiple face-to-face sessions of PT is challenging in a busy primary care setting. For instance, a typical well-child visit lasts 8 to 18 minutes during which time clinicians are expected to complete activities including a physical exam, assessment of developmental milestones met, assessment of the child’s nutritional status, and deliver any treatments and immunizations. Providing in depth parenting guidance to families struggling with children’s emotional or behavioral problems may be unrealistic [18,19]. Further, clinicians’ time for pediatric visits related to behavioral problems is reimbursed at a significantly lower rate than general medical visits [19]. The use of Internet-based methods to deliver PT is a feasible and potentially cost-effective approach to address the challenges of delivering PT in primary care settings. Internet delivery addresses issues related to parent participation by increasing accessibility, availability, and parent-controlled access.

    The ezPARENT Program

    The ezParent program is a delivery adaptation of the evidence-based, group delivered Chicago Parent Program (CPP). The program is a prevention intervention designed to promote parenting competence and prevent child behavior problems in children 2- to 5-years old [20,21]. Like the group-based program, the six modules of the ezParent program teach parents evidence-based strategies for encouraging good behavior and decreasing misbehavior in children. See Figure 1 for a list of module content. The ezParent Program was originally named the electronicCPP and changed to the ezParent Program in 2016. For a full description of the development of the program see Breitenstein et al [20].

    The ezParent program is a self-administered program that is downloaded onto an Android tablet computer as an app. It is one of the first adaptations of an evidence-based group delivered program to incorporate tablet-based technology. The program is set up so that users move sequentially through the modules (ie, parents must complete the first module prior to the second module being unlocked). Once a module is unlocked, parents can access the module and return to any portion of that module at a later time. We estimate that initial completion of one module would take approximately 1 hour [20].

    Like other PT programs, rather than focusing primarily on education or increasing knowledge, the ezParent program assists parents in acquiring a range of evidence-based parenting skills [13,20]. There are several unique aspects of the ezParent program developed to promote parent engagement and support learning. These are (1) video vignettes of parents interacting with their children, which support vicarious learning and present examples of the various parenting strategies, (2) knowledge questions, which assess parent understanding of the module content and provide added information when needed, (3) interactive “game” activities, which are intended to be fun and keep parents stimulated and engaged, and (4) module practice assignments, which provide opportunities for parents to practice what they are learning with their children [20].

    Consistent with the group-based CPP, the ezParent program is designed to be culturally and contextually relevant for low-income, ethnically diverse parents of young children [21]. During the development of the tablet program, we worked with low income and ethnic minority parents to assure the content was relevant and that the app was easy to use, attractive, and interesting [20]. The use of mobile technology allowed us to design the program to be interactive and accessible, a particularly salient feature for African American and Hispanic parents who are among the most active users of the mobile Internet [22,23]. Therefore, parent training via an app on mobile devices may be an ideal method to increase the reach and accessibility of parent training programs, particularly among low-income African-American and Hispanic families.

    Figure 1. ezPARENT program home page and module topics.
    View this figure

    Study Purpose and Hypothesis

    The purpose of this study was to establish the feasibility and efficacy of the ezParent program. To examine feasibility, we examined parent satisfaction with the program and the percent of modules completed relative to percent of face to face sessions attended in prior studies of the group-based CPP. To test efficacy, we examined the effect of the ezParent program against an attention control condition for improving parenting and child behavior in a sample of low-income ethnic minority parents of young children recruited from a primary care setting. We hypothesized that the percent of ezParent modules completed would be comparable to or greater than the percent of group-based CPP session completed; compared with an attention control group, parents in the ezParent intervention group would report using more positive discipline strategies, greater parenting self-efficacy, and decreased parenting stress and child behavior problems; and effect sizes for improvement in parent and child behaviors would be comparable to or greater than the group-based CPP.


    Methods

    Trial Design

    The study used a two-group randomized control trial (RCT) design with repeated measures follow-up. Subjects were randomly assigned to either the intervention (ezParent group) or attention control condition (health promotion group). Data collection in both conditions was at baseline and at 12 and 24 weeks post baseline. This study was reviewed and approved by the University’s institutional review board.

    Parents in the ezParent group received the six module ezParent program to be completed over a 12-week period (allowing approximately 2 weeks per module). During the same time period, parents in the health promotion control group received information on various topics (ie, nutrition, exercise, finance, safety, medical information, and entertainment) via a website that included links and portable document format handouts for each topic. These materials were the same health promotion handouts and websites distributed at the primary care pediatric recruitment site. The health promotion condition was designed to control for attention related to technology use over the same period of time.

    Sample

    Parents were eligible to participate in the study if they were the parent or legal guardian of a child between the ages of 2 to 5 years, the child was receiving or eligible to receive Medicaid insurance (eg, All Kids Assist or Share in Illinois), and the child received medical care at the pediatric primary care recruiting site. Only one parent per family participated in the study. If a parent had more than one child between the ages of 2 and 5 years, the parent selected one child to report on their behavior over the length of the study. Currently, the ezParent program is only available in English; therefore, parents who were unable to speak and read English were excluded.

    Parents were recruited between October 2013 and June 2014 from a large, urban, primary care pediatric clinic located on the near west side of Chicago. According to the clinic, most of the families they serve are African American (50%) or Latino (30%) and over 65% of families have low incomes or receive Medicaid. We advertised broadly at the primary care site using project flyers that contained information about the purpose of the study, inclusion criteria, expectations for participation, and contact information for parents to directly contact the study staff. An interest form was included in the flyer; parents completed the form indicating their willingness to be contacted by the study staff. If a parent was eligible and interested in participating, then a 2-hour meeting was scheduled in our research offices to complete the participant consent, baseline data collection, and randomization to condition.

    Figure 2 depicts the participant flow in the trial. Two hundred eighty-seven parents were assessed for eligibility. Of the 118 parents eligible for the study, 70.3% (83/118) of parents consented to participate, completed baseline assessments, and were randomized to either the ezParent or health promotion control condition. Of the 83 parents randomized, 79 parents completed follow-up assessments (95.2% retention rate).

    The sample was predominantly comprised of single, ethnic minority (African-American or Hispanic) mothers of young children earning annual incomes of less than $20,000 (see Table 1). There were no significant demographic differences between the intervention and control groups.

    Variables and Measures

    Module Completion and Satisfaction

    Parents earn a “module badge” at the end of each module and after they review the practice assignment. After completion of the module, parents are alerted with a pop-up in the program congratulating them on earning the module badge. Parents can view their earned badges in the “my badges” section of the program. The ezParent digital platform provides time stamps of when parents earn the module badge. Intervention dose was defined by parent receipt of the module badge. For this analysis, we included badges that were earned prior to the 12 week post baseline data collection appointment corresponding with the intervention completion period (eg, 6 modules × 2 weeks per module).

    Satisfaction was measured using an end of program survey administered 6 months after baseline. Parents rated how helpful they found the ezParent program (very helpful, a little helpful, or not at all helpful), their overall satisfaction with the program (very dissatisfied, dissatisfied, satisfied, or very satisfied), and whether they would recommend the ezParent program to other parents (not recommend, recommend, or highly recommend). Parents in the control group rated their perception of helpfulness (very helpful, a little helpful, or not at all helpful) of the information provided in the health promotion website (eg, nutrition, exercise, finances, safety, medical, and entertainment) and whether they would recommend the website to other parents (not recommend, recommend, or highly recommend).

    Figure 2. CONSORT flow diagram.
    View this figure
    Table 1. Study participant demographics (N=79; frequency [%]).
    View this table
    Parent Outcomes

    Parenting self-efficacy, behavior, and stress were assessed using self-report measures. The 38-item Toddler Care Questionnaire (TCQ) was used to measure parenting self-efficacy [24]. The TCQ measures parent self-efficacy in managing situations and tasks that are specific to raising young children. TCQ scale scores range from 38 (not at all confident) to 190 (very confident). Reliability (Cronbach’s alpha) for the TCQ for this sample was 0.94.

    Parent discipline strategies were measured using the 40-item Parenting Questionnaire (PQ) [25,26]. The PQ includes three discipline scales measuring parental warmth (Warmth), extent to which they follow through on discipline (Follow Through), and use of corporal punishment (Corporal Punishment). Parents rate each item on a scale of 1 (almost never) to 5 (very often). Reliability (Cronbach’s alpha) for the PQ scales were 0.88 (Warmth), 0.81 (Follow Through), and 0.66 (Corporal Punishment).

    The 36-item Parenting Stress Index-Short Form (PSI-SF) was used to measure parenting stress. The PSI-SF is derived from the validated 101-item PSI and mirrors the strong validity of the PSI [27]. Parents respond to items on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) resulting in a total stress score. Higher scores on the PSI-SF indicate higher parenting stress. The PSI-SF also includes a published cut-off score for clinically significant stress based on scores above the 85th percentile [28]. Reliability (Cronbach’s alpha) for the PSI-SF for this sample was 0.92.

    Child Behavior Problems

    Parents reported child behavior problems using the 36-item Eyberg Child Behavior Inventory (ECBI) [29]. The ECBI is for parents of children ages 2- to 16-years old and assesses problem behavior on two scales, the Intensity Scale and the Problem Scale. The Intensity Scale assesses the frequency of 36 problem behaviors on a 7-point scale ranging from 1 (the behavior never happens) to 7 (the behavior is always happening). The Problem Scale assesses parent perception of each of the behaviors as being problematic for him or her as a parent (yes or no). The ECBI includes cut off scores for each scale indicative of clinical significant child behavior problems based on scores that are 1.5 standard deviations (SD) above the mean (the 93rd percentile) [29]. Reliability (Cronbach’s alpha) for the ECBI for this sample was 0.92 (Intensity Scale) and 0.93 (Problem Scale).

    Group-Based CPP Archival Data

    The data obtained in this study were compared with group-based CPP archival data collected on a racially, ethnically, and socioeconomically comparable sample of 267 parents that received the intervention as part of a RCT [15]. There were no significant differences between the group-based intervention group (n=267) and the ezParent intervention group (n=40) for income or race. Data were compared on intervention dose (ie, percent of ezParent module completion versus percent of face-to-face parent sessions attended) and effect size changes in parenting self-efficacy (TCQ), parent discipline strategies (PQ), and child behavior (ECBI). Comparison of the ezParent and group-based CPP effect size estimates is important in establishing the comparability of the magnitude of treatment effects of the intervention using different delivery methods. The group-based CPP includes 12, 2-hour group sessions facilitated by two trained leaders [21]. The six modules of the ezParent program parallel the content of the first 10 sessions of the group-based CPP (see Textboxes 1 and 2) [20]. The 11th and 12th group sessions do not include new content and are used to help parents synthesize the knowledge and skills acquired from the prior sessions, these sessions correspond with the review materials included in the ezParent Program.


    Textbox 1. ezParent program.
    View this box

    Textbox 2. CPP-group based topics.
    View this box

    Procedures

    At their first study appointment, parents were enrolled in the study by completing the informed consent process and responding to all demographic and survey instruments. Consents and survey data were collected on an Internet-based data collection app developed for this study. The goal of using Internet-based data collection was to increase the efficiency and ease of data collection and entry. Prior to recruitment, study staff tested the data collection app for ease of use and accuracy of data collection. Once established, the data collection app was launched. Total time for completion of the surveys at baseline was 35 to 40 minutes. Randomization to group assignment was made using stratified random sampling based on time sequential cohorts. The randomization table was built into the data collection app and group assignment was made after all baseline data were acquired. Parents were assured that the study assignment was random and had no connection with their responses to the survey questions. After study assignment was revealed, all parents in the control and intervention groups were given Android tablets to use for the duration of the study.

    Intervention Fidelity

    Several strategies were implemented to ensure intervention fidelity (eg, strategies to monitor and enhance the reliability and validity of the intervention) [30]. After study assignment was revealed, the study staff provided a structured training (using written and verbal information) to all participants on how to use the tablet. Additional training was then provided specific to assigned condition. Following training, parents were prompted to verbalize their understanding and conducted return demonstrations to assess ability to use the tablet and the health promotion website (control) or ezParent app (intervention).

    Parents in the ezParent condition received automated text message reminders to complete the ezParent program if they had not completed the module content. Automated text messages also included positive encouragement regarding completion of the program and of practice assignments. These messages were consistent across all intervention parents.

    Analytic Approach

    Parent participants were included in the analysis nondependent on their adherence or nonadherence to the ezParent program. Parents (n=4) who were lost to follow up after enrollment were not included because we were unable to collect any follow up data for outcome analysis (see Figure 2). Parent completion of the ezParent modules was calculated as the percentage of the six modules completed by each of the intervention group parents (n=40). For comparison to the group-based CPP, intervention attendance was calculated as the percentage of sessions 1 through 10 of the CPP group attended by intervention group parents (n=267). This analysis is based on content completion not on the amount of time spent in either the group-based or tablet-based program. Content across the ezParent modules and the CPP group sessions is comparable (see Textboxes 1 and 2). The ezParent and group based rates of participation were compared using 95% confidence intervals (CI). Satisfaction data were summarized using item frequencies.

    Chi square tests were used to compare baseline data between parents in the ezParent and health promotion control group and determine if there were significant differences. Parent and child data were analyzed using a repeated measures analysis of variance (RM-ANOVA) with simple contrasts to determine if there were significant intervention effects on the outcome measures from baseline to 12 weeks post baseline (T1 to T2) and from baseline to 24 weeks post intervention (T1 to T3).

    Effect sizes for between group comparisons were calculated for all outcome variables using Cohen’s d [31]. For comparison to the group-based CPP, we calculated effect sizes from data published by Breitenstein and colleagues [15] (effect size comparisons for the PSI-SF are not included as this measure was not part of the group-based evaluation). Effect sizes were calculated T1 to T2 and for T1 to T3 using the following equation:

    ( ͞xt1 − ͞xt2) − ( ͞xc2 − ͞xc1) /SDpooled

    where ͞xt2 = treatment group mean at time 2; ͞xt1 = treatment group mean at time 1; ͞xc2 = control group mean at time 2; ͞xc1 = control group mean at time 1; and SDpooled = the pooled SDs of all reported groups (eg, treatment and control) [31].


    Results

    ezPARENT Module Completion

    Table 2 presents the percent of the sample completing each module (ie, earned a module badge) during the first 3 months of the study and the percent of parents in the archival comparison sample attending the face to face CPP group-based sessions. All parents in the ezParent sample earned the module 1 completion badge. Across all six modules, the average ezParent module completion by parent was 85.4% (34/40) of modules (95% CI = 74.5%-96.4%). In contrast, average parent attendance for the face-to-face CPP groups was 50.6% (135/267) of sessions (95% CI = 44.6%-56.6%), and 24% (64/267) of the archival CPP sample failed to attend any sessions [15]. The ezParent dose rate was higher than the upper CI for the group CPP dose, suggesting that parent participation rates of the ezParent and group based CPP were significantly different (P<.05).

    Table 2. ezParent module (n=40) and CPP group-base (n=267) Dose.
    View this table

    Satisfaction

    Of the parents in the ezParent group, 87.5% (35/40) reported the ezParent program being very helpful and 12.5% (5/40) a little helpful. In the control group parent reports of helpfulness of the topics on the health promotion website (control group) ranged from 46.2% (18/39) to 74.4% (29/39) very helpful; 20.5% (8/39) to 28.2% (11/39) a little helpful; and 2.6% (1/39) to 5.1% (2/39) not at all helpful. The nutrition topic in the website was the highest ranked as very helpful. Of parents in the ezParent condition, 42.5% (17/40) reported that it was a little bit hard to regularly use the ezParent program, 2.5% (1/40) very hard, and 52.5% (21/40) not at all hard. Satisfaction with the ezParent program was high, with 80.0% (32/40) of parents reporting they were very satisfied and 20.0% (8/40) reporting they were satisfied with the program.

    Parents in the ezParent group were more likely to recommend the ezParent Program to other parents than parents recommending the health promotion site (Kendall’s Tau-b = 0.25; P<0.05). Overall, 82.5% (33/40) of the ezParent group reported they would highly recommend, 15.0% (6/40) would recommend, and 2.5% (1/40) didn’t know if they would recommend the program to another parent. In contrast, 59.0% (23/39) of parents in the health promotion control group reported they would highly recommend, 35.9% (14/39) would recommend, 2.6% (1/39) would not recommend, and 2.6% (1/39) didn’t know if they would recommend the website to another parent.

    Parent and Child Outcomes

    Table 3 presents means and SDs for each variable by condition at each data collection time point. At baseline, there were no significant differences between the ezParent and health promotion control groups for parent behavior (PQ), self-efficacy (TCQ), and stress (PSI-SF), or child behavior (ECBI). At baseline, 6.3% (5/79) of the sample scored above the PSI cut off score (>109) [28]; 25.3% (20/79) scored above the ECBI problem scale cut off score (>14); and 19% (15/79) of the sample scored above the ECBI intensity scale cut-off score (>130) [29].

    From T1 to T3, there was a significant difference between conditions for parenting warmth on the PQ (F1,77 = 4.82, P<.05). There were no significant differences between the intervention and control conditions on parents’ reports of their follow through on discipline, use of corporal punishment, parenting stress, parenting self-efficacy, or child behavior problems.

    In Table 4, we present effect size estimates for the ezParent program and comparison effect sizes for the group-based CPP [15]. A Cohen’s d of .2 is considered a small effect, .5 a medium effect, and .8 or larger a large effect [31]. Effect sizes from T1 to T3 were in the small range for parent warmth (PQ), parent corporal punishment (PQ), follow through (PQ), parenting stress (PSI-SF), and child behavior problems based on the ECBI intensity scale. With the exception of parenting self-efficacy (TCQ) and corporal punishment (PQ), ezParent effect sizes were comparable or greater than those of the group-based CPP.

    Table 3. Mean and standard deviation of outcome variables for intervention (n=40) and control (n=39) groups.
    View this table
    Table 4. Comparison of between group effect size estimates of ezParent Program (n=40 control; n=39 intervention) and group CPP (n=237 control; n=267 intervention).
    View this table

    Discussion

    Principal Findings

    In this study we evaluated the feasibility, acceptability, and preliminary efficacy of the tablet-based adaptation of the CPP, the ezParent program, in a sample of low-income, ethnic minority parents of 2- to 5-year-old children seen in primary care. The CPP is an evidence-based parenting skills program originally designed to be delivered in a face-to-face parent group format [21]. However, like many preventive interventions targeting low-income families that require face-to-face involvement, participation rates have typically been low [14,16]. Moreover, pediatric primary care providers, who serve a large population of families receiving Medicaid, are often unable to devote the time needed to provide parenting skills training and support. These problems diminish the impact and reach of evidence-based parenting programs. Given the challenges of using face-to-face formats for parenting skills training, we were particularly interested in examining the amount of intervention (ie, dose) parents would receive using a self-administered digital format for acquiring parenting skills training compared with a face-to-face group format.

    Overall, the results indicate that the ezParent program is feasible and acceptable in a low-income, ethnic minority population of parents. Compared with a control group of parents using a health promotion website, ezParent program parents were more likely to report that they would recommend the program to another parent. Compared with a comparable archival sample of parents enrolled in the face to face group-based CPP, parents enrolled in ezParent received a higher dose of parenting content. Specifically, parents in the ezParent group completed over 85.0% (34/40) of the modules compared with parents in the face-to-face format who attended on average only 50.6% (135/267) of the parent group sessions. Moreover, all of the ezParent participants completed at least one module whereas 24.0% (64/267) of parents enrolled in the face-to-face group format, never attended a single session. These findings are consistent with a recent review of digitally delivered parenting interventions reporting that 41.7% to 99.2% of participants completed their digitally delivered parent training interventions [32]. Furthermore, these data demonstrate that a digital format for delivering parenting skills training to a predominantly low-income ethnic minority population of parents is feasible and acceptable, and may greatly extend the reach for helping more parents and young children in primary care.

    Although module completion rates in this study were promising, there is more work to be done to understand patterns of parent use of the ezParent program. A benefit of using technology is the ability to track all aspects of parent usage of the app. Digital tracking data can provide important information regarding parent behavior in interacting with the intervention, specifically, how and when parents use the ezParent program and what content they access. For example, information regarding the number of visits parents make to components of the program, time stamps of visits to each portion of the program, and the frequency which parents return to completed content. In future work, we will conduct detailed analysis of this usage data. Indeed, research of Internet-delivered interventions is beginning to emerge related to patterns of program use and intervention outcome [33,34]. A fine tune analysis of usage metrics will expand our knowledge related to dose and program effectiveness.

    Overall, we found the effects of the ezParent program on parenting and child behavior problems were in the small range and many were comparable to the effect sizes obtained from archival samples using the CPP in the face-to-face format. Although significant improvements were found for parent use of warmth in their discipline, no improvements were found for parent use of corporal punishment, following through on discipline, parenting stress, and child behavior problems relative to the control group. There are three possible reasons for these results. First, the power calculation for this study was based on estimated effect sizes for intervention dose rather than parent and child outcomes. Much larger sample sizes would be needed to detect significant differences based on small effects, which are typical for prevention studies [35,36]. Therefore, we may not have adequate power to detect differences between the intervention and control parents in this analysis.

    Second, although we recruited from a population of families with social risk factors related to parent and child dysfunction (eg, low income), the parents in our study reported low levels of parenting stress and relatively few child behavior problems at baseline. Less than one-quarter of the parents reported child behavior problems exceeding the cut point for clinically significant problems and less than 6.3% (5/79) reported clinically significant levels of parenting stress. Therefore, floor effects in this relatively healthy population may have made it difficult to detect improvements over time. Greater effects may be more detectable in a referred population of parents and young children.

    Third, the lack of effects may be due to the limited amount of time allotted for assessing parent and child behavior change. It is possible that more time for parents to absorb the new information, practice the new skills, and observe changes in themselves and their children may yield greater improvements in discipline strategies, parenting self-efficacy, and child behavior.

    Study Limitations and Future Directions

    Although our findings indicate feasibility and acceptability of the ezParent program and comparable effect sizes to face-to-face delivery for parents recruited from primary care we recruited from one location from a relatively healthy population. It is possible that parents involved in this study were highly motivated to participate and learn from the intervention. Future studies of the ezParent program will employ a larger sample size from multiple primary care sites and longer term follow-up to establish the efficacy of the ezParent program in modifying parent and child behaviors. In addition, to lay the ground work for full scale implementation in primary care sites, implementation, and cost effectiveness of the ezParent program in primary care will be determined.

    Using an archival sample for a comparison group of delivery methods presents methodological limitations because of the lack of experimental control across the two groups. Although not a purpose of the original evaluation of the ezParent program, we felt it was important to demonstrate an initial comparison of dose and outcome to the group-based CPP. Our analysis suggests they may be comparable but a true test would require a randomized design comparing group-based CPP to ezParent. Indeed, a RCT structured as an equivalence or noninferiority trial with an adequately powered sample size would allow a head to head comparison of the two formats, directly compare intervention outcomes and dose, and allow us to understand which parents are likely to benefit from each delivery format.

    Another potential limitation is the use of parent report to evaluate the intervention. A more robust measure of intervention efficacy could include multi-informant and multimethod data. However, previous findings from RCTs of the group-based CPP have found significant findings across three sources of data: parent self-report, teacher report, and parent-child observation [15,37]. In these studies, the parent self-report findings have consistently agreed with other methods of measuring parent and child outcomes.

    It is important to understand how parents use the information and program. In this analysis we assessed one metric of program usage – module completion. Although module completion provides an overall picture of parent dose it does not provide us with specific information related to patterns of use, time spent on the program, and other important data of ezParent use. In addition, we assessed program use from baseline to the 12-week follow-up even though parents had access to the program throughout the 24-week study period. Therefore, as previously noted, an analysis of usage data is critical in understanding patterns using the ezParent intervention and parent’s ongoing use of information after the initial module completion. This information will help understand the change mechanisms of the intervention. Further, demonstrating the processes and patterns related to using tablet-based behavior change interventions is important to the field of eHealth behavioral interventions.

    Acknowledgments

    This study was supported by a grant from The Robert Wood Johnson Nurse Faculty Scholars Program (Grant #73016). SB is the study PI. She conceived the study and drafted this manuscript, DG revised the paper, and all authors provided feedback. DA collected the data and developed protocols and assisted in developing manuscript. LF and EO performed the analysis and reviewed the results. The authors gratefully acknowledge the parents participating in the study, the staff at the recruitment site, and to all the families that have informed this work.

    Conflicts of Interest

    Under an agreement between Rush University Medical Center and Dr. Deborah Gross. Dr. Gross is entitled to revenue from sales of the Chicago Parent Program and the adaptation described in this article. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflict of interest policies.

    References

    1. Campbell SB, Spieker S, Burchinal M, Poe MD, NICHD Early Child Care Research Network. Trajectories of aggression from toddlerhood to age 9 predict academic and social functioning through age 12. J Child Psychol Psychiatry 2006;47:791-800. [CrossRef] [Medline]
    2. Merikangas KR, He J, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2010;49:980-989 [FREE Full text] [CrossRef] [Medline]
    3. Shaw DS, Lacourse E, Nagin DS. Developmental trajectories of conduct problems and hyperactivity from ages 2 to 10. J Child Psychol Psychiatry 2005;46:931-942. [CrossRef] [Medline]
    4. Halfon N, Houtrow A, Larson K, Newacheck PW. The Changing Landscape of Disability in Childhood. The Future of Children 2012;22:13-42. [CrossRef] [Medline]
    5. Slomski A. Chronic mental health issues in children now loom larger than physical problems. JAMA 2012;308:223-225. [CrossRef] [Medline]
    6. Tremblay RE, Nagin DS, Seguin JR, Zoccolillo M, Boivin M, Pe´russe D, et al. Physical aggression during early childhood: trajectories and predictors. Pediatrics 2004;114:e43-e50. [CrossRef] [Medline]
    7. Humphries ML, Keenan KE. Theoretical, developmental & cultural orientations of school-based prevention programs for preschoolers. Clin Child Fam Psychol Rev 2006;9:135-148. [CrossRef] [Medline]
    8. Berkout OV, Gross AM. Externalizing behavior challenges within primary care settings. Aggression and Violent Behavior 2013;18:491-495. [CrossRef]
    9. Polaha J, Dalton WT, Allen S. The prevalence of emotional and behavior problems in pediatric primary care serving rural children. J Pediatr Psychol 2011;36:652-660 [FREE Full text] [CrossRef] [Medline]
    10. Pearl ES. Parent management training for reducing oppositional and aggressive behavior in preschoolers. Aggression and Violent Behavior 2009;14:295-305. [CrossRef]
    11. Barlow J, Smailagic N, Ferriter M, Bennett C, Jones H. Group-based parent-training programmes for improving emotional and behavioural adjustment in children from birth to three years old. Cochrane Database Syst Rev 2010(3):CD003680 [FREE Full text] [CrossRef] [Medline]
    12. Lundahl B, Risser HJ, Lovejoy MC. A meta-analysis of parent training: moderators and follow-up effects. Clin Psychol Rev 2006;26:86-104. [CrossRef] [Medline]
    13. Kaminski JW, Valle LA, Filene JH, Boyle CL. A meta-analytic review of components associated with parent training program effectiveness. J Abnorm Child Psychol 2008;36:567-589. [CrossRef] [Medline]
    14. Heinrichs N, Bertram H, Kuschel A, Hahlweg K. Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: barriers to research and program participation. Prev Sci 2005;6:275-286. [CrossRef] [Medline]
    15. Breitenstein SM, Gross D, Fogg L, Ridge A, Garvey C, Julion W, et al. The Chicago Parent Program: comparing 1-year outcomes for African American and Latino parents of young children. Res Nurs Health 2012;35:475-489 [FREE Full text] [CrossRef] [Medline]
    16. Nix RL, Bierman KL, McMahon RJ, Conduct Problems Prevention Research Group. How attendance and quality of participation affect treatment response to parent management training. J Consult Clin Psychol 2009;77:429-438 [FREE Full text] [CrossRef] [Medline]
    17. Talmi A, Fazio E. Commentary: promoting health and well-being in pediatric primary care settings: using health and behavior codes at routine well-child visits. J Pediatr Psychol 2012;37:496-502 [FREE Full text] [CrossRef] [Medline]
    18. Olson LM, Inkelas M, Halfon N, Schuster MA, O'Connor KG, Mistry R. Overview of the content of health supervision for young children: reports from parents and pediatricians. Pediatrics 2004;113(6 Suppl):1907-1916. [Medline]
    19. Meadows T, Valleley R, Haack MK, Thorson R, Evans J. Physician "costs" in providing behavioral health in primary care. Clin Pediatr (Phila) 2011;50:447-455. [CrossRef] [Medline]
    20. Breitenstein SM, Shane J, Julion W, Gross D. Developing the eCPP: adapting an evidence-based parent training program for digital delivery in primary care settings. Worldviews Evid Based Nurs 2015;12:31-40. [CrossRef] [Medline]
    21. Gross D, Garvey C, Julion W, Fogg L. Preventive parent training with low-income, ethnic minority families of preschoolers. In: Briesmeister J, Schaefer C, editors. Handbook of parent training: helping parents prevent and solve problem behaviors. Hoboken, N.J: John Wiley & Sons; 2007:31-40.
    22. Smith A. Pew Research Center Internet and American Life Project Website. 2010. Technology trends among people of color   URL: http://www.pewinternet.org/Commentary/2010/September/ [accessed 2011-08-15] [WebCite Cache]
    23. Washington J. Pew Research Center Internet and American Life Project Website. 2011. For minorities, new "digital divide" seen   URL: http://usatoday30.usatoday.com/tech/news/2011-01-10-minorities-online_n.htm [accessed 2016-01-05] [WebCite Cache]
    24. Gross D, Rocissano L. Maternal confidence in toddlerhood: its measurement for clinical practice and research. Nurse Pract 1988;13:19-28. [Medline]
    25. Gross D, Fogg L, Garvey C, Julion W. Behavior problems in young children: an analysis of cross-informant agreements and disagreements. Res Nurs Health 2004;27:413-425. [CrossRef] [Medline]
    26. McCabe KM, Clark R, Barnett D. Family protective factors among urban African American youth. J Clin Child Psychol 1999;28:137-150. [CrossRef] [Medline]
    27. Abidin R. Parenting Stress Index, 3rd edition. Odessa: Psychological Assessment Resources, Inc; 1995.
    28. Abidin R. Parenting Stress Index 4th Edition-Professional Manual. Lutz: Psychological Assessment Resources, Inc; 2012.
    29. Eyberg S, Pincus D. Eyberg Child Behavior Inventory & Sutter-Eyberg Student Behavior Inventory-Revised: Professional Manual. Odessa: Psychological Assessment Resources, Inc; 1999.
    30. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Treatment Fidelity Workgroup of the NIH Behavior Change Consortium. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol 2004;23:443-451. [CrossRef] [Medline]
    31. Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd edition. Hillsdale: Lawrence Erlbaum Associates, Inc; 1988.
    32. Breitenstein SM, Gross D, Christophersen R. Digital delivery methods of parenting training interventions: a systematic review. Worldviews Evid Based Nurs 2014;11:168-176. [CrossRef] [Medline]
    33. Whitton AE, Proudfoot J, Clarke J, Birch M, Parker G, Manicavasagar V, et al. Breaking open the black box: isolating the most potent features of a Web and mobile phone-based intervention for depression, anxiety, and stress. JMIR Mental Health 2015;2:e3. [CrossRef]
    34. Van Gemert-Pijnen Julia Ewc, Kelders SM, Bohlmeijer ET. Understanding the usage of content in a mental health intervention for depression: an analysis of log data. J Med Internet Res 2014;16:e27 [FREE Full text] [CrossRef] [Medline]
    35. Wilson SJ, Lipsey MW. School-based interventions for aggressive and disruptive behavior: update of a meta-analysis. Am J Prev Med 2007;33(2 Suppl):S130-S143 [FREE Full text] [CrossRef] [Medline]
    36. Merzel C, D’Afflitti J. Reconsidering community-based health promotion: promise, performance, and potential. Am J Public Health 2003;93:557-574. [CrossRef]
    37. Gross D, Garvey C, Julion W, Fogg L, Tucker S, Mokros H. Efficacy of the Chicago parent program with low-income African American and Latino parents of young children. Prev Sci 2009;10:54-65 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    CI: confidence interval
    CPP: Chicago Parent Program
    ECBI: Eyberg child behavior inventory
    PT: parent training
    PQ: parenting questionnaire
    RCT: randomized controlled trial
    RM-ANOVA: repeated measures analysis of variance
    PQ: parent questionnaire
    PSI-SF: parenting stress index-short form
    SD: standard deviation
    T1: baseline
    T2: 3 months post baseline T3: 6 months post baseline
    TCQ: toddler care questionnaire


    Edited by G Eysenbach; submitted 06.10.15; peer-reviewed by P Enebrink, J Doty; comments to author 29.10.15; revised version received 30.11.15; accepted 03.01.16; published 20.04.16

    ©Susan M. Breitenstein, Louis Fogg, Edith V. Ocampo, Diana I. Acosta, Deborah Gross. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 20.04.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.