JOURNAL OF CHILD AND ADOLESCENT PSYCHOPHARMACOLOGY

Volume 26, Number 8, 2016

Mary Ann Liebert, Inc.
Pp. 740749
DOI: 10.1089/cap.2016.0048

Review Article

Digital Parent Training for Children with Disruptive Behaviors:

Systematic Review and Meta-Analysis of Randomized Trials
Amit Baumel, PhD,1,2 Aditya Pawar, MD,1,2 John M. Kane, MD,1,2 and Christoph U. Correll, MD1,2

Abstract

Objectives: Digital-based parent training (DPT) programs for parents of children with disruptive behaviors have been
developed and tested in randomized trials. The aim of this study was to quantitatively assess the efficacy of these programs
versus a control condition.
Methods: We conducted a systematic review and random effects meta-analysis of peer-reviewed randomized studies
comparing DPT targeting children with disruptive behaviors versus a control group (wait list or no treatment).
Results: Altogether, seven studies (n = 718) were meta-analyzed. Compared to the control groups, DPT resulted in significantly greater improvement in child behavior (effect size [ES] = 0.44, 95% confidence interval [CI] = 0.210.66, studies = 7),
parent behavior (ES = 0.41, 95% CI = 0.250.57, studies = 6), and parental confidence (ES = 0.36, 95% CI = 0.120.59,
studies = 4). The improvement in child behavior was moderated by age group and severity of clinical presentation, which
overlapped 100%. While DPT was superior to control conditions in studies of young children (mean age <7 years) with a
clinical range of disruptive behaviors (ES = 0.61, 95% CI = 0.400.82, studies = 4), results were nonsignificant in studies of
older children (mean age >11 years) with a nonclinical range of symptoms (ES = 0.21, 95% CI = -0.01 to 0.42, studies = 3).
Analyses yielded similar results of higher ESs favoring studies of young children with clinical range disruptive behaviors for
parent behavior and parental confidence, but the differences were not significant. Results further suggested that in studies of
younger children, interactive programs (e.g., computerized programs) were more effective in improving child behavior
compared to noninteractive programs (e.g., watching video clips) ( p < 0.05).
Conclusion: Although additional studies are needed, DPT holds promise as a potentially scalable evidence-based treatment
of children with disruptive behaviors that can save human resources.
Keywords: parent, training, technology, conduct, oppositional, treatment
Introduction

isruptive behavior disorders and related difficulties (e.g.,

aggression, defiance, noncompliance) are among the most
prevalent types of psychiatric problems affecting children and adolescents (Comer et al. 2013). In an epidemiological study conducted in 2009, the median 12-month prevalence of conduct
disorder or oppositional defiant disorder was 6% (Merikangas et al.
2009), and several studies reported disruptive behaviors to be
among the most common reasons youth were referred to mental
healthcare (e.g., Egger and Angold 2006; Zisser and Eyberg 2010).
Given the prevalence and wide-ranging consequences of disruptive behavior disorder problems to families and society, research
has focused on the development of effective interventions for their
prevention and treatment. Among interventions, behavioral parent
training (BPT) is considered the first choice treatment (Eyberg et al.
2008; Chorpita et al. 2011). However, barriers to receiving such
interventions exist, including the lack of trained staff to provide this
evidence-based treatment, stigma related to receiving mental health
1
2

treatment, and the difficulties to access and engage in treatment in

terms of costs, time, and location (Kazdin and Blase 2011; Kazdin
and Rabbitt 2013). These barriers lead to poor quality of care for
youth with disruptive behaviors. For example, research suggests
that psychotropic medications are prescribed for a majority of youth
with disruptive behaviors without attempting psychological intervention, such as BPT (Olfson et al. 2012, 2014), exposing youth to
the risk of concerning antipsychotic adverse effects, including potentially life-shortening cardiometabolic risks (Correll et al. 2009;
Maayan and Correll 2011; Galling et al. 2016).
Technology-based programs offer a host of features that can
address most of these barriers and might be particularly useful for
parents of youth with relevant disruptive behavior problems. First,
technology-based interventions move the intervention outside of
traditional clinics and into peoples homes, increasing access and
reducing stigma. Services can be available 24/7, on demand, which
might be useful for people whose schedules are difficult and unpredictable. Furthermore, scalability of technology-based services
results in a significant decrease of program costs. Accordingly, the

Psychiatry Research, Northwell Health, Zucker Hillside Hospital, Glen Oaks, New York.
Hofstra North Shore LIJ School of Medicine, The Feinstein Institute for Medical Research, Hempstead, New York.

740

DIGITAL PARENT TRAINING: A META-ANALYSIS

advantages of technology-based solutions have been tested by researchers studying BPT for youth with disruptive behaviors. Specifically, research suggests that technologies could play a significant role
in parent training by engaging families into services (Breitenstein et al.
2014; Sourander 2015), training of staff, and in serving as the primary
delivery source of BPT ( Jones et al. 2013).
These suggestions are in line with the fact that BPT was among
the first to test technology in its early and primitive forms (e.g.,
videotapes) as the primary delivery source of an intervention
(Webster-Stratton et al. 1988). Subsequently, digital-based parent
training (DPT) programs for disruptive behaviors were developed
and tested in randomized trials. There are a few narrative reviews
on DPTs (e.g., Breitenstein et al. 2014; Jones et al. 2013), which
introduce the ways technology can be used to improve BPT for
promoting childrens mental health. None of these articles, however, systematically reviewed or meta-analyzed the results of randomized studies of DPT for youth with disruptive behaviors. The
aim of this meta-analysis was, therefore, to meta-analytically assess
the effects of DPTs for treating youth with disruptive behaviors to
inform stakeholders regarding this interventions potential and to
identify main challenges for future consideration.
Methods
This review and following meta-analysis were carried out in line
with the Meta-Analysis Reporting Standards (Publications and
Communications Board Working Group on Journal Article Reporting 2008) and PRISMA statement (Liberati et al. 2009) for
reporting meta-analysis.
Selection of studies
Six criteria were used to select studies for inclusion. First, the study
had to report on a parent training intervention targeting their childs
disruptive behavior problems (but not targeting criminal activities or
child maltreatment), measured using a valid scale with established
reliability and validity properties (e.g., Eyberg Child Behavior Inventory [ECBI]; Eyberg and Robinson 1983). Accordingly, we did
not include studies of general parent training targeting child rearing,
as their focus is not specifically to improve behavioral problems.
Attention-deficit/hyperactivity disorder (ADHD) studies were included if the primary goal was to target childrens externalizing behavior. Second, the program was designed to use digital media or
software programs (e.g., DVD, online based), primarily to replace
human support, and not to be primarily used within a therapy setting
(e.g., group, face-to-face) or in parallel to sessions with a therapist or a
coach. Accordingly, studies investigating the efficacy of therapistassisted BPT or investigating traditional therapy delivered through
teleconference were excluded. Third, the study reported on families
in which the targeted children did not have developmental delay or
health impairment. Fourth, the study reported meta-analyzable outcomes, as detailed in the coding section, regarding child behavior,
parent behavior, or parental confidence in self-efficacy. Fifth, the
study had at least one treatment and one control group, drawn from the
same population, with at least five participants per group. Sixth, the
study was published in English and in a peer-reviewed source.
Computer searches of PubMed, Psychinfo, and Embase databases were conducted for all published studies from 1974 to December 21, 2015. We used a combination of keywords for parent,
training, technologies (e.g., online, internet, computerized, mobile), and disruptive behavior problems (see Appendix for complete
Psychinfo search terms) and keywords for randomized trials,
aiming to start with a broad and sensitive search strategy (Robinson

741
and Dickersin 2002). A search for reference to technology was also
conducted by author name, using the names of known experts in the
area of BPT programs (e.g., Dumas, Eyberg, Forehand, Jones,
Kazdin, Patterson, Sanders, Webster-Stratton). Additional articles
were identified by hand searching the references of retrieved articles and previous reviews. The first two authors, a clinical psychologist (A.B.) and psychiatrist (A.P.), independently assessed all
potentially relevant articles for inclusion. Any disagreements were
resolved through discussion and consensus.
Coding of studies
Eligible studies were reviewed, and data on dependent measures
and study characteristics were retrieved in accordance to previous
large meta-analysis of parent training (Lundahl et al. 2006) and as
detailed below. Individual effect sizes (ESs) were extracted and
coded for three dependent outcomes: (i) child behavior was computed based on standardized parent report measures, such as the
ECBI (Eyberg and Robinson 1983), (ii) parent behavior was computed based on parent reports on their behaviors and interactions with
the child using measures such as the Parenting Scale (Arnold et al.
1993), and (iii) parental confidence (in self-efficacy) was computed
based on parental reports using measures, such as the Parenting
Sense of Competence scale ( Johnston and Mash 1989) or the Parenting Task Checklist, each of which assess parents task-specific
self-efficacy (Sanders and Woolley 2005). In studies with more than
one measure for either one of the three dependent outcomes (child
behavior, parent behavior, parental confidence), one pooled ES per
outcome was computed based on all available relevant measures.
Several study design and participant characteristics were coded
and categorized based on conceptual and data distribution aspects.
Childs mean age was coded into two groups: young children (37
84 months) and middle school children (138162 months). Severity
level of undesirable child behavior at the beginning of treatment
was coded as clinical when data indicated a clinical range of
disruptive behavior disorders in the sample and was coded as
nonclinical if data indicated that the study did not include mostly
children in the clinical range of disruptive behavior disorders,
based on standard measures or inclusion criteria. Family socioeconomic status (SES) was coded into low, medium, and high. We
based the coding on data, including parental employment rates,
income, and education (e.g., 80% of sample households income
above average was coded as high; sample consisting 73.3% of
household income below the median annual income in the United
States was coded as low). Interventions were coded into either
noninteractive when the intervention was delivered through the use
of noninteractive digital videos or audio (e.g., viewing a series of
clips) or interactive when the intervention enabled the participants
to control pacing, choose different pathways to growth, or receive
feedback (Segal et al. 2003; Baumel and Muench 2016). Studies
were independently coded by the first and second authors revealing
100% agreement on each of the categories described above;
therefore, inter-rater reliability was not computed further.
Assessment of methodological quality and procedures
Methodological quality was assessed with the Cochrane Collaboration risk of bias tool that addresses different domains of
methodological quality, including random sequence generation,
incomplete outcome data, and other biases (Higgins et al. 2011).
The other bias domain was assessed through the following criterion: balance/imbalance of demographic and clinical measurements baseline characteristics across study groups. Since blinding

742

BAUMEL ET AL.

of study participants and study staff is not feasible in these nondrug

studies (Higgins et al. 2011), we do not present this assessment
item. Two reviewers (A.B., A.P.) independently assessed the
methodological study quality. Any disagreements were resolved
through discussion and consensus.
Data analysis
We organized our findings in six separate analytic data files,
each relating to one of the three examined outcomes (child behavior, parent behavior, parental confidence) and one of the two
time frames (i.e., immediate treatment effect and post-treatment
follow-up effect). Immediate treatment effect related to the difference in outcome between the intervention and control group
immediately after the intervention. Post-treatment follow-up effect
related to the within treatment group difference in the intervention
arm from the end of treatment until the end of follow-up time.
Cohens d ESs (Wilson and Lipsey 2001) were calculated using
Comprehensive Meta-Analysis software (version 3.3.070) using
random-effects models (DerSimonian and Laird 1986). Unlike
fixed-effects models, random-effects models assume that the true
ES may vary between studies due to different population parameters. Therefore, compared with the fixed effects model, the weights
assigned under random effects are more balanced (Borenstein et al.
2007). All ESs are organized in a way that a larger positive effect reflects bigger positive improvement in the relevant outcome,
either in the intervention group versus the control condition
(treatment effect adjusted for control effect) or at the end of posttreatment follow-up compared to end of the intervention (maintenance effect in the intervention group). Because post-treatment
follow-up effects were calculated by examining the difference
between outcome measures at the end of the intervention and the
outcomes at the end of post-treatment follow-up, a nonsignificant
outcome indicates that the effect achieved at the end of treatment
was sustained through the follow-up period despite cessation of the
intervention. On the contrary, a significant change with a negative

FIG. 1.

effect would mean that the effect achieved at the end of treatment
was not sustained through the follow-up period and that prior gains
were diminished or lost after the end of the active intervention. We
used the means and standard deviations reported in the studies
instead of already computed ESs or significance measurements.
Z-scores were used to assess the significance of pooled effects
sizes, and homogeneities of ESs were assessed with the Q statistic
(Hedges and Olkin 2014) and I2 to describe the percentage of
variation due to heterogeneity (Higgins and Thompson 2002).
Potential moderators were evaluated by computing Qbetween tests.
Results
Search results
The electronic and manual searches resulted in 7195 hits after
removing duplicates. Through the first screening process, 63 articles were identified and retrieved for detailed evaluation (Fig. 1).
Characteristics of included studies
A total of seven studies with 718 participants (329 in the intervention condition, 389 in the control condition) met all inclusion
criteria. Selected characteristics of these studies are presented in
Table 1. In all studies, the control condition groups did not receive
any treatment during the intervention time. Four studies were
aimed at children with an average age ranging between 3.9 and 6.8
years and three were aimed at children with an average age between
11.8 and 14 years. While all four studies targeting younger children
consisted of samples in the clinical range of elevated disruptive
behavior symptoms, all three studies targeting older children consisted of samples with data suggesting a magnitude of disruptive
behaviors that was mostly below the clinical range of symptoms.
Regarding the SES of the samples, two out of the three studies
aimed for younger children with SES information included samples
within the range of high SES and two out of the three studies aimed
for older children had samples within the range of low SES.

Flow diagram of study selection.

743

312 (6.8)

Enebrink
et al. (2012)

Kacir (2000)

At least four identified

behaviors from a list of
problematic behaviors
(e.g., trouble at school)

1 SD above the mean of

ECBI

Parent concerns regarding

children behavior
problems. (ECBI problems
mean < 12)

Clinical symptoms
of the sample

1218 (14.4) 58% of participants

demonstrated some
elevations (scores of 9 or
greater) on the EXBI
problems.

(13.1)

915 (11.8)

Cefai et al. (2010)

Irvine et al. (2015)

Child agea

Study

The intervention was based on a Swedish parent training program, Comet, that
targets parents reactions and activities to promote behavioral change in the
parent and then the child (Kling et al. 2010).
The intervention consisted of 7 online interactive sessions distributed over 10
weeks (some session homework took two to three weeks to complete) relating
to topics such as extinction of negative behavior and positive interaction.
Sessions were composed of written text, videos of interactions between a
parent/child, and illustrations. Other features included downloadable material
(e.g., reward system), multiple-choice questions with direct feedback, online
diary keeping parents reports on their training and past week, and the
possibility to pose questions to the research assistant and write comments to
homework or to a monitored parenting discussion forum. The research
assistants gave feedback (e.g., reinforcement on progress) through the website
on work and distributed a new session each week.

The Parenting Wiselys main aim was to modify parentchild interactions that
reinforce antisocial behavior, using refraining and cognitive restructuring
methods to foster behavior change (Gordon and Stanar 2003).
The intervention consisted of a self-administered software program (CD-ROM)
with scenario-based learning design and a supplementary workbook. After
selecting a problem, parents viewed a video clip depicting a family struggling
with that problem. Parents selected a solution to the problem, viewed a video
enactment of their solution, and participated in a critique of that choice.
Strategies such as contracting, contingency management, I-statements, active
listening, assertive discipline, and praise were presented. After viewing the
most effective solution, a series of multiple-choice questions reviewed the
concepts and skills depicted in that section. Parents completed the program in a
little over 2 hours.

Interventiontechnology implementation

230 (90, 140) The program teaches adaptive parenting skills through scenario-based learning
(problem-based learning).
The computer program was used within community centers. It comprised two
sessions in which the parents chose topics out of the available 9 (e.g., bedtime,
fighting) and watched a short video (per topic) depicting a problematic
situation. After the video played, parents were asked to choose which of the
three parental responses was closest to what they would do. Only one choice
was correct and led to a positive outcome. The program features also
included on-screen Q&As that were designed to mimic a parenttherapist
counseling session and to consider how behaviors affected the outcome. Parents
were also encouraged to make up a personal action plan by selecting up to three
parenting skills relevant to that problem situation to practice at home.
38 (19, 19) The intervention was similar to Cefai et al. (2010)

86 (46, 40)

86 (40, 46)

Nb

WLC

WLC

WLC

WLC

I/
/
2/
1

I/
2/
2/
1

I/
1/
1/
3

I/
2/
2/
2

3m

6m

3m

Control Codesc Fo-up

Table 1. Selected Characteristics of Randomized Trials Examining the Effects of Digital Parent Training
Targeting Children with Disruptive Behaviors

(continued)

ECBI, PBQ

ECBI, PS, Parental

Self Efficacy

ECBI, PPI

ECBI, PSOC

Measuresd

744

210 (5.7)

Porzig-Drummond
et al. (2015)

Elevated disruptive behavior

based on ECBI (ECBI
problems mean > 15)

Elevated disruptive behavior

based on ECBI (ECBI
problems mean > 15)

Parent concerns about the

childs emotional or
behavioral adjustment
(ECBI problems
mean > 15).

Clinical symptoms
of the sample

116 (60, 56)

62 (29, 33)

100 (45, 55)

Nb
WLC

I/
1/
1/
2

N/
1/
1/
3

N/
1/
1/

6m

6m

6m

Control Codesc Fo-up

The intervention program was based on the 1-2-3 magic parenting program
WLC
focusing on established behavioral, emotional, and cognitive principles,
teaching the parents techniques, and providing guidance on ways to reduce
childrens disruptive behaviors (Phelan 2010).
The program consisted of 2 digital videos (DVD/online) viewed over 2 weeks,
followed by 2 weeks when parents were asked to act. Each video included
lecture components by the author, explaining the programs parenting strategies
and their application at home and public settings. Features included numerous
role-played video vignettes, demonstrating maladaptive parentchild
interactions, and the more adaptive parenting techniques taught through the
program. Total viewing time for both videos was 3 h 46 minutes. The first video
addressed stop behaviors, such as straight thinking, and controlling obnoxious
behavior. The second video addressed start behaviors, such as tactics for
encouraging good behavior, and strategies for building self-esteem. Parents also
received tip sheets, which summarized the main points of the program.
The intervention was based on the Triple P (Sanders 1999). The online program is NT
an eight-module, interactive self-directed software that took an average of 8
weeks to complete. The intervention provided instruction in the use of core
positive parenting skills presented in sequenced modules (e.g., managing
misbehavior). The online program features included video-based modeling of
parenting skills, diverse parent testimonials describing their experiences,
personalized content, including goal setting, interactive exercises to prompt
parental problem solving, downloadable worksheets and podcasts to review
session content, and automated text messaging and e-mail prompts to increase
the likelihood of program completion.

The intervention was based on the Triple P program, which directs the parent to
promote nurturing behaviors, a low conflict environment, and childrens social
and emotional well-being through positive parenting practices (Sanders 1999).
The program included seven online downloadable podcasts, ranging from 9 to 14
minutes, discussing parenting skills revolving around disruptive behaviors (e.g.,
dealing with aggression, sharing). The format of these podcasts was
conversational, in which the presenter asked questions of the parenting expert
relevant to the topics discussed. Podcasts were available for participating
parents in 3 phases over 2 weeks (2 were available immediately, the next 3 after
5 days, and the final 2 an additional 5 days later). Parents were e-mailed when a
new set of podcasts was available and were given an additional 2 weeks to
implement the strategies. Podcasts remained available throughout the study.

Interventiontechnology implementation

ECBI, PS, PTC

ECBI, PSI-SF

ECBI, PS, PTC,

CAPES confidence

Measuresd

Dashes indicate that no data were reported and/or obtained.

Age is presented in years. Mean is presented in brackets.
b
The first value refers to the total number of parents in the study; the first number in brackets refers to the total number of parents in the treatment group and the second value in brackets refers to the number of parents
in the control group.
c
The first grouping is a type of technology (I = interactive, N = noninteractive); the second grouping is a clinical symptom level (1 = clinical; 2 = not clinical); the third grouping is child age (1 = bellow mean age of 7,
2 = above mean age of 11.5); the fourth grouping is SES (1 = low, 2 = middle, 3 = high).
d
Only instruments that were used to calculate ESs are included.
CAPES, Child Adjustment and Parent Efficacy Scale (parenting confidence scale); CBCL, Child Behavior Checklist externalizing scale; ECBI, Eyberg Child Behavior Inventory; FAQ, frequently asked questions;
Fo-up, follow-up; HSQ-M, Home Situations Questionnaire Modified version; MCQ, multiple choice questions; NT, no treatment; PBQ, Parent Behavior Questionnaire; PCIT, ParentChild Interaction Therapy; PLOC,
the Parental Locus of Control scale; PPI, The Parenting Practices Interview; PS, Parenting Scale; PSI-SF, Parenting Stress Index-Short Form; PSOC, Parenting Sense of Competence; PTC, Parenting Task Checklist;
Triple p, positive parenting program; Q&A, questions and answers; SES, socioeconomic status; WLC, wait list control.

29 (4.7)

210 (6.1)

Morawska
et al. (2014)

Sanders
et al. (2012)

Child agea

Study

Table 1. (Continued)

DIGITAL PARENT TRAINING: A META-ANALYSIS

All interventions were based on theoretically driven evidencebased content, targeting parents beliefs and behaviors around child
rearing activities and discipline. Two of the three studies aimed for
older nonclinical samples used the same intervention in different
samples treated in different countries (Kacir and Gordon 2000; Cefai
et al. 2010). These three studies were also based on the same theoretical basis of Parenting Adolescents Wisely (Gordon et al., unpublished work, 1996) and intervention design of scenario-based learning.
Five interventions included interactive programs and two included
noninteractive programs. Finally, one intervention included research
assistant activity to remotely engage users with the intervention
website (Enebrink et al. 2012). Six studies provided follow-up information: three in a clinical child sample at 6 months postintervention
(Sanders et al. 2012; Morawska et al. 2014; Porzig-Drummond et al.
2015) and three in a nonclinical sample, two at 3 months postintervention (Kacir and Gordon 2000; Cefai et al. 2010), and one at 6
months postintervention (Irvine et al. 2015b).
Methodological quality
The quality assessments are summarized in Figure 2. In two
studies (Cefai et al. 2010; Irvine et al. 2015a) it was unclear whether
randomization was conducted with appropriate methods and whether
the allocation was concealed from relevant staff. These studies,
however, provided analysis of groups at baseline showing no significant differences in the variables measured, suggesting that the
randomization procedure was adequate. In one study (Irvine et al.
2015a), a gap was found between the simplicity of the recruitment
process, which was conducted remotely and included US $40 compensation for filling out the assessments, and the participation itself
within the intervention group, which required participants to use the
program within community centers. This gap resulted in low rates of
participants beginning treatment within the intervention group. In
one study (Kacir and Gordon 2000), the data at baseline pointed to an
imbalance on several outcome measurements favoring the control
group, which, in turn, yielded more conservative outcome ESs.
Treatment effects: intervention versus control groups
Overall, compared to the control condition, digital-based parent
training resulted in significantly greater improvement in child be-

745
havior (ES = 0.44, 95% confidence interval [CI]: 0.210.66, studies = 7, n = 718), parent behavior (ES = 0.41, 95% CI: 0.250.57,
studies = 6, n = 632), and parental confidence (ES = 0.36, 95% CI:
0.120.59, studies = 4, n = 532) (Fig. 3). The funnel plots were
examined and did not indicate a significant publication bias.
The Q-statistic for heterogeneity showed a trend toward significance for child behavior [Q(6) = 11.99, p = 0.06, I2 = 49.98], being
not significant for parent behavior [Q(5) = 2.73, p = 0.74] and for
parental confidence [Q(3) = 5.21, p = 0.16].
We subgrouped the studies by clinical (n = 4) and nonclinical
(n = 3) severity of the disruptive behavior problems and conducted an
additional subgroup analysis for media type (interactive vs. noninteractive) with regard to all outcome measures (Table 2). Since all
studies aimed at middle school children included only interactive
features and consistently provided homogenous results (Qw < 0.35),
we compared the media type (interactive vs. noninteractive) only in
studies that focused on young samples within the clinical range.
Digital PT had a moderate ES advantage regarding child behavior in studies of young children with clinically significant disruptive behavior (ES = 0.61, 95% CI: 0.400.82) and a small ES
with a trend toward significance for a middle school nonclinical
sample (d = 0.21, 95% CI: -0.01 to 0.42). Analyses yielded similar
results of higher ESs favoring studies of young children with
clinically significant disruptive behavior for parent behavior and
parental confidence, but the differences were not significant.
In a subgroup analysis of early-onset children with clinically
significant disruptive behavior, a significant difference was found
regarding child behavior between interactive and noninteractive
programs (Qb = 4.48, p < 0.01). Interactive programs had a large ES
advantage (ES = 0.82, 95% CI: 0.541.11), whereas noninteractive
programs had a small ES advantage (d = 0.36, 95% CI: 0.050.68)
compared to control groups. While a similar trend was found for
parental confidence (Qb = 2.88, p < 0.1), this calculation was based
on only one study per group (i.e., interactive vs. noninteractive).
Post-treatment follow-up effects:
within intervention group
Follow-up effect analysis revealed no significant change during
the nonintervention phase (postintervention follow-up in all studies
with information) on child behavior (ES = 0.11, 95% CI: -0.03 to
0.25, studies = 6, n = 180), parental behavior (ES = 0.00, 95% CI:
-0.23 to 0.22, studies = 4, n = 128), or parental confidence (ES = 0.14,
95% CI: -0.03 to 0.30, studies = 3, n = 111). That is, the effects found
at the end of treatment were sustained during the follow-up period,
that is, despite cessation of the intervention.
Discussion

FIG. 2. Risk of bias assessments using Cochrane Collaborations tool.

The aim of this systematic review and meta-analysis was to

summarize the reported effectiveness of DPT for childrens disruptive behaviors to inform researchers and clinicians regarding this
interventions potential and to identify the main challenges in this
field for future consideration. Out of the seven randomized studies
meeting inclusion criteria, five were published after 2011 and three
were published between 2014 and 2015, indicating recent progress
made in this area. All interventions were guided by evidence-based
theoretical frameworks, targeting parents perceptions around discipline and child rearing behaviors and providing guidance regarding adequate and inadequate parenting behaviors.
Overall, we found that DPT resulted in positive improvements
for the three outcome measurements with overall small to medium ESs of 0.44 for child behavior, 0.41 for parent behavior, and

746

BAUMEL ET AL.

FIG. 3. Forrest plots of standardized mean difference ESs (Cohens d, positive favoring the intervention), 95% confidence interval
lower and upper limits, and significance measurement. The random model ES is computed against the control studies and also an overall
ES is computed. In relevant variables, ESs are also presented for clinical and nonclinical subgroups. ES, effect size.
0.35 for parental confidence in self-efficacy. We identified two
groups of studies, however, one targeting young children within
clinical range of disruptive behaviors and the other targeting
older children mostly below the clinical range of disruptive behaviors, which differed significantly in their efficacy. The results
point to a medium ES of 0.61 for child behavior in studies of
young children within the clinical range of symptoms and a small
ES of 0.21with a trend toward significance for child behavior in
studies of older children not within the clinical range of targeted
symptoms. Although not significant, the same trend of difference
between these two study samples was apparent for parent behavior and confidence.
There are several explanations for this finding that should be
addressed. First, there are three important baseline differences
between these two study groups: targeted childrens age, the reported range of clinical symptoms, and the SES of the samples. As
has been shown in a previous comprehensive meta-analysis of

parent training for disruptive behaviors (Lundahl et al. 2006),

higher childrens age, lower clinical symptom severity, and lower
SES are all associated with significantly less improvement on
childs behavior reports. Since digital PTs for older children targeted a nonclinical sample with lower SES on average, it was not
feasible to distinguish between these overlapping characteristics.
This implies that the smaller ESs found for this group might be due
to the combination of these three baseline factors. Second, all interventions targeting older children were practically based on the
same program design and theoretical framework, that is, Parenting
Adolescents Wisely (Gordon et al., unpublished work, 1996).
Therefore, one cannot rule out the possibility that another programs features would yield different outcomes. To further examine the potential of DPT for older children in improving childrens
behavior and other program outcomes, different samples in terms of
SES and clinical range of symptoms, as well as different intervention programs need to be examined. These studies will enable

DIGITAL PARENT TRAINING: A META-ANALYSIS

747

Table 2. Meta-Analyses of Studies Examining the Effects of Technology-Based Parent Training

for Children with Disruptive Behaviors
k
Child behavior
Overall
7
Clinical
4
Nonclinical
3
Clinical subgroup analysesa
Type
Interactive
2
Noninteractive
2
Parent behavior
Overall
6
Clinical
4
Nonclinical
2
Clinical subgroup analysesa
Type
Interactive
2
Noninteractive
2
Parental confidence
Overall
4
Clinical
2
Nonclinical
2
Clinical subgroup analyses
Type
Interactive
1
Noninteractive
1

Cohens d effect size

0.44
0.61
0.21

95% CI
0.210.66
0.400.82
-0.01 to 0.42

Qw

I2

Qb

3.87
5.67
1.89

0.000
0.000
0.058

11.99*
4.75
0.18

49.98
36.9
0.00

7.06**

4.48*
0.82
0.36

0.541.11
0.050.68

5.60
2.29

0.000
0.02

0.04
0.24

0.00
0.00

0.41
0.49
0.30

0.250.57
0.280.70
0.050.55

5.05
4.61
2.38

0.000
0.000
0.02

2.73
0.99

0.00
0.00
0.00

0.57
0.39

0.290.85
0.080.71

3.97
2.47

0.000
0.01

0.00
0.31

0.00
0.00

0.36
0.51
0.23

0.120.59
0.240.78
0.010.46

2.96
3.66
2.04

0.003
0.000
0.04

5.21
2.88*
0.00

42.44
65.29
0.00

3.82
1.29

0.000
0.20

1.37

0.68

1.04

2.88*
0.73
0.26

0.361.11
-0.14 to 0.66

The Qw statistic estimates within group homogeneity. The Qb statistic estimates whether the difference between the effect sizes in the subgroups is
significant.
a
All subgroup analyses were conducted with random-effects analyses.
d, effect size; k, number of studies; CI, confidence interval.
*p < 0.10; **p < 0.05; ***p < 0.01; ****p < 0.001.

researchers and service providers to better estimate the potential of

digital PT to support families with older children and/or identify
subgroups responding better to one or another approach.
Nonetheless, the ESs found overall for DPT for children resemble
those reported for self-directed parent training in previous metaanalysis (Lundahl et al. 2006). This finding strengthens the use of
digital programs as a delivery method of self-directed parental intervention that can preserve resources, increase access, and is
widely scalable. The latter feature is highly relevant, as research
suggests that antipsychotics are prescribed for youth with disruptive
behaviors instead of or without attempting behavioral interventions,
which have been found to be all too often inaccessible (Olfson et al.
2012, 2014). Moreover, the use of digital programs, mostly online,
opens new possibilities not existing in previous self-directed designs (e.g., books), such as the applicability to examine parents use
of the program and self-reports embedded within the design to better
manage and evaluate the usability and fidelity of the treatment.
Subsequently, our meta-analysis revealed that interactive programs resulted in a larger ES (0.82) for childs behavior compared
to noninteractive programs (0.36) in the sample of younger children
within the clinical range of symptoms. Since interactive programs
as a delivery method offer much more tailored design in comparison to regular self-directed programs, this finding does not seem
unreasonable. This finding points to the need to further investigate
the effect of these two designs on the efficacy of parent training for
disruptive behaviors, which in turn could help a great deal in the
understanding of the way different delivery methods and designs
affect programs success.

Relating to studies design, two considerations for future research should be noted. First, our systematic review revealed a lack
of randomized trials comparing digital led and therapist led behavioral programs (individual or group behavioral interventions,
but without a technological component), which is necessary to
understand the therapeutic impact of digital programs. Another
relevant yet missing comparator would be the usual care of youth
with disruptive behavioral problems that all too often do not include
formal evidence-based psychological interventions (Knapp et al.
2012; Rosato et al. 2012). Comparison with usual care would enable us to gauge the real-world clinical utility of digital-based
parent training. Such studies should focus not only on efficacy but
also on sustainability and cost effectiveness of the face-to-face and
digitally based approaches. Second, while the reviewed programs
efficacy was evaluated using standardized methods, none of these
studies provided data suggesting that the program resulted in earlier
engagement with services, or engagement with care of a population
that would not otherwise receive these services due to causes such
as stigma, lack of access, or costs. Hence, following the establishment of these programs efficacy, their promise in providing
different pathways to care that enable better access and use of
services should clearly be examined.
Several limitations of this meta-analysis require consideration.
First, the number of studies included in our analysis was small,
although it is above the median for reviews listed in the Cochrane
Database of Systematic Reviews, which is six (Borenstein et al.
2009b). Moreover, the observed effects were consistent among the
studies of the two identified subgroups (clinical, nonclinical), and

748
therefore, the limitation of realizing the true impact of dispersion
(Borenstein et al. 2009a) was less crucial. Nonetheless, more studies
are needed to examine whether the calculated ES is stable across
different samples and digital interventions. Second, the results are
limited to parents experience of their children, which limits our
understanding of DPT effects on childrens behavior in other settings. This limitation, however, is the result of the main outcome
measures used within most parent training studies, since the primary
focus of parent training is to manage child behaviors while the child
is under parental guidance and supervision (Lundahl et al. 2006).
Third, none of the studies targeting younger children focused on a
sample with a low SES, a population which is of great importance in
this field (Gortmaker et al. 1990; Loeber et al. 1995). Therefore, we
cannot rule out the possibility that for families with low SES the
studied interventions are not as effective. Future examinations of
digital interactive parent training programs targeting early onset of
disruptive behaviors in families with low SES are needed.
Conclusions
Despite these limitations, this study presents evidence that
digital-based parent training for disruptive behaviors is an effective
and promising intervention tool, with current data mainly supporting its use in young children experiencing a clinically significant
range of symptoms.
Clinical Significance
As technology provides important opportunities to extend the
reach of services, our hope is that this review will not only provide
empirical information about the current state of the field but also
information regarding further directions for research, enabling more
families and youth to benefit from effective and evidence-based
psychological interventions for disruptive behaviors of youth.
Disclosures
Dr. A.B. and Dr. A.P. have no financial interests to disclose.
Dr. J.M.K. has been a consultant for Alkermes, Eli Lilly, EnVivo
Pharmaceuticals (Forum), Forest, Genentech, H. Lundbeck. Intracellular Therapeutics, Janssen Pharmaceutica, Johnson and
Johnson, Otsuka, Reviva, Roche, Sunovion, and Teva. He has received honoraria for lectures from Janssen, Genentech, Lundbeck,
and Otsuka. Dr. J.M.K. is a Shareholder in MedAvante, Inc., and
Vanguard Research Group. Dr. C.U.C. has been a consultant and/or
advisor to or has received honoraria from AbbVie, Acadia, Actavis,
Alkermes, Eli Lilly, Genentech, Gerson Lehrman Group, IntraCellular Therapies, Janssen/J&J, Lundbeck, MedAvante, Medscape, Otsuka, Pfizer, ProPhase, Reviva, Roche, Sunovion, Supernus,
and Takeda. He has received grant support from Bristol-Myers
Squibb, Otsuka, and Takeda.
References
Arnold DS, OLeary SG, Wolff LS, Acker MM: The Parenting Scale:
A measure of dysfunctional parenting in discipline situations.
Psychol Assess 5:137144, 1993.
Baumel A, Muench F: Heuristic evaluation of Ehealth interventions:
Establishing standards that relate to the therapeutic process perspective. JMIR Ment Health 3:e5, 2016.
Borenstein M, Hedges L, Rothstein H: Meta-analysis: Fixed effect vs.
random effects. http://www.metaanalysis.com/downloads/Metaanalysis_fixed_effect_vs_random_effects_sv.pdf.2007.

BAUMEL ET AL.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (eds): When
does it make sense to perform a meta-analysis? In: Introduction to
Meta-Analysis. Hoboken (New Jersey), Wiley, 2009a, pp 357364.
Borenstein MH, Higgins L, Rothstein J:. Introduction to Meta-Analysis. Hoboken (New Jersey), Wiley, 2009b.
Breitenstein SM, Gross D, Christophersen R: Digital delivery methods
of parenting training interventions: A systematic review. Worldviews Evid Based Nurs 11:168176, 2014.
Cefai J, Smith D, Pushak RE: Parenting wisely: Parent training via
CD-ROM with an Australian sample. Child Fam Behav Ther 32:
1733, 2010.
Chorpita BF, Daleiden EL, Ebesutani C, Young J, Becker KD, Nakamura
BJ, Phillips L, Ward A, Lynch R, Trent L: Evidence-based treatments
for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clin Psychol Sci Pract 18:154172, 2011.
Comer JS, Chow C, Chan PT, Cooper-Vince C, Wilson LA: Psychosocial treatment efficacy for disruptive behavior problems in very
young children: A meta-analytic examination. J Am Acad Child
Adolesc Psychiatry 52:2636, 2013.
Correll CU, Manu P, Olshanskiy V, Napolitano B, Kane JM, Malhotra
AK: Cardiometabolic risk of second-generation antipsychotic medications during first-time use in children and adolescents. JAMA 302:
17651773, 2009.
DerSimonian R, Laird N: Meta-analysis in clinical trials. Control Clin
Trials 7:177188, 1986.
Egger HL, Angold A: Common emotional and behavioral disorders in
preschool children: Presentation, nosology, and epidemiology. J
Child Psychol Psychiatry 47:313337, 2006.
Enebrink P, Hogstrom J, Forster M, Ghaderi A: Internet-based parent
management training: A randomized controlled study. Behav Res
Ther 50:240249, 2012.
Eyberg SM, Nelson MM, Boggs SR: Evidence-based psychosocial
treatments for children and adolescents with disruptive behavior. J
Clin Child Adolesc Psychol 37:215237, 2008.
Eyberg SM, Robinson EA: Conduct problem behavior: Standardization
of a behavioral rating. J Clin Child Adolesc Psychol 12:347354, 1983.
Galling B, Roldan A, Nielsen RE, Nielsen J, Gerhard T, Carbon M,
Stubbs B, Vancampfort D, De Hert M, Olfson M: Type 2 diabetes
mellitus in youth exposed to antipsychotics: A systematic review
and meta-analysis. JAMA Psychiatry 73:247259, 2016.
Gordon DA, Stanar CR: Lessons learned from the dissemination of
Parenting Wisely, a parent training CD-ROM. Cogn Behav Pract
10:312323, 2003.
Gortmaker SL, Walker DK, Weitzman M, Sobol AM: Chronic conditions, socioeconomic risks, and behavioral problems in children
and adolescents. Pediatrics 85:267276, 1990.
Hedges LV, Olkin I: Statistical Method for Meta-Analysis. London,
Academic Press, 2014.
Higgins JP, Altman DG, Gtzsche PC, Juni P, Moher D, Oxman AD,
Savovic J, Schulz KF, Weeks L, Sterne JA: The Cochrane Collaborations tool for assessing risk of bias in randomised trials. BMJ
343:d5928, 2011.
Higgins JP, Thompson SG: Quantifying heterogeneity in a metaanalysis. Stat Med 21:15391558, 2002.
Irvine AB, Gelatt VA, Hammond M, Seeley JR: A randomized study
of internet parent training accessed from community technology
centers. Prev Sci 16:597608, 2015a.
Irvine AB, Russell H, Manocchia M, Mino DE, Glassen TC, Morgan R,
Gau JM, Birney AJ, Ary DV: Mobile-Web app to self-manage low back
pain: Randomized controlled trial. J Med Internet Res 17:e1, 2015b.
Johnston C, Mash EJ: A measure of parenting satisfaction and efficacy. J Clin Child Psychol 18:167175, 1989.
Jones DJ, Forehand R, Cuellar J, Kincaid C, Parent J, Fenton N, Goodrum N: Harnessing innovative technologies to advance childrens

DIGITAL PARENT TRAINING: A META-ANALYSIS

mental health: Behavioral parent training as an example. Clin Psychol
Rev 33:241252, 2013.
Kacir CD, Gordon DA: Parenting adolescents wisely: The effectiveness of an interactive videodisk parent training program in Appalachia. Child Fam Behav Ther 21:122, 2000.
Kazdin AE, Blase SL: Rebooting psychotherapy research and practice
to reduce the burden of mental illness. Perspect Psychol Sci 6:21
37, 2011.
Kazdin AE, Rabbitt SM: Novel models for delivering mental health
services and reducing the burdens of mental illness. Clin Psychol
Sci 2013. DOI: 10.1177/2167702612463566.
, Forster M, Sundell K, Melin L: A randomized controlled
Kling A
effectiveness trial of parent management training with varying
degrees of therapist support. Behav Ther 41:530542, 2010.
Knapp P, Chait A, Pappadopulos E, Crystal S, Jensen PS: Treatment of
maladaptive aggression in youth: CERT guidelines I. Engagement,
assessment, and management. Pediatrics 129:e1562e1576, 2012.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gtzsche PC, Ioannidis
JP, Clarke M, Devereaux P, Kleijnen J, Moher D: The PRISMA
statement for reporting systematic reviews and meta-analyses of
studies that evaluate health care interventions: Explanation and
elaboration. Ann Intern Med 151:W-65W-94, 2009.
Loeber R, Green SM, Keenan K, Lahey BB: Which boys will fare
worse? Early predictors of the onset of conduct disorder in a sixyear longitudinal study. J Am Acad Child Adolesc Psychiatry 34:
499509, 1995.
Lundahl B, Risser HJ, Lovejoy MC: A meta-analysis of parent
training: Moderators and follow-up effects. Clin Psychol Rev 26:
86104, 2006.
Maayan L, Correll CU: Weight gain and metabolic risks associated
with antipsychotic medications in children and adolescents. J Child
Adolesc Psychopharmacol 21:517535, 2011.
Merikangas KR, Nakamura EF, Kessler RC: Epidemiology of mental
disorders in children and adolescents. Dialogues Clin Neurosci
11:720, 2009.
Morawska A, Tometzki H, Sanders MR: An evaluation of the efficacy
of a triple P-positive parenting program podcast series. J Dev Behav
Pediatr 35:128137, 2014.
Olfson M, Blanco C, Liu S-M, Wang S, Correll CU: National trends in
the office-based treatment of children, adolescents, and adults with
antipsychotics. Arch Gen Psychiatry 69:12471256, 2012.
Olfson M, Blanco C, Wang S, Laje G, Correll CU: National trends in
the mental health care of children, adolescents, and adults by officebased physicians. JAMA Psychiatry 71:8190, 2014.
Phelan TW: 1-2-3 Magic: Effective Discipline for Children 212.
ParentMagic, Inc., Glen Ellyn (Illinois); (2010).
Porzig-Drummond R, Stevenson RJ, Stevenson C: Preliminary evaluation of a self-directed video-based 1-2-3 Magic parenting program:
A randomized controlled trial. Behav Res Ther 66:3242, 2015.

749
APA Publications and Communications Board Working Group on
Journal Article Reporting Standards: Reporting standards for research in psychology: Why do we need them? What might they be?
Am Psychol 63:839851, 2008.
Robinson KA, Dickersin K: Development of a highly sensitive search
strategy for the retrieval of reports of controlled trials using
PubMed. Int J Epidemiol 31:150153, 2002.
Rosato NS, Correll CU, Pappadopulos E, Chait A, Crystal S, Jensen PS:
Treatment of maladaptive aggression in youth: CERT guidelines II.
Treatments and ongoing management. Pediatrics 129:e1577e1586,
2012.
Sanders MR. Triple P-Positive Parenting Program: Towards an empirically validated multilevel parenting and family support strategy
for the prevention of behavior and emotional problems in children.
Clin Child Fam Psychol Rev 2:7190, 1999.
Sanders MR, Baker S, Turner KM: A randomized controlled trial evaluating the efficacy of Triple P Online with parents of children with
early-onset conduct problems. Behav Res Ther 50:675684, 2012.
Sanders MR, Woolley M: The relationship between maternal selfefficacy and parenting practices: Implications for parent training.
Child Care Health Dev 31:6573, 2005.
Segal D, Chen PY, Gordon DA, Kacir CD, Gylys J: Development
and evaluation of a parenting intervention program: Integration of
scientific and practical approaches. Int J Hum Comput Interact 15:
453467, 2003.
Sourander A: Challenges in childrens enrolment to psychosocial
services. Eur Child Adolesc Psychiatry 24:361363, 2015.
Webster-Stratton C, Kolpacoff M, Hollinsworth T: Self-administered
videotape therapy for families with conduct-problem children:
Comparison with two cost-effective treatments and a control group.
J Consult Clin Psychol 56:558, 1988.
Wilson DB, Lipsey MW: The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychol Methods
6:413429, 2001.
Zisser A, Eyberg S: Parent-child interaction therapy and the treatment
of disruptive behavior disorders. In: Evidence-Based Psychotherapies
for Children and Adolescents. Weisz, JR, and Kazdin, AE, eds. New
York, Guilford Press, 2010, pp. 179193.

Address correspondence to:

Amit Baumel, PhD
Psychiatry Research, Northwell Health
The Zucker-Hillside Hospital
75-59 263rd Street
Glen Oaks, NY 11004
E-mail: abaumel@northwell.edu

Appendix
Psychinfo Search Terms
(Time limits were set through the search engines platform)
(program OR educat* OR psychoeducat* OR train* OR self-training
OR guid* OR self-guided OR skill* OR manag* OR therap* OR psychotherapy* OR treat* OR interven* OR self-help OR self-directed) AND
(parent*) AND
(online* OR computer* OR internet* OR video* OR web-based
OR website* OR mobile* OR smartphone* OR text-messaging OR

texting OR sms OR digital* OR tech* OR ehealth OR e-health OR

mhealth OR m-health) AND
(Conduct OR CD OR behavior* OR behaviour* OR Oppositional OR defiant OR ODD OR anger OR aggressi* OR discipline*
OR undiscipline* OR Impulse Control Disorder OR Impulse
Control Disorders OR attention deficit OR attention-deficit OR
attention-deficit-disorder OR ADHD OR hyperactiv* OR overactiv* OR inattent*)