Effectiveness Applied Behaviour AJNHJHBlysis Interventions

THE EFFECTIVENESS OF APPLIED
BEHAVIOUR ANALYSIS
INTERVENTIONS FOR PEOPLE
WITH AUTISM SPECTRUM
DISORDER
- AN UPDATE OF SECONDARY
LITERATURE
Report to the New Zealand Ministry of Health

30 September 2009
UPDATE OF REVIEWS: APPLIED BEHAVIOUR ANALYSIS FOR PEOPLE WITH AUTISM SPECTRUM DISORDER
2
PRINCIPAL AUTHOR
Marita Broadstock
ACKNOWLEDGEMENTS
Acknowledgements
This systematic review was undertaken by staff of the New Zealand Guidelines
Group, and completed in September 2009. The scope of the review was agreed with
the Living Guidelines Group projects sponsor, Joanna Curzon (Ministry of Health)
and was funded by the Ministry of Health to contribute to the work of the Living
Guideline Group.
Marita Broadstock (Senior Researcher) conducted the review and prepared the
report and Evidence Tables. Margaret Paterson (NZGG Information Specialist)
conducted the search strategy and managed document retrieval and referencing. A
list of excluded publications is available upon request. Anne Lethaby (Acting
Manager, Research Services) provided methodological input and peer review.
Please cite this report as:
3
New Zealand Guidelines Group. The effectiveness of applied behaviour analysis

interventions for people with autism spectrum disorder an update of secondary literature.
Wellington; 2009.
4
SYNTHESIS OF RECENTLY PUBLISHED SECONDARY

LITERATURE ON APPLIED BEHAVIOUR ANALYSIS
BACKGROUND AND SCOPE

In 2008, the New Zealand Ministries of Health, and Education, commissioned two
technical reviews on interventions and strategies for people with Autism Spectrum
Disorder (ASD) grounded in the principles of applied behaviour analysis (ABA)
(Ministries of Health and Education, 2007). These two reviews were completed in
parallel by Uniservices and New Zealand Guidelines Group (NZGG) respectively and
based on research published to 31 December 2007. These reports represent the
prime evidence for the Living Guideline Group (LGG) to consider in determining
whether any revision of Recommendations relevant to ABA is needed in the New
Zealand Autism Spectrum Disorder Guideline (2008). However, as this field has been
the subject of significant research interest, it was considered valuable to supplement
the New Zealand commissioned research with a synthesis of other high level
evidence that has emerged since the reviews search cut-off period.
In a hierarchy of evidence for studies of intervention effectiveness, the highest level
evidence (Level I) consists of systematic reviews and meta analyses which include
(but are not limited to) randomised controlled trials (RCTs) (level II evidence)
(National Health and Medical Research Council, 2008). This current update is limited
to level I evidence. Such secondary evidence offers the opportunity to consider a
range of approaches to reviewing and synthesising the evidence relating to ABA and
compare conclusions with those of the two comprehensive parallel technical reviews.
OBJECTIVES AND RESEARCH QUESTION

The objective of this update is to critically appraise secondary research evidence
published since December 2007 relating to the effectiveness of interventions and
strategies grounded in the principles of applied behaviour analysis for people with
autism spectrum disorder.
ABA-based interventions can be defined as those in which the principles of learning
theory are applied in a systematic and measurable manner to increase, reduce,
maintain and/or generalise target behaviours (Ministries of Health and Education,
2007). Well-established principles and techniques of ABA include (a) reinforcement,
(b) shaping, (c) chaining, (d) fading, (e) response and stimulus prompting, (f)
discrimination training, (g) programming, and (h) functional assessment (Broadstock
& Lethaby, 2008).
The clinical question employed for this update of secondary literature was that used
for the parallel technical reviews (Ministries of Health and Education, 2007):
To what extent are interventions and strategies based on the principles of applied
behaviour analysis effective in leading to the following outcomes for people with
autism spectrum disorders:
5
social development and relating to others

development of cognitive (thinking) skills
development of functional and spontaneous communication which is used in
natural environments
engagement and flexibility in developmentally appropriate tasks and play and
later engagement in vocational activities
development of fine and gross motor skills
prevention of challenging behaviours and substitution with more appropriate
and conventional behaviours
development of independent organisational skills and other behaviours
generalisation of abilities across multiple natural environments outside the
treatment setting
maintenance of effects after conclusion of intervention
improvement in behaviours considered non-core ASD behaviours, such as
sleep disturbance, self mutilation, aggression, attention and concentration
problems.
REVIEW METHODS
A systematic method of literature searching, selection and appraisal was employed in
the preparation of this report, consistent with New Zealand Guidelines Group (NZGG)
review processes (Broadstock & Lethaby, 2008).
The search was limited to articles published in the English language between
January 1 2008 and August 6 2009 (when the search was conducted). Sources
included general bibliographic databases (Medline, PsychINFO, EMBASE, CINAHL,
ERIC, Cochrane Library) as well as various health technology assessment/guideline
databases, and cross-checking of references from retrieved references.
Selection criteria were based on those used for the NZGG technical review
(Broadstock & Lethaby, 2008), modified to solely identify on secondary studies.
Studies were included if they:
were secondary research (systematic reviews and meta-analyses)
were published on or after January 1st 2008
had a clear review question
used at least two searching sources
and reported on studies:
of eligible interventions (studies which evaluated interventions which were
predominantly based on the principles of applied behaviour analysis)
considering comparators including usual care, another intervention or
application of interventions
considering individuals with a diagnosis of autism spectrum disorder or where
results are reported separately for this group.
Research papers were excluded if they:
6
were non-systematic reviews, correspondence, editorials, expert opinion

articles, comments, case reports, book chapters, articles published only in
abstract form, conference proceedings, news items, unpublished work
were primary studies (except where reported in included systematic reviews
did not provide separate analyses/syntheses of results for eligible
interventions and eligible population
were not deemed appropriate to the research question or nature of review,
including those reporting on outcomes solely relating to safety; the
acceptability of, or ethical, economic or legal considerations relating to ABA;
or the impact on persons other than those diagnosed with ASD.
STUDY SELECTION AND APPRAISAL

Selection criteria were applied by a single reviewer to abstracts/titles identified by the
search strategy, and again to those retrieved as full text, to identify the final set of
included papers for critical appraisal. All included studies represented level I
evidence (National Health and Medical Research Council, 2008) as specified in the
selection criteria, being systematic reviews or meta analyses of studies that included
randomised controlled trials.
Included studies were formally critically appraised using NZGGs quality checklists
which are based on the GATE Frame tools designed by Effective Practice,
Informatics and Quality improvement (EPIQ), within the School of Population Health
at the University of Auckland. Twenty questions addressed methodological quality,
including whether the reviews methods were internally valid, precision of results, and
the applicability/external validity of the review. An overall quality code was applied
based on the appraisal to summarise the quality of each included study. Overall
study quality was categorised qualitatively based on summary questions on the
GATE checklist and using one of the following descriptors: very good, good, or
fair.
Details of each study were entered into Evidence Tables, including aspects of
methodology, results, authors conclusions, reviewers additional comments, and the
summary study quality descriptor. The included studies are presented alphabetically
by first author within two intervention type categories, (1) applied behaviour analysis
interventions (defined broadly), and (2) Picture Exchange Communication System
(PECS) interventions (see Evidence Tables).
DATA SYNTHESIS
Studies were narratively synthesised to determine the strength of evidence. Strength
of evidence is determined by three domains (West, King, & Carey, 2002):
- quality (the extent to which bias was minimised);
- quantity (magnitude of effect, numbers of studies, sample size or power);
- consistency (the extent to which similar findings are reported).
7
KEY FINDINGS AND CONCLUSIONS
Overview
Of 461 articles identified by the search strategy, 35 publications were retrieved as full
text. Twelve of these met selection criteria and were deemed eligible for critical
appraisal and inclusion in this synthesis.
Of the 12 included secondary studies, 10 reported on evaluations of interventions
broadly based on applied behaviour analysis, and two reported on evaluations of the
Picture Exchange Communication System (PECS) intervention specifically (see
Table).
The 10 reviews reporting on ABA-based interventions (in general) used varying
descriptive terms for included interventions, reflecting that there is no universally
accepted definition of ABA-based interventions. For example, some studies refered
specifically to Early Intensive Behavioural Interventions (EIBI) as being restricted to
interventions based on the work of Lovaas (1987), whereas others included as EIBI
any intensive behavioural intervention directed at children in pre-school years. In this
report the broader definition is used.
Of the 10 reviews, four included meta analyses (Sigmund Eldevik et al., 2009;
Ospina et al., 2008; Reichow & Wolery, 2009; Spreckley & Boyd, 2009b); all four
were rated as being of good (Reichow & Wolery, 2009) or very good quality
(Sigmund Eldevik et al., 2009; Ospina et al., 2008; Spreckley & Boyd, 2009b).
Four of the 12 reviews (33%) were classified as being of very good quality (Blue
Cross and Blue Shield Association, 2009; Sigmund Eldevik et al., 2009; Ospina et al.,
2008; Spreckley & Boyd, 2009b), another third were of good quality (Svein
Eikeseth, 2009; Howlin, Magiati, & Charman, 2009; Reichow & Wolery, 2009; Rogers
& Vismara, 2008), and the remaining final four systematic reviews were of fair
quality, including two considering ABA in general (Case-Smith & Arbesman, 2008;
Seida et al., 2009), and two reporting on PECS (Ostryn, Wolfe, & Rusch, 2008;
Sulzer-Azaroff, Hoffman, Horton, Bondy, & Frost, 2009).
It should be noted that as these reviews were all published across an 18 month
period on the same broad topic area, they tend to report on the same studies, and
also overlap with the studies included in the parallel technical reviews of NZGG and
Auckland Uniservices. For this reason it is important to be aware that the results and
the studies appraised should not be summated as independent sources of evidence
as this would misrepresent the quantity of studies and give the individual studies
undue weight.
No additional primary studies published post December 2007 and eligible for
inclusion in either of the two technical reviews were identified by the 12 reviews, and
so the evidence catchment area is broadly the same as that considered by the
parallel technical reviews, although some included studies prior to 1998, the earliest
date for publications considered in the parallel technical reviews. In considering
secondary reviews which consider overlapping but different selection criteria and
methodological approaches, attention should be given to the consistency of
8
conclusions. More weight/emphasis should be placed on the reviews that are of

higher quality. For this reason, the following synthesis will consider the findings and
conclusions of the very good quality reviews first.
Reviews of very good quality

Four reviews were rated as being of very good quality using the systematic critical
appraisal process outlined above. All considered ABA interventions in general,
though the focus was on studies of early intensive behavioural analysis.
The systematic review published by Blue Cross and Blue Shield Associations
(2009) Technical Evaluation Centre considered evidence published 1987 - July 2008.
The review considered the use of EIBI based on applied behaviour analysis. It
excluded single-case experimental studies (SCED), arguing that such designs lack
generalisability across individuals, and are of limited value in evaluating EIBI due to
methodological constraints.
The reviewers included 16 studies, including 2 RCTs. They concluded that,
weaknesses in research design, differences in the treatments and outcomes
compared, and inconsistent results, mean that the impact of EIBI versus other
treatments on outcomes for children with autism cannot be determined. They argued
that the weakness of the evidence base and variability in findings meant that they
could also not reach conclusions about the impact of EIBI on any specific
domains/outcomes, or on whether greater intensity of EIBI can lead to better
outcomes.
The authors called for research to identify what aspects of an intervention and its
delivery lead to greatest positive effect, including content, technique, intensity,
starting and ending age. A greater emphasis was said to be needed on randomized,
controlled trials with substantially larger sample sizes, uniformity of outcomes and
instruments, and consistent treatments.
The systematic review by Eldevik and Hastings (2009) considered nine controlled
group studies of EIBI in their meta analyses. They reported moderate to large effect
sizes for adaptive behaviour composite (ABC) and full-scale IQ scores, respectively,
compared with controls or eclectic comparators. It is concluded that, in the absence
of other interventions with established efcacy, EIBI should be an intervention of
choice for children with autism. However the authors cautioned that the improved
outcomes for people receiving EIBI may be due to the EIBI group having received, in
general, more frequent and more total hours of supervision and training than
comparison groups. They noted that this remains a threat to the validity of their
conclusions and called for RCTs where the comparison intervention is of similar
intensity and where staff receive similar training and supervision.
Another SR with meta analysis rated as being very good quality was authored by
Ospina et al (2008). It considered literature published to May 2007 and included 101
controlled studies across a range of interventions ranging from behavioural to
developmental. Those interventions relating to ABA specifically included those based
on Direct Trial Training (DTT) and Lovaas (1987) therapy. With respect to ABA, the
9
reviewers concluded from descriptive synthesis that there was some support for
discrete trial training in terms of motor and functional skills but not for communication
skills. From meta analyses of 3 controlled clinical trials the authors reported that
Lovaas therapy appeared to show benefits when compared to special education on
several domains (adaptive behaviour, communication and interaction, comprehensive
language, daily living skills, expressive language, overall intellectual functioning,
socialisation) with the exception of non-verbal intellectual functioning. Based on a
meta-analysis of two RCTs, they found no difference in communication skill outcomes
for children receiving EIBI compared with a Developmental Individual Difference
Relationship based intervention. Lower level evidence from meta-analysis of two
retrospective cohort studies showed greater effects for high intensity versus low
intensity Lovaas.
These findings were tempered by the limitations of the evidence base, with the
authors noting that, these findings are based on pooling of a few, methodologically
weak studies with few participants and relatively short-term follow-up. The
heterogeneity of the study populations, interventions and outcomes was also noted,
raising questions as to the appropriate use of meta analysis to synthesise these
results. The overall conclusion from across interventions was that there is no clear
answer regarding the most effective therapy to improve symptoms associated with
ASD. The authors called for replication in RCTs to substantiate the use of Lovaas
(1987) and to assess the effect of treatment intensity on outcomes of children with
ASD. They further recommended that clinical management be guided by individual
needs and availability of resources.
The fourth review of very good quality also included a meta analysis (Spreckley &
Boyd, 2009b), and reported on applied behaviour interventions (ABI) for preschool
children with ASD. Randomised or quasi-randomised trials were included, and of the
13 studies meeting initial selection criteria, only six met minimum quality criteria and
four had adequate data to be included in the formal meta analysis. Results
suggested no significant improvement on outcomes for those in ABI programs
compared with standard care on cognitive outcomes, expressive language, receptive
language, or adaptive behaviour. Low to moderate heterogeneity existed across
outcomes which prompts uncertainty about the suitability of meta analytic techniques
here, a limitation the authors acknowledged given high variability in participant
characteristics and programme content. Other limitations of the evidence base
disussed by the authors included difficulty in establishing genuine control groups,
lack of strict selection criteria, and limited information on participant retention. The
authors concluded that there is inadequate evidence that applied behaviour
intervention has better outcomes than standard care for children with autism.
Summary
These high quality reviews offer some agreement and some divergence in their
interpretations of the evidence base. Two reviews, which included meta analyses,
gave tentative evidence of positive benefits of ABA-based interventions over
comparison treatments from a limited evidence base. EIBI was reported as having
beneficial effects on adaptive behaviour and IQ compared with control or eclectic
comparators (Sigmund Eldevik et al., 2009). Some support was reported for discrete
trial training in terms of motor and functional skills but not for communication skills.
Lovaas (1987) therapy was presented as having benefit over no treatment, and
greater effect for higher intensity treatment (Ospina et al., 2008). However these
conclusions are qualified by methodological weaknesses of the individual studies and
the small number of studies and participants. In particular, the lack of control for the
intensity of treatment delivered between intervention and comparator is problematic.
In contrast, the two other reviews suggested there was inadequate evidence of
effectiveness of ABA-based early interventions. These conclusions were based on a
lack of significant results in the meta analysis of four studies by Spreckley and Boyd
(2009b). The Blue Cross and Blue Shield Association (2009) report argued that the
limitations and inconsistent results of the evidence base precluded making any
conclusions about the relative effectiveness of EIBI compared with alternative
treatments, about its effectiveness for particular outcome domains, or about the
impact of intensity on outcomes. The report also argued that study heterogeneity
ruled out the use of meta analysis.
All four reviews emphasised the need for additional research addressing
methodological weaknesses, and suggested the need for appropriately powered
RCTs to systematically investigate what characteristics of treatment lead to the best
outcomes. Particularly called for was the use of comparison interventions of similar
intensity and quality to those of EIBI.
Reviews of good quality

Four reviews were rated as being of good quality, having more flaws and limitations
than the very good reviews discussed above, often in terms of a limited search
strategy, lack of systematic quality assessment, and/or lack of detail and precision in
reporting study characteristics and synthesising results. However these studies are
still of reasonably good quality and of value in contributing to the evidence base.
A systematic review of comprehensive psycho-educational research (Eikeseth,
2009) included 20 studies on ABA-based interventions. The reviewers concluded that
intensive ABA treatment carried out by trained therapists is effective in enhancing
global functioning in pre-school children with autism, and those with PDD-NOS.
In Howlin et als (2009) review of 11 controlled (any comparator) group EIBI studies,
in 9 of 10 studies, IQ scores were significantly higher for children at follow-up in the
EIBI group compared with those in the control/comparator group, with moderate to
large effect sizes in the majority, but not all studies. However the authors also noted
that children receiving EIBI received significantly more (mean) hours of intervention
11
per week than those in the comparison group (30 c.f. 19 hours, respectively). It was
concluded that whilst EIBI is highly effective for some children, there is wide
individual variation in response, and some failed to make progress at all. Howlin et al
(2009) also noted that chronological age and length of intervention did not appear to
be related to outcome, with varying results for initial language level as a predictor.
Initial IQ strongly predicted improvement after EIBI treatment in four studies, but was
unassociated with outcome in a fifth.
A systematic review and meta analysis by Reichow and Wolery (2009) evaluated 13
controlled group studies on EIBI interventions based on the Lovaas (1987) model
(notably, Eldevik and Hastings, 2009, criticised this as too restrictive a definition for a
review of EIBIs effectiveness). Meta analyses of 12 studies considering change in IQ
within treatment group suggested a moderately large effect size for improvement at
followup, however the authors cautioned that this finding was limited by excluding
reference to a comparison group. As such, improvements cannot be attributed to EIBI
exclusively; for example, they could relate to maturation effects. As there was
significant variability between study characteristics, the appropriateness of using
meta analytic techniques for synthesising results is questionable. Descriptive and
summative syntheses suggested that EIBI can be an effective treatment, on average,
for some children with autism compared with control/various comparators. However
individual data typically were not presented and the authors noted that the
intervention has not worked for all children, and gains in one domain may not be
accompanied by gains in another domain within individuals. Moreover, comparison
groups were of limited quality and poorly described. The authors concluded that it is
not possible to determine whether EIBI is more or less effective than other treatment
options without comparisons between EIBI and empirically validated treatment
programs.
The systematic review by Rogers and Vismara (2008) considered early
interventions including psychopharmacological ones for young children with autism,
reporting on seven ABA based interventions. From four controlled studies evaluating
Lovaass (1987) treatment, the authors argued that this intervention meets criteria for
a well-established psychosocial intervention for improving the intellectual
performance (specifically IQ scores) of young children with ASD. Well established
interventions were defined as those requiring treatment manuals, clearly specified
participant groups, and either (a) two independent well-designed group studies
showing the treatment to be better than placebo or alternative treatment, or
equivalent to an established effective treatment, or (b) nine or more single-subject
design studies using strong designs and comparison to an alternative treatment
(Chambless et al., 1998). There was less consistency in the data for improvements in
behavior, adaptive skills, and language skills. Overall, the review concluded that
focused daily early intervention programs of several different kinds are beneficial for
young children with autism, however the lack of comparative studies precludes
determining which comprehensive treatment approach is best. Further, the authors
suggested that other well-known interventions may be as or more efficacious as
Lovaass (1987) model but they have not been rigorously evaluated.
Summary
The review by Eikeseth (2009) concluded that intensive ABA treatment carried out by trained
therapists is effective in enhancing global functioning in pre-school children with autism, and those
with PDD-NOS. Rogers and Vismara (2008) suggested that Lovaas-based (1987) EIBI met criteria
for a well-established intervention for improving the IQ of young children with ASD, but that there
was less consistency for improvements in behavior, adaptive skills, and language. Howlin et al
(2009) and Reichow and Wolery (2009) both concluded that whilst EIBI is highly effective in
increasing IQ for some children, some appear to make no progress, suggesting wide individual
variation in response. The former reviewers also found that age and length of intervention do not
appear to predict response, but that initial language level did in some studies, and initial IQ
appears to be related to impact on outcome in most studies evaluated.
Despite these broadly positive conclusions about the potential for benefit among (at least some)
children with ASD, the reviewers tempered their findings with reference to the methodological
limitations of the evidence base. Major concerns centred around the need to disentangle the
effects of programme content from those of programme intensity, which has not been well
controlled for. To address this, Howlin et al (2009) called for equivalence trials, where high quality
interventions are compared witho other high quality interventions of a similar intensity. Other calls
for further research echoed those of the very good studies reported earlier, including the need to
systematically assess characteristics of children for whom EIBI is most effective, optimal duration
and intensity of treatment, and age of commencement of treatment, so that better targetted and
cost-effective interventions could be developed (Howlin et al., 2009).
Another repeated theme concerned the limitations of the comparator or control groups with which
ABA based interventions are compared. These included their lack of standardization, being poorly
defined, lack of measures of procedural fidelity, and lack of data on any supplemental treatments
participants may also receive. The apparent superiority of EIBI may be an artifact of the lack of
rigorous empirical evaluation of alternative interventions of good quality.
As fully randomised controlled studies in this area were extremely rare, the nonrandom assignment
of participants to treatment groups was also said to severely limit conclusions about treatment
effectiveness. This is because factors affecting group allocation (such as parental chioce) may
have contributed to improved outcomes (such as increased parental motivation and support), thus
confounding any effects attributed to the treatment.
Only one review (Rogers & Vismara, 2008) made specific reference to cultural factors, observing
that treatment programmes for children with autism have been developed for and evaluated
primarily with children from European American backgrounds, and generalization of effects across
ethnic groups from this research is premature. It was suggested that cultural issues may moderate
the effects of autism intervention programs.
Reviews of fair quality

Four reviews were rated as being of fair quality, two considering ABA interventions broadly defined,
and the only two reviews considering a specific ABA based intervention, both relating the the
Picture Exchange Communication System. These intervention categories will be considered
separately, as they are in the Evidence Tables.
Applied Behaviour Analysis interventions in general
Two reviews on ABA interventions were classified as being of fair quality due to flaws in
methodology and reporting such that results are likely to be susceptible to bias. It should be noted
that some of these studies had supplementary or alternative goals to providing comprehensive
systematic reviews of the literature. Whilst the narrative critiques of the literature in these reviews
may be of interest, conclusions about the evidence base from these studies may not be reliable.
The review by Case-Smith and Arbesman (2008) reviewed ASD interventions relevant to
occupational therapy. The authors concluded that behavioural interventions are successful in
training children in basic academic and life skills, but noted that generalisation of skills to natural
environments was unproven. They also commented on the lack of studies into the promotion of
effective transition to work and independent living. Discussing barriers to widespread application
of behavioural interventions, the reviewers pointed to the time and resources required to implement
a 30- to 40-hour/week intervention as well as the lack of denition of the most appropriate
candidates.
Seida et al (2009) conducted an umbrella review of systematic reviews of psychosocial
interventions for people with ASD. Broadly favourable outcomes from behavioural interventions
versus no treatment were reported. Critiquing the literature, the authors noted the absence of data
on the relative effectiveness of treatment options, and the lack of research attention to factors in
choice of treatment including cost, convenience and family burdens.
Picture Exchange Communication System

Two systematic reviews of fair quality specifically considered interventions based on Picture
Exchange Communication System (PECS).
The review by Ostryn et al (2008) considered 15 PECS studies concluded that whilst PECS is
widely implemented it lacked a strong empirical base. Criticisms of the literature included lack of
reporting of statistical significance and precision of results, lack of scope and definition of key
outcomes, and lack of data on maintenance. The authors recommended that PECS was best used
as an initial intervention to teach manding as part of a multimodal system but is not recommended
as a long term intervention.
The other PECS review by Sulzer-Azaroff et al (2009) concluded from 34 studies that
professionals and parents can teach individuals to successfully initiate exchanges of pictures for
tangible and non tangible reinforcers.
REFERENCES
The references for this report and the appended Evidence Tables are presented below.
Publications included for critical appraisal in this review are indicated by ** prefacing the citation.
15
EVIDENCE TABLES
Evidence tables for included Level I secondary studies assessing the effectiveness of applied behaviour analysis (ABA) for people with
autism spectrum disorder.
1. Applied Behaviour Analysis interventions in general
Reference, Evidence level, aim and Inclusion and Results and authors conclusions Comments and quality
country, search method exclusion criteria score
design
Blue Cross Evidence level: I Inclusion criteria: 16 studies met selection criteria, including 2 RCTS, 9 nonrandomized, Clearly defined
Blue Shield reported on the use of comparative studies, and 5 single-arm studies. No studies were found that methodology though
(2009) Aim: EIBI compared to another included children only two databases
Technology To conduct a systematic treatment strategy; with Aspergers disorder; 4 studies explicitly included children with PDD or considered
Evaluation review of the research attempted to identify PDD-NOS. Study quality coded and
Centre literature on the use of features of EIBI that had rated, apparently by
early intensive the most impact on its The strongest evidence was provided by two randomized, controlled trials single reviewer.
US behavioural interventions effectiveness; or (Smith et al. 2000; Sallows and Graupner 2005) and they compared Justification given for
based on applied sought to identify children different interventions, had small sample sizes, and came to different excluding SCED studies
SR behavioural analysis most likely to benefit from conclusions. including lack of
(EIBI) among young EIBI. 3 of 4 studies examining the impact of pretreatment cognitive functioning generalisability across
children with ASD. Asked: found that it significantly predicted outcomes, while one (a randomized, individuals, inability to
(1) How effective is EIBI in Exclusion criteria: controlled trial) did not. Some studies suggested that younger age at the use blinding, variability
improving the functioning sample size <10, start of therapy is a predictor of better outcomes (e.g., Howard et al. of the ASD condition
of children with autism including single-subject 2005), while others found no difference based on initial age (e.g., Magiati over time, and potential
spectrum disorders, and studies; et al. 2007). for carryover effect
how does it compare to interventions were very The findings on whether more intense treatment leads to better outcomes between intervention
other early intervention poorly described; were inconsistent, and no conclusions were drawn. and comparator (given
approaches? (2) Can interventions were not that interventions are
patient characteristics be comprehensive, where a Authors conclusions: intended to have long
identified that predict number of domains Unfortunately, only two randomized, controlled trials have been lasting effects)
better outcomes from affected by ASD were conducted. Weaknesses in research design, differences in the treatments Comprehensive Tables
EIBI? addressed; and outcomes compared, and inconsistent results mean that the impact of including details of
(3) Does the effect of EIBI intervention within a EIBI versus other treatments on outcomes for children with autism cannot study characteristics,
vary with the intensity of treatment group was be determined. outcomes, results, and
treatment? heterogeneous, The body of evidence overall is too weak to reach conclusions regarding aspects of quality.
combining a variety of any of the domains/outcome areas, and too variable to assess whether Thorough critical
Search period: methods; greater intensity of EIBI can lead to better outcomes. discussion of included
From 1966 (varied, see experimental intervention The heterogeneity of the interventions used in studies on EIBI and the studies and of recent
below) July 2008. was not intensive (< 20 significant methodological weaknesses preclude the use of meta-analysis systematic reviews
A more narrowly defined hrs/wk); and of reaching any general conclusions about the effectiveness of EIBI Discussion of the
update was performed in study did not directly among children with ASD. methodological
January 2009 using measure outcomes About half of the studies followed children for approximately 2 years or limitations of studies
Medline but identified no through a direct less, and some for only 1 year. This is not sufficient follow-up time to Suggestions made for
additional eligible studies. assessment of the childs assess the potential impact of an intervention over a lifetime. future research
16
design
Note that selection criteria achievement; Research is needed to identify those characteristics of treatment - content,
then excluded studies study was technique, intensity, starting and ending age, etc. - that maximize its Overall quality: Very good
published <1987. published before effectiveness. (Need) a greater emphasis on randomized, controlled trials;
1987, when the substantially larger sample sizes; uniformity of outcomes and instruments;
Search strategy: seminal article on and consistent treatments that do not vary widely within the experimental
Medline (from 1966) EIBI by Lovaas or control group.
PsycINFO (from 1970) was released. The cost of continuing the current course of assuming that EIBI works may
(search terms provided). not be obvious. EIBI is costly financially for society and requires a large
Study quality was appraised time commitment from children, their families, and their teachers or
References of retrieved and graded (as good, fair or therapists. However, these programs may not appear to pose any harm for
papers were cross poor) using detailed criteria the children themselves. Nevertheless, the opportunity costs could be
checked to identify and formal grading systems. high, indeed, of providing suboptimal care to these children, simply
additional papers. because we as a society do not know what works best. The children may
be treated with an intervention that is not as effective as the alternatives.
17
design
Case-Smith Evidence level: I Inclusion criteria: 49 studies met selection criteria. Six categories of research topics were Search strategy
& Arbesman Provided evidence identified, including intensive behavioral intervention, which is reported on here. restricted to databases,
(2008) Aim: for an intervention no search terms given.
To identify, evaluate, and approach used with EIBI using one-on-one discrete trial training is widely applied to children with No Tables
USA synthesize the research children or ASD and has evidence of moderate to strong effects (Lovaas, 1987; Sallows Narrative discussion of
literature on interventions adolescents with & Graupner, 2005; Smith, Groen, & Wynn, 2000). results with minimal
SR for ASD of relevance to ASD; Positive behavioral support has moderate to strong positive effects in methodological critique
occupational therapy. Peer reviewed; reducing problem behaviors in children with ASD. A systematic approach and minimal detail of
Addressed a that includes prevention of problem behaviors by applying consistent quantitative results.
Search period: performance area instruction and consequences to behavior, modifying the environment to Framed selection and
1987 2007 or intervention promote appropriate behaviors, and collecting data to monitor childrens discussion of studies in
approach within the progress appears to be highly effective (Horner et al., 2002). terms of relevance to
Search strategy: domain of practice of occupational
CINAHL occupational Authors conclusions: therapists.
Cochrane SRs database therapy; Although behavioral interventions are successful in training children in
ERIC RCTs, non basic academic and life skills concepts, it is not known how well these skills Overall quality: Fair
Medline randomized clinical transfer into the natural environment, and the ndings are mixed regarding
PsychINFO trials, and before- changes in behavior (Smith et al., 2000). Barriers to widespread application
Social Science Abstracts and-after, one are as follows: (1) time and resources required to implement a 30- to 40-
Sociological Abstracts group designs. hr/wk intervention and (2) denition of the most appropriate candidates
RehabData (Bassett, Green, & Kazanjian, 2000).
Latin American and Exclusion criteria: When children exhibit problem behaviors, functional analysis is essential to
Caribbean Health Case determine the basis for the behavior.
Sciences Literature series, single subject Level I studies of interventions to promote adolescents and young adults
EBSCOHost design, case reports and success in work and independent living were virtually absent from the
(search terms not expert opinion, narrative research literature.
provided). reviews, consensus
statements;
Used
qualitative methods to the
exclusion of quantitative
methods;
Had
serious design limitations.
Single reviewer extracted

data. Used a hierarchy of
evidence based on study
design to rank studies.
18
design
Howlin et al. Evidence level: I Inclusion criteria: 11 studies (reported in 13 reports) met selection criteria, including one RCT Clearly defined
(2009) Case-control comparison (Smith et al, 2000). methodology and
Aim: study of EIBI for children comprehensive search
UK To examine the findings with autism; Intervention intensity strategy
from controlled early UCLA affiliated The mean intervention period for the 9 studies which reported No rating of study
SR intensive behavioural (home or clinic intensity clearly was 27.4 months (SD=10.7, range 14-48 mths). quality
intervention (EIBI) studies based) or UCLA- Intensity (number of hours of intervention per week) varied greatly by Detailed Tables and
published in peer modeled (home- individual within study. Estimated that EIBI children received significantly discussion of results
reviewed journals. based) EIBI more hours of intervention per week than controls (EIBI Mean=29.8 hrs/wk Good attempt to draw
program; cf Control Mean=19.1 hrs/wk, p=.007). out patterns of results
Search period: At least 10 people and explain findings.
1985 May 2007 in EIBI group; Impact of intervention on IQ Discussion of the
age < 6 years at IQ did not vary at baseline between treatment groups, but at final follow up methodological
Search strategy: start of treatment; was significantly higher for EIBI group compared to the Control group in 9 of limitations of included
CINAHL intervention at least 11 studies, with estimated effect sizes moderate to large in the majority of studies
Cochrane library 12 hrs/wk; studies, but not all studies. Suggestions made for
ERIC intervention for 12 future research
Medline mths or longer Variability in outcome
PsychINFO adequate data on There was considerable variation between individuals, with a minority Overall quality: Good
Embase IQ or standard achieving marked improvement and even educational independence, whilst
(search terms provided). outcome measures. the majority had less dramatic improvements. Some failed to make progress
at all.
Searched websites of Exclusion criteria:
several major health, SCED or case Predictors of outcome
autism and research series study without Length of intervention did not appear to be related to outcome, and
organisations. References comparison group, most impact on IQ occurred in the first year of the intervention after which
of review papers were studies focused on a the effect tended to plateau.
cross checked to identify specific behaviour, use of Initial (baseline) IQ strongly predicted improvement in 4 studies, but was
additional papers. pharmacological or non unassociated in another study that investigated this.
ABA interventions, Chronological age did not predict outcome (though age range tended to
outcomes for/views of be limited to 3-5.5 yrs) which does not support suggestions that EIBI must
therapists or parents. start at 3 years.
Mixed results were found for initial language level which was associated
Summaries/analyses based on with outcome in 4 of 7 studies investigating this.
comparisons of the published
group means. Methodological limitations
Comparisons were made difficult by varying time lag of follow-up after
ending of intervention, and use of different instruments and scoring
reported (raw, standard, age equivalents).
Few researchers have assessed outcome in terms of behavioural
difficulties/severity of autism, or impact on family life.
Few studies reported on quality of alternative therapies in the Control arm,
which tended not to be autism specific.
Authors conclusions:
There is little question that EIBI is highly effective for some children,
however gains are not universal.
19
design
Crucial need to systematically assess for which children is EIBI most and
least effective. Clearer evidence concerning the optimal duration of
therapy and the age at which is should begin could result in the
development of better targeted, more cost effective programmes.
Need for more RCTs, and need for equivalence trials, where high quality
interventions are compared with each other, rather than comparing high
quality interventions with low quality/low intensity interventions. Failure to
control for time in intervention means that for some studies, EIBI may be
more effective due to differences in intensity, not quality.
20
design
Eikeseth Evidence level: I Inclusion criteria: 25 studies met selection criteria, including 20 which evaluated interventions Comprehensive search
(2009) study was based on Applied Behavior Analysis (ABA) treatment, 3 studies evaluated strategy, but no search
Aim: published in a peer- Project TEACCH, and 2 studies evaluated the Colorado Health Sciences terms or dates given to
Norway Evaluates comprehensive reviewed journal; Model. Only those relating to ABA are discussed here. allow replication
psycho-educational children had a Very detailed Tables,
SR research on early mean age of 6 Scientic merit describing studies,
intervention for children years or less at 1 ABA study, an RCT, received Level 1 scientic merit (Smith, Groen, & interventions, and
with autism. intake; Wynn, 2000) discussion of results
participants 4 ABA studies received Level 2 rating (Cohen, Amerine-Dickens, & Lacks detail on
Search period: received Smith, 2006; Eikeseth, Smith, Jahr, & Eledevik, 2002, 2007; Howard, participant
Not reported comprehensive Sparkman, Cohen, Green, Stanislaw, 2005; Remington et al., 2007) characteristics at
psycho-educational 9 received Level 3 evidence support baseline.
Search strategy: interventions; 6 ABA outcome studies were classied as having insufcient (level 4) Discussion of the
ERIC studies contained scientic value. methodological
Medline outcome data. limitations of included
Psyclit Magnitude of treatment effects studies
(search terms not Comprehensive psycho- 4 ABA studies received Level 1 rating demonstrating that children Suggestions made for
provided). educational interventions were receiving ABA made signicantly more gains than control group children future research
dened as interventions on standardized measures of IQ, language and adaptive functioning directions
References of recent addressing social behaviors, (Cohen et al., 2006; Eikeseth et al., 2002, 2007; Howard et al., 2005; Insufficient quantitative
papers were cross communication and Sallows & Graupner, 2005). Several studies also included data on information on the
checked to identify ritualistic/stereotyped maladaptive behavior, personality, school performance and changes in extent of the gains
additional papers. behaviors. diagnosis. experienced, the clinical
3 ABA studies received Level 2 rating (Eldevik et al., 2006; Lovaas, 1987; meaningfulness of these
Researchers known in the Exclusion criteria: Smith, Groen, & Wynn, 2000), demonstrating that ABA treated children gains and whether there
field were contacted by None reported. made signicantly more gains than the comparison group on one were subgroups of
email and asked for standardized measures of IQ or Adaptive Functioning. children who did not
relevant articles recently Outcome studies were graded 5 ABA studies received Level 3 rating. improve.
published or in press. according to their (i) scientic
value and (ii) according to the Authors conclusions: Overall quality: Good
magnitude of results Recommended practice parameter
documented in the studies. ABA treatment is demonstrated effective in enhancing global functioning
in pre-school children with autism when treatment is intensive and carried
Scientic Merit was evaluated out by trained therapists.
based on: (a) diagnosis, (b) ABA treatment is demonstrated effective in enhancing global functioning
study design, (c) dependent in children with PDD-NOS (one Level 1 study; Smith, Groen, & Wynn,
variables and (d) treatment 2000).
delity. Four levels were used. Guideline practice parameter
ABA can be effective for children who are up to 7 years-of-age at intake
(one Level 2 study; Eikeseth et al., 2002, 2007).
21
Reference, Evidence level, aim and Inclusion and exclusion criteria Results and authors conclusions Comments and quality
country, search method score
design
Eldevik et al Evidence level: I Inclusion criteria: 9 controlled designs met selection criteria, including one RCT Moderately broad
(2009) intervention generally adhered to (Smith et al, 2000). Howard et al (2005) included both a search strategy
Aim: defined common EIBI elements; Control and Comparison group and so 10 effect sizes were Excluded Sallows and
Wales & To conduct a systematic aged between 2-7 years at intake; calculated. Four studies had a Comparison group, and six had Graupner (2005) as it
Norway literature search for diagnosed with autism, PDD NOS; a Control group. compared two EIBI
studies reporting effects of full-scale measure of intelligence groups of somewhat
SR/MA Early Intensive Behavioral and/or standardized measure of The average effect size was 1.10 for change in full-scale different intensity
Intervention (EIBI). Also adaptive behaviour at intake and intelligence (95% CI .87, 1.34) and .66 (95% CI=.41, .90) for varying in programme
aimed to replicate and post intervention; change in adaptive behavior composite. These effect sizes are delivery (clinic- or
extend (with intervention lasted 12-36 mths; generally considered to be large and moderate, respectively. parent-directed)
methodological included Control (no intervention, As the two sets of comparative data from Howard et al were Used more inclusive
improvements) the MA of considerably less intensive one, or compared to the same intervention group and not independent, definition of EIBI than
Reichow and Wolery poorly described intervention) or an analysis excluding the control group from Howard et al Reichow and Wolery
(2009). Comparison (non EIBI intervention (2005) was also conducted, with effect sizes not significantly (2009)
of similar duration and intensity) altered. Cited papers of those
Search period: group. retrieved which were
- March 2008 Homegeneity was tested and the Q statistic was not significant, excluded and noted why
Exclusion criteria: indicating that studies could be combined, and a fixed effect Detailed account of
Search strategy: Primarily using nonverbal intelligence model was used. No statistical or visual evidence of publication outcome measures
ERIC measure; bias was observed. considered
PsycINFO Case study or case series; Approached authors for
PubMed Data from people in another included Authors conclusions: individual data
(search terms provided). study. EIBI produces large to moderate effect sizes for changes Fully detailed Tables,
in IQ and ABC scores for children with ASD when describing design,
References of retrieved Inter-rater reliability was high for selection and compared with no intervention controls and eclectic allocation to group
papers were cross coding of articles. Approached authors for raw provision. The results support the clinical implication that method, interventions,
checked to identify individual data which was re-analysed. at present, and in the absence of other interventions with outcome scores
additional papers. Completed a meta-analysis yielding a established efcacy, Early Intensive Behavioral Moderator variables
standardized mean difference (SMD) effect Intervention should be an intervention of choice for were not considered.
size for two available outcome measures: children with autism. One author is from
change in full-scale intelligence and/or The difference in outcome between EIBI and the Lovaas institute.
adaptive behavior composite. Effect sizes comparison intervention may be due to differences in the
were computed using Hedgess g. amount and frequency of supervision and training. There Overall quality: Very good
was insufficient data to control for this in the present
study, however the EIBI group in general received more
frequent and more total hours of supervision and training.
This remains a threat for the validity of conclusions about
the superiority of EIBI in relation to comparison
intervention.
Randomized controlled trials comparing EIBI to other
interventions are still needed. In particular, studies are
needed where the comparison intervention is of similar
intensity and where staff receive similar training and
supervision.
22
Reference, Evidence level, aim and Inclusion and exclusion criteria Results and authors conclusions Comments and quality
country, search method score
design
Ospina et al. Evidence level: I Inclusion criteria: 101 studies met selection criteria, and evaluated 8 broad types of Clearly defined
(2009) RCTs, non randomized controlled intervention defined on a continuum of behavioural to methodology and
Aim: trials, and prospective and developmental interventions, the most relevant reported here being extensive and
Canada To identify, appraise and retrospective cohort studies with a the ABA category, which included discrete trial training (DTT) and comprehensive search
synthesise the evidence control group; UCLA/Lovaas therapy as sub-categories. There were 31 ABA strategy
SR/MA on the effectiveness of published in English; studies, including 12 trials and 9 cohort/observational studies. Review reports
behavioural and reporting on the efficacy of any extensive details of
developmental behavioural or developmental Descriptive analyses individual studies
interventions for intervention; The reviewers judged studies to be at predominantly high risk including sample
improving core symptoms for individuals with ASD (including of bias and reported inconsistent results across various characteristics, as well
of ASD. those with dual diagnoses); interventions. as assessments for
reporting on outcomes relating to Discrete Trial Training effects were inconsistent, but motor and each study across
Search period: the core features of ASD (though functional outcomes were often positive compared to speech- several quality domains.
May 2007 additional outcomes may also be related outcomes which were generally negative. For Lovaas, 7 Careful synthesis of
reported). of 8 non-RCTs reported positive findings, whereas 3 of 4 RCTs results.
Search strategy: reported no significant findings. The reviewers note that as Discussion of the
22 electronic databases, Exclusion criteria: non-RCT evidence can be more prone to bias and overestimate methodological
including: Individuals with Retts Disorder, or treatment effects, this observation has serious implications for limitations of review
BioSYS Previews Childhood Disintegrative Disorder. interpreting the results from non-RCTs. approach and studies.
CINAHL Heterogeneity was moderate for some outcomes (for overall In particular, notes risk
Cochrane central register Two independent reviewers made the intellectual functioning, non verbal intellectual functioning, of expectancy bias for
of controlled trials final study selection, extracted data socialization). studies with non active
Embase using pre-tested checklists, and reached controls (eg, wait list)
ERIC consensus on study quality. Meta- Meta Analyses where participants (and
Medline analyses of the study results were Meta-analyses of 3 controlled clinical trials showed that Lovaas their families) in control
ProQuest Dissertations conducted when two or more trials treatment was superior to special education on measures of group would not expect
and theses assessed the same intervention, used adaptive behaviour (WMD = 11.8; 95%CI, 6.94 to 16.67), improvement.
PsychINFO similar comparison groups, and had data communication and interaction (WMD = 16.63; 95% CI, 11.25 Moderate heterogeneity
Web of Science for common outcomes of interest. If the to 22.01), comprehensive language (WMD = 12.84; 95% CI, evident for some
(search terms provided). same measure was reported, reported 6.38 to 19.30), daily living skills (WMD = 5.61; 95% CI, 0.54 to outcomes.
weighted mean differences (WMD); 10.67), expressive language (WMD = 15.05; 95% CI, 6.19 to Suggestions made for
Hand searching journals, otherwise, reported standardised mean 23.90), overall intellectual functioning (SMD = 0.95; 95% CI, future research
cross-checking of differences (SMD). SMD of 0.2 indicated 0.44 to 1.46), and socialization (WMD = 9.17; 95% CI, 2.16 to
reference lists, databases a small effect, 0,5 a medium, and 0.8 a 16.19). Overall quality: Very good
of theses and large effect. High-intensity Lovaas was superior to low-intensity Lovaas on
dissertations, and measures of intellectual functioning in two retrospective cohort
contacting experts in the studies (SMD = 0.92; 95% CI, 0.61 to 1.24).
field were used to identify No statistically significant differences were found for: Lovaas
additional papers. versus special education for non-verbal intellectual functioning
in a meta analysis of three controlled clinical trials (SMD = 7.83;
95% CI, 22.86 to 18.52); or for Lovaas versus Developmental
Individual-difference Relationship-based intervention for
communication skills based on two RCTs (SMD = 0.73; 95% CI,
20.26 to 1.72).
23
design
Authors' conclusions:
There is no clear answer regarding the most effective therapy to improve
symptoms associated with ASD.
The evidence seems to provide some support for discrete trial training in
terms of motor and functional skills but not for communication skills.
Lovaas' therapy showed benefits when compared to ''no treatment'' and
evidence from meta-analysis of retrospective cohort studies showed greater
effects for High versus Low intensity Lovaas. Whilst Lovaas may improve
some core symptoms of ASD compared to special education, these findings
are based on pooling of a few, methodologically weak studies with few
participants and relatively short-term follow-up. Replication in RCTs is
needed to substantiate the use of Lovaas and to assess the effect of
treatment intensity on outcomes of children with ASD.
There is considerable potential for heterogeneity in the population,
intervention, comparator and outcomes of interest, as ASD is a spectrum
disorder, therapy is not always reported in detail, comparators are difficult to
control for, and outcomes are somewhat subjective.
As no definitive behavioural or developmental intervention improves all
symptoms for all individuals with ASD, it is recommended that clinical
management be guided by individual needs and availability of resources.
24
design
Reichow and Evidence level: I Inclusion criteria: 14 studies met selection criteria, although due to patient overlap, only 13 Limited detail provided
Wolery evaluated EIBI based on publications were appraised, reporting on 14 distinct samples. The studies on search strategy
(2009) Aim: the UCLA/Lovaas (1987) included 10 studies allowing between group analyses, including two RCTs restricting ability for
To provide a 3-part model; (Smith et al, 2000; Sallows and Grupner, 2005). replication.
USA comprehensive synthesis participants had Detailed Tables, figures
of the early intensive diagnoses of autistic Descriptive analyses Precision details
SR/MA behavioral intervention disorder, ASD, PDD, or EIBI is an effective intervention for children with autism. On average, (confidence intervals)
(EIBI) for young children PDD-NOS; children present fewer or less severe autism symptoms after intervention. given in Figure only
with autism based on the participant samples Effect size differences between treatment groups Discussion of the
Lovaas University of receiving EIBI had a The between group (standardized mean difference) effect sizes suggest methodological
California at Los Angeles mean chronological age < children receiving EIBI made more gains than children receiving minimal limitations of included
Young Autism Project 84 months at the behavioral intervention, eclectic treatment, or treatment as usual. studies and of approach
method (Lovaas, 1987). beginning of treatment; Children receiving EIBI made large gains on multiple domains of of reviewers.
The synthesis included: mean duration of EIBI behavior, and made better progress than children with autism who receive Studies used a range of
(a) descriptive analyses, was 12 months; less intense behavioral intervention or other treatments. measures of IQ
(b) effect size analyses, at least one child outcome Meta analysis for change in IQ within treatment group Included quasi
and (c) a meta-analysis. measure was reported; A meta-analysis was conducted on 12 samples using standardized mean experimental studies
experimental (e.g., change effect size for IQ within the intervention group (because without an equivalent
Search period: pre-/post-test multiple- comparison groups varied across studies). Using a random effects model, control group in the MA.
Not reported group design) or quasi- the mean effect size was 0.69. This is statistically signicant (p < 0.001) Heterogeneity present,
experimental designs (i.e., and represents a large effect. This suggests EIBI is, on average, an which raises questions
Search strategy: nonequivalent control effective intervention for increasing IQ scores for children with autism, about whether
An electronic database group, one-group though the effect is likely to be inflated by publication bias. combining results
search, databases not pre-/post-test design); Tests of homogeneity suggest that there was significant variability through a meta analysis
identified. (Search terms publication in English in a between studies. was appropriate.
not provided). peer-reviewed journal. Moderating variables identified a priori suggest the greatest results on IQ Suggestions made for
sample adjustment. change might be seen when supervisory staff were trained using the future research
References from review UCLA model (p<0.01). directions
articles and eligible
reports were cross Overall quality: Good
checked to identify
additional papers.
Hand search of selected

journals.
Contact with experts.
25
design
Study characteristics and Authors conclusions:
effect size data were coded While these ndings were strong, the nonrandom assignment of
using a manual. Two participants to groups limit conclusions about the superiority of EIBI to
independent coders for 4 of other treatments. Further, the comparison groups often lacked
the studies had mean standardization within the group, were poorly defined, had no measures
interobserver agreement of of procedural fidelity, and had no data on whether participants received
92%, range 85-93%. Studies supplemental treatments.
rated on experimental rigour. The ndings suggest EIBI can be an effective treatment, on average, for
Meta analysis conducted for some children with autism. However the intervention has not worked for
studies reporting within-group all children, and individual data typically were not presented. It is unclear
comparisons using if individuals making change in one domain (e.g., IQ) also made gains in
standardized mean change another (e.g., adaptive behavior).
effect size with small sample These results should be taken with caution because of the small number
adjustment. of studies, and because the standardized mean change effect size is
calculated without reference to a comparison or control group. Thus, the
threats to internal validity of history, maturation, lack of procedural delity,
and instrumentation threats cannot be eliminated. Thus, while the effect
sizes were often large, they cannot be attributed to EIBI exclusively. Effect
sizes tend to be inflated when using standardised mean change effect
sizes (Morris, 2000). Without comparisons between EIBI and empirically
validated treatment programs, it is not possible to determine if EIBI is
more or less effective than other treatment options.
26
design
Rogers and Evidence level: I Inclusion criteria: 22 studies met selection criteria. Studies of non ABA interventions, including Limited search strategy,
Vismara study involved psychopharmacological interventions, are not discussed here. 1 database and cross-
(2008) Aim: comprehensive 5 RCTs, included 4 relating to ABA were identified: Jocelyn et al (1998), checking
What is the empirical (addressing core Drew et al (2002), Smith et al (2000), and Sallows and Graupner (2005). Detailed Tables,
US evidence supporting deficits in autism) Non-randomized controlled group design studies included 2 of Lovaas- describing studies,
efficacy of early treatment based interventions: Eikeseth et al (2002), Cohen et al (2006); and one of interventions, and
SR intervention or young approaches for an EIBI approach using a mixture of didactic and naturalistic behavioral discussion of results
children with autism. children with teaching approaches by Howard et al (2005). Discussion of
autism; methodological
Search period: predominantly ages Results suggest that young children with autism, as a group, demonstrate limitations of included
1998 - 2006 5 years or younger; accelerated developmental gains in response to focused daily studies and findings.
controlled group interventions of several different kinds. Significant increases in language Detailed suggestions
Search strategy: designs or single- and communication abilities in the treated group occurred in most studies made for future
PsycINFO subject multiple and interventions with many targeted hours per week resulted in research directions,
(search terms provided). baseline designs increases in IQ at the group level as well. especially around
using 3 or more The best-designed, controlled studies evaluating Lovaass treatment met mediators and
References of all subjects with criteria for a well-established psychosocial intervention for improving the moderators of effect
reviewed articles were published data. intellectual performance of young children with autism spectrum Practice
cross checked to identify disorders, based on the significant increase in IQ reported in these four recommendations are
additional papers. Exclusion criteria: studies compared to control groups. Although some of these studies also offered for
studies that did not report reported significant improvements in behavior, adaptive skills, and psychologists.
as outcomes analyses of language skills, reviewers argue that there is less consistency in the data Discussed the value of
child progress using in these areas. both RCTs and SCED
general measures of No treatment met the probably efficacious criteria, although 3 studies in the pathway
childrens language or treatments met criteria for being possibly efficacious. of intervention
intellectual development; Most studies were either Type 2 or 3 in terms of their methodological rigor development.
studies targeting only one based on Nathan and Gormans (2002) criteria.
domain (eg; unwanted Overall quality: Good
behaviours); Authors conclusions:
case reports; The evidence suggests that early intervention programs are beneficial for
studies whose data were children with autism, often improving developmental functioning and
published only in book decreasing maladaptive behaviors and symptom severity at the level of
chapters; group analysis.
studies included in review Lack of comparative studies prevents determining which comprehensive
by Rogers (1998). treatment approach is best for young children with autism. Other well-
known interventions may be as or more efficacious as Lovaass model but
Studies were graded they have not been rigorously evaluated.
according to methodological Because the majority of studies did not include ethnically diverse
criteria based on Chambless participant groups, generalization of effects across groups is premature.
et al (1998) as being (i) well Given the few randomized controlled treatment trials that have been
established or (ii) probably carried out, the few models that have been tested, and the large
efficacious. Studies also differences in interventions that are being published, it is clear that the
graded using Nathan and field is still very early in the process of determining (a) what kinds of
Gorman (2002) criteria as interventions are most efficacious in early autism, (b) what variables
Type 1 through to Type 6, moderate and mediate treatment gains and improved outcomes following
ranging from most to least intervention, and (c) the degree of both short-term and long-term
27
design
rigorous. improvements that can reasonably be expected for a child with autism.
28
design
Seida et al Evidence level: I Inclusion criteria: 30 studies met selection criteria representing five intervention domains, Broad search strategy
(2008) Systematic reviews including one of interventions based on behavioural theory (n=9) reported on Use of two reviewers
Aim: (defined as having here. and quality checklist
Canada To present an umbrella defined a search Only secondary studies
review of the clinical strategy, and All 9 SRs on interventions based on behavioural theory were rated as having considered
SR (of SRs) findings of systematic attempted to low methodological quality. Only sparsely detailed
reviews of psychosocial synthesise data of Tables, listing
interventions for ASDs. primary studies The reviews with meta-analyses showed favourable outcomes for the intervention domains
quantitatively or behavioural intervention. Reductions in problem behaviour were found in and the outcome areas
Search period: qualitatively); 3 reviews and an increase in adaptive, cognitive, and language skills was where positive,
May 2007 Data reported for observed in 1 review. negative or unclear
participants with In the reviews without meta-analysis, positive ndings were reported for results were found. No
Search strategy: autistic disorder, intelligence, developmental gains, functional skills, and communication data provided to allow
25 electronic databases Asperger outcomes. The authors of 3 reviews concluded that there is uncertainty estimation of effect or
including Medline syndrome, atypical about whether behavioural interventions produce normal functioning and precision of findings.
(listed with search terms autism, high improvement on various intelligence and developmental measures. The Suggestions made for
in an online supplement). functioning autism, remaining 2 reviews provided no information on efcacy outcomes; future research,
PDD-NOS, and/or instead trends in the use of behavioural treatments over time were including on reporting
References of articles and suspected autism; described. and conduct of SRs
of personal collections Have covered a
were cross checked to psychosocial Authors conclusions: Overall quality: Fair
identify additional papers. intervention aimed The reviews reported positive outcomes for many of the interventions,
at improving the suggesting that some form of treatment is favourable over no treatment.
functioning of Little evidence for the relative effectiveness of these treatment options.
individuals in any of Many of the systematic reviews had methodological weaknesses that
the impairments make them vulnerable to bias.
characteristic of Even if differences in the therapeutic effectiveness of the interventions
ASD. exist, differences in cost, convenience and family burdens associated with
the interventions are likely to be important factors in individual decision-
Exclusion criteria: making.
editorials, Future studies and reviews that break down the characteristics of the
correspondence, individuals with autism and the components of the programmes are
abstracts and review needed in order to provide more meaningful and stronger conclusions.
summaries
Abstracts scanned by single

reviewer but eligibility criteria
applied on full text
independently by two
reviewers. Studies were
graded for methodological
quality by two reviewers using
the Overview Quality
Assessment Questionnaire.
Data extracted by single
reviewer, and verified by
another.
29
design
Spreckley & Evidence level: I Inclusion criteria: 13 studies met selection criteria, 6 were trials with adequate methodologic Broad database search
Boyd (2009) Systematic reviews, quality (PEDro6), and 4 of these had adequate data to be included in the strategy, but no other
Aim: randomized or quasi- meta analysis (Smith et al, 2000; Sallows and Graupner, 2005; Eikeseth et al search technique
Australia To review the randomized controlled 2002; and Eikeseth et al, 2007). Applied minimum
effectiveness of applied trials (RCT); quality criteria for
SR/MA behavior intervention Preschool children with Meta-analysis of 4 studies concluded that, compared with standard care, ABI inclusion in the MA
(ABI) programs for diagnosis of ASD or programs did not signicantly improve outcomes of children in the Tables described
preschool children with PDD; experimental group compared with those who received standard care: design, interventions,
autism spectrum disorder Interventions focused on For cognitive outcomes, a standardized mean difference (SMD) of 0.38 comparators, and
(ASD) in their cognitive, ABI approaches to (95%CI -0.09 to 0.84; P=.1) intensity.
adaptive behavior, and behavioural For expressive language; SMD of 0.37 (95%CI - 0.09 to 0.84; P=.11) Estimates of precision
language development. management; For receptive language; SMD of 0.29 (95%CI - 0.17 to 0.74; P=.22) provided.
Interventions delivered to For adaptive behavior; SMD of 0.30 (95%CI -0.16 to 0.77; P=.20). Too few studies
Search period: parents/carers and/or included to consider
From 1982 but varied directly to the child by Heterogeneity (measured by I2) was low for cognitive and receptive language sub-group or sensitivity
(see below) - Nov 2007 special educators, outcomes, and moderate for expressive language and adaptive behaviour analyses or to consider
teachers, speech outcomes. moderator variables.
Search strategy: pathologists, Follow-up
Cochrane Library psychologists, or allied Authors conclusions: correspondence about
Medline (from 1996) health professional Currently there is inadequate evidence that ABI has better outcomes than this article from Smith
PsycINFO (from 1985) students; standard care for children with autism. Appropriately powered clinical et al (2009) suggested
CINAHL (from 1982) Interventions occurred trials with broader outcomes are required. that the Sallows and
AMED (from 1985) between 18 mths and 6 Limitations of the meta-analysis and evidence base include: high Graupner (2005) study
(search terms provided). years; variability in the studies included (in ages of children involved, programme be excluded from the
Cognitive, language, content), difficulty in establishing control groups all received some form of MA as its comparison
and/or adaptive intervention), poor homogeneity, limited information on retention in the group was a lower
behaviour outcomes interventons groups, and lack of strict selection criteria for participants. intensity form of ABA.
measured. What is too often forgotten is that the overwhelming majority of children Spreckley & Boyds
with ASD change over time as part of their development. (2009b) reply argued
Exclusion criteria: that they were
Studies with PEDro score evaluating standard
<6; (i.e., high intensity) ABA
RCTs with no useful data with any standard or
for meta-analysis; and/or eclectic comparator.
those that did not include Limitations and
discrete trial training heterogeneity of the
(DTT) as part of their evidence base raises
intervention. questions about
whether a meta
2 reviewers independently analysis was
applied the Physiotherapy appropriate.
Evidence Database (PEDro)
Scale of quality assessment to Overall quality: Good
critically appraise the studies.
Data synthesis used RevMan,
reported effect sizes and
standardised mean differences
30
design
(SMD).
31
2. PECS
design
Ostryn et al. Evidence level: I Inclusion criteria: 15 studies met selection criteria. Most studies were single subject within Broad search strategy
(2009) English language; subjects or multiple baseline designs, with one RCT (Yoder and Stone, 2006) Detailed Tables
Aim: Used experimental and one non-randomised, between-group experimental study (Carr and Felce, describing outcome
USA To examine research or quasi- 2007). measures and results,
involving the use of PECS experimental but lacking data on
SR with individuals with ASD design; Noted that whilst positive outcomes were reported in all but four participant
to identify dependent Included individuals studies, though these were not defined in terms of statistical significance. characteristics and
variables and outcomes aged under 18 Noted that acquisition rates (through phases of PECS 1-6) were reported interventions.
reported in the research, years old with as being fast by several studies, though fast was not defined. Narrative discussion of
and to apply the ASDs; Commented that outcomes such as spontaneous communication were results framed around
communication evaluated PECS; only defined in two studies in relation to visual prompts. how outcomes
competence paradigm to published in a peer- Outcomes associated with joint attention were measured in only three measures in PECS
the PECS research. reviewed Journal. studies. studies relate to the
Notes that few studies examined maintenance of PECS outcomes. communication
Search period: Exclusion criteria: Where they are reported, results are encouraging. competence paradigm.
1985 2007 Not reported No formal checklists
The authors applied the communication competence paradigm to PECS used to assess or grade
Search strategy: Narrative synthesis outcomes and argue that additional strategies to those of manding or tacting quality.
ERIC could enhance PECS. The authors suggest that PECS strategies may be too Minimal synthesis of
PsychINFO limited for some individuals and people with ASD should be encouraged to use findings, although note
PubMed any communication attempts, including speech, gestures, and vocal that this was not
(search terms provided). approximations. specifically the aim of
the review.
Also searched PECS Authors conclusions: Suggestions made for
website. References of Results of this study reveal that the PECS is widely implemented with future research.
retrieved papers were individuals having ASDs but without a strong empirical base. Suggestions for how to
cross checked. Suggest that PECS is best used as an initial intervention to teach use PECS as part of a
manding and the basic elements of communicative exchange, but that multimodal approach
other communication systems may be needed to permit communication including other
beyond this. communication systems
PECS is not recommended as a long term intervention and is best
implemented as part of a multimodal system for when picture Overall quality: Fair
communications are more socially appropriate.
32
design
Sulzer- Evidence level: I Inclusion criteria: 34 studies met selection criteria, including two RCTs (Howlin et al. 2007) Limited search strategy,
Azaroff et al. Articles including (Yoder and Stone 2006b) (Yoder and Stone 2006a) and one non randomised particularly with respect
(2009) Aim: data and protocol experimental study (Carr and Felce 2007). Studies reported on 386 study to search terms and
To synthesise key containing the key participants, the majority of whom had received diagnoses of ASD. reporting.
UK features of the published, features of PECS Narrow definition of
peer-reviewed, data- (as specified by The authors presented a number of summarising statements in the Results PECS.
SR based research reports of Frost and Bondy). section asserting positive research findings, however these were not Lacks detail of outcome
Picture Exchange attended by quantitative supporting data, analysis or synthesis of study measures and follow-up
Communication System Exclusion criteria: findings. For example, researchers reported improvement in communication periods of inidivudal
(PECS) applications, to Papers written in among the vast majority of their participants. Also, several investigators studies.
answer the question how languages the authors provided evidence that learning to use PECS was associated with some of Results and Discussion
solid is the scientific couldnt read (n=3) their participants increasing their speaking and social approaching and a focussed on narrative,
evidence supporting Studies describing number of investigators cited decreases in disruptive or dangerous imprecise description of
PECS effectiveness? different pictorial behaviours. studies.
interventions to the Detailed Tables
Search period: original PECS. Thus whilst these and similar statements stated that there were examples of Emphasised the need
Not reported positive findings, the authors did not demonstrate the extent to which such for additional reporting
The review summarised the findings occurred or the precision of this effect, how effects varied between in future research
Search strategy: studies in a narrative way, with studies and individuals, or why. Note that two of the
ERIC little quantification. authors were the
PsycINFO The authors urged PECS researchers to report more methodological details, original developers of
Science Direct and identified questions to be explored in future research. PECS.
Google scholar. Minimal
search terms used Authors conclusions: Overall quality: Fair
including PECS, Bondy, Analysis of the available research leads to the conclusion that the
Frost (who developed majority of participants who lacked functional comunication skills did
PECS). acquire extensive functional vocabularies.
Analysis of the studies strongly supports the conclusion that by adhering
References of retrieved to the PECS protocol, professionals and parents can teach individuals to
papers were cross successfully initiate exchanges of pictures for tangible and non tangible
checked to identify reinforcers.
additional papers.
Key:
ABI Applied behaviour intervention ABA Applied behaviour analysis
CI Confidence interval DTT Discrete trial training
EIBI Early intensive behavioural intervention Hrs/wk Hours per week
IQ Intelligence quotient MA Meta analysis
PEDro Physiotherapy evidence database PDD NOS Pervasive developmental disorder not otherwise specified
PECS Picture exchange communication system RCT Randomised controlled trial
SCED Single case experimental design SD Standard deviation
SMD Standardised mean difference SR Systematic review
TEACCH Treatment and education of autistic and related communication-handicapped children UCLA University of California, Los Angeles
WMD Weighted mean difference
33

Effectiveness Applied Behaviour AJNHJHBlysis Interventions

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Effectiveness Applied Behaviour AJNHJHBlysis Interventions

Încărcat de

Drepturi de autor:

Formate disponibile

THE EFFECTIVENESS OF APPLIED

Report to the New Zealand Ministry of Health

Please cite this report as:

New Zealand Guidelines Group. The effectiveness of applied behaviour analysis

SYNTHESIS OF RECENTLY PUBLISHED SECONDARY

BACKGROUND AND SCOPE

OBJECTIVES AND RESEARCH QUESTION

social development and relating to others

were non-systematic reviews, correspondence, editorials, expert opinion

STUDY SELECTION AND APPRAISAL

KEY FINDINGS AND CONCLUSIONS

conclusions. More weight/emphasis should be placed on the reviews that are of

Reviews of very good quality

Reviews of good quality

Reviews of fair quality

Picture Exchange Communication System

Single reviewer extracted

Hand search of selected

Contact with experts.

Abstracts scanned by single

S-ar putea să vă placă și