Generalization of RtI

Procedures to Written
Language
Merilee McCurdy, Ph.D.
University of Nebraska - Lincoln
 Description From Program
As schools begin to implement RtI procedures, it is
important to realize that RtI procedures can be
used with all academic areas, including writing
and mathematics. This presentation will review
writing assessment issues such as norming
writing skills, universal screening procedures,
and progress monitoring techniques. In addition,
writing interventions that can be used within RtI
frameworks will be identified for each writing
skills area.
 Research in Reading
To date, much of the literature and research in RtI has been done in the
area of reading.

Why?
Research has shown that those children who become adults with low
levels of literacy are at a disadvantage in a society that has high demands
for effective literacy skills in the work place.
Torgeson, J.K. (2000)

There has not been a significant change in the reading performance of

students from 1992 to 2005 (National Center for Educational Statistics,
2003).

A 1997 study documented that at least 20% of the population in the

United States had reading difficulties (Lyon & Moats, 1997).
 RtI and Reading
Research has focused efforts on understanding
reading deficits and identifying effective reading
programs or interventions.

This concentration of RtI procedures in the area of

reading has been appropriate as the field seeks to
understand and develop initial RtI procedures.

Unfortunately, this intense focus has not been

similar in other academic areas, especially for
written language.
 The Significance of Writing
Writing has become the neglected element of school
reform (National Commission on Writing, 2004)

The Commission asserts that students must struggle

with details, wrestle with facts, and reword raw
information and dimly understood concepts into
language they can communicate to someone else.

In short, students must write if they are to learn.

The National Commission on Writing. (2003). The
neglected R: The need for a writing revolution.
 The Significance of Writing
The National Commission on Writing for Americas
Families, Schools, and Colleges (2004) described writing
as a threshold skill for employment and promotion and
indicated that people who cannot write well are less
likely to be hired, retained, and/or promoted.
National Commission on Writing for Americas Families, Schools, and
Colleges. (2004). Writing: A ticket to workor a ticket out. NY: The
College Board

Writing skills are often needed for demonstrating

learning (e.g., responding to exam items) and
communicating across a variety of employment settings.
 The Significance of Writing:
Percentage of students, by writing achievement level, grades 4, 8, and 12: 1998
and 2002

At or
Below At At At above At or above
Basic Basic Proficient Advanced Basic Proficient
Grade 4

1998 16 61 22 1 84 23

2002 14 58 26 2 86 28
Grade 8

1998 16 58 25 1 84 27

2002 15 54 29 2 85 31
Grade 12

1998 22 57 21 1 78 22

2002 26 51 22 2 74 24
Nebraska Statewide Writing Assessment
All Students
2005 - 2006

Grade
4 8 11
Levels
Nebraska Statewide Writing Assessment
Students in Special Education
2005 - 2006

Grade
4 8 11
Levels
 RtI and Writing
Schools appear to be having difficulty preparing
students to meet societys demands in the area of
writing.
The RtI process can assist with:
Identifying students who are at-risk in the area of writing
Providing or developing intensive interventions in the area of
writing
Monitoring weekly progress of student progress
Using data to make decisions regarding students
needs following a predetermined length of
time
 RtI and Writing
In many ways, RtI procedures developed for the area of
reading can be easily generalized to writing

However, due to the lack of research on written

language, many questions remain.
 Questions
Questions regarding RtI and writing assessment:
1) How much writing time is needed for a valid evaluation
of writing?
2) Is writing performance stable across assessments?
3) How can story organization and story development be
evaluated using CBA techniques?
4) Which variables should be used for initial decision
making?
5) Which variables should be used for ongoing decision
making?
6) Are some assessment variables useful independently or
in a combination with others?
7) What type of writing prompts should be used?
 Questions
Questions regarding RtI and writing interventions:
1) How are Tier 1 services evaluated?
2) What are empirically based interventions for writing?
3) Can interventions be delivered in small groups?
4) How much intervention time is required to observe
changes?
5) Which variables are more reliable and valid for
progress monitoring?
6) What gains should be expected?
7) How can students be motivated to write?
 RtI Writing Team: Lincoln Public
Schools
What we have achieved so far.
Established an RtI Writing Team (2004)

Collected normative data

Grades K-2 (2005-2006)
Grades 3-5 (2006-2007)

Initiated small pilot studies

 RtI Writing Team: Lincoln Public
Schools
Future Plans.
2007-08
Pilot K-5 writing procedures
Small pilots at high school and middle schools to determine
normative data collection procedures
Evaluate research based interventions
Train others to score writing probes (reliably)

2008-2009
Implement RTI writing district-wide at elementary schools
Begin norming at middle schools
others will score if meet criteria for training

2009-2010
?????
 Core Components for Quality RtI
Implementation
School-wide buy-in and implementation plan
Team Leadership
Integration of Services
Implementation Infrastructure
Parent Involvement
Universal Screening and Assessment **
Individual Progress Monitoring **
Planned Service Delivery Decision Rules
Scientifically Supported Instruction **
Intervention Delivery
SLD Verification
 Assessment
Curriculum-Based Measurement (CBM)
Content of the assessments are based on the
instructional curriculum.
Measures are presented in a standardized format.
Material for assessment is controlled for difficulty by
grade levels.
Measures are generally brief.
Shapiro (2004)

Development of Norms
Universal Screening
Progress Monitoring
 General Assessment Procedures for
Writing
The student is given a story starter in the form
of a starting sentence or partial sentence
Provides the student with an idea to write about

The student is asked to think about their story

for one minute and to write for three minutes

At the end of the writing period, the examiner

collects the students writing
 Differences between Reading and
Writing CBM
Reading Writing

Takes one minute Takes four minutes

Administered Can be administered in

individually groups

Must be scored Permanent products

concurrently with allow for later scoring
performance
 Other data to consider
Classroom writing products

Classroom writing assessments/rubrics

Statewide writing assessments

Reading performance

Motivation!
 Development of Norms
LPS Normative Procedures
N=250 per grade; randomly selected by
district
One story starter administered at fall, winter,
spring (different starter each period)
Same starter used with each grade (1st/2nd)
Kindergarten
Letter writing (fall and winter)
Story starter in spring
Used a scoring team to increase reliability
 NORMATIVE VARIABLES BY GRADE
TLW TWW CWS CMIWS %CWS

K Winter X
K Spring X X X X X
1 Fall X X X X
1 Winter X X X X
1 Spring X X X X
2 Fall X X X X
2 Winter X X X X
2 Spring X X X X

TLW Total Letters Written TWW Total Words Written

CWS Correct Writing Sequences CMIWS Correct minus Incorrect Writing Sequences
%CWS Percent Correct Writing Sequences
 Universal Screening
The practice of assessing all students to identify
those who are not making academic or behavioral
progress at expected rates.
National Association of State Directors of Special Education, Inc. (2005)

A classroom-wide, school-wide, or district-wide assessment

used to identify students who are at risk for academic
failure or behavioral difficulties and could potentially
benefit from specific instruction or intervention.

A critical prerequisite to providing early school-based

prevention and intervention services for students at risk
for or with academic, behavioral, or emotional difficulties.
Glover & Albers (2007)
 Universal Screening
Assessments are in place that provide teachers with
information about the progress of all students to
determine which students need closer monitoring,
additional intervention, and/or adjusted learning
opportunities

Norms are necessary to (1) identify students who need

additional supports, (2) develop growth expectations,
and (3) set entrance and exit criteria.

Develop norming procedures based on school resources

 Universal Screening
Scoring: It doesnt take that long!
Requires between 22 and 31 seconds to score TWW
Requires between 25 and 37 seconds to score WSC
Requires between 57 and 151 seconds to score CWS

If all three scores are used, it takes between

1 minutes and 2 minutes to score one
writing sample.

Gansle, Noell, VanDerHeyden, Naquin, & Slider, 2002

Malecki & Jewell, 2003
Difficulties regarding RtI Writing and
Universal Screening
Several components of writing to consider:
Spelling
Grammatical usage
Mechanics of writing
Numerous variables to consider!!

It has not yet been determined which variables:

are most sensitive to change over time
are most valid and reliable for a given grade
level
 Writing Variables
Production-dependent Production independent
Correctly Written Words % Correct Writing Sequences
Total Words Written % Correctly Spelled Words
Correct Writing Sequences Mean Length of Correct Writing
Sequences
Mature Words
Letter Sequences Correct
Total Words Spelled Correctly

A measure of both fluency and accuracy = Correct Minus Incorrect

Writing Sequences (CMIWS), an accurate-production measure
How to set growth expectations?
Low and variable levels of growth on many variables.
From AIMSweb norms:

TWW Fall to Winter Winter to Spring

2nd grade +5 +7
3rd grade +6 +2

CWS
2nd grade +7 +5
3rd grade +7 +5
Data represent growth across entire normative sample
 Progress Monitoring
The practice of assessing students to determine if
academic or behavioral interventions are producing
desired effects.
National Association of State Directors of Special Education, Inc. (2005)

Provides critical information about student progress that is

used to ensure the use of effective educational practices
and to verify that students are progressing at an
adequate rate.

Assessments are in place that provide teachers with

information about which students are benefiting from
intervention and which need additional supports.
 Progress Monitoring Procedures
1. Based upon the norms you have decided to use and
each students screening results, set a goal for each
student.
This goal should reflect an average gain per week as determined
by the normative data.

2. Once the students intervention has begun, monitor the

students progress at least once per week. For writing,
may want to include more than 1 writing sample.

3. Graph the students scores on a chart.

 Progress Monitoring Procedures
4. Review the chart weekly to determine whether progress
is being made.

5. After the student has been in an intervention for a

specified amount of time:
Look at the level and the rate of progress
Determine whether the goal was attained
Plan for next steps

These are general RtI procedures and should be the

same for regardless of academic area!!
 The Complexity of Writing:
Sample Student Data
TWW CWS

25 10 Baseline
Intervention Intervention
20 Baseline 8
6
Words

15

CWS
10 4
5 2

0 0
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Session Session

WSC CMIWS

Intervention 3.5 Intervention

12
3
10 Baseline
Baseline 2.5
8
Words
Words

2
6 1.5
4 1
2 0.5
0 0
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Session Session
Further Complexity of Writing

CMIWS

8
6
4
2
CMIWS

0
-2 1 2 3 4 5 6 7 8
-4
-6
-8
Session
 Advantages of Writing Variables
Although numerous variables add to the
complexity of assessment, they aid in identifying
specific difficulties.

Awareness of specific difficulties further assists

in developing/identifying interventions for the
next phase of intervention (if necessary)
 Scientifically Supported Instruction

Core instruction and interventions have

been validated through scientific research

Must demonstrate that districts have a

process for identifying interventions with
scientific evidence of effectiveness.
 Do Not Forget Tier 1
Tier 1 must be evaluated to examine effectiveness of
curriculum for most students. This can be
accomplished through universal screening.

When teachers use a scientifically supported

curriculum and appropriate instructional practices,
they are preventing many problems from occurring.
Nebraska Department of Education Technical
Assistance Document (2006)
 Interventions in RtI
Basic Message:

Intervention delivery for writing is very similar to

reading procedures.

However

MORE RESEARCH IS NEEDED ON INTERVENTIONS

THAT CAN BE USED IN THE SCHOOLS!!
 What makes a good Writer?
Writing extends beyond mastering grammar, punctuation
and the ability to identify parts of speech.

Writing is best understood as a complex intellectual activity.

It requires students to stretch their minds, sharpen their analytical
capabilities, and make accurate and valid distinctions. (National
Commission on Writing)

Writing is a way for students to demonstrate what they know

AND a way to help them understand what they know.
 Areas for Intervention
Three primary writing stages:
Prewriting
Planning
Story Development
Idea Development

Writing
Story Organization
Grammar
Spelling
Fluency

Post Writing
Editing/Revising
Challenges to improving students
writing
Time for writing in and out of the classroom

Assessment or measurement of progress and

results

Integrating technology into the teaching and

learning of writing

Support for teaching and other classroom issues

The National Commission on Writing. (2003). The Neglected

R: The need for a writing revolution.
 Characteristics of Strong Writing
Instruction
Writing Strategies - involves teaching students strategies for planning,
revising, and editing their compositions
Summarization - involves explicitly and systematically teaching students how
to summarize texts.
Effect Size above .50

Collaborative Writing - uses instructional arrangements in which adolescents

work together to plan, draft, revise, and edit their compositions.
Specific Product Goals - assigns students specific, reachable goals for the
writing they are to complete.
Word Processing - uses computers and word processors as instructional
supports for writing assignments.
Sentence Combining - involves teaching students to construct more complex,
sophisticated sentences.
Prewriting - engages students in activities designed to help them generate or
organize ideas for their composition.
Inquiry Activities - engages students in analyzing immediate, concrete data to
help them develop ideas and content for a particular writing task.
Process Writing Approach - interweaves a number of writing instructional
activities in a workshop environment that stresses extended writing
opportunities, writing for authentic audiences, personalized instruction, and
cycles of writing.
 Tier 2 and 3 Instruction: Identifying
Target Skills
Generate hypotheses based on accuracy or
fluency concern:

Questions to ask when identifying target skill:

Does the student have letter identification and letter
formation skills?
Can the student produce legible text?
Did the student use the planning time?
Does the student produce an adequate amount of
text?
Does the student produce a grammatically correct
story?
Does the student understand story organization?
Could motivation be impacting performance?
 Classwide Fluency Strategies
Classwide and individual performance
feedback strategies are very successful at
increasing student writing production.

Students must practice writing to improve

writing skills!
 Building Effective Interventions
Writing Program composed of:
Direct Instruction
Goal Setting
Writing Practice

Feedback
Individual Feedback
Charting
Rewards

McCurdy, M., Skinner, C. H., Watson, T. S., & Shriver, M. D. (accepted).

Examining the effects of group contingencies on the writing
performance of middle school students with learning disabilities in
writing. School Psychology Quarterly.
 Evaluating Published Programs
Examine for:
Technical Adequacy
Research Support
Grade appropriateness/match to need
Relationship to 6 Traits of Writing
Price
Grade Name Description (from website) Writing Trait Other
K12 Teaching From the basic sentence to the formal Ideas Publisher:
345 Competence essay, Teaching Competence in Pro-ed Inc.
in Written Language is the program to Organization
Written choose to teach writing at any level. www.proedinc
Language- This program works because it Voice .com
Second teaches both basic and advanced
Edition writing skills in an easy to learn, Word Choice Price: $62.00
step-by-step format. Each lesson also
includes social (pragmatic) uses of Convention
writing and proofreading and error
recognition.
This program is different from other
programs in the field because it can
be used either:
As a step-by-step, structured
hierarchy of writing skills, from the
most basic to advanced, or
As a resource to plug into any level
where an individual or group needs
focused work. From the original 44
lessons, the program has been
modified and expanded to a total of
80 lessons for individuals or groups
at different levels of ability.
 Intervention Resources
Strategic Instruction Model (SIM)
Center for Research on Learning
http://www.ku-crl.org/sim/strategies.shtml

National Center on Accelerating Student Learning

http://kc.vanderbilt.edu/casl/index.html

Mnemonics for Planning and Editing

Cognitive Strategy Instruction; Dr. Bob Reid UNL
http://www.unl.edu/csi
 Intervention Delivery: Resources!
Interventions can be delivered in small groups
Behavior management procedures will be necessary
Motivation and engagement is a concern

Consider and evaluate comprehensive reading

programs
These programs may also impact writing performance
May serve as the first supplementary intervention
Must monitor for improvement or lack of improvement
 LPS Goals of Pilot Year (2007-2008)
Identify any modifications to district RtI procedures
Identify appropriate assessment variables for progress
monitoring and decision making
Incorporate district assessment data into decision
making structure
Establish structured decision making criteria for
measuring adequate progress or referral to special
education
Locally validate multiple interventions for use with
writing concerns
Prepare for the future
Norming at upper levels (3, 5, or 10 minute writing times?)
Procedures at upper levels
Interventions for writing skills in the upper grades
 References
Daly, E., Glover, T., McCurdy, M. (2006). Response to intervention: Technical
assistance document. Lincoln, Nebraska: Nebraska Department of
Education.

Gansle, K. A., Noell, G. H., VanDerHeyden, A. M., Naquin, G. M., & Slider, N. J.
(2002). Moving beyond total words written: The reliability, criterion validity,
and time cost of alternative measures for curriculum-based measurement
in writing. School Psychology Review, 31, 477- 497.

Glover, T. A., & Albers, C. A. (2007). Considerations for evaluating universal

screening assessments. Journal of School Psychology, 45, 117-135.

Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve
writing of adolescents in middle and high schools - A report to Carnegie
Corporation of New York. Washington, DC: Alliance for Excellent Education.

Lyon, G.R. & Moats, L.C. (1997). Critical conceptual and methodological
considerations in reading intervention research. Journal of Learning
Disabilities, 6, 578-588.
 References
Malecki, C. K., & Jewell, J. (2003). Developmental, gender, and practical
considerations in scoring curriculum-based measurement writing probes.
Psychology in the Schools, 40, 379-390.

McCurdy, M., Skinner, C. H., Watson, T. S., & Shriver, M. D. (accepted).

Examining the effects of group contingencies on the writing performance of
middle school students with learning disabilities in writing. School
Psychology Quarterly.

National Association of State Directors of Special Education, Inc. (2005).

Response to intervention: Policy considerations and implementation. NASDE.

National Center for Educational Statistics. (2003). NAEP 2002 writing report
card for the nation and the states (Report NCES 1999-462). Washington,
DC: Office of Educational Research and Improvement.

National Commission on Writing for Americas Families, Schools, and Colleges.

(2004). Writing: A ticket to workor a ticket out. NY: The College Entrance
Examination Board
 References
National Commission on Writing in Americas Schools and Colleges.
(2003). The neglected R: The need for a writing revolution. NY:
The College Entrance Examination Board.

Shapiro, E. S. (2004). Academic skills problems: Direct assessment and

intervention (3rd ed.). NY: The Guilford Press.

Torgesen, J.K. (2000). Individual differences in response to early

interventions in reading: The lingering problem of treatment
resisters. Learning Disabilities Research and Practice, 15, 55-64.
 Contact Information

Merilee McCurdy, Ph.D.

Assistant Professor of School Psychology
Licensed Psychologist

University of Nebraska - Lincoln

234 Teachers College Hall
Lincoln, NE 68588-0345

mmccurdy2@unl.edu