TEACHERS AND STUDENTS BELIEFS

ABOUT THE USE OF RUBRICS TO

EVALUATE WRITING
BY CARINA FLORES CALYECA

Purpose
This study based on Markees
theoretical framework explores the
implementation of English writing
rubrics by attempting to
understand professors, and
students different beliefs
regarding this implementation.

Research Questions
RQ1: Who proposed the implementation of
the writing rubrics in this context? Why and
what did they think of the implementation?
RQ 2: What are higher education English
teachers beliefs about the use of rubrics as
tools to assess writing? Do they use them?
When do they use them? Do they use them
to teach writing or only to evaluate? Have
they changed them or adapted them?
RQ 3: What do students report about the
implementation of the writing rubrics?

Theoretical Framework

A) WHO?

F) HOW?

B) WHAT?

C) WHERE?
Cultural,
political,
administrative,
educational and
institutional
subsystems.

Markees
Theoretical
Framework

E) WHY?

D) WHEN?

Cooper,
1982

Rubrics
Descriptive scoring schemes that are
developed by teachers or other evaluators
to guide the analysis of the products or
processes of students efforts (Brookhart,
1999).

Types of rubrics
1. Holistic rubrics
2. Analytic rubrics
3. Primary trait rubrics
4. Multiple trait rubrics

Primary Trait Scoring Rubric for Writing Mechanics

Trait

Sentences
and
paragraph
s

Word
choice

Spelling

Punctuatio
n

Exceptional

Usage of

sophisticated
sentence patterns.
Paragraphs indicate
shift in thought and
are used to make
sequence of events
clear.

Words are used

correctly and
precisely.

Spelling is correct,
including complex
and irregular words.
A range of
punctuation
including commas,
apostrophes,
colons and
semicolons is used
accurately and
effectively.

Acceptable

Simple and some

complex sentences
are used.

Some paragraphing
to show sequence
of events/ideas.

Acceptable
vocabulary.
Words are
technologically
appropriate.

Spelling is generally
accurate.
Periods and
capitals are used
correctly and
punctuation is
beginning to be
used within the
sentence.

Amateur

Unsatisfactory

Sentence structure is
usually correct.
Simple sentences are
used.
Little attempt made to
paragraph writing.

Simple vocabulary.

Incorrect
vocabulary.

Frequent spelling errors.

Spelling errors
interfere with
understanding.

Frequent punctuation
errors.

Insufficient or
lacks punctuation.
Incorrect use of
capital letters.

Sentences do not
make sense.
No paragraphing.

Letter-writing : Advising a friend about work

Category

Salutation and Salutation and

closing have no
closing
errors in
capitalization and
punctuation.

3
Salutation and
closing have 1-2
errors in
capitalization and
punctuation.

Salutation and
closing have 3 or
more errors in
capitalization and
punctuation.

Salutation and/or
closing are
missing.

Complies with less

than 75% of the
requirements for a
friendly letter.

Format

Complies with all

Complies with
the requirements
almost all the
for a friendly letter. requirements for a
friendly letter.

Complies with
several of the
requirements for a
friendly letter.

Content and
accuracy

The letter contains The letter contains

the 3 accurate
2 accurate facts
facts about the
about the topic.
topic.

The letter contains The letter contains

1 accurate facts
no accurate facts
about the topic.
about the topic
.

Grammar and
Spelling
(conventions)

Writer makes no
errors in grammar
or spelling

Writer makes 3-4

errors in grammar
and/or spelling

Writer makes 1-2

errors in grammar
and/or spelling.

Writer makes
more than 4 errors
in grammar and/or
spelling

Advantages of the use of the rubrics

1. Timely feedback (Steven and Levi, 2005)
2. Detailed feedback: strenghts and weaknesses
(Steven and Levi, 2005)

3. Encourage critical thinking

(Goodrich, 2000 & Steven and Levi, 2005)

4. Self and peer evaluation

(OMalley & Valdez, 1998; Saddler & Andrade, 2004)

5. Teachers expectations
(Stevens and Levi, 2005; INTEL 2012)

6. Refine teaching skills

(UNSW, 2014)

7. Objective assessment
(Goodrich, 2010)

Disadvantages of the use of the rubrics

1. Task-specific evaluative criteria (Pophman, 1997)
2. General evaluative criteria (Pophman, 1997)
3. The excessive lenght in a rubric
4. Rubric administration

(Pophman, 1997)

(Kohn, 2006)

5. Rubrics are not self-explanatory.

(Goodrich, 2010)

Method
Case study based on a mix methods
exploratory sequential desing (Cresswell,
2007).
This type of approach focuses first on
collecting qualitative data and then
collecting quantitative data in order to test
or generalize the initial findings (Creswell,
2011)

Context of Research: A public University in

Central Mexico. English was established as a
mandatory course as part of curricula.
Participants: Rubric designers, teachers, and
students.
Instruments: Semi-structured interviews and
semi-structured questionnaires.
Data collection and analysis: Content analysis
principles. (Cohen,Manion, & Morrison, 2000)

Results
1. The implementation of rubrics to assess writing
skills: Markees Theoretical Framework.
2. Designers rationale underlying rubrics
implementation.
3. Teachers beliefs about the use of the rubrics
including what they think about the A2 language
level required to students.
4. Students beliefs about the use of the writing
rubrics summarizing strengths and limitations
they reported.

A) WHO?

F) HOW?

B) WHAT?

Markees
Theoretical
Framework

E) WHY?

D) WHEN?

C)
WHERE?

2. Designers rationale underlying rubrics

implementation.
First, to standardize evaluation since every teacher
had their own grading system and some of them
were described as flexible and some others as
too strict;
Second, to be fair with students work and give
effectively feedback to students because as
Stevens and Levi (2005) claim that rubrics are
useful tools to give effectively feedback if it is
given immediately.
And third, evaluation based on competences should
be carried out through rubrics (OMalley &
Valdez,1998).

3. Teachers perspectives about

the use of the rubrics:
Purposes
Strengths
Weaknesses

8
6
4

3
2

2
0

grade

6
4
2
0

evaluate inform studentsmandatory

5
3

grade

evaluate

useful

sometimes, 14 T

6
5
4
3

2
1
0

time consuming

low level

adapt

Chart Title

25

22

Strenghts

20

15

12

11

10

14

13

Weaknesses

12
10

8
6
4

2
0
lack validity
time consuming
ambiguous
training to use them

4. Students perceptions about the rubrics:

advantages and disadvantages
#

when they used the

rubrics

Before and after writing

Advantages

Disadvantages

A guide to know what to write

Writing style
limited
They contain a
lot of academic
vocabulary

A guide to know what to write

Required by the teacher,

before writing

To be informed about aspects

to be evaluated

To be informed about aspects

to be evaluated

Lack of use

A guide to know what to write

Lack of use

To be informed about aspects

to be evaluated

Before writing

A rare use: self-learning

Before writing and I

understand what to do

Before writing

General conclusions
There is little experience with rubrics in this
university.
2. Center-periphery and Problem-solving
models for a top down decision made by
the coordinators and a group of teachers to
standardize assessment , give feedback
and evaluate competences.
3. Third, more than 50% of the teachers, in this
study, adopted the rubrics mainly to grade
and evaluate.
1.

General conclusions
4. From 40 to 45% of the teachers did
not adopt the rubrics mainly because
they considered they were time
consuming, confusing and
inadequate for the context.
5. Students reported this was the first
experience with rubrics and identified
more strengths than weaknesses.

Implications
Raise teachers awareness about
rubrics not only as assessment tools
but also as learning and teaching tools
for self and peer evaluation.

Directions for further research

First, to research on how teachers who did
not use the rubrics usually evaluate their
students writing tasks.
Second, to continue exploring teachers
best practices on the use of the rubrics in
this context.

Thank you!

adopters

resisters

suppliers

ACTORS
implementers

Change
agents
clients

Framewor
k

Types of Social Change

Origin of the new idea
Recognition of the
need for change

Internal to the social

system

External to the social

system

Internal
Recognition is by
members of the social
system

I. Immanent change

II. Selective contact

change

External
Recognition may be by
change agents outside
the social system

III. Induced immanent

change

IV. Directed contact

change

Based on Rogers (1962) and Shoemakers (1971)

work,
Framewor
k

S-shaped diffusion
curve
based on Cooper
(1982)

Framewor
k

Psychological profiles

Early-adopters

Laggards

Framewor
k

Five different approaches to

effecting change
1. The social interaction model.
2. Center-periphery model.
3. Research, development, and
diffusion model.
4. Problem-solving model.
5. The linkage model.
Framewor
k

ACTORS
adopters
suppliers

implementers

Coordinators who identified

the need to implement the
rubrics
The coordinator and
assistants
Coordinators and teachers
who designed the rubrics

clients

Teachers and students.

Change
agents

Coordinators and teachers

who managed the innovation.

resisters

Teachers and students who

did not use the rubrics.
Framework

Immanent change
Coordinator: at that time, we were concerned about
fairness. Some teachers were very flexibleothers were
being too strict and hard on students.
Selective contact change
Coordinator: we have to do it by rubricsespecially if
you are working beyond abilities, beyond attitudes and
values, the very famous competences, you have to do it.
Framewor
k

This study has been carried out in the English Language

Department of a large public university in the center of Mexico
where English language is taught as a mandatory course for
undergraduate students and who at the end of the last and
forth course are expected to achieve an A2 language level
based on the Common European Framework.
There are 75 teachers who belong to this English Language
Teaching department and who teach at different schools and
different timetables every quarter and; therefore, they do not
meet frequently.
Framewor
k

The implementation of the rubrics to assess

writing at this public university was carried out
in two phases.
1. In 2008 coordinators provided teachers with
an analytic rubric to evaluate different types
of writing tasks.
2. In 2011 two multiple-trait rubrics to assess
writing tasks were implemented.
Framewor
k

Teachers degree

1; 2%
17; 38%
27; 60%

MA
BA
PhD

Framewor
k

Teachers who adopted the rubric: adopters

12; 41%
17; 59%

MA
BA

Teachers who sometimes resisted: laggards

1; 6%

6; 38%

9; 56%

MA
BA
PhD

Framewor
k

Center-periphery & Problem-solving

models.
Coordinator: we were concerned about the
number of teachers we had at that time. We were
thinking that some teachers were very flexible in
their grading systems and that other teachers
were were being too strict and too hard on
students and none of the two extremes was fair.
Framework
Framewor