education

sciences
Article
Insights Chinese Primary Mathematics Teachers
Gained into their Students’ Learning from Using
Classroom Assessment Techniques
Xiaoyan Zhao 1,2, * , Marja van den Heuvel-Panhuizen 1,3,4, * and Michiel Veldhuis 3,5, *
1 Freudenthal Institute, Faculty of Science, Utrecht University, 3584 CC Utrecht, The Netherlands
2 School of Teacher Education, Nanjing Normal University, Nanjing 210023, China
3 Freudenthal Group, Faculty of Social and Behavioural Sciences, Utrecht University, 3584 CS Utrecht,
The Netherlands
4 Faculty of Education, Art and Culture, Nord University, 8049 Bodø, Norway
5 iPabo University of Applied Sciences, 1061 AD Amsterdam, The Netherlands
* Correspondence: 69005@njnu.edu.cn (X.Z.); m.vandenheuvel-panhuizen@uu.nl (M.v.d.H.-P.);
m.veldhuis@uu.nl (M.V.)




Received: 13 March 2019; Accepted: 11 June 2019; Published: 18 June 2019


Abstract: In this study, we explored the insights that Chinese primary mathematics teachers gained
into their students’ mathematical understanding from using classroom assessment techniques
(CATs). CATs are short teacher-initiated targeted assessment activities proximate to the textbook,
which teachers can use in their daily practice to make informed instructional decisions. Twenty-five
third-grade teachers participated in a two-week program of implementing eight CATs focusing on the
multiplication of two-digit numbers, and filled in feedback forms after using the CATs. When their
responses described specific information about their students, emphasized the novelty of the gained
information, or referred to a fitting instructional adaptation, and these reactions went together
with references to the mathematics content of the CATs, the teachers’ responses were considered
as evidence of gained insights into their students’ mathematics understanding. This was the case
for three-quarters of the teachers, but the number of gained insights differed. Five teachers gained
insights from five or more CATs, while 14 teachers did so only from three or fewer CATs, and six
teachers showed no clear evidence of new insights at all. Despite the differences in levels of gained
insights, all the teachers paid more attention to descriptions of students’ performance than to possible
instructional adaptations.

Keywords: classroom assessment techniques; mathematics education; multiplication; primary

school; China

1. Introduction

1.1. Assessment in the Hands of Teachers

Any instructional decision making—and thus any form of teaching—requires in one way or another
information about students’ learning [1]. The more reliable and valid this information is, the better
teachers can find a foothold for these decisions. For generating such information, many approaches
are possible, ranging from standardized externally developed tests to teacher-made assignments.
Contrary to the low reliability that has been attributed to teachers’ judgments of students’ performance
in the past (see, for example, Parkes [2]), nowadays, teacher assessment that is aimed at gaining
insights into their students’ progress is highly valued and seen as crucial for adapting the teaching
to the students’ abilities and needs. Teachers are also seen as being in a good position for collecting

Educ. Sci. 2019, 9, 150; doi:10.3390/educsci9020150 www.mdpi.com/journal/education

Educ. Sci. 2019, 9, 150 2 of 19

information about their students’ learning [3]. Teacher-led assessment activities that are interwoven
with instruction and fully integrated in the teachers’ daily teaching practice, such as questioning,
observing students, and giving quizzes or teacher-made written assignments, can provide insights
about students’ thinking and about what productive and actionable next instructional steps might
be taken [4]. When the assessment focuses on figuring out what students know, or what difficulties
students have, for the purpose of making decisions about further instruction, it is considered as
formative assessment. Formative assessment in which the teacher has the lead is often referred to as
classroom assessment [5–10].
What information can be collected through classroom assessment depends largely on what
assessment activities are conducted. Helpful assessment activities are those which offer teachers
a window into the students’ thinking to uncover their mathematical conceptions and skills [10].
Therefore, much attention has been paid to gaining knowledge about how mathematics teachers can
improve their assessment activities to acquire adequate information about their students’ development
(see, for example, Schoenfeld [11]). Research has shown that using various oral questioning strategies
and written tasks, and then analyzing students’ responses, offers mathematics teachers opportunities
to reveal their students’ understanding [11,12]. In particular, challenging students with open-ended
problems enables teachers to diagnose students’ understanding and reveal their methods of problem
solving [12,13]. Other measures that make assessments by teachers more informative are using
rubrics [14] or concept maps [15] as frameworks for analyzing students’ responses. Both measures
were found to assist teachers in identifying gaps in their students’ understanding of the particular
mathematical topics under investigation.

1.2. Assessment Techniques

Assessment techniques are assessment activities by which mathematics teachers can gauge what
their students do and do not know, so that they can adjust their teaching to their students’ needs.
These assessment techniques can be characterized as short, feasible, and for teachers, often well-known
activities, which are fully embedded in teachers’ teaching practice [16]. Also, several other researchers
and educators [17–21] have investigated such assessment techniques.
Wiliam et al. [16,20] investigated a large number of assessment techniques that are used to support
primary and secondary teachers’ formative assessment practice with the goal of making instructional
decisions either for direct use or for later teaching. Not surprisingly, again, asking questions turned
out to be very helpful. However, the fact that teachers distinguished different types of questions
for different moments in a lesson was a new approach to questioning students. At the beginning of
a teaching sequence, range-finding questions were used to find out students’ previous knowledge
(see, for example, “how many fractions can you find between 1/6 and 1/7?” [16] (p. 21)). During a
lesson hinge point, questions were used to indicate the direction of the remainder of the lesson (see,
for example, six polygons were shown, and the students were asked to indicate how many lines of
symmetry each polygon has [20] (p. 103)). Finally, exit pass questions, which are asked before students
were leaving the classroom, were meant to make decisions about the next lesson. Furthermore, to allow
all the students to answer at the same time, Wiliam et al. suggested the use of ABCD cards and mini
whiteboards [20]. Then, when a question was asked, all the students could show their answers by
holding up a card or writing their answer on the whiteboard.
Similar assessment techniques were also discussed by Keeley and Tobey [18], who considered these
techniques as useful to give insight into students’ factual, conceptual, and procedural mathematical
understanding for a broad range of mathematics teachers from Kindergarten to Grade 12. In Andersson
and Palm’s [17] study, it came to the fore that the primary mathematics teachers involved paid the
most attention to those assessment techniques that helped them best to collect information about their
students’ knowledge and skills. A similar finding was revealed by Wylie and Lyon [21], who conducted
research with mathematics and science teachers in high school and found that the most used assessment
techniques were asking questions, organizing classroom discussions, and using written tasks.
Educ. Sci. 2019, 9, 150 3 of 19

Characteristic of the aforementioned studies on assessment techniques is that the techniques are
general in nature. They can be applied to any subject and to any mathematical topic. When teachers
are provided with such examples of assessment techniques, it can happen, as was found by James and
McCormick [22] (p. 976), that some teachers understand the “spirit” behind the assessment techniques,
and thus are able to adapt them to their teaching, but that others just catch the “letter” of them and
carry them out in a ritualized and mechanistic way. The latter may be the result of providing teachers
with assessment techniques that are not directly related to the content the teachers are teaching at that
moment. To avoid this, and to have assessment techniques that can generate indications for further
instruction, the techniques should be content-dependent.
A study in which this content-dependent approach was adopted is that of Phelan et al. [23].
The aim of their study was supporting teachers to assess students’ learning in pre-algebra. To find out
what had to be assessed, an expert panel was organized to map algebra knowledge and its prerequisites.
Such a map was used to design the questions and tasks that could provide teachers with the necessary
information. This innovative content-dependent approach to assessment, which differs from providing
teachers with general assessment guidelines, turned out to be rather successful, and apparently had a
positive impact on students’ learning [23].
To make the assessment even closer to the teaching at hand, Veldhuis and Van den
Heuvel-Panhuizen [19] took the textbook used by the teachers as a starting point. They designed brief
and targeted activities, called classroom assessment techniques (CATs), that teachers could use in their
daily practice to reveal information about students’ learning of a particular mathematical concept or
skill. The ultimate goal of CATs is providing teachers with deep insights into students’ mathematical
thinking to make adequate instructional decisions. This requires a skillful way of questioning [24,25],
or, in the words of Heritage and Heritage [26] (p. 187), “questioning lies at the epicenter of formative
assessment.” For the CATs, this implies that they were designed to serve as an eye-opener for teachers
to acquire knowledge about their students’ learning that they did not have before. This goes beyond
knowing whether students are able to flawlessly carry out particular calculation procedures. Instead,
the CATs are intended to delve deeper and reveal whether and how students understand the underlying
concepts of problems and see the relationships between problems, and to what extent they are flexible
in solving problems. Therefore, rather than just repeating the tasks in the textbook, CATs present
the content to be assessed from a different perspective and in an unfamiliar way. In addition to the
content-dependency, what is innovative of the CATs is in particular that teachers are offered a new
perspective for looking at students’ understanding. This makes CATs different from the usual ways of
assessing students, but at the same time, these new activities are close to the known daily teaching
practice. Moreover, to make the CATs manageable for the teachers, they have a format that supports
teachers to gather the students’ information efficiently and makes the assessment feasible to carry out.
The two main formats that Veldhuis and Van den Heuvel-Panhuizen [19] used for their CATs were
red/green cards and worksheets. With the students responding to a question by holding up a red or a
green card, the teacher can quickly gather information about the group as a whole. The worksheets,
mostly containing a few problems on a specific mathematical concept or skill, are meant to provide
teachers with more information on individual students’ strategy use.

1.3. A New Approach to Assessment in Mathematics Education in China

Mathematics education in China has a deeply-rooted examination culture [27]. External
examinations at the school level and teacher-made end-of-chapter tests at the classroom level used
to be the main aspect of teachers’ assessment activity [28]. In 2001, the Ministry of Education [29]
formally launched a new approach to assessment aimed at improving teaching and learning. Since
then, mathematics teachers are encouraged to get a comprehensive understanding of students’ learning
by employing various approaches: for example, written tests, oral tests, open questions, activity
reports, observations, interviews, exercises in and after class, and portfolios [30,31]. However, Cai and
Wang [32] found that Chinese mathematics teachers in primary education put much more emphasis on
Educ. Sci. 2019, 9, 150 4 of 19

providing information to students than on getting information from them. Furthermore, when taking
action to understand students’ thinking, teachers are more likely to do so before lessons than during
or after lessons [33,34]. Moreover, when teachers plan their teaching, textbooks serve as the main
source, rather than findings from assessing their students’ learning [32,35]. Also, the exercises in
textbooks have an important role in the decisions that teachers make about assessment activities. Yet,
such exercises may be more suitable for summative assessment than for classroom assessment [36].
According to Liu [36], this may lead to teachers focusing on assessing the result of learning, namely
what basic knowledge and skills students have acquired, instead of assessing how students developed
their mathematical thinking during the learning process. Furthermore, studies have revealed that
only very limited attention has been paid to improving teachers’ assessment practice to get more
information about students’ learning [37,38].
Taking into account the promising international findings about the use of assessment techniques,
we explored whether this approach to assessment could assist Chinese primary mathematics teachers
in their assessment practice. Specifically, as a sequel to the studies carried out in the Netherlands [19]
in which classroom assessment techniques (CATs) for primary mathematics education were developed
and teachers were supported in using CATs, we investigated the use of CATs in China. Six third-grade
mathematics teachers of two primary schools in Nanjing, China, participated in a pilot study [39].
The focus of this pilot study was on assessing the topic of division, in particular three-digit numbers
divided by a one-digit number. In line with the Dutch studies [19], the CATs in the Chinese pilot
study were also based on a textbook analysis and formulated in such a way that they were not just
a repetition of what is in the textbook. In this way, CATs might give teachers access to a deeper
level of students’ skills and understanding. It was found that teachers recognized that it can be very
revealing to challenge students with questions that are not completely prepared by the textbook. Also,
they appreciated the use of red/green cards for providing quick information. In general, teachers were
positive about the CATs as a way to reveal their students’ learning in an effective and efficient way.

1.4. The Present Study

Based on these experiences, we set up a study to investigate whether this positive finding holds
for a larger group of teachers, and for a different mathematical topic. In particular, we wanted to
explore the insights that Chinese primary school mathematics teachers may gain from carrying out
CATs. Precisely, our focus was on whether the teachers, through using CATs, could acquire knowledge
about their students’ learning that they did not have before. The research question of the present study
was: What new insights can Chinese primary mathematics teachers gain about their students’ understanding of
mathematics from using CATs?
Since we already had experiences with the Chinese mathematics curriculum in Grade 3 through
the pilot study, we chose to do the present study in this grade, too. However, to extend our knowledge
about the use of CATs in Chinese mathematics classrooms, we changed the topic of investigation.
We stayed in the domain of number and operations, but instead of on division, the focus in the present
study was on multiplication, in particular on what insights teachers can gain from using CATs to assess
students’ understanding of the multiplication of two-digit numbers.

2. Methods
In the study, Chinese teachers were asked to use a number of CATs in their regular teaching of
multiplication during the first two weeks of the second semester of Grade 3. Teachers were informed
about the CATs through a teacher guide and two researcher-led meetings. Data on how teachers
used the CATs and what insights they got from them were gathered through feedback forms and a
teacher-written final report
 Educ. Sci. 2019, 9, 150 5 of 19

2.1. Participants
For practical reasons, we decided to set up the study in Nanjing. We contacted three local
teaching research offices, which are responsible for inspecting the educational quality of the schools
andSci.
Educ. providing professional
2019, 9, x FOR PEER REVIEW development to primary school teachers in their administrative districts. 5 of 21
One of these offices volunteered to participate. To include various schools in terms of the school’s
reputation,
selected educational
by this quality,research
local teaching and location,
office.nine
Theout of 403 primary
Grade schools
mathematics in its district
teachers were
and their selected
students
ofbythese
this local
nineteaching
schools research
took part office.
in theThe GradeThus,
study. 3 mathematics
our sample teachers and their
consisted of 25students
teachersofand
thesetheir
nine
schools took
students in 25part in theInstudy.
classes. all theThus, ourthe
classes, sample
sameconsisted
textbookofseries
25 teachers and their
was used, namely students in 25 classes.
the Sujiaoban (苏
In all the classes,
教版) textbook [40]. the same textbook series was used, namely the Sujiaoban (苏教版) textbook [40].

2.2.Multiplication
2.2. MultiplicationofofTwo-Digit
Two-DigitNumbers
Numbers
Fordeveloping
For developing thethe CATs,
CATs, wewe firstfirst investigated
investigated when whenand and
howhow the topic
the topic of multiplication
of multiplication of two-of
two-digit
digit numbers numbers was addressed
was addressed the Sujiaoban
in theinSujiaoban textbook.
textbook. We found
We found that that
this this
topictopic
was wasdealtdealt
withwith
in
in the first chapter of the book meant for the second semester of
the first chapter of the book meant for the second semester of Grade 3. This chapter covers nine Grade 3. This chapter covers nine
lessonstaught
lessons taught over
over around
around two two weeks,
weeks,consisting
consistingofofso-called
so-called new new lessons
lessons andand revision
revisionlessons. A new
lessons. A
lesson mostly starts with a new type of problem, which is presented
new lesson mostly starts with a new type of problem, which is presented as a context problem, as a context problem, followed by
the corresponding bare number problem. Then, examples are given
followed by the corresponding bare number problem. Then, examples are given of how to solve this of how to solve this problem
type, and
problem finally,
type, andexercises are offered
finally, exercises areto practice
offered to this.
practiceA revision lesson generally
this. A revision includesincludes
lesson generally exercises
for rehearsing
exercises and discussing
for rehearsing what thewhat
and discussing students have learned
the students in earlier
have learned in lessons. The main
earlier lessons. Thecontent
main
components addressed in this chapter include, among others, multiplication
content components addressed in this chapter include, among others, multiplication with multiples with multiples of 10,
ofthe
10,structure of theofmultiplication
the structure the multiplication algorithm, and the
algorithm, andratio aspect
the ratio of multiplication.
aspect of multiplication.
Multiplication with multiples of 10 is presented in a new lessonstarts
Multiplication with multiples of 10 is presented in a new lesson and and withstartsa with
context problem
a context
in which Uncle Li is sending 10 boxes of bell peppers, with
problem in which Uncle Li is sending 10 boxes of bell peppers, with 12 peppers in each box. 12 peppers in each box. This context
This
problem is followed by the corresponding bare number problem.
context problem is followed by the corresponding bare number problem. The students need to find The students need to find out how
out how many peppers are sent in total. The textbook shows that one method of solving 12 × 10 is use
many peppers are sent in total. The textbook shows that one method of solving 12 × 10 is to make to
of 12use
make × 1.ofBy 12 seeing both multiplications
× 1. By seeing both multiplications with their
with answers,
their answers,the students
the studentsbecome acquainted
become acquainted with
the strategy
with the strategyof using an analogous
of using an analogous problem,
problem,thatthat
is, using a problem
is, using a problem whose whoseanswer is known
answer is known or easy
or
to calculate to find the result of an analogous problem. After this,
easy to calculate to find the result of an analogous problem. After this, the textbook provides threethe textbook provides three sets
of exercises
sets of exercises of multiplications
of multiplications withwithmultiples
multiples of 10:
of 10: × 1× =,
16 16 1 =,1616× ×1010=;=;70 70××66=, 70 ×
=, 70 60 =;
× 60 =;55××40 40=,=,
50 × 40
50 × 40 =. =.
InInthethenext
next
new new lesson,
lesson, the the structure
structure of theofmultiplication
the multiplication algorithmalgorithm is introduced.
is introduced. Here, specialHere,
special attention
attention is paid toishow
paidthe to result
how the of result of the multiple-of-10
the multiple-of-10 part of the part of the multiplication
multiplication is notated,isnamely
notated,
namely without writing down a zero and leaving the one-position
without writing down a zero and leaving the one-position empty (Figure 1). This structural empty (Figure 1). This structural
understandingisisfurther
understanding furthersupported
supportedby byexercises
exercisesininwhich
whichthe thestudents
studentsare areprovided
providedwith withan anempty
empty
structureofofthe
structure themultiplication
multiplicationalgorithm
algorithmthat thatthey
theyhave
havetotofillfillinin(Figure
(Figure2). 2).InInaddition,
addition,the thestudents
students
have to explain what they need to calculate
have to explain what they need to calculate in each step. in each step.

2 4 2 4 2 4
× 1 2 × 1 2 × 1 2
The amount
4 8 in 2 boxes 4 8 The amount
4 8
2 4 0 in 10 boxes 2 4 0
The amount
2 8 8 in 12 boxes

Structureofofthe
Figure1.1.Structure
Figure themultiplication
multiplicationalgorithm.
algorithm.

2 2 3 2 4 3
× 2 3 × 3 1 × 2 1

Figure 2. Exercises with empty multiplication structure.

In the subsequent new lesson, the students are prompted to further strengthen their
understanding of the structure of the multiplication algorithm. To achieve this, the textbook offers
 × 1 2 × 1 2 × 1 2
The amount
4 8 in 2 boxes 4 8 The amount
4 8
2 4 0 in 10 boxes 2 4 0
The amount
2 8 8 in 12 boxes
Educ. Sci. 2019, 9, 150 6 of 19
Figure 1. Structure of the multiplication algorithm.

2 2 3 2 4 3
× 2 3 × 3 1 × 2 1

Figure
Figure 2.
2. Exercises
Exercises with
with empty
empty multiplication
multiplication structure.
structure.

In the
In thesubsequent
subsequent newnew lesson, the students
lesson, are prompted
the students to further strengthen
are prompted to furthertheir understanding
strengthen their
of the structure of the multiplication
understanding structure of the algorithm. To achieve
multiplication this, the
algorithm. Totextbook
achieveoffers onlytextbook
this, the the start offers
of the
algorithm
only for 24
the start × 53algorithm
of the (Figure 3).for The 24students
× 53 (Figurehave3).
to The
complete
studentsthe remaining steps ofthe
have to complete theremaining
algorithm.
Educ. Sci. 2019, Right
steps
Educ.
9,
Educ. Sci. 2019, 9, x FOR after
of
Sci.
x FOR
Educ.
the
2019,
PEER
Sci.PEER
this,
9, x
REVIEW the
algorithm.
FOR
2019, REVIEW
textbook
PEER Right
9, x FOR PEER REVIEWREVIEW provides
after this,a description
the textbook in words
provides of
a the steps
description to be
in taken
words
6 of when
of
21
6 of 21 the carrying
steps
66 of to
of 21
21
out the multiplication algorithm of two-digit numbers. The students
be taken when carrying out the multiplication algorithm of two-digit numbers. The students are told are told to first choose the digit
to
to first
first choose
choose tothe
in
to thedigit
the
first
firstdigit
ones
choosein
choose the
theones
inplace
the
the ones
of
digit
digit place
place
the in of
inlower
the
the the
ofones
the lower
lower
number
ones placeto
place number
number
ofmultiply
of the to
to multiply
the lower
lower multiply
the upper
number
number the
the upper
upper number,
tonumber,
to multiply
multiply number,
and
thethen
the and
and
upper
upper then
then
donumber,
the same
number, and
andforthen
the
then
do
do the
the same
samedo for
for
digit the
thethe
in digit
digit
the
same tens in
in
for the
place
the tens
tens
of
digit theplace
place
lower
in the of
of the
the
number.
tens lower
lower
place After
of number.
number.
this,
the for
lower After
After
every this,
this,
number. for
for
calculated
do the same for the digit in the tens place of the lower number. After this, for every calculated every
every
After calculated
calculated
product,
this, for they
every have to write
calculated
product,
product, theytheyproduct,
thehavelastto
have
product, they
write
todigit
write
they have
of the
the
have to
thelast
last write
digitthe
digit
product
to write of
in
the last
of the
the digit
lastproduct
product
same of the
of in
in
digitcolumn product
thethe
the same
same
as
product in
in the
the column
column
digit same
as
thechosen
same column
as the
the digit
digitthe
from
column as the
the digit
chosen
chosen
aslower from
from chosen
number.
digit chosen from
Finally,
from
the
the lower
lower number.
number.
the
the Finally,
Finally,
students
lower the
the
need
number. students
students
to add
Finally, theneed
need
thetwo to
to add
add
products.
students the
the
need two
two
the lower number. Finally, the students need to add the two products.to products.
products.
add the two products.

2 4 22 44
× 5 3 ×× 55 33
7 2 77 22
Figure
Figure 3.
3. Multiplication
Multiplication
Figure
Figure 3.
Figure 3. algorithm
algorithm for
Multiplication
Multiplication which the
for algorithm
3. Multiplication which
algorithm
algorithm start
the for
start is
is given.
for which
for which
whichgiven.
the
the start
the start is
start is given.
is given.
given.

The
The ratio
ratio aspect
aspectThe
Theof
The ratio
of aspect
multiplication
multiplication
ratio
ratio aspect
aspect of of multiplication
is
is dealt
dealt with
of multiplication
multiplication withis is
in dealt
revision
indealt
is revision
dealt withlessons
with
with in revision
lessons
in
in revision
halfway
halfway
revision lessons
lessonsand
lessons halfway
at
at the
andhalfwaythe end
halfway end and
andof
andofatat
the
the
at the end
the
the end of
end of the
of the
the
chapter.
chapter. In chapter.
In the
the first
first In
problemthe
problem
chapter. In the first
first problem
problem
offered
offered to
to theoffered
theoffered to
students,
students,to thethe students,
they
theystudents,
have
have they
to
to they
find
findhave
have
out
out tohow
the first problem offered to the students, they have to find out how many pencils there find
to
how out
find
many
many how
out many
how
pencils
pencils many pencils
there
there pencilsthere are
there
are
are in
in 10,
10, 20, in
40,
20,are
40,
are 10,
inand
and
in 20,
10,80
10, 40,
80
20,
20, and
boxes
boxes
40, 80
40, and boxes
when
when
and 80 inwhen
in
80 boxes one
boxesonewhen in one
box,
box,
when there
there
in box,
in one
one are
arethere
10 are 10
pencils.
10 there
box,
box, pencils.
there are
arepencils.
The
The
10 problemThe problem
problem
10 pencils.
pencils. is
is presented
The
The is presented
presented
problem
problem in
is aa
inpresented
is in a ratio
presented in
in aa
ratio
ratio table table
table (Figure
(Figure
ratio (Figure
4).
4).
table The
The 4). The
first
first
(Figure first
column
column
4). The column
shows
shows
first shows
that
that
column in
inthat
five
showsfivein five
boxes,
boxes,
that boxes,
in there
there
five there
ratio table (Figure 4). The first column shows that in five boxes, there are 50 pencils in total. The are
are
boxes, are
50
50 50 pencils
pencils
pencils
there are in
in50 in
total.total.
total. The
pencilsThe The
in students
total. The
students
students have
havehave
students
to
to fill
students to fill
fill have
in
in in the
the
the
have to remaining
fill
remainingin
in the
remaining
to fill theempty
emptyempty
remaining
cells.
remaining cells.
cells. empty
In
In the
empty In the
the cells.
end,
end, end,
cells. In
they
they they
In the
thehave
have have
end,
end, tothey
to to explain
have
explain
havewhat
explain
they towhat
to
what explain
they
they
explainthey what
can
can can they
what discover
they can
can
discover
discover from
fromfrom
the
the
discover the
ratio
ratio ratio
table.
from table.
table.
the The
The The
ratiofocus
focus focus
table. in
in in
this
this
The this
problem
problem
focus problem
in is
is
this on
onis on
the
the
problem the external
external
external
is on
discover from the ratio table. The focus in this problem is on the external ratio between the number ratio
ratio
the ratio between
between
between
external the
the
ratio the number
number
number
between the of boxes
number
of
of boxes
boxes and
andofand
of
of of pencils,
pencils,
ofboxes
pencils,
boxes and
and or
orofinor
in
of in other
other
other
pencils,
pencils, words,
words,
or words,
or in on
on the
in other
other on
the the functional
functional
functional
words,
words, on
on the
the relationship
relationship
relationship
functional
functional between
between between
relationship
relationship them.
them. them.
This
This is
between
between This
is even
even
them.
them. is This
even
This isclearer
is even
even
clearer
clearer in
in the in the
clearer
next next
in ratio-table-like
the
next ratio-table-like
theclearer next
ratio-table-like problemproblem
ratio-table-like
problem (Figure
in the next ratio-table-like (Figure(Figure
problem
problem 5),
5), in 5), in
(Figure
which
in(Figure which
which 5), 5),
the
theininthe students
which
students
students
which the the
are are explicitly
students
explicitly
are students are
explicitlyare asked
askedasked
explicitly
to
to
explicitly to multiply
asked
asked to to
multiply
multiply two two
given
twomultiply
given given numbers.
numbers.
numbers.
two given Also,
Also,Also,
this
this
numbers. this functional
functional
functional
Also, this relationship
relationship
relationship
functional is
is is emphasized
emphasized
emphasized
relationship
multiply two given numbers. Also, this functional relationship is emphasized by the notation of the is by
by by
the
the the
emphasized notation
notation
notationby of
of
the of
the
the the
notationfollowing
of the
following following
following accompanying
following accompanying
accompanying
accompanying exercises exercises
exercises
exercises
accompanying exercises and
and and
and ,,,inin which
inwhich
whichthe,,the
in
the whicharethe
in students
students
students
which arestudents
are
required
the to fillare
students in
are
required
required to
to fill
fill
the in
in the
the
empty
required empty
empty
toframes.
fill frames.
inframes.
the empty frames.
required to fill in the empty frames.

Figure
Figure 4. Figure
4. Ratio
Ratio table 4.
4. Ratio
Ratio table
table problem.
Figure problem. table problem.
problem.

Figure Multiplying two numbers.

Figure
Figure 5. Figure 5.
5. Multiplying
Multiplying Multiplying
5.two
two numbers.two
numbers.
Multiplying two numbers.
numbers.
2.3. CATs for Assessing Multiplication of Two-Digit Numbers
2.3.
2.3. CATs
CATs for Assessing
for2.3.
Assessing
2.3. CATs
CATs for Multiplication
Multiplication
for Assessing of
of Two-Digit
Two-Digit of
Assessing Multiplication
Multiplication Numbers
Numbers
of Two-Digit
Two-Digit Numbers
Numbers
To provide teachers with a tool for getting insights in their students’ learning and actionable
To
To provide To provide
provide teachers
teachers with
with teachers
aa tool
tool forwith
for aa tool
getting for getting
toolinsights
getting insights in insights
in their
their in their
students’
students’ students’
learning
learning andlearning
and actionable
actionable and actionable
cluesTo forprovide
their next teachers
steps in with
teaching, for
we getting
developed insights
eightin their
CATs students’
(see Appendix learning
A): fiveandCATsactionable
in the
clues
clues for
for their
their next
next
clues steps
forsteps
their in
in teaching,
nextteaching,
steps we
we
in developed
developed
teaching, we eight
eight CATs
CATs
developed (see
(see
eight Appendix
Appendix
CATs (see A):
A): five
five
Appendix CATs
CATs A):in
in the
the
five CATs in the
format of the red/green cards, and three in a worksheet format. Each CAT contains two similarin
clues for their next steps in teaching, we developed eight CATs (see Appendix A): five CATs the
tasks
format
format of theformat
of the red/green
red/greenof the red/green
cards,
cards, and
and cards,
three
three in
in and
a
a three
worksheet
worksheet in a worksheet
format.
format. Each
Each format.
CAT
CAT Each
contains
contains
format of the red/green cards, and three in a worksheet format. Each CAT contains two similar tasks CAT
two
two contains
similar
similar two
tasks
tasks similar tasks
for
for doing
doing two
two assessments,
for
forassessments,
doing
doing two ifif necessary.
necessary.if
two assessments,
assessments, Here,
Here,
if exemplarily,
exemplarily,
necessary.
necessary. Here, we
we explain
explain four
Here, exemplarily,
exemplarily, four
we CATs
CATs in
we explain
explain detail.
infour
detail.
four CATs
CATsThree
Three
in are
are Three
in detail.
detail. Three are
are
meant
meant for
for assessing
assessing
meant multiplication
multiplication
for assessing with
with multiples
multiples
multiplication of
withof 10
10 (CAT-1),
(CAT-1),
multiples of the
the
10 structure
structure
(CAT-1), of
of
the
meant for assessing multiplication with multiples of 10 (CAT-1), the structure of the multiplication the
the multiplication
multiplication
structure of the multiplication
algorithm algorithm
algorithm (CAT-3),
(CAT-3),
algorithm and(CAT-3),
and the ratioand
the ratio
(CAT-3), the
aspect
theof
aspect
and ratio
of aspect
aspect of
multiplicationof multiplication
multiplication
ratio (CAT-4).
(CAT-4). Finally,
multiplication (CAT-4).
Finally, nearFinally,
near
(CAT-4). the endnear
the end
Finally, of thethe
of the
near end
end of
chapter,
chapter,
the of the
the chapter,
chapter,
when
when students
students
when have
have learned
learned
students the
the
have multiplication
multiplication
learned the algorithm
algorithm
multiplication for
for two-digit
two-digit
algorithm numbers,
fornumbers,
when students have learned the multiplication algorithm for two-digit numbers, it is assessed two-digit it
it is
is assessed
assessed
numbers, it is assessed
Educ. Sci. 2019, 9, 150 7 of 19

for doing two assessments, if necessary. Here, exemplarily, we explain four CATs in detail. Three are
meant for assessing multiplication with multiples of 10 (CAT-1), the structure of the multiplication
algorithm (CAT-3), and the ratio aspect of multiplication (CAT-4). Finally, near the end of the chapter,
when students have learned the multiplication algorithm for two-digit numbers, it is assessed whether
the students’ understanding goes beyond mechanically carrying out the algorithm (CAT-8). To show
the possible ways of collecting information with the CATs, we chose two CATs of each format: CAT-1
and CAT-4, which had a red/green card format, and CAT-3 and CAT-8, which had the individual
worksheet format. In addition, CAT-1, CAT-3/CAT-4, and CAT-8 were meant to be used at the beginning,
in the middle, and by the end of teaching with this chapter, respectively. A further reason for discussing
these CATs anticipates on our finding that the CATs differed in the degree to which they gave teachers
insights. Choosing these CATs gave us the opportunity to provide a fair picture of what CATs can bring
about.
Educ. Sci.CAT-1
2019, 9, and
x FORCAT-4 were CATs that helped fewer teachers gain insights than CAT-3 and CAT-8.
PEER REVIEW 7 of 21

2.3.1. CAT-1:
2.3.1. CAT-1: Family
Family problems
Problems
Multiplications with
Multiplications with multiples
multiplesofof10 10are
areoften
oftenconsidered
considered asasrather
rathereasy
easyproblems.
problems. Solving
Solving 12 ×1210×
by thinking of the analogous problem 12 × 1 and adding a zero
10 by thinking of the analogous problem 12 × 1 and adding a zero is not hard. However, is not hard. However, understanding
why this simplewhy
understanding adding of a zero
this simple works
adding of is something
a zero works else. To reallyelse.
is something grasp the content
To really grasp component
the content
component of multiplication with multiples of 10, a deeper understanding of the 10-basedsystem
of multiplication with multiples of 10, a deeper understanding of the 10-based number number is
necessary. Just being able to put a zero at the end of a result, in the case
system is necessary. Just being able to put a zero at the end of a result, in the case of problems withinof problems within the
number
the range
number rangeof two-digit
of two-digit numbers
numbers (e.g., using
(e.g., using 7070× ×6 6totofind
find7070××60),
60),does
doesnot
not guarantee
guarantee that the
that the
students comprehend this content component of multiplication. Therefore,
students comprehend this content component of multiplication. Therefore, using the exercises in the using the exercises in
the textbook in which the numbers are below 100 has limited value for
textbook in which the numbers are below 100 has limited value for assessing whether students truly assessing whether students
truly understand
understand multiplication
multiplication with multiples
with multiples of 10. Students
of 10. Students have learned
have learned to add oneto add
zeroone zero
in the in the
chapter,
chapter, and in the assessment based on these problems, they have to add
and in the assessment based on these problems, they have to add one zero, too. Students can pass one zero, too. Students can
pass test
this this by
testcarrying
by carrying out out mechanically
mechanically whatwhattheythey
havehave practiced.
practiced. ToTolearn
learnmore
moreabout
about students’
students’
understanding, we
understanding, we developed
developed aa CATCAT in in which
which the the scope
scope wentwent beyond
beyond the the two-digit
two-digit number
number range.range.
If students understand the 10-based number system, they can use
If students understand the 10-based number system, they can use the analogy strategy also for the analogy strategy also for aa
broader collection
broader collection of of problems.
problems.
CAT-1 (Figure 6) has
CAT-1 (Figure 6) has the
the red/green
red/green card
card format
formatand and starts
starts with
with the
the multiplication
multiplication97 97×× 8,
8, to
to which
which
the answer is given. Then, several related multiplication problems follow,
the answer is given. Then, several related multiplication problems follow, such as 970 × 8000. These such as 970 × 8000. These
problems are
problems are at
at first
firstsight
sightnotnoteasy
easytotosolve
solveby bymental
mentalcalculation.
calculation.InInthe theCAT,
CAT, students
students areare
notnotasked to
asked
solve these problems, but only whether they think they are able to solve
to solve these problems, but only whether they think they are able to solve them. They show theirthem. They show their answer
by raising
answer by the green
raising the(“Yes”) or the red
green (“Yes”) or card (“No”).
the red By tallying
card (“No”). the green
By tallying theand redand
green cards,redthe teacher
cards, the
gets an immediate
teacher gets an immediateoverview of students’
overview responses.
of students’ In this way,
responses. he/she
In this way,observes whether students’
he/she observes whether
understanding
students’ of multiplication
understanding with multiples
of multiplication of 10 goesofbeyond
with multiples 10 goesmechanically adding one
beyond mechanically zero and
adding one
whether
zero and they
whethersee the
they analogy
see theand thinkand
analogy theythink
can make
they canusemakeof it. use of it.

It is known that 97×8 equals 776.

Do you think you can solve the following problems?
(Yes- Green card ; No- Red card )

a) 97×80 b) 97×800 c) 97×8000 d) 970×8000

Figure 6. CAT-1:
CAT-1: Family problems.

2.3.2. CAT-3:
2.3.2. CAT-3: Breaking
Breaking down
down aa multiplication
Multiplication
Knowing how
Knowing how ananalgorithm
algorithmis isbuilt
builtupup
cancan help
help for for using
using it. Therefore,
it. Therefore, in the in the chapter,
chapter, much
attention is paid to the structure of the multiplication algorithm. Students were explained howhow
much attention is paid to the structure of the multiplication algorithm. Students were explained the
the results
results of multiplications
of multiplications withwith multiples
multiples ofare
of 10 10 are notated,
notated, theythey
hadhad to fill
to fill in an
in an empty
empty structure
structure of
of the multiplication algorithm, and they were taught how to carry out the algorithm
the multiplication algorithm, and they were taught how to carry out the algorithm step-by-step. step-by-step.
However, being
However, being able
able to
to write
write down
down thethe algorithm perfectly and
algorithm perfectly and even
even arriving
arriving at at the
the correct
correct answer
answer
does not necessarily mean that students understand what they are doing and understand the
structure of multiplications with two-digit numbers.
CAT-3 (Figure 7) has a worksheet format and is meant to give teachers an extra opportunity to
assess whether their students can identify the components of a multiplication and how they
understand what is behind the algorithm. In this CAT, the same numbers are used as in the textbook,
 Educ. Sci. 2019, 9, 150 8 of 19

does not necessarily mean that students understand what they are doing and understand the structure
of multiplications with two-digit numbers.
CAT-3 (Figure 7) has a worksheet format and is meant to give teachers an extra opportunity
to assess whether their students can identify the components of a multiplication and how they
understand what is behind the algorithm. In this CAT, the same numbers are used as in the textbook,
namely 24 × 53. However, now students have to unravel this multiplication instead of carrying it out.
By using distributive and associative properties, this can lead to four sub-multiplications, namely 3 × 4,
3 × 20, 50 × 4, and 50 × 20, or in any other order. The teacher hands out the worksheet and checks
Educ. Sci. 2019, 9, x FOR PEER REVIEW
students’ responses after class and uses the gained information for decisions about further instruction
EVIEW Educ.
in the
Educ. Sci. 2019,
next
Sci. 9,
9, xx FOR
2019,lessons.FOR PEER
PEER REVIEW
REVIEW 8 of 21 88 of
of 21
21
24×53 means that you have to calculate

53 means that you have to calculate 24×53

24×53 means
means that
that you
you have
have to
to calculate
calculate a) 3×4 and 3×20 and __×

a) 3×4 and 3×20 and __×4 and a) 3×4

__×20
a) 3×4 and 3×20
and 3×20 and and __×4__×4 and __×20
and __×20 b) 4×__ and 4×__ and __×

b) 4×__ and 4×__ and __×3 and b)

b) 4×__
20×50
4×__ and
and 4×__
4×__ and
and __×3
__×3 and 20×50
and 20×50 20×50 20×__ 4×
c) and and
20×50 20×__ 4×50 20×50 20×__ 4×50 __×3
c) and and c)__×3
and c) 20×50 and
and 20×__ and
and 4×50 and
and __×3 Figure 7. CAT-3: Breaking down a m
Figure 7. CAT-3: Breaking down a multiplication.Figure
Figure 7.
7. CAT-3:
CAT-3: Breaking down a multiplication.
CAT-3: Breaking down a multiplication.
2.3.3. CAT-4: Completing the ratio table
he ratio table 2.3.3.
2.3.3. CAT-4:
CAT-4: Completing
CAT-4: Completing the
Completing the ratio
Ratiotable
ratio Table
table The ratio table problems provided in the textbook, in w
The
The ratio
ms provided in the textbook, ratiointable
which
table problems
studentsprovided
problems just havein
provided tothe
in textbook,
multiply
the textbook, numbers
the in
in which in the topjust
which students
students rowhave
just by 10to
have toormultiply
20, maythe
multiply not really reveal how
the
10 or 20, may notnumbers in
in the
really reveal
numbers how
the top row
row by
topstudentsby 10 or
or 20,
20, may
understand
10 or 20, maythe
may not really
really reveal
notrelationships
not really reveal between
reveal how the
how students
how numbers
students understand
students in the
understand the
understand ratio
relationships focus is on verti
table.
the relationships
relationships The
he ratio table. Thebetween
between the
focus is the
on numbers
numbers
thevertical
numbers inin the
inthe ratio
theratio
reasoning ratio table.
table.
and The
The
is based
table. The focus
focus
on is
is on
is on
knowing
focus howreasoning
vertical
on many
vertical
vertical areand
reasoning
reasoningin one
and unit.
is
is based
and In knowing
is based
on
based fact,
on a ratio
on knowing
knowinghowtable is not necessa
how
many
In fact, a ratio table
how ismany
not
many inare
onein
are necessary
are one
forunit.
inunit.
one In
In afact,
Insolving
fact,
unit. ratio
these
fact, aa ratio
table table
is notis
problems.
ratio table not
necessary
It
is seems
not to be
necessary just
for
for solving
necessary a format
for solving
solving to
these these notate
problems.
these multiplication
problems.
It seems
problems. It to be problems with
It seems
seems
ate multiplicationtojust
to be just
just aawith
bea format
problems format
to notate
format to notate
of multiplication
to multiplication
multiplesnotate 10. CAT-4 problems
multiplication problems
(Figure with developed
waswith
8) multiples
problems with of 10. for
multiples
multiples assessing
of
CAT-4
of 10. CAT-4
(Figure
10. CAT-4 the ratiodeveloped
(Figure
8) was
(Figure aspect
8)
8) was
wasof multiplication an
developed
for assessing
he ratio aspect of developed
multiplicationfor assessing
the ratio the
aspect
and has athe
for assessing ratio
of
broader aspect
multiplicationof multiplication
and
operationalization
ratio aspect hasof
of multiplication a ratio
andand
broader has the
a use of
broader the
operationalization ratio table.
operationalization
of ratio
and has a broader operationalization of and of
the
io table. ratio
use ofand
ratio thethe
and use
ratio
the of
of the
the ratio
table.
use ratio table.
table.
You are NOT allowed to calculate how man
there are in one box. Can you still find out h
are NOT allowed to calculate how many pencils You
You are
are NOT
NOT allowed
allowed toto calculate
calculate howhow many
many pencils
pencils pencils there are in the following boxes?
there
there are
e are in one box. Can you still find out how many are in
in one
one box.
box. Can
Can you
you still
still find
find out
out how
how many
many (Yes- Green card ; No- Red card )
cils there are in the following boxes? pencils
pencils there
there are
are in
in the
the following
following boxes?
boxes?
s- Green card ; No- Red card ) (Yes-
(Yes- Green
Green card
card ;; No-
No- Red
Red card
card )) Number of boxes 6 11 12 13
mber of boxes 6 11 12 13 17 18Number
22 23
Number of
of boxes
boxes 6
6 11
11 12
12 13
13 17
17 18
18 22
22 23
23 Number of pencils 72 132
mber of pencils 72 132 Number
Number of
of pencils
pencils 72
72 132
132
Figure 8. CAT-4: Completing the r
Figure 8. CAT-4: Completing the ratio table. Figure
Figure 8.
8. CAT-4:
CAT-4: Completing
CAT-4: Completing the
Completing the ratio
ratio table.
table.
This CAT has a red/green card format and is called ‘Co
reen card format and is This
This CAT
CAT
called
CAT has aaa red/green
has
‘Completing
has red/green
red/greenthe ratio cardtable’.
card
card format
format
format and
Itand
is
and isiscalled
is
meant called challenge
‘Completing
to ‘Completing
called ‘Completing studentstheratio
the
the in their
ratio
ratio table’.
table’. work
table’. It
It is
is with
It meant
is meant
meant the
to ratio table and g
to
to
r work with the challenge
ratiochallenge
challenge andstudents
tablestudents
students givein in their
their
inteachers
their work
workwork withthe
with
extensive
with theratio
ratiotable
information
the ratio table
table students’
and
ofand
and give
give understanding
give teachers
teachers
teachers extensive
extensive
extensive of the ratio aspect
information
information
information of of multiplication
of
understanding ofof CAT is that the number of pencils in one box is not given. Mor
f the ratio aspect students’ understanding
of multiplication.
students’ ofthe
The difficulty
understanding theratio
the foraspect
ratio
ratio aspect
the of of
multiplication.
students
aspect of multiplication.
in this
multiplication. TheThedifficulty
The difficulty
difficultyfor the
for students
for the
the students
students in thisinCAT
in this
this
pencils in one boxCATis
is that
CAT notisthe
thatnumber
that
isgiven. the of pencils
number
Moreover,
the number of
they
of in one
pencils
are
pencils inbox
in
not oneis
one
allowedbox
box notis
togiven.
is not given.
calculate
not given. this
Moreover, number.
Moreover,
Moreover,theythey areThe
they arenumbers
not
are allowed
not
not allowed toincalculate
allowed the
to ratiothis
to calculate
calculate table have been ch
rs in the ratio tablenumber.
this number.
thishave
number. The numbers
been TheThe in
numbers
chosen
numbers the
in such ratio
in the
in athe table
way ratio
ratio havetable
thattable been have
students chosen
have are been prompted
in such
chosen a way
into find
that
such other
students
a way
been chosen in such a way that students are methods
are
that to
prompted
students fill in tothe empty cells. Fo
are
find
prompted
ethods to fill in the othercells.
empty
prompted methods
to
to find
find to fillmethods
Forother
example,
other in the
methodsif inempty
to
sixfill
to cells.
in
in the
boxes
fill For
there
the example,
empty
empty are cells.pencils,
72 if For
cells. in
Forsix then
example,
example, you
boxes there if also
if ininare know
six72
six boxes
boxes how
pencils,there
theremany
thenareyou
are there
72 are in 12 boxe
72
w how many there also
pencils,
areknow
pencils, 12how
inthen
then you
boxes.
youmany
also
also there
know
Similarly,
know arehow
ifinin12six
how boxes.
many
many boxes Similarly,
there
therethereare
are areinif12
in 72
12 pencils
inboxes.
six
boxes.boxes andthere
Similarly,
Similarly, in 11 ifboxes
areif 72
in sixthere
in pencils
six boxes
boxes arethere
and 132,
in
there then
11are
are 72you can also know
boxes
72
re are 132, then youthere
pencils areand
can also
pencils 132,
andknowin then
11 you
boxes
directly
in 11 boxes can
therealsoare
howare
there know
132,
many directly
then
132, pencils
then you you how
can
there
canare many
also 17 boxes.
pencils there Reasoning
are in 17 and
boxes.
alsoinknow directly how many pencils there are in
know directly how many calculating
pencilsReasoning
there similar
areandin to this means
calculating similarcalculating
17
17 boxes.
to
boxes. similarthat
this Reasoning
means
Reasoning to this themeans
and
and calculating
ratio table
calculatingthat the is ratioonly
similar
not
similar table
to
to this is not
used
this vertically,
means
means onlythat
used
that but
the
the also
vertically,
ratio horizontally.
ratio table but is
table also
is not To
only a
nothorizontally.certain
only usedused degree, the tex
To a certain degree, the textbook
also givesalso aagives opportunities this richer method ofopportunities
using the ratio table, when the stude
ntally. To a certainvertically,
degree,
vertically, but
the
but also
also horizontally.
textbookhorizontally. To
To certain
opportunities
certain degree,
to elicit
degree, thetotextbook
the elicit
textbook thisalsoricher
also gives
givesmethod of using
opportunities the
to ratio
to elicit
elicit
ng the ratio table,this
table,
when
this richer
when
richerthe method
the students
students
method of using
ofare are
asked
using the
thetoratio
asked reflect
ratio table,
to reflect
table,onon when
whatthey
what
when the
the students
they discovered
discovered
students are
areinasked
inthe to
to reflect
theratio
asked ratio table.on
table.
reflect on what
Inparticular,
In they
particular,
what they this is the case w
discovered
ble. In particular, this in
in the
this is the case
discovered ratio
when
the table.
table. In
problems
ratio particular,
such
Insuch as
as
particular, this
this is
is the case
are
theare when
when problems
followed
followed
case by
by
problems such
such as
as ,and thereisisare
andthere an
anopportunity to di
are
opportunity
followed
, and there is an opportunity by to discuss that ,
followed by to discuss that, this this
and is equal
there
and isthere
equalis to
an
is to 16
an 16 × 20. This
opportunity way
to of reasoning
discuss that about
this is the
equal ratio
to 16 table
× 20. is
This
× 20. This to discuss that this is equal to 16 × 20. This
opportunity in any case explicitl
e ratio table is in way
any of
of reasoning
waycase explicitlyabout
reasoning the
the ratio
promoted
about table
table is
in CAT-4
ratio and
is in
in any case teachers
case explicitly with
explicitly promoted
can provide
any extra
promoted in information
in CAT-4
CAT-4 and
and can providetheir students’ u
about
can provide
rmation about their teachers with
with extra
students’
teachers extra information
understanding
information about
of the
about their
ratio aspect
their of multiplication.
students’
students’ understanding
understanding of of the
the ratio
ratio aspect
aspect of of
multiplication.
multiplication.
2.3.4. CAT-8: Solving problems without algorithm
2.3.4.
2.3.4. CAT-8:
lems without algorithm CAT-8: Solving
Solving problems
problems without
without algorithm
algorithm The exercises provided in the chapter are mainly on so
The exercises provided in the chapter are mainly on solvingcontext,
without by using
multiplication the algorithm.
problems, with orBy the end of the
Educ. Sci. 2019, 9, 150 9 of 19

any case explicitly promoted in CAT-4 and can provide teachers with extra information about their
students’ understanding of the ratio aspect of multiplication.

2.3.4. CAT-8: Solving Problems without Algorithm

The exercises provided in the chapter are mainly on solving multiplication problems, with or
without context, by using the algorithm. By the end of the chapter, the expectation is that most students
are quite able to correctly perform the algorithm. However, after lots of practice, it could happen that
students carry out every step of algorithm perfectly, but merely in a mechanical way. CAT-8 (Figure 9)
has a worksheet format and allows the teacher to assess whether students really understand what a
multiplication
Educ. Sci. 2019, 9, xmeans andREVIEW
FOR PEER thus also what the algorithmic procedure actually implies. 9 of 21

Here you see a multiplication problem. You have to solve it

WITHOUT using the algorithm. Write down how you solved it.

In this way I solved the problem Answer

59×62

Figure 9. CAT-8:
CAT-8: Solving
Solving problems
problems without algorithm.

In this
this CAT,
CAT,students
studentshave
haveto to
solve multiplication
solve multiplicationproblems
problemswithout usingusing
without the algorithm. The main
the algorithm. The
idea isidea
main thatiswhen
that students cannotcannot
when students solve a solve
multiplication problemproblem
a multiplication without without
using theusing
algorithm, they will
the algorithm,
probably
they will not have anot
probably sufficient
have aunderstanding of multiplication,
sufficient understanding which mightwhich
of multiplication, put them in trouble
might put themwhen
in
learningwhen
trouble to solve more complicated
learning to solve more multiplication
complicated problems with, problems
multiplication for example, three-digit
with, numbers
for example, or
three-
decimal
digit numbers.
numbers Examining
or decimal the worksheets
numbers. Examining after
theclass offers the
worksheets teacher
after classclues
offersabout whetherclues
the teacher and
what instructional
about whether andsupports students need
what instructional beforestudents
supports finishing thisbefore
need chapter.
finishing this chapter.

2.4. Teacher
2.4. Teacher training
Training
To familiarize
To familiarizethe
theteachers
teacherswith
withthe
theCATs
CATs and
andassist them
assist themin using thethe
in using CATs in class,
CATs two two
in class, two-hour
two-
hour meetings were organized during the first two weeks of the second semester of Grade 3. The
meetings were organized during the first two weeks of the second semester of Grade 3. first
The first
meeting took
meeting took place
place before
before the
the teaching
teaching ofof the
the chapter
chapter on
on multiplication
multiplication started.
started. Each
Each teacher
teacher received
received
a package including a teacher guide with the material (PowerPoint slides, red/green
a package including a teacher guide with the material (PowerPoint slides, red/green cards, cards, worksheets)
needed for carrying
worksheets) needed out the CATsout
for carrying in their teaching
the CATs of this
in their chapter.
teaching ofDuring the meeting,
this chapter. Duringsome general
the meeting,
information about classroom assessment was presented, and the CATs that could
some general information about classroom assessment was presented, and the CATs that could be used in the comingbe
week were discussed. The second meeting began with sharing the teachers’ experiences
used in the coming week were discussed. The second meeting began with sharing the teachers’ in using the
CATs. After in
experiences that, the the
using CATs for the
CATs. second
After that,week werefor
the CATs discussed.
the second week were discussed.

2.5. Data
2.5. Data Collection
Collection
During the
During the two-week
two-week intervention,
intervention, eacheach of
of the
the 25
25 teachers
teachers was
was observed
observed at at least
least twice
twice regarding
regarding
their use
their use of
of the
the CATs.
CATs. As As aa result,
result, wewe could
could see
see that
that thethe teachers
teachers basically
basically conducted
conducted the the CATs
CATs as as
expected. To know what insights teachers might gain into students’ learning, they
expected. To know what insights teachers might gain into students’ learning, they were asked to fill were asked to fill in
feedback
in feedbackforms
formsandand write a final
write report.
a final report.Teachers
Teachers filled inin
filled the feedback
the feedback forms
forms every
everytime
timethey
theyused
useda
a CAT, and wrote the final report after finishing the chapter. Specifically, the teachers commented the
CAT, and wrote the final report after finishing the chapter. Specifically, the teachers commented on on
feedback
the form
feedback on on
form whether
whether using
using the CAT
the CAT helped
helpedthem themgain
gainnew
newinformation
information about their students’
about their students’
learning, whether
learning, whether theythey adapted
adapted their
their further
further instruction,
instruction, and and what
what thethe new
new information
information and and the
the
instructional adaption looked like. In the final report, among others, the teachers
instructional adaption looked like. In the final report, among others, the teachers were asked to were asked to suggest
two CATs
suggest twotoCATs
be included in the textbook
to be included as assessment
in the textbook exercisesexercises
as assessment and explain why they
and explain why had chosen
they had
these CATs.
chosen these CATs.

2.6. Data Analysis

To get an overall picture of the teachers’ experiences with the CATs, we first scanned all the
filled-in feedback forms. The responses of the 25 teachers on the feedback forms and final reports
were included in the final analysis. To answer the research question regarding teachers’ insights into
Educ. Sci. 2019, 9, 150 10 of 19

2.6. Data Analysis

To get an overall picture of the teachers’ experiences with the CATs, we first scanned all the
filled-in feedback forms. The responses of the 25 teachers on the feedback forms and final reports
were included in the final analysis. To answer the research question regarding teachers’ insights into
students’ mathematics learning, we had to identify which teachers got insights when using the CATs.
Therefore, all the teachers’ responses were gathered and translated into English. First, all the responses
were scrutinized by the three authors separately. Each author identified the information the teachers
gained about their students and had to decide whether there was evidence of gaining insights. In order
to award criteria for having gained insights, the authors had to specify why they thought so. Next,
for each CAT, the authors’ decisions on whether the teachers’ responses showed indications of insights,
and the reasons for making these decisions were compared and discussed. In some cases, it was easy
to agree on whether there was evidence of teachers gaining insights. When the decisions differed,
they were discussed until 100% agreement was reached. Then, we checked all of our decisions again,
which led to some changes of our earlier decisions. During this checking process, we also finalized the
formulation of the criteria for indicating gaining insights.
In the end, this resulted in four unique criteria:

1. Referring to the mathematical content the CAT is supposed to assess. For this, teachers can use their
own words or give a clear description of the purpose of the CAT by using (partly) the wording
that appeared in the teacher guide. However, this criterion is not met when teachers only refer to
the CAT in general terms not mentioning the mathematics assessed.
2. Providing specific information about students. This includes mentioning the proportion of students
showing a particular performance on the assessed content or describing the difficulties students
encountered with this content.
3. Describing the novelty of the gained information about students. This means that teachers learn
something “new”, “unexpected”, “surprising”, or “that was not known before” about students’
understanding of the assessed content.
4. Explaining an instructional adaptation matching the findings from the CAT. Such an instructional
adaptation has to correspond to the information about the assessed content as revealed by using
the CAT; general phrases such as “providing additional exercises” or “give extra instruction” are
not sufficient.

Showing that one has learned something from doing an assessment is a multifaceted phenomenon.
It can be expressed in different ways. Teachers can say something about the performance of their
students, can emphasize that they discovered new information in the students’ performance, or can
discuss their decisions about further teaching. All these responses can indicate that teachers have
learned something from assessing their students with the CATs. Yet, to fully classify a teacher’s
response as having gained insights from a CAT, a first requirement is that the teacher refers to the
mathematical content the CAT is supposed to assess. Just talking about students’ performance in
general terms is not sufficient. So, our final decision rule to qualify a teacher’s response as having
gained insights is that it should meet Criterion I and at least one from Criterion II, III, and IV. Based
on this decision rule, a final round of checking was carried out by the first author. This resulted in
qualifying 57 teachers’ responses out of the total 200 possible responses (25 teachers x eight CATs)
as having gained insights. Table 1 provides examples of the qualifications of teachers’ responses
about CAT-1.
Educ. Sci. 2019, 9, 150 11 of 19

Table 1. Examples of the teachers’ responses about CAT-1 Family problems, and whether they were
qualified as showing having gained insights. CAT: classroom assessment technique.

Example of Teacher’s Response Criterion Met Having Gained Insights

Example 1: Teacher S14
“In my expectation, the vast majority of the students can find the correct
answer without being disturbed by the increasing number of zeroes.
However, the reality is that students felt difficult when the numbers
I, II, III Yes
became bigger and bigger [II, III]. Therefore, it illustrated that the
students were not flexible enough when solving by mental calculation
the problem with multiples of 10 [I]. Also, it means that the students
lack the ability of reasoning and generalization [I].”
Example 2: Teacher S09
“The students could not recognize such analogous problems very well
[I]. Most students showed their cards based on their ‘gut feeling’ [II]. I, II Yes
Only a small proportion of the students could find the rule [II]. As the
number of zeroes increased, the accuracy went down [II].”
Example 3: Teacher S04
“Most of the students could give the answer quickly, but some II No
individual students need help [II].”
Example 4: Teacher S10
“This CAT helps the students to extend the boundary of knowledge.
[no criterion
The difficulty level is appropriate. The students showed strong interests No
applicable]
when doing the CAT. In general, the CAT is helpful to extend students’
knowledge and to improve their initiatives.”

3. Results

3.1. Teachers Gaining Insights from Using the CATs

Table 2 gives an overview of the CATs used by the teachers (see the white cells) and whether the
teachers gained insights into their students’ understanding of multiplication of two-digit numbers
(see the white cells containing a X). Of the 25 teachers involved in the study, 22 used all the eight CATs,
one (H02) used seven CATs, and two (H04, N05) used only five CATs. In total, 193 responses of using
the CATs were collected, of which 57 responses (30%) clearly showed that the teachers gained insights.
Five teachers gained insights from five or more CATs (the High Insight group), 14 teachers did this only
from three or fewer CATs (the Some Insight group), and six teachers did not show clear evidence of
insights, no matter how many CATs they used (the No Insight group). CAT-3 and CAT-8 appeared to be
the most informative CATs for teachers; for these respective CATs, there were nine and 11 teachers
who showed evidence of gaining insights. Gaining insights here means that clear indications could be
identified in a teacher’s response to a particular CAT: that the teacher referred to specific information
about his/her students, or emphasized the novelty of the gained information, or referred to a fitting
instructional adaptation, and these reactions went together with references to the mathematics content
of the CATs.
Educ. Sci. 2019, 9, 150 12 of 19

Table 2. Overview of whether clear evidence of teachers (n = 25) gaining insights into their students’
understanding of multiplication by using the CATs was identified 1 .

CAT
Insight Teacher Total of X
1 4 7 2 5 6 3 8

High H01 3 3 3 3 3 3 3 7
Insight H02 3 3 3 3 3 3 6
H03 3 3 3 3 3 5
H04 3 3 3 3 3 5
H05 3 3 3 3 3 5
Some S01 3 3 3 3
Insight S02 3 3 3 3
S03 3 3 3 3
S04 3 3 3 3
S05 3 3 3 3
S06 3 3 3 3
S07 3 3 2
S08 3 3 2
S09 3 3 2
S10 3 1
S11 3 1
S12 3 1
S13 3 1
S14 3 1
No N01 0
Insight N02 0
N03 0
N04 0
N05 0
N06 0
Total of 3 5 5 5 6 8 8 9 11 57
1Black cell means the teacher did not use the CAT; empty white cell means the teacher used the CAT but no clear
evidence of the teacher gaining insights from using the CAT was identified; white cell with “3” means the teacher
used the CAT and clear evidence of the teacher gaining insights from using the CAT was identified.

To give more information about the specific insights the teachers appeared to gain into their
students’ mathematical understanding, we now focus on the teachers’ responses to the four earlier
described CATs (1, 3, 4, and 8). Based on the four criteria for gained insights, in the next sections,
we discuss for each of the CATs why we considered particular responses of the teachers as indications
for having gained insights.

3.2. Four Examples of Teachers Gaining Insights from Using CATs

3.2.1. Insights from Using CAT-1: Family Problems.

All teachers, except one (H04), returned a response for CAT-1. In five teachers’ responses,
clear evidence was found that they gained insights about whether, and to what extent, their students
understood multiplication with multiples of 10 beyond the two-digit number range. Four of the
teachers who gained insights directly referred to the mathematical content that CAT-1 is supposed to
assess. For example, they reported about students’ analogous thinking (H03, S09) or about students’
flexible use of known rules of multiplication with multiples of 10 (H01, S14). Teacher S06 described that
CAT-1 aims to assess “a method” that “makes use of the given problems” and that shows its “advantage
when the number of zeroes increases”. In our view, this indicates that the teacher recognized what the
task is about, resulting in a good lens for diagnosing what students do or do not understand. In their
responses, the five teachers dealt with their students’ performance when the numbers in the problems
Educ. Sci. 2019, 9, 150 13 of 19

contained more than two digits. For three teachers (H01, S06, S14), this came down to reporting
that most students were able to deal with this and that only a few students “could only solve [until]
97 × 80 and 97 × 800” (H01). Three teachers (H03, S09, S14) reported that fewer students provided
correct answers when the numbers became bigger, and that “only a minority of the students could
determine the rule” (S09). This came as a surprise to one teacher (S14), who explained that “in my
expectation, the vast majority of students would find the correct answers without being disturbed by
the increase in the number of zeroes”. Only two teachers (H03, S06) mentioned how they would adapt
their instruction; they were going to include analogous problems for students to practice with.
In 19 teachers’ responses, the mathematical content assessed by CAT-1 was not mentioned.
When teachers described students’ performance, it was in very general terms. Some teachers
reported to be satisfied with their students’ performance; for example, students were able to calculate
“according to the given characteristics” (S05), other teachers pointed out their students’ shortcomings
in understanding “the rule” (H02, H05, S01). Similarly, when teachers wrote about their instructional
adaptations, they often used general terms such as “more exercises” (S07) or “extra instruction”
(N05). Interestingly, two teachers (S11, N04) decided not to adjust their further teaching because they
considered the content of CAT-1 to be too similar to what is in the textbook. In contrast, another teacher
(N01) provided as a reason for not adapting her instruction that “there is no such type of exercise in
the textbook”.

3.2.2. Insights from Using CAT-3: Breaking down a Multiplication

Twenty-four teachers provided a response for CAT-3. Nine teachers were found to have gained
insights into whether their students could identify the components of a multiplication of two-digit
numbers and how their students understand what is “behind” the algorithm. Only one of these
teachers (S11) reflected on what is assessed in CAT-3 in her own words: “breaking down the
multiplication problem into four components is factually the same as showing how the algorithm
works to calculate the multiplication of two-digit numbers”. The remaining teachers who gained insights
referred to the mathematical content assessed by CAT-3 in terms of “the meaning of multiplication of
two-digit numbers” (H03) or “the meaning behind the algorithm” (H01). When describing students’
performance, most teachers (H01, H02, S03, S04, S09, S11, S12) made a clear distinction between
students’ understanding of the structure of the multiplication algorithm and students’ ability to
apply the procedures. For example, one teacher (S12) reported that most of her students “master the
procedure of calculating, but their understanding about how it works is not good enough”. Another
teacher (S11) found that “25 out of the 39 students could answer all the blanks correctly”. In contrast,
the remaining teachers reported that at least half of the students were unable to break down 24 × 53,
although they were able to find its result. Two teachers (H01, H04) said this was not what they
expected. Another teacher (S03) also expressed her surprise, as she “thought the students would
not even understand the question”, but “the situation in fact was a bit better”. For further teaching,
this teacher wanted to pay more attention to help the students “understand the meaning of each step
of performing the algorithm”.
The remaining 15 teachers did not refer to the mathematical content assessed in CAT-3. Instead,
they mainly focused on describing students’ performance. Most teachers reported that their students
had difficulties in breaking down the problem—for example, “students could only break down 24 × 53
into 24 × 50 + 24 × 3” (S06)—or difficulties in understanding the question—for example, “students had
never been trained to break down a multiplication problem into four parts” (S05). Overall, students in
these 15 classes did not perform well on CAT-3. However, about half of the teachers would not adjust
their further teaching. The main reason they gave was that CAT-3 was too different from what they
taught in class about the multiplication of two-digit numbers, and it therefore could “disturb students’
thinking” (N01) or “make students confused” (N02).
Educ. Sci. 2019, 9, 150 14 of 19

3.2.3. Insights from Using CAT-4: Completing the Ratio Table

Twenty-three teachers provided a response about their use of CAT-4. Also for this CAT, five teachers’
responses reflected gained insight into their students’ understanding. Four of these teachers’ responses
(H01, H02, S03, S07) reflected the aim of finding out how students use the different relationships
between the provided numbers in the ratio table. One teacher (S04) only referred to vertical reasoning
in the ratio table: “most students still needed to calculate how many pencils there are in one box in
order to find the total number”. In the other four classes (H01, H02, S03, and S07), the teachers reported
that many of their students could solve the problems by observing the provided numbers of boxes and
establishing relationships. One teacher (H02) realized that “vertical and horizontal reasoning can be
considered at the same time” and she planned “in future teaching, to encourage students to think and
solve problems from multiple perspectives”.
When the remaining 18 teachers referred to what CAT-4 aims to assess, it was in very general terms;
they mentioned “the relation between the numbers” (S02), “multiple perspectives” (S05), or “other
strategies” (S09). Regarding students’ performance on CAT-4, five teachers reported their satisfaction
with students’ reactions. In the other classes, the majority of students “was limited in their thinking”
(S10) or “just made a guess” (N03). Based on CAT-4, 13 teachers wrote they would provide “more
exercises”. Two of them (S05, S13) in fact considered reasoning horizontally in a ratio table not to be
“something every student needs to master”. The teachers who indicated not making instructional
adaptations gave different explanations. For example, “it is not the content in the textbook” (N01) or
“it is not necessary to explain such difficult content in Grade 3. It is of course good if students can
understand; it is also fine if they do not understand” (S09).

3.2.4. Insights from Using CAT-8: Solving Problems without Algorithm

All teachers returned a response for CAT-8. In 11 teachers’ responses, clear evidence was identified
of gained insights into students’ capability of solving multiplication problems of two-digit numbers
without using the algorithm. In their responses, these teachers referred to the mathematical content
assessed by CAT-8, for example about their students’ understanding of multiplication, students’
flexible use of different solutions, or students having a ‘mindset’ about solving multiplication problems.
In particular, these teachers described their students’ performance of providing different solutions to
solve 59 × 62. For example, one teacher (H02) found that some students understood the connection
between Lattice multiplication and breaking down the multiplication. Another teacher (S01) wrote that
“a small part of my students could solve the problem by using the distributive property”. Contrastingly,
another teacher (S05) found that “part of the students thought ‘without using the algorithm’ meant
‘no accurate answer being required’, [and] therefore they only made an estimation of the product”.
Furthermore, two teachers (H01, S10) reported that their students were used to writing down the
algorithm when given a multiplication problem, and they did not know how to start now. Only one
teacher (H05) really described the novelty of her gained insights. She had expected her students to not
be able to solve 59 × 62 without using the algorithm, but in fact many of them used the method of Lattice
multiplication. Regarding instructional adaptation, two teachers (H02, H04) valued the “openness and
flexibility” (H02) of CAT-8 to “give students more space to think and imagine freely” (H04) and would
use it in future teaching.
Of the 14 teachers with no evidence for insights, most only reported briefly that their students
could not solve problems without using the algorithm. Some teachers did refer to CAT-8 as aiming
to “develop students’ divergent thinking” (S09), to “extend students’ learning” (S12), or to “remind
students to solve problems in different ways” (S11). Three teachers (S03, S13, N06) seemed not to
understand what CAT-8 aims to assess. They thought that the assessed mathematical content was
students’ ability to apply the properties of multiplication; for example “it is difficult for students to
understand the distributive property” (S13). Another teacher (S03) made it clear that “students are
going to systematically learn the properties [in Grade 4]”. Seven teachers used general phrases to
describe their instructional adaptation; for example, “providing additional instruction for those who
Educ. Sci. 2019, 9, 150 15 of 19

are able to learn more” (N03) or “providing extra exercises to revise this content” (N01). The other
seven teachers would not make any instructional adaptation. Two of them (S11, N04) did this because
their students did not have problems in solving the problems, and the remaining five teachers felt
using the algorithm was more suitable to solve these problems.

4. Conclusions and Discussion

In this study, we investigated the insights into students’ understanding that Chinese primary
mathematics teachers gained by using CATs. CATs are meant to give teachers access to their students’
understanding of a particular mathematical concept or skill, by posing questions that purposely present
the mathematical content from a different perspective and in an unfamiliar way. In this way, CATs may
provide teachers with new lenses to observe and understand their students’ learning. In addition,
the gained insights may offer teachers clues to answer the question of what to do next in their teaching.
With respect to the latter, all the teachers in our study, no matter the extent of gained insights, paid more
attention to describing what they found out about their students’ performance than to indicating
possible instructional adaptations. This echoes the finding of Heritage et al. [41] that teachers are better
at drawing inferences about students’ understanding from assessment information than at deciding
the next instructional steps.
Although using CATs to assess students’ learning was quite new for the participating Chinese
mathematics teachers, most of them did gain insights into students’ mathematics understanding of
multiplication of two-digit numbers. Gaining insights was evidenced by either describing specific
information about their students, emphasizing the novelty of the gained information, or giving
suggestions for a fitting instructional adaptation, while at the same time referring to the mathematics
content of the CATs. Three-quarters of the teachers gave signs of having gained insights. However,
another finding was that the teachers differed greatly in the extent to which they gained these insights.
Five teachers acquired insights from five or more CATs, while 14 teachers did so only from three or
fewer CATs, and six teachers did not show clear evidence of insights, no matter how many CATs
they used. These teachers were respectively assigned to the High Insight, Some Insight, or No Insight
group. The teachers belonging to the High Insight group could clearly recognize that offering students
problems that differ slightly from the way they are presented in the textbook can be an eye-opener for
the teachers and can offer them the opportunity to acquire knowledge about their students’ learning
that they did not have before. In line with what was found by James and McCormick [22] for the
‘Assessment for Learning’ practice, these teachers seemed to have adopted the CATs as a means
to assess students according to the ‘spirit’ of the CATs, whereas the No Insight teachers seemed to
have implemented the CATs in a more mechanical way, more according to the ‘letter’ of the CATs,
following the prescribed procedure of the CATs and carrying them out in class without getting a better
understanding of students’ mathematics learning. In contrast with the teachers belonging to the High
Insight group, who clearly favored the revelatory capacity of assessment tasks that differ in a specific
way from the teaching tasks, the teachers in the No Insight group did not value them as such, and did
not appear to notice the difference. Several of these teachers considered the CATs to be (too) similar to
the textbook, and did not want to repeat what they had already taught. Yet, others emphasized that the
CATs were too different or too difficult compared to their regular teaching. The latter group of teachers
appeared to hold a strict view of what should and could be assessed, which might lead to them not
perceiving the CATs as helpful for obtaining more insight into students’ understanding regarding
the topic that is at hand in the textbook and their teaching. As a result, they might not have used
the CATs and the elicited information optimally, and thus did not show evidence of gained insights
from using the CATs. A possible reason why teachers did not gain insights may have to do with the
short duration of the intervention. A two-week support is a very limited time for teachers to become
accustomed to using the CATs and recognizing the valuable information revealed by using them.
Perhaps more insights would have been observed if a longer intervention had taken place. Moreover,
the teachers, including those in High Insight and Some Insight groups, were not involved in designing
Educ. Sci. 2019, 9, 150 16 of 19

the CATs, and this could also be a reason that they did not notice or value the information about
students’ learning. When teachers can take part in the development of the CATs and are supported
regarding seeing the connections between the CATs and the textbook content, this might lead to gaining
more insights into students’ understanding. In addition, that the local teaching office, rather than the
individual teachers, decided to participate in this study may have influenced the teachers’ willingness
regarding using CATs and filling in feedback forms. Also, the felt need of teachers to finish their
planned regular teaching may have been a factor that teachers considered when deciding whether to
use the CATs, and might have led to not always getting the most out of the CATs.
Having only a short intervention and not having the teachers involved in designing the CATs
are shortcomings of our study that should be kept in mind when interpreting our findings. Also,
it is important to take into account that our conclusions about whether the teachers obtained insights
from using CATs were based on their self-reported data. Further data collection, such as observing
teachers using CATs in class and directly asking them about their gained insights, could shed new
light on possibly gathered new information about their students’ mathematics learning. Furthermore,
examining students’ responses and worksheets could be included in order to triangulate teachers’
reported insights. Another limitation of our study is that the CATs used in the context of Chinese
primary mathematics education so far, including the pilot study [39], were designed based on one
particular textbook series and only involved teachers from the city of Nanjing. Whether Chinese
primary teachers who use different mathematics textbook series or who are from different regions
can get new insights from implementing CATs remains unclear. Further studies are necessary in
this respect.
A strong recommendation for additional studies is in any case to have a design in which teachers
are included in developing CATs. Moreover, future research may investigate why some teachers gain
less insights than others. To know more about this, it could be worthwhile to explore whether teachers
with different assessment profiles [38,42] benefit differently from using CATs. For example, recently,
it has been shown that significant differences exist between Chinese expert teachers in primary school
mathematics education and their non-expert colleagues in their perception and reported behavior
of understanding their students’ mathematics thinking [34]. Future research may also focus on how
to use the gained insights for making instructional decisions, because this is an issue to which the
teachers in our study did not pay much attention. In line with this, new research may be conducted to
examine whether and how teachers’ gained insights affects students’ performance.
Despite the aforementioned limitations and the questions that still have to be answered, we think
we can conclude that our study provides evidence that Chinese primary school mathematics teachers
may gain insights into their students’ understanding of mathematics from using assessment techniques
such as CATs. Yet, for the majority of the teachers in our study, it seems to be necessary to offer them
more time and support to get acquainted with this assessment approach. Also, more opportunities
could be provided to support teachers to see the connections between CATs and the textbook.
Using the CATs implies a strong formative approach to assessment. For Chinese primary school
mathematics teachers, who often put more emphasis on providing information to students than
on getting information about students [32], this may mean a change of perspective. Our CATs
may help them develop a more formative approach to assess students’ learning. More research is
certainly necessary, especially studies that investigate how teachers’ culturally-based beliefs about
teaching affects their formative use of assessment and that examine how to support teachers to become
independent users of formative assessment.

Author Contributions: Conceptualization, Methodology, Analyses, Writing: X.Z., M.v.d.H.-P., M.V.; Funding
Acquisition, X.Z. (China Scholarship Council) and M.v.d.H.-P. (Netherlands Organization for Scientific Research).
Funding: This research was funded by China Scholarship Council, grant number 201206860002; and the
Netherlands Organization for Scientific Research, grant number NWO MaGW/PROO: Project 411-10-750.
Acknowledgments: The authors thank Lianhua Ning from Nanjing Normal University, China, for helping to
contact schools, and all the teachers involved in this study for their cooperation and contribution.
 Educ. Sci. 2019, 9, x FOR PEER REVIEW 18 of 21

Educ. Sci.Educ.
2019,Sci.9,2019,
150 9, x FOR PEER
Table A1. The eight CATs for multiplication of two-digit numbers.
REVIEW 18 of 21 17 of 19

Title Format Example Purpose

Table
Educ. Sci. 2019, 9, x FOR PEERA1. The eight CATs for multiplication of two-digit numbers.
REVIEW 18 of 21
It is known that 97×8 equals 776. Assessing whether
Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the
Title Format
Do you think you can solve the following problems?
Example students can recognize
Purpose
Educ. Sci. 2019, 9, x FOR PEER
Family REVIEW
Table (Yes- Green card ; No- Red
A1. The eight CATs for multiplication of two-digit numbers. card ) 18 of 21
study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to
Red/Green It is known that 97×8 equals 776. analogous
Assessing problems
whetherand
problems
publish the results. a) 97×80
Do you think you b) 97×800can solve the c) 97×8000
following problems? d) 970×8000
Educ. Sci. 2019,
Title 9, x FOR PEERcards
REVIEW
Format Example of two-digit numbers students are aware
can
Purpose of the
18
recognize of 21
(CAT-1)
Family Table A1. The(Yes- eight GreenCATs cardfor ; No- multiplication
Red card ) .
Red/Green It is known that 97×8 equals 776. relationship
analogous Assessing among
problems whether the
and
problems Do CATs
a) 97×80 you think b)for youmultiplication
97×800 can solve
005 the following
c) 97×8000 problems?
d) 970×8000 results
Appendix A Title
Educ. (CAT-1)
Sci.Family
Format
Table
cards
2019, 9, x FOR PEER
A1. The eight
REVIEW
Example of two-digit numbers are of
.students these
Purpose
aware canofproblems
the
recognize
18 of 21
(Yes- Green card ; No- Red card )
Red/Green It is Here
known you that see 97×8 fiveequals
bags with 776. a different weight. Assessingproblems
relationship
analogous whether
among theand
Title
problems Format Howthink
Doa)you many you kilograms
can Example
solve approximately
the following in one bag?
problems? Purpose
Educ. Sci. 2019, 9, x FOR PEERcardsREVIEW 97×80 b) 97×800005 c) 97×8000 d) 970×8000 students
results of these
are can recognize
aware of18the
problems of 21
Family (Yes-
Table A1. The Iteight Do Greenyou think
CATs card for the
; No- answer
Red776.
multiplication cardis 30, ) 40 or of35?two-digit numbers .Assessing
(CAT-1) is known that 97×8 equals Assessing whether
whetherand
How many Red/Green Here
(Yes- you
Green seecancardfivesolve
;bags
No- the with
Red a different
card weight.
) problems? analogous
relationship problemsamong the
problems Table A1. The eight CATs for multiplication of two-digit numbers.
Red/Green
Do you think
How many
a) 97×80
you
b)kilograms
97×800 approximately
following
c) 97×8000 in one bag?
d) 970×8000 students
students cancan recognize
find the
Title
Family
approximately? cardsA1. The(Yes-
Format
Table eight Green
Bag CATs1 card Bag for
; No- Example
2 multiplication
Red
Bag card
3 005 )Bag 4 ofBag two-digit
5 . are Purpose
numbersresults aware
of these of the
problems
(CAT-1) Red/Green Do you think the answer is 30, 40 or 35? analogous problems and
cards It is(Yes-
known that
approximate
Assessing
relationship average
whether
among of a
the
Titleproblems
How (CAT-2)
many Format a) 97×80 33HerekgGreen you97×8
b) 3497×800kg equals
see
card five
37bags
;Example
No- Red
kg 776.with
c) 97×8000 card
37 kg a)different weight.
36d)kg970×8000 Assessing whether
Purpose
Title cards
Format
Red/Green Do youHow thinkmany you can Example
solve
kilograms 005the following problems?
approximately in one bag? are
series
students
results aware
Purpose
of ofcan
these offind
numbers thethe
problems
(CAT-1)
approximately? students can recognize
Family (Yes- Bag
It is known Do
Green 1 you card
that Bagthink;2No-
97×8 the Red
Bag
equalsanswer
3card
776.Bag is) 30, 4 40 Bag or 35?5 relationship
Assessing among
whether the
cards
Red/Green Here you see five bags with a different weight. approximate
analogous Assessing
Assessing average
problemswhether
whether of astudents can
and
(CAT-2)
How manyRed/Green
problems Do you 33(Yes-
thinkGreen
kg you
34 kg can card 37; kg
solve No-theRed
006 card ) problems?
following
37 kg 36 kg
Family problems Red/Green a) How
97×80 many b) kilograms
97×800 005
approximately
c) 97×8000 in one
d) bag?
970×8000 results
students
recognize
seriesof
students these
can
of problems
recognize
analogous
numbers
can find the problems and are
Family cards (Yes- Do Green you card ; No- Red card ) are aware of the
(CAT-1)approximately?
(CAT-1) cards
Red/Green HereBag you1think
seeBag the2bags
five answerBagwith is
3 a30, 40 4or 35?
Bag
different Bag 5
weight. aware
analogous
Assessing ofproblems
the relationship
whether andof a among the
problems cards (Yes- Green card approximate
relationship average
among the
How many
(CAT-2) Here How
you
a) 97×80 many
see
33 kilograms
kgab)multiplication
97×800
34 kg ; No- 37 Red
approximately
problem.
c) kg97×8000card 37 )Youkgin do one
36not
d) bag?
kg have to
970×8000 results ofthe
these
cards
Red/Green calculateDo you thethink
result. the You answer
only
006
005 is 30,to40fillorin35? results
are
students aware
series
of these of of
can find
numbers
problemsthe problems
(CAT-1)
approximately? Bag 1 Bag 2 Bag 3haveBag 4 the blanks.
Bag 5 Assessing whether
How many cards (Yes-
Here Green
you see card five ; No-
bags Redwith carda )
different weight.
relationship
approximate among
averagethe of a
(CAT-2) 24×53 means
33 kg that
34 kg you have
37 kg to calculate
37 kg 36 kg Assessing whether
Breaking down Red/Green
approximately? Here you Howsee
Bag
3×4 1
many a multiplication
Bag
kilograms 005
2 3×20 Bag
problem.
006
approximately
3 Bag 4
Youindo
__×4 Bag
one
5
notbag?
have to students
results series cannumbers
of these
of find the
problems
cards a)
calculate Do the
youresult.and the
think You only have
answer and
is 30, to 40 fill in
or the
35? and __×20
blanks. students
approximate understand
averagestudents how
of a
Howamany Here Assessing whether can find the
multiplication
(CAT-2) Worksheet 33 kg you34 see kgfive 37 bags kg with37a kg different weight. Assessing whether
How many Red/Green
36 kg
approximately? 24×53
b) Here (Yes-
How
4×__ meansGreen
many and card
kilograms
that you4×__; No-
have Red
approximately
toand card__×3
calculate ) in oneand bag?20×50 the multiplication
Assessing
series
approximate of whether
numbers average of a series of
(CAT-3) Red/Green you see a multiplication 006 problem. You do not have to students can find the
Breaking down cards Do
3×4 you think the 3×20answer is 30, 40 or 35? algorithm works
approximately?
(CAT-2) a) calculateBag 1 the and result.
Bag 2 20×__ YouBagonly
3 and haveBag __×4 4to fill Bag
in the5and __×20
blanks. students
Assessing understand numbers
whether how
a multiplication
How many cards
Worksheet c) 20×50 (Yes- Green andcard ; No- Red and card 4×50 ) and __×3 approximate average of a
(CAT-2) Red/Green Here 24×53 33 kg
you
4×__ means
see a34 kgthat
multiplication
4×__ 37 006
you have
kg problem.to
37calculate
kg__×3 You36dokgand not have to
20×50 the
students multiplication
Assessing can find whetherthe
b) and and series of numbers
(CAT-3)down
approximately?
Breaking calculate Bagthe 1 Bag 2You Bag only3 have Bag to 4fill __×4in Bag
the 5blanks.
cards a) 3×4 result. and 3×20 and and __×20 approximate algorithm
students works ofhow
average
understand a
a (CAT-2)
multiplication c) 20×50
Worksheet Here you kg aand
33 see 34 kg 20×__
multiplication 37 kg007 and37 kg4×50
problem. You 36 do kgnotand __×3
have to
24×53 means that you have to calculate Assessing
series
the whether
of numbers
multiplication
Breaking down b)
calculate 4×__
the result. and You 4×__
only 006
have and
to fill in __×3
the blanks.and 20×50
(CAT-3) You are
a) 3×4 NOT allowed and to 3×20 calculateand how many __×4pencils and __×20 students understand how
a multiplication Worksheet 24×53 there are in onethat box.you Canhave you still find out 4×50 how many algorithm works
Breaking down a c) 20×50
means and 20×__ 006
007to calculate
and and __×3 Assessing whether
Breaking down pencils
Here
b) you 4×__there are
see aand in the
multiplication following
4×__ problem. boxes?
and __×3 You do not andhave20×50
to the multiplication
Assessing whether students understand
(CAT-3) a) 3×4 and 3×20 and __×4 and __×20 students Assessing understand whether
workshowalgorithm works
multiplication Worksheet (Yes- Green card ; No- Red card )
calculate
You are NOT the result.
allowedYou only have
to calculate how to many
fill in the blanks.
pencils
aCompleting
multiplication the Worksheet 20×50 20×__ 4×50
howalgorithm
the multiplication
(CAT-3) Red/Green b) c)
there
Number
are in oneand
of boxes
box. Can you stilland
6 have 11problem.
find out
12 13You17
how many
18and22 __×3
23 students can make use of
the multiplication
Here
24×53 4×__
youmeanssee and 4×__
a multiplication and
to007 __×3 do and
not 20×50
have to
ratio table
(CAT-3) pencils there arethat in the you following calculate
boxes? Assessing whether
Breaking down cards calculate thecardresult. You only have ) to fill in the blanks. the ratio table
algorithm works by
(CAT-4) the
(Yes-
a) Number
You Green
3×4
20×50 areofNOT pencils
and ; No-
allowed 20×__
Red
72to
3×20 card
132
calculateand how __×4
4×50 many pencils __×20
and __×3 Assessing
students understandwhether how
c) and and and
aCompleting
multiplication Worksheet
Red/Green 24×53 there means
Number
are in one
ofthere
boxes
thatbox. youCan have you
007
6 following
tostill
11 12boxes?
calculate
find out how many
13 17 18 22 23
reasoning
Assessing
students canhorizontally
whether
make use of
Breaking
ratio down
table b) 3×4 pencils
4×__ are
and 3×20 in the
4×__ and __×4 __×3 20×50
and __×20 the multiplication
(CAT-3) cards a)You are NOT
(Yes-
Number Green and
allowed
of pencils card ; No- to
72calculate
and
132cardhow
Red ) many pencils and students the understand
ratio
Assessing table byhow
whether
a multiplication
(CAT-4) the Worksheet
Completing algorithm works
c)there are in one
20×50 and box. 20×__ Can you 007 still
008 and find out 4×50 how many __×3
and 20×50 the multiplication
reasoning horizontally
Completing (CAT-3)
the Red/Green b) pencils4×__
Number thereofare andboxes
in the 4×__ 6
following and
11 boxes? 12__×3 13 17and 18 22 23 students
Assessing can make students
whether use of can make
ratio table Red/Green You(Yes- areGreen
Are NOT allowed
the statements
card ; No- to calculate card how
correct?
Red ) many pencils
algorithm
Assessing works
whether
ratioCompleting
table(CAT-4) the cards cards there
c) Number
20×50
are in of
one andpencils
box. 20×__
Can 72
you 132
still andfind out4×50 how many and __×3 use theof ratio
the table
ratio by
table by reasoning
Red/Green pencils (Yes- thereGreen cardthe ;following
No- Red 008 card ) students can make use of
(CAT-4) Number of are
boxes in 6 007
11 boxes?
12 13 17 18 22 23 reasoning horizontally
horizontally
ratio
Bigger table
or (Yes- Green card ; No- Red card ) Assessingwhether
Assessing whether
Completing cards
Red/Green Are the ofstatements correct? the ratio table by
(CAT-4)the
YouNumberareThe
a) NOT pencils
allowed of
product to72 132is how
calculate
24×45 smaller manythan pencils1200.
smaller? Red/Green there 007 studentscan
students canmake estimate use theof
ratio table cards Number b)are
(Yes- The in boxes
of
Green one
product box.
cardCan ;6No-
of you
18×3211
Red still 12find
card
is008 bigger out
13 ) than how
17 many18 22 23
600. reasoning horizontally
(CAT-5) cards pencils there are in the following boxes? product
the ratiobytable reasoning
by
Bigger or You are NOT
Number Are ofthe allowed to
pencilsstatements 72 calculate
132
correct? how many pencils Assessing whether
(CAT-4) Red/Green (Yes-
there
Green
a) areThe
card
in product
one box. of
; No-
Can
Red
24×45
card
you still
)
is smaller
find out how thanmany 1200.
Assessing whether
Completing
smaller? the 008 reasoning
students can horizontally
estimate the
Bigger or smaller? Red/Green
cards b) (Yes-
pencils
Number there
The Green
are in the
of product
boxes card ; No-
of following
18×32
6 11009Red
is boxes?
bigger
12 card13 than ) 17 600. 18 22 23 students can make use ofstudents can
ratio tableor Red/Green
(CAT-5)
Bigger (Yes- Green card ;
Are the statements correct?No- Red card ) product Assessing
Assessing
Assessing
whether
by whether
reasoning
whether
Completing
(CAT-5) the cardscards
Red/Green Number of pencils 72 132 the
Assessingratio
estimate the table
whether by
product by reasoning
(CAT-4)
smaller? Red/Green Here a) The product of 24×45
008 is smaller than 1200. students
students cancan make use ofthe
estimate
(Yes-you
Number Green
of see card
boxes multiplication
;6No-11 Red problems
card13
12 ) with 17 their
18 answers.
22 23 reasoning horizontally
ratio table cards Do b)
Are you The
see product
the statements a mistake? of 18×32
correct? 009 is bigger than 600. students can quickly
Bigger
(CAT-5)or cards Number of pencils 72 132 Assessing
the
productratio by whether
table by
reasoning
(CAT-4)
Quick check of Red/Green (Yes-
a) The Green card ;of
product No- Red card
24×45 ) check thecan correctness of
smaller? Red/Green (Yes- Green card ; No- Rediscard smaller ) than 1200. Assessing
students
reasoning whether the
estimate
horizontally
cards Hereb) Theyou product
see multiplicationof 18×32 008 problems
is bigger with thantheir600.answers. Assessing whether students can quickly
Bigger
Quick check answers
of or
(CAT-5) Red/Green 50×200=1000
Do you see a mistake? 37×34=1250 009 23×38=874 Assessing
students
product the whether
result
can of
quickly
bycorrectness
reasoning of the result of
Red/Green
cards a)AreThe theproduct
statements of 24×45 correct? is smaller than 1200. checkcan the
answers smaller?
Quick (CAT-6)
check of (Yes- Green card ; No- Red 008card ) students
multiplication
check estimate
the correctness
Assessing problems
whether the
of
cards
cards
Red/Green b)
(Yes- The
Here product
Green
you see card of ; 18×32
No- Red
multiplication is bigger
card
problems ) thanwith 600.their answers. multiplication problems without
(CAT-6) (CAT-5)
answers Are the statements
Do you see a mistake? correct? 009 product
without the
students by reasoning
performing
resultcan ofquicklythe
Bigger or cards 50×200=1000 37×34=1250
010 23×38=874 Assessing performing whether the algorithm
(CAT-6)
Quick
smaller? check of Red/Green a)(Yes-
(Yes- TheGreen
Here Green
product
you seecard
card of; No-
;24×45
multiplication
No-Red Redis cardsmaller
card) ) than 1200.
problems with their answers.
Assessing
multiplication
check
students
algorithm
can whether
problems
the correctness
estimate theof
Bigger or Red/Green
cards b) The product of 18×32 009 is bigger than 600. Assessing
students canwhether
quickly
answers
(CAT-5) Red/Green Do you see
50×200=1000 a mistake? 37×34=1250 23×38=874
without
product performing
the result
by whether
reasoningof the
cards a)
(Yes-youTheGreen product of 24×45 010is smaller than 1200. Assessing
smaller?
Quick check of
(CAT-6) Here see card ; No- Redproblems
multiplication card ) with their answers. students
check thecan
multiplication estimate
correctness
algorithm problemsthe
of
cards
Red/Green b) The product of 18×32 is bigger than 600. students canreasoning
quickly
(CAT-5)
answers Do you see a mistake? product theby result of
50×200=1000 009
37×34=1250 without
Assessing
23×38=874 performing
whether the
Quick check of cards (Yes- Green card ; No- Red card
010) check the correctness
Assessing whether of
students can use the
(CAT-6) Red/Green multiplication
Assessing
students algorithm
can problems
whether
use the
answers
Fruit language Here you see multiplication problems
009 with their answers. the performing
associative result ofdistributive
andquickly property of
cards 50×200=1000 37×34=1250 23×38=874 without
students can the
Worksheet Do you see a mistake?
010 associative
Assessing and
whether
(CAT-6)
(CAT-7)
Fruit language (Yes-
multiplication
multiplication
Assessing problems
whether to restructure a
Quick check of Worksheet Here Green
you seecard ; No- Red card
multiplication )
problems with their answers. check algorithm
the
distributive
students correctness
canproperty
use the of
of
(CAT-7) Red/Green Do you see a mistake? without
students multiplication
performing
can quickly the problem
answers 50×200=1000 010
37×34=1250 23×38=874 the result
multiplication
associative
Assessing of
and
whetherto
Quick
Fruit check of
language cards (Yes- Green card ; No- Red card ) check the algorithm
correctness of
(CAT-6)
Educ. Sci. 2019, 9, x FORRed/Green
PEER REVIEW
Worksheet multiplication
restructure
distributive
students problems
property
can a19 ofof21
answers
(CAT-7) 50×200=1000 37×34=1250 23×38=874 the result ofuse the
cards without performing
multiplication
Assessing
multiplication
associative to the
problem
whether and
(CAT-6)
Fruit language 010 multiplication problems
Worksheet algorithm
students
Assessing can
restructure
distributive use
whethera
propertythe of
(CAT-7) Here you see a multiplication problem. You have to solve it without
Assessing performing
whether the
010
WITHOUT using the algorithm.011 Write down how you solved it.
Assessing
associative
students
multiplication havewhether
multiplication aand
deep
problem tostudents have a
Fruit language students algorithm
can use the
Solving problems
Solving Worksheet deep
distributive
understanding understanding
property
restructure of thea of of the
without using(CAT-7)
the In this way I solved the problem Answer associative
multiplication and
operation and whether
problems
Fruit language Worksheet 011 multiplication
multiplication
Assessing
multiplication whether to
operation
problem
algorithm Worksheet 59×62 they have,
distributive instead of
property of the algorithm,
without
(CAT-7) using Worksheet and restructure
whether
students can useahave,
they the
(CAT-8) other strategies
multiplication
Assessing whether toavailable to solve
the algorithm 011 multiplication
instead of the
associative problem
algorithm,
and
Fruit language multiplication
restructure
students can use a the problems
(CAT-8) Worksheet other strategies
distributive available
property of
(CAT-7) multiplication
associative problem
and
Fruit language 011
012 to solve multiplication
multiplication to
Worksheet distributive
problemsproperty of
(CAT-7) restructure a
References 011 multiplication to
multiplication problem
restructure a
References
1. Shavelson, R.J. The Basic Teaching Skill: Decision Making (R & D multiplication
Memorandum No.104); problem
Stanford University:
011
1. Shavelson, R.J. The Basic Teaching Skill: Decision Making (R & D Memorandum No.104); Stanford
Stanford, CA, USA, 1973.
University: Stanford, CA, USA, 1973. 011
2. Parkes, J. Reliability in classroom assessment. In Sage Handbook of Research on Classroom Assessment;
2. Parkes, J. Reliability in classroom assessment. In Sage Handbook of Research on Classroom
McMillan, J.H., Ed.;
Assessment; Corwin
McMillan, J.H.,Press: Thousand
Ed.; Corwin Press: Oaks, CA,Oaks,
Thousand USA,CA,2013;
USA,pp. 107–123.
2013; Available online:
pp. 107–123.
http://sci-hub.tw/10.4135/9781452218649.n7 (accessed on 11 March 2019).
Available online: http://sci-hub.tw/10.4135/9781452218649.n7 (accessed on 11 March 2019).
3. Harlen, W. Assessment of Learning; Sage: London, UK, 2007. Available online: http://sci-
hub.tw/10.4135/9781446214695 (accessed on 11 March 2019).
4. Shepard, L.A.; Penuel, W.R.; Pellegrino, J.W. Using learning and motivation theories to
coherently link formative assessment, grading practices, and large-scale assessment. Educ. Meas.
Issues Pract. 2018, 37(1), 21–34, doi: org/10.1111/emip.12189.
5. Black, P.; Wiliam, D. Assessment and classroom learning. Assess. Educ. Princ. Policy Pract. 1998,
Educ. Sci. 2019, 9, 150 18 of 19

3. Harlen, W. Assessment of Learning; Sage: London, UK, 2007; Available online: http://sci-hub.tw/10.4135/
9781446214695 (accessed on 11 March 2019).
4. Shepard, L.A.; Penuel, W.R.; Pellegrino, J.W. Using learning and motivation theories to coherently link
formative assessment, grading practices, and large-scale assessment. Educ. Meas. Issues Pract. 2018, 37(1),
21–34. [CrossRef]
5. Black, P.; Wiliam, D. Assessment and classroom learning. Assess. Educ. Princ. Policy Pract. 1998, 5, 7–74.
[CrossRef]
6. Brookhart, S.M. Classroom assessment: Tensions and intersections in theory and practice. Teach. Coll. Rec.
2004, 106, 429–458. [CrossRef]
7. De Lange, J. Framework for Classroom Assessment in Mathematics; NICLA; WCER: Madison, WI, USA, 1999.
8. Shepard, L.A. The role of assessment in a learning culture. Educ. Res. 2000, 29(7), 4–14. [CrossRef]
9. Stiggins, R.; Chappuis, J. Using student-involved classroom assessment to close achievement gaps. Theory
Pract. 2005, 44(1), 11–18. [CrossRef]
10. Wiliam, D. Keeping learning on track: Classroom assessment and the regulation of learning. In Second
Handbook of Research on Mathematics Teaching and Learning; Lester, F.K., Ed.; Information Age Publishing:
Greenwich, UK, 2007; pp. 1053–1098.
11. Schoenfeld, A.H. Summative and formative assessments in mathematics supporting the goals of the common
core standards. Theory Pract. 2015, 54, 183–194. [CrossRef]
12. Lin, P.J. Conceptualizing teachers’ understanding of students’ mathematical learning by using assessment
tasks. Int. J. Sci. Math. Educ. 2006, 4, 545–580. [CrossRef]
13. Panizzon, D.; Pegg, J. Assessment practices: Empowering mathematics and science teachers in rural
secondary schools to enhance student learning. Int. J. Sci. Math. Educ. 2008, 6, 417–436. [CrossRef]
14. Gallego-Arrufat, M.J.; Dandis, M. Rubrics in a secondary mathematics class. Int. Electron. J. Math. Educ.
2014, 9, 73–82.
15. Jin, H.; Wong, K.Y. Mapping conceptual understanding of algebraic concepts: An exploratory investigation
involving grade 8 Chinese students. Int. J. Sci. Math. Educ. 2015, 13, 683–703. [CrossRef]
16. Leahy, S.; Lyon, C.; Thompson, M.; Wiliam, D. Classroom assessment: Minute-by minute and day by day.
Educ. Leadersh. 2005, 63(3), 18–24.
17. Andersson, C.; Palm, T. The impact of formative assessment on student achievement: A study of the effects
of changes to classroom practice after a comprehensive professional development programme. Learn. Instr.
2017, 49, 92–102. [CrossRef]
18. Keeley, P.; Tobey, C.R. Mathematics Formative Assessment: 75 Practical Strategies for Linking Assessment,
Instruction, and Learning; Corwin Press: Thousand Oaks, CA, USA, 2011.
19. Veldhuis, M.; Van den Heuvel-Panhuizen, M. Supporting primary school teachers’ classroom assessment
in mathematics education: Effects on student achievement. Math. Educ. Res. J. 2019. Available online:
https://link.springer.com/article/10.1007/s13394-019-00270-5 (accessed on 3 June 2019).
20. Wiliam, D. Embedded Formative Assessment; Solution Tree: Bloomington, IN, USA, 2011.
21. Wylie, E.C.; Lyon, C.J. The fidelity of formative assessment implementation: Issues of breadth and quality.
Assess. Educ. Princ. Policy Pract. 2015, 22, 140–160. [CrossRef]
22. James, M.; McCormick, R. Teachers learning how to learn. Teach. Teach. Educ. 2009, 25, 973–982. [CrossRef]
23. Phelan, J.; Choi, K.; Niemi, D.N.; Vendlinski, T.; Baker, E.L.; Herman, J. The effects of POWERSOURCE©
assessments on middle-school students’ math performance. Assess. Educ. Princ. Policy Pract. 2012, 19,
211–230. [CrossRef]
24. Davidson, A.; Herbert, S.; Bragg, L.A. Supporting elementary teachers’ planning and assessing of
mathematical reasoning. Int. J. Sci. Math. Educ. 2018. [CrossRef]
25. Martino, A.M.; Maher, C.A. Teacher questioning to promote justification and generalization in mathematics:
What research practice has taught us. J. Math. Behav. 1999, 18, 53–78. [CrossRef]
26. Heritage, M.; Heritage, J. Teacher questioning: The epicenter of instruction and assessment. Appl. Meas.
Educ. 2013, 26, 176–190. [CrossRef]
27. Zhang, D.; Lee, P.Y. Examination culture and mathematics teaching. In Proceedings of the ICMI–China
Regional Conference of Mathematical Education, Beijing, China, 5–8 August 1991.
28. Li, Y. Purpose, function and types of mathematics assessment in China. ZDM 2000, 32, 192–200. [CrossRef]
Educ. Sci. 2019, 9, 150 19 of 19

29. Ministry of Education of the People’s Republic of China (MoE). Jichu jiaoyu kecheng Gaige Gangyao (Shixing)
[Curriculum Reform Outline of Basic Education (Trial Version)]. Available online: http://old.moe.gov.cn/
publicfiles/business/htmlfiles/moe/s8001/201404/167343.html (accessed on 11 March 2019).
30. Ministry of Education of the People’s Republic of China (MoE). Quanrizhi Yiwu Jiaoyu Shuxue Kecheng Biaozhun
(Shiyangao); [Mathematics Curriculum Standards of Nine-Year Compulsory Education (Trial Version)];
Beijing Normal University Press: Beijing, China, 2001.
31. Ministry of Education of the People’s Republic of China (MoE). Yiwu jiaoyu shuxue kecheng biaozhun
(2011 nian ban) [Mathematics Curriculum Standards of Nine-Year Compulsory Education (2011 Version)].
Available online: http://ncct.moe.edu.cn/2014/curriculumPolicy_1115/3175.html (accessed on 11 March 2019).
32. Cai, J.; Wang, T. Conceptions of effective mathematics teaching within a cultural context: Perspectives of
teachers from China and the United States. J. Math. Teach. Educ. 2010, 13, 265–287. [CrossRef]
33. Cai, J.; Ding, M.; Wang, T. How do exemplary Chinese and U.S. mathematics teachers view instructional
coherence? Educ. Stud. Math. 2014, 85, 265–280. [CrossRef]
34. Zhu, Y.; Yu, W.; Cai, J. Understanding students’ mathematical thinking for effective teaching: A comparison
between expert and nonexpert Chinese elementary mathematics teachers. Eurasia J. Math. Sci. Technol. Educ.
2018, 14, 213–224. [CrossRef]
35. Li, Y.; Chen, X.; Kulm, G. Mathematics teachers’ practices and thinking in lesson plan development: A case
of teaching fraction division. ZDM 2009, 41, 717–731. [CrossRef]
36. Liu, J. Xiaoxue Dinianji Shuxue Jiaokeshu Zhong De Xiti Yanjiu—Jiyu Zhongguo Xinjiapo Jiaokeshu Bijiao De
Shijiao; [Study in Mathematics Textbook Tasks for Lower Grades in Primary School—From the Perspective
of Comparison Between Textbooks in China and that in Singapore]. Master’s Thesis, Northeast Normal
University, Changchun, China, 2012.
37. Gu, F.; Gu, L. Characterizing mathematics teaching research specialists’ mentoring in the context of Chinese
lesson study. ZDM 2016, 48, 441–454. [CrossRef]
38. Zhao, X.; Van den Heuvel-Panhuizen, M.; Veldhuis, M. Chinese primary school mathematics teachers’
assessment profiles: Findings from a large-scale questionnaire survey. Int. J. Sci. Math. Educ. 2018, 16,
1387–1407. [CrossRef]
39. Zhao, X.; Van den Heuvel-Panhuizen, M.; Veldhuis, M. Teachers’ use of classroom assessment techniques in
primary mathematics education—an explorative study with six Chinese teachers. Int. J. STEM Educ. 2016,
3, 19. [CrossRef]
40. Jiangsu Phoenix Education Publishing House. Sujiaoban Jiaokeshu; (Xiaoxue Shuxue Sannianji Xiace)
[Sujiaoban Textbook (Mathematics Textbook for Grade 3 in Primary Education, Volume 2)]; Jiangsu Phoenix
Education Publishing House: Nanjing, China, 2014.
41. Heritage, M.; Kim, J.; Vendlinski, T.; Herman, J. From evidence to action: A seamless process in formative
assessment? Educ. Meas. Issues Pract. 2009, 28(3), 24–31. [CrossRef]
42. Veldhuis, M.; van den Heuvel-Panhuizen, M. Primary school teachers’ assessment profiles in mathematics
718 education. PLoS ONE 2014, 9, e86817. [CrossRef]

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).