Documente Academic
Documente Profesional
Documente Cultură
Ronen Cohen
Boise State University
Educational Technology
Course 505: Evaluation for Educational Technologies
Fall 2014
Summary
Another International School has been in existence since 2008. The Student
Report system has changed several times since then (there is no documentation of the
changes), and with the arrival of a new curriculum coordinator (in August 2013) and a
new Primary Principal (in August 2014), further modifications have been introduced.
The Student Report system includes student evaluations from both homeroom and
single-subject teachers.
The goals of this evaluation were (1) to determine if the program successfully
achieved the goals and objectives as outlined in the schools Reporting Guidelines
(Appendix A), and (2) to identify the perceptions and experiences of stakeholders
(teachers, parents, and administrators) so that they inform future modifications to the
Student Report system.
Drawing on the programs goals and objectives, the evaluator met with various
stakeholders and sent out a survey to determine what different stakeholders would like
the evaluation to focus on. Once these were identified, the evaluator met with
stakeholders once more and collaboratively brainstormed evaluation questions, which
drove the evaluation. Questions were then collated and agreed upon by the evaluator
and the Primary Principal and Curriculum Coordinator. In order to solicit both
quantitative and qualitative data, the evaluator designed online surveys based on the
evaluation questions for the three distinct groups of stakeholders (administrators,
teaching staff, and parents). Once completed and collected, data was then analyzed.
As mentioned above, the Student Report program at Another International School has
gone through frequent changes in the past. However, it is worth noting that the new
leadership demonstrated a commitment to assessing current practices and designing
and implementing long-term procedures. Based on survey responses, the
administrators possess a clear idea about what a comprehensive assessment and
reporting policy includes, and already begun making amendments to past practices,
introducing new assessment and reporting documents to create clearer and more
standardized systems. This evaluation revealed that in general, parents view the
Student Report as a clear and comprehensive document. This being said, the Student
Report system only partially meets the program goals.
Some of the concerns raised by different stakeholders indicate that in order to
align the Student Report program with the programs goals and objectives, more work
needs to be done. More assessment and reporting documents need to be created,
published, shared with all stakeholders, and implemented. Curricular work must
continue in order for the school to provide stakeholders with measurable outcomes and
conceptual understandings, and to include them in the Student Reports. Streamlining
reporting practices would allow for more objective and consistent products. Due to
parent responses about ambiguity and lack of substantive information in single subject
teachers reports, it would be beneficial to further examine these reports and explore
how they can portray a more accurate and comprehensive picture of students
achievement and progress. Finally, a thought should be given to how the Student
Report can better complement conferences and goal setting as well as the procedures
and practices that are already in place.
The following table displays the reporting and conferencing schedule for the
academic year 2014-2015 as outlined in the Reporting and Conference Timeline
document:
Date
11/08/14
30/09/14
6/10/14
30/01/15
4/02/15
10/04/15
29/04/15
18/06/15
Event
Beginning of academic year
Mid-Semester One Report
(literacy, mathematics, personal/social development, and
first unit of inquiry)
Student/Parent/Teacher Conference
Semester One Report
(literacy, mathematics, personal/social development, and
second and third units of inquiry)
Student/Parent/Teacher Conference
Mid-Semester Two Report
(fourth unit of inquiry)
Student-led Conference
Semester Two Report and last day of academic year
(literacy, mathematics, personal/social development, and
fifth and sixth units of inquiry)
Evaluation Method
Participants
In order to identify if the programs goals and objectives have been met, three
groups of stakeholders were selected to be included in the Student Report evaluationadministrators, teachers, and parents. The first group included the Primary School
Principal and the PYP coordinator. The role of these administrators is to design and
oversee the Student Report program. The second group consisted of 14 Primary School
homeroom teachers and 24 single subject teachers. The teachers role, aside from
teaching, is to report on each of their students progress. The third group was composed
of 119 sets of parents of Primary School students. The group was chosen because of
their role as clients, and their experiences and perceptions were of major importance to
understanding the effectiveness of the program.
Procedures
The Student Report evaluation idea was first introduced to the administrators,
who gave their consent for it to take place. The evaluator and administrators then met in
person and collaborated online to extract and finalize the programs goals and
objectives. Due to the short timeline for the evaluation, the evaluator and the Primary
School Principal arrived at a decision to narrow the scope of the evaluation to include
the three abovementioned stakeholder groups, and design one survey per group. Due
to the small number of stakeholders in each group, no population sample was selected.
All members of the three groups were asked to fill out a population-specific online
survey created on Google Forms.
The evaluator introduced the evaluation to teachers during a staff meeting and
conducted a short session with the goal of ensuring teachers are aware of the program
goals and to brainstorm evaluation questions to guide the evaluation. Teachers were
also asked to fill out a survey to indicate what they would like the evaluation to focus on.
The evaluator collated responses, and created evaluation questions. After the
administrators approval, the three surveys were sent out to administrators, teachers
and students parents. The evaluation took place between reporting sessions and had
no perceived impact on teaching and/or learning.
Data Sources
As mentioned in the Procedures section of this report, the evaluation included three
surveys, which were directed toward the three identified stakeholders at the schooladministrators, teachers and parents. Each of the surveys included three sections:
1. A general information section, which was composed of questions designed to
identify and sort through the population (number of years at the school, grade level
involvement, etc.);
2. A main section, with a number of questions in regards to the identified programs
goals and objectives; and,
3. A final section, which asked respondents to identify the programs strengths,
weaknesses, and asked for ideas for its improvement.
Surveys were created using Google Forms and included a combination of open-ended
questions (paragraph responses), rating questions (one answer responses), and ticking
questions (multiple option responses).
Results
Surveys
The primary data for the evaluation of Another International School comes from
the three online surveys taken by the schools administrators, teachers and parents.
Survey questions were related to the evaluation questions, which came from the
identified programs goals and objectives. Questions for each stakeholder intended to
identify practices and perceptions (as explained below), and questions across the three
surveys were designed to check and identify gaps between (1) administrators
perceptions and teachers understanding and practice, and between (2) teachers
practice and parents perceptions.
Administrator Survey
The survey was sent to the programs administrators electronically. Both
administrators (100%) completed the survey. Questions on the survey were quantitative
and qualitative, and related to the programs goals and objectives. They were designed
to solicit their opinion of the programs alignment with the Board of Directors policies
indicated the following: 16 out of 20 (80%) set up goals with their students and indicate
a 40% difference from administrators responses, 11 out of 20 (55%) set up meeting
with parents (a negligible 5% difference), and 15 out of 20 (75%) adjust instruction and
provide additional support based on the information they included in their reports (25%
difference and a significant 45% difference respectively). In order to compare responses
from administrators and teachers, the average administrators percentage was used.
For example, the percentage rate for meet with parents (50%) was achieved at using
the mean of 31% (the lowest percentage) and 91% (the highest percentage).
Parent Survey
The survey was sent to parents electronically, with a note from the PYP
coordinator briefly explaining what the evaluation is about. 31 out of 119 families (27%)
filled out the survey. No reminders were sent after the initial e-mail. The questions on
the parent survey were intended to solicit information about how they use the Student
Report as well as to identify their level of satisfaction with it.
Figure 5 in Appendix B, Survey questions: Is the report accurate and
comprehensive?, relates to the programs third objective and synthesizes stakeholders
responses about the accuracy and comprehensiveness of reporting about students
academic, social and personal development. The percentage scale used in the chart
indicates respondents level of agreement to the statements. Respondents were given a
rating scale from strongly disagree to strongly agree as well as a not applicable (NA)
option. The NA option was provided for stakeholders who are new at the school and at
the time of taking the survey had not yet received a report. In the analysis of the
responses, NA responses were omitted and percentage was re-calculated to reflect the
new parts of wholes. In this chart, responses are grouped into either agree (composed
of agree and strongly agree responses) or disagree (disagree and strongly disagree),
whereas 100% in the chart represents the entire group responding with agree and
strongly agree.
Discussion
The purpose of this formative evaluation was to determine whether Another
International Schools Student Report system successfully achieves the programs
objectives and to identify the perceptions and experiences of stakeholders so that these
inform future modifications to the Student Report system. The evaluation utilized both
qualitative and quantitative data in the form of online surveys. This report presented and
discussed the evaluation process and the data found and analyzed through the surveys.
Responses from the three surveys suggest that teachers reporting practices are
mostly accurate and comprehensive and that existing procedures and expectations
need to be further developed and implemented in order to properly align the Student
Report system with the programs stated objectives. A discussion of specific results as
they relate to each program objective follows.
Objective #1: Report cards are a part of the schools comprehensive assessment
and reporting policy that is consistent with IBO guidelines and Board of Directors
policies
Project Cost
This evaluation project was conducted by a staff member of Another International
School as part of his graduate studies. No costs were incurred to the school as a result
of this evaluation; therefore the costs below represent the total savings the school would
have spent on an outside evaluator.
Evaluation activities in this section include meeting with different stakeholders at
Another International School, travel costs, time to prepare surveys and analyze the
data, and time for preparation of the final evaluation report.
Daily Rate:
Travel (City Center to the schools location):
Per Diem:
Miscellaneous Supplies:
The school will report student progress to parents and guardians on a timely basis after
each quarter of the school year. A comprehensive plan for school/home
communications is essential if clear understandings are to take place. Therefore, a plan
with combinations of written and oral communications has been devised in order to
insure that clear and concise information is transferred. Teachers will also use a variety
of methods to be sure parents know the progress of each student.
2.
Appendix B: Charts
Figure 1: Administrators level of agreement: Report card policies vs. Board of Directors
policies and IBO guidelines.
Administrators
level
of
agreement:
Consistency
with
guiding
documents
Board
of
Directors
policy
Agree
Strongly
Agree
International
Baccalearuate
guidelines
Administrators
40%
Teacher
20%
0%
Parents
Academic
achievement
Academic
progress
Areas
for
improvement
100%
80%
Administrator
perceptions
60%
Teacher responses
40%
20%
0%
60%
40%
Teachers'
responses
20%
0%
set up goals
meet
with
parents
adjust
instruction
provide
additional
support
Administrators
40%
Teacher
20%
Parents
0%
Academic:
Comprehensive
Academic:
Accurate
Social:
Social:
Accurate
Personal:
Comprehensive
Comprehensive
Personal:
Accurate
(Please note: Narrative responses such as NS, none, not so far, or left blank were omitted)
1.
How
many
years
have
you
been
at
Stonehill?
year
(This
is
my
first
year)
2-3
years
4-5
years
More
than
5
years
2.
What
do
you
believe
a
comprehensive
assessment
and
reporting
policy
includes?
3.
Do
you
believe
the
current
PYP
reporting
policy
is
consistent
with
the
IBO
guidelines?
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
4.
Do
you
believe
the
current
PYP
reporting
policy
is
consistent
with
GC
policy
on
reporting?
Strongly
Disagree
Disagree
Neutral
Agree
5.
How
should
report
cards
support
learning?
6.
Please
rate
the
following
statements
according
to
your
experiences.
Strongly Agree
Never
Rarely
Sometimes
Most
of
the
time
Always
7.
Based
on
your
experience,
how
many
teachers
use
the
information
they
include
in
their
reports
to
do
the
following
0%
-
10%
11%
-
30%
31%
-
50%
51%
-
70%
71%
-
90%
91%
-
100%
8.
Please
indicate
your
level
of
agreement
towards
the
following
statements:
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
N/A
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
N/A
development
is
COMPREHENSIVE.
9.
How
is
academic
development
being
measured
in
the
PYP?
10.
How
is
social
development
being
measured
in
the
PYP?
11.
How
is
personal
development
being
measured
in
the
PYP?
12.
Do
you
believe
reporting
is
consistent
across
the
PYP?
Please
explain
your
answer
13.
What
mechanisms
are
in
place
to
ensure
reporting
is
clear
and
consistent
across
the
PYP?
14.
Are
there
clear
standards
and
benchmarks
for
reporting
at
each
grade
level?
How
do
you
think
this
affects
reporting?
15.
Does
the
school
keep
adequate
records
of
student
learning
and
progress?
Never
Rarely
Sometimes
Most
of
the
time
Always
16.
How
would
you
define
adequate
record
keeping?
17.
What
do
you
think
Stonehill's
PYP
report
card
system's
strengths
are?
18.
What
do
you
think
Stonehill's
PYP
report
card
system's
weaknesses
are?
19.
What
changes/additions
would
you
like
to
see
in
Stonehill's
PYP
report
cards?
------------------------------------------
I don't know
Personal development is referred to explicitly in the PSPE Scope and Sequence which is tracked and assessed
in the Early Years. It is measured throughout the PYP in the report card, but not using PSPE outcomes.
Do you believe reporting is consistent across the PYP? Please explain your answer
Reporting is not consistent. With an absence of articulated, specific curriculum, teacher's and grade level's
expectations are not aligned. The report format lack specific outcomes, skills, understandings and therefore
relies on narratives, which individual teachers approach differently across the school.
No. Different teachers provide varying quality of reports. Some teachers write detailed individual reports that
demonstrate an in-depth knowledge of the child's progress and development, while other teachers write
impersonal generic reports with an over-reliance on 'cut and pate'.
What mechanisms are in place to ensure reporting is clear and consistent across the PYP?
There are guidelines issued at the beginning of the reporting process.
Report guidelines were revised in September 2014 to offer a higher level of consistency with regards to the
writing of comments. Staff meetings have been dedicated to reporting so as to better inform teachers of the
process, timelines and expectations. Peer-editing and revision processes encourage consistency and quality of
comments. The size of the text boxes have been decreased to encourage quality rather than quantity for all
teachers and subjects
Are there clear standards and benchmarks for reporting at each grade level? How do you think this affects
reporting?
There are not clear standards and benchmarks for reporting as these have not been articulated through the
curriculums as yet. This has a significant effect on reporting, again as the majority of the reports are based on
narrative comments from teachers as opposed to clear and specific outcomes.
No. This is in development. Clear standards and benchmarks will improve the quality of reporting.
(Please note: Narrative responses such as NS, none, not so far, or left blank were omitted)
1. How many years have you been teaching at Stonehill?
1 year (This is our first year)
2-3 years
4-5 years
Academic achievement
Student progress
Personal growth
Emotional growth
Social growth
Other
Set up goals
Other
Disagree
Neutral
Agree
Strongly
agree
N/A
What changes/additions would you like to see in the PYP report card system?
Does not directly reflect learning outcomes in scope and sequence documents - same language would be
useful for teachers and parents
Specifics outcomes that are taken directly from the unit planners.
Have more student involvement leading to more ownership of the report. Easier the older the kids get.
Have space/numbers dedicated not just to skills and knowledge, but also attitudes
I would like to see the report card include measurable learning outcomes and the narrative, not just the
narrative.
Reduce the number of times the reports are due? I don't really work with reports so I am guessing at this.
While the report is being prepared it would help if I can view it as it would be visible when it is printed out to
parents. This would give me a clear picture of all the information that goes onto the document.
Make it simpler for teachers to complete Make it more objective (Less teacher personality)
Access to all so that we can read the EAL and subject teachers reports. It can also be useful to read previous
homeroom teachers reports to see if recurring issues are arising.
Fewer report cards per year. With two (or is it three?) scheduled conferences each year, and regular
communication with the parents (with other conferences as needed), I think that two report cards/ year is
adequate. We should drop the interim reports, as we are still in the process of organizing the routines of the
class and assessing the students then.
More outcomes oriented so that it cuts the guess work out of what teachers mean. It would also lessen the
inconsistencies between teachers reporting styles and streamline our reporting process.
I am sure I could say more after seeing a full year of student reports.
Specialist reports are very generic and should have more focused information about the student they are writing
about and not just state what the learning objectives for the lessons have been or make general statements for
the whole grade level. ELL / LS and Homeroom reports should be combined into one report so that the SAME
information is shared with the parent. Two different viewpoints for the same child is not professional for the
school or useful for the parent.
Integration of S&S, standards, or continuum of learning, so that we can mark where kids are and speak to the
specific areas of strength or weakness.
(Please note: Narrative responses such as NS, none, not so far, or left blank were omitted)
1. How many years has your child/children attended Stonehill?
1 year (This is our first year)
2-3 years
4-5 years
2. Please indicate your child's age group If you have more than one child in the PYP, please tick all applicable
options
Early Childhood (P1-P3)
Never
Sometimes
Always
Other
5. Please rate the following statements with regards to the report card.
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
N/A
6. Is there anything else you would like to tell us about the report card, your interaction with it, etc.?
7. What do you think Stonehill's PYP report card system's strengths are?
8. What do you think Stonehill's PYP report card system's weaknesses are?
9. What changes/additions would you like to see in Stonehill's PYP report cards?
-----------------------------------------Is there anything else you would like to tell us about the report card, your interaction with it, etc.?
I think they are far too long...they could be simplified a lot.
The report cards uptill this year have been of no real use to track my child's progress at school. Comments have
been very generic and vague with no real indicators of how my child has been doing. There is more 'fluff' than
stuff in the reports. Comments from special classes ( ICT, Info Litt, Art, Music, Lang B) have all be absolutely
useless with the comments having little to no value towards my child's work and progress at school in these
areas. There needs to be more effort made from these teachers to be more specific and report accurately about
each individual child and not have the parent feel the report is a generic statement that can be found in any
other report for that grade. There are no references made to specific goals or achievements my child makes in
these classrooms.
General observations are fine and on the mark but it can be too fluffy. While I think some of the personal
observations can be insightful there is generally far less depth around academics I like how IIB (and thus report
cards) encourage curiosity and ways of thinking I still don't believe the balance is yet right with fundamentals
(reading, writing, spelling, maths)
Report card together with a parent-teacher converstation is the best in my opiion.
While I found the report card useful for some ares I found personal meeting with Home Teacher and Learning
support teacher more useful.
I love the deatiled report cards and the effort put into them. It is a great feedback for our child and us...
I felt the evaluation of my child in the report card was too general. Individual strengths and weakness did not
become clear enough to me which would have in turn helped more for supporting the child at home.
No, I find it appropriate.
I would like the ratings to be more differentiated to better judge how my kid is doing.
No it's pretty comprehensive
I read it a couple of times and underline the important information.
Occasionally it seems to me that teachers may have cut and pasted between reports because he/she or
his/hers are sometimes wrong.
I think it's good to have one, I always like to receive it and to ask my child to explain what I don't understand.
I would like to know what is the scale of the report is it according the end of the year or to present time of the
child level
This is our first year at stonehill and so far I only got the first intermediate report before the end of the year
report.
Very positive but sometimes a little bit too superficial
What do you think Stonehill's PYP report card system's strengths are?
Looks at the whole child.
Gives information on the UOI, Ib Profiles is clearly seperated in different areas like language, maths,... Clean
and simple overview
I can only talk about the report card from our previous IB school. It has information about how the child is doing
in each area/subject and also social and emotional development. Information well organise by subject.
They cover various specialists which I like
It gives us a link between home and school and it helps to see our children development from every teachers
point of view.
I can't comment much as this is my child's first year at Stonehill and we haven't seen any report card but as far
as I know I have heard mixed reviews.
Very clear and to the point
Very detailed and comprehensive.
To put the achievements of the child first is a good motivator.
Addresses all levels of kids ability
Very structured. Not judgmental for the child, focused on areas of improvements.
NONE but we hope it will change soon. The report card THIS YEAR from the homeroom has been somewhat
better and the credit for that goes to the homeroom teacher.
Each teacher gets to share progress
It's comprehensive information from most teachers, only a few are sparsely written.
It's very personal and the teacher's always provide insightful feedback. Some schools reporting systems are
very generic with very little "Comments" or "Feedback" regarding your specific child.
PYP report card is pretty straight forward. It gives a comprehensive a glance report of my child's performance.
I can't think of one. Except that you ate focusing on it. Good. They need work.
I am very impressed about the report card because most of the teacher look and see each of the child's details
about their strengths and weakness, and reported very accurately.
I like the details that are provided, rather than letter grades
Short and simple
Quite easy to understand
Teacher comments are the piece I pay attention to most.
It is child focused and detailed
The full written statements from the teachers.
Give good feedback about the kid's behavior and development during the year, specially on the social side.
The biggest strengths is it is very comprehensive and explains in details on how the progress is taking place.
What do you think Stonehill's PYP report card system's weaknesses are?
As a parent, I would like to see a breakdown of subjects and areas of improvement clearly specified. I
understand that the children need positive feedback from parents and teachers, but a reality check for parents
would be great!
It does not focus with accuracy the areas or subjects that need to be improved (goals) and what are the guide
lines for parents to help.
I can't comment much as this is my child's first year at Stonehill and we haven't seen any report card.
Lack of reporting regarding second languages, PE, Arts, Science. It's too general
I don't see any
Homeroom teachers reporting sections are enormous, and way to time consuming for the teachers to fill out. A
concise written portion on each subject would be enough
We had a report card with the wrong name in it....personal error? Attendence is always incorrect
Don't Know haven't had one yet.
Could have more examples?
The annotation system is not clear for the parents as well as for the teacher who couldn't answer our questions.
Too much jargon
Too much written. I'd rather systematically have examples of the activities my child does to support the report
cards different parts, and I'd like to have at least half an hour with teachers to talk about it (especially achieved
parts and areas of improvement). In Stonehill, when we have teachers/parents conferences, it is no school and
it's a long coday of conferences for teachers. In other schools, it take place after school during a whole week,
we don't have to rush and teachers take time to really deepen the situation and to share with parents, not just to
talk about what they already wrote in the report card. It is interesting for both parents AND teachers, it's a
chance to know more about the child. Parents would write their availability and it would not be more than 2
interviews per evening for instance. It's important for parents ! It has been frustrating to just have a small slot,
while children are spending so much time at school. I know it's a concession from teachers to their private life
but I think it's part of the job and it's the only opportunity to work together. Good luck !
There are none.
The grading system is sometimes difficult to understand.
It would be good if it could be more extensive on the Academic development.
See above. There are no frameworks or measures or data with which to assess child's results. Neither is there
an indication of progression.
I would like to see some quantitative information. The report cards are fairly generic - and seem like they could
apply to any child.
Behind a lot of sentences sometimes the content gets too complicated.
At times, the sheer size of the report is daunting. (So long!) But I would much rather have this then a reporting
system with little to no information regarding your child's strengths, weakness, feedback etc.
Language is sometimes difficult to understand for second language parents. There is a lot of writing to interpret.
It should explain the weakness in more details and suggest more corrective steps to be taken by the parents.
I don't think the ratings allow me to fully understand how my kid is progressing.
There has been no real indication of how my child is actually doing in class. The comments are all blanket
statements which seem applicable to any child in the classroom. This is especially with rregard to the reports
from the other lessons ( Music, Art, ICT, Info Literacy, Lang B, Host Culture) These must be more specific to
each child. We are not interested in a write up on what the whole grade level has worked on during these
lessons. That is what the blog is for. Instead we want to see more specific information about what MY CHILD is
achieving or not in the class. Parent teacher conferences MUST also include a meeting time with these
specialist teachers. If my child is spending at least an hour a week in these classrooms, it should be a
requirement for the specialist teachers to also meet with parents and conference with them. These lessons are
not optional like an ECA activity and so a report card must have a follow up conference with specialists too.
What changes/additions would you like to see in Stonehill's PYP report cards?
A chance for the child to comment on what they thought of there report
Shortened!
I think they are doing a good job.
More to the point
More differentiation.
Organisation of the parents teachers conferences to talk about the report card.
I would like to see how my child performed in comparison to the other kids. It can be in the form of percentile,
grade or anything better.
More focus for next coming months, areas to develop or to be proud of
I would like to grade each subject with digits not letter.
Set of action (goals) that the child needs improve in each area or subject.
Would like picture of grade to be attached, would like to see kids goals and achievments listed, comparison to ib
learning expectations would also be nice
A completely new on that is SMART
Comprehensive anotation
Specialist Comments MUST be more individualized and not sound like they are the same for all the children.
More detail, less "fluff"