Part 6.

Formative Evaluation Plan

6a. Expert Review
The subject matter expert (SME) for this project is Sherilyn Skiba. Sheri was chosen
because she has been teaching AP Statistics for ten years at Eastlake High School in
Libertys neighboring school district, Lake Washington SD. Sheri had already been used
as a resource for Libertys start up AP Statistics Course to help determine which
textbook would be the best choice. Sheri Skiba was contacted on July 25, 2013 and
responded back almost immediately. The survey associated with the interview of the
SME can be found within section 7a.

6b. One-to-One Evaluation

This evaluation process involves testing out the project materials with several students
before the project is actually assigned to a complete classroom. During the one-to-one
evaluation process, there are specific things the designer is looking for such that the
project is improved before actual administration. The designer is mainly seeking
feedback regarding formatting and grammar errors. In addition, the designer needs
information about the flow of the project. The designer may wish to ask clarifying
questions of the student, thus there is designer and student interaction.
Inquiry about the following would be essential feedback from each student. To obtain
that feedback, the student would be presented with both a computer version and a
paper copy of the ID project such that they could write comments. Three or four
students would be chosen to complete the one-to-one evaluation process. Selection of
a variety of students would be ideal. Each student would be interviewed individually so
that previous student comments do not influence other student comments (Smith,
2005).
1. Identify any instructions that are not clear to follow such that you can obtain a
result without guidance.
2. Locate and identify any misspelled words.
3. Will these instructions lead you to designing a box plot without assistance? If not,
please indicate which step needs clarification.
4. If you need instructor assistance to make a box plot, at which point in the
instructions does this occur?
5. When looking at the visuals provided in the project, do you understand what they
all mean? Please list any visuals that do not appear relevant.
6. Please identify any links that do not go to the expected location.

6c. Small Group Evaluation

Throughout this evaluation stage of the ID project, the designer will have a small group
of students, or learners, attempt completion of the project. At this time, all revisions
should have been completed based upon the results of the one-to-one evaluation.
During the small group evaluation process, the designer should refrain from interacting
with the student evaluators. However, if none of the students in the group can make
sense of a step in the project, then the designer may clarify. Likewise the designer
should make note that the step in question needs revision (Smith, 2005).
During this evaluation process, four students would be selected for analysis of the AP
Stats ID project. Students would work in a group of four to see if they could complete
the activity while the designer observes and only interacts if no one in the group can
make sense of a particular instruction. At that time, students should be instructed to call
over the designer if no one in the group can determine the intention of the instructions.
For this particular AP Stats project, essential questions to refer to the students include:
1. Does the flowchart lead you to a box plot that you can make with paper and
pencil? If not, which steps are confusing?
2. Does the video for the calculator instructions lead you to a box plot result?
3. Does the video for the excel instructions lead you to a box plot result?
4. Which steps were unclear to more than one person in the group?
5. How should that instruction change to become more clear?
6. Did all students in the group have the skills necessary to complete the project? If
not, then what was the cause of the issue that didn't allow student completion?
7. Will these instructions lead you to designing a box plot without assistance? If not,
please indicate which step needs clarification.
8. Please identify any links that do not go to the expected location.

6d. Field Trial

The field trial process is essential to determine if all revisions from the one-to-one and
small group evaluations were corrected to achieve a positive result. Throughout the field
trial, the designer should use a larger number of students to conduct the trial. At this
time the project should actually be presented in the environment where regular
instruction occurs (Smith, 2005).
1. Will these instructions lead you to designing a box plot without assistance? If not,
please indicate which step needs clarification.
2. Please identify any links that do not go to the expected location.
3. Are all the students successful in all objective categories?
4. Does the video for the calculator instructions lead you to a box plot result?
5. Does the video for the excel instructions lead you to a box plot result?
6. How long did it take each student to complete the activity?
7. Is there anything the instructor noticed that need revision?
8. Please add any additional comments about this project.

2|E d T e c h 5 0 3 : I n s t r u c t i o n a l D e s i g n P r o j e c t

Part 7. Formative Evaluation Report

7a. Evaluation Survey or Rubric
The questions were designed such that a yes response means the SME agrees with the
designer about that particular aspect of the project and determines success. The point
scale then determines ID Project success with a higher score. The SME will be asked to
rate each response on a scale of 1-5 as shown below.
1 no, I completely disagree
2 somewhat disagree
3 unsure
4 somewhat agree
5 yes, agreed
The following are the list of questions that were presented to the SME, Sheri Skiba.
1. Are these skills AP Statistic students need to have?
2. Are the examples and practice exercises relevant to this course?
3. Is the instruction provided sufficient for students to learn this topic in an online
venue?
4. Do you believe the instructional tools are consistent with how an AP Statistics
student can learn?
If you do not answer a 5 for any particular question, then can you please comment on
how improvement can be made?
Question
1. Are these skills AP Statistic students need to have?

2. Are the examples and practice exercises relevant to

this course?
3. Is the instruction provided sufficient for students to
learn this topic in an online venue?
4. Do you believe the instructional tools are consistent
with how an AP Statistics student can learn?
3|E d T e c h 5 0 3 : I n s t r u c t i o n a l D e s i g n P r o j e c t

Rating/Comments

7b. Expert Review Report

Question
1. Are these skills AP
Statistic students need to
have?

Rating/Comments
4/ The actual creating of the graphical display is a very small
part of the purpose of this topic. The AP exam focuses much
more on the analysis. Box plots are particularly useful when
comparing two sets of data and are rarely used in isolation.
Often on the AP Exam students are actually given several box
plots and expected to be able to discuss and compare spread,
center, shape and outliers. They should be able to discuss in
detail what the quartiles represent and use that information to
compare sets of data. They should also know how to create a
modified box plot which clearly shows outliers. The Excel and
hand methods are enrichment activities as students will most
likely create the box plots with a calculator on the exam.
2. Are the examples and 4/ As noted in the comment above, the focus should not be on
practice exercises relevant the mechanics of creating the graphical display but on the
to this course?
analysis. AP students are very quick to be able to create the
box plots with any of the methods, so I would suggest that you
use at least two sets of data such different seasons or different
teams so they focus on the important analysis skills. It is also
important for them to know when it is best to use box plots and
when a different graphical display would be more appropriate.
3.
Is
the
instruction 5
provided
sufficient
for
students to learn this topic
in an online venue?
4. Do you believe the 4\ I think spending this much time on mechanics isnt
instructional
tools
are necessary.
consistent with how an AP
Statistics
student
can
learn?

7c. Designer's Response of Review

Based on the above feedback from the SME, it is good to know that the Instructor Guide
was successful in clearly identifying the steps needed to complete the activity. The SME
review however, clearly noted that this much time on such a small aspect of the course
is not necessary. Moreover, it may not have been clear to the SME that this activity is
intended to occur at the beginning of the AP Statistics course as perhaps a preassessment activity to prepare students for the statistics course. Another use of this
activity could be for remediation for any student that has not learned about box plots in
other courses or has forgotten how a box plot is created. Clear knowledge of the
creation and analysis of a box plot is necessary for future activities in AP Statistics such
that students use these ideas for statistical inference later in the course.
4|E d T e c h 5 0 3 : I n s t r u c t i o n a l D e s i g n P r o j e c t