Formative Assessment Items and Tasks in OAS

Phase I of Formative Item Bank

December 5 and 6, 2012

Background on RT3 Assessment Initiatives

Purpose
To provide assessment resources that reflect the rigor of the CCGPS To balance the use of formative and summative assessments in the classroom To promote student learning

RT3 Assessment Initiatives

Development of a three-prong toolkit to support teachers and districts and to promote student learning An assessment literacy professional learning opportunity that focuses on implementation of formative instruction Initial Pilot January 2013 A set of interim benchmarks in ELA, Math, and selected grades for Science and Social Studies plus additional items Initial Pilot Spring 2013 An expansive bank of formative instructional assessment items/tasks based on CCGPS in ELA and Mathematics as a teacher resource - Phase I Release Fall
2012; Phase II release Fall 2013
3

Formative Instructional Practices

Summative Assessment Re-Design Teach Formative Assessment Formative Assessment/ Diagnostic

Common Core State Standards

Design Teach
Formative Assessment

Re-Design Teach

Formative Assessment

Re-Design Teach

Formative Instructional Practices

Conducted during instruction (lesson, unit, etc.) Identifies student strengths and weaknesses Helps teacher determine next steps Review Differentiation Continuation Supplies information to provide students with detailed feedback Assessment for the purpose of improving achievement LOW stakes
5

Purpose of the Formative Item Bank

The purpose of the Formative Item Bank is to provide items and tasks for teachers to use in order to assess students knowledge while they are learning the curriculum. To provide an opportunity for students to show what they know and show teachers what students do not understand To provide teachers good examples of tasks/items that assess the rigor of the Common Core.

Formative Instructional Assessment Items

Assess students conceptual and skill-level understanding Tasks require students to
Apply the content they know to real world problems Express logical reasoning by writing, showing their work and/or explaining their answer

Items/Tasks for Formative Instructional Use

Aligned to CCGPS for Grades 3 - High School ELA & Math Format similar to PARCCs Common Assessments prototypes In other words, items/tasks are more rigorous and require students to show what they know and how they know it

Item Development Process (same as used for state tests)

Written to align with curriculum Piloted in RT3 classrooms in spring 2012 Reviewed with student data and approved by Georgia Educators

Formative Item Bank

Item types
About 1700 Formative Classroom Assessment Items
About 750 Phase I items Phase I available in OAS as of Nov. 30, 2012

Some multiple choice items Most are open-ended or constructed-response format Holistic Rubric Exemplar Paper (i.e.,ideal response) Anchors used in scoring Student Exemplars/Responses DOK targets DOK 2; 19% DOK 3; 41% DOK 4; 39%
9

The Essence of the Common Core State Standards

The Common Core State Standards focus on students learning how to think, not just doing by rote memorization or quick scan-reading, and sets higher expectations on student comprehension and their ability to articulate their knowledge.

10

Capturing the Essence of ELA Common Core

ELA CCSS require students to use evidence from what lies within the four corners of the text and make valid claims that can be proven with the text. Students must be more adept at explaining that evidence orally and in writing.
11

 In order to be college-, career-, and life-ready, students need to be familiar and comfortable with a broad range of genres and formats and much greater emphasis on informational text.
In grades 3- 5: there is a call for more scientific, technical, and historic texts. In grades 6-8: more literary nonfiction including essays, speeches, opinion pieces, literary essays, biographies, memoirs, journalism, and historical, scientific, technical, and economic accounts.
12

Capturing the Essence of ELA Common Core

Capturing the Essence of Mathematics Common Core

Math CC standards require students to understand why they are doing the things they are doing. Students should
be able to explain why they're doing what they're doing and justify why they are doing it, not just explain what they are doing
13

Pilot
Conducted in spring 2012 Students assigned ELA forms could respond within OAS Students assigned Mathematics multiplechoice forms could respond within OAS. Students assigned Mathematics constructed response items responded on a paper-pencil answer document.

14

Extended Response Items

Performance-based tasks May address multiple standards, multiple domains, and/or multiple areas of the curriculum May allow for multiple correct responses and/or varying methods of arriving at a correct answer Scored through use of a rubric and associated student exemplars
15

Grade 3 ELA: Reading Passage (Literary)

16

Grade 3 ELA: Reading Passage (Informational) and Item 1

17

Grade 3 ELA: Rubric for Item 1

18

Grade 3 ELA: Exemplar for Item 1

19

Grade 3 ELA: Sample Student Response for Item 1

Student Response

Teacher Feedback

Rating based upon rubric

20

Grade 3 ELA: Sample Student Response for Item 1

21

Grade 3: Sample Student Response for Item 1

22

Grade 3: Sample Student Response for ELA Item 1

23

Grade 3 ELA:

Performance Data for Item 1

Item Type: Extended Response DOK: 3 Number of Students in Pilot: 90 Response Data
4s: 3s: 2s: 1s: 0s: 1 for 1.11% 6 for 6.67% 27 for 30% 53 for 58.89% 3 for 3.33%
7.78% proficient

24

Grade 6 Math: Item

25

Grade 6 Math:
Exemplar & Sample Student Response

26

Grade 6 Math: Sample Student Response

27

Grade 6 Math: Performance Data

Item Type: Extended Response DOK: 3 Number of Students in Pilot: 87 Response Data
4s: 3s: 2s: 1s: 0s: 0 for 0% 5 for 5.75% 34 for 39.08% 39 for 44.83% 9 for 10.34%
5.75% proficient

28

Scaffolded Items
Include a sequence of items or tasks Designed to demonstrate deeper understanding May be multi-standard and multi-domain May guide a student to mapping out a response to a more extended task Scored through use of a rubric and associated student exemplars
29

Grade 3 Math: Scaffolded Item

30

Grade 3 Math: Performance Data

Item Type: Scaffolded DOK: 3 Number of Students in Pilot: 44 Response Data
4s: 3s: 2s: 1s: 0s: 5 for 11.36% 3 for 6.82% 18 for 40.19% 15 for 34.09% 3 for 6.82%
18.18% proficient

31

Overall ELA Pilot Summary Data

Grade Number of students and percent falling into each score point Total student N/ %

0 3 4 5 6 7
475

1
1613

2
713

3
202

4
45 3048

15.60%
323

52.90%
1518

23.40%
814

6.60%
199

1.50%
83

100%
2937

11.00%
367

51.70%
1100

27.70%
901

6.80%
518

2.80%
125

100%
3011

12.20%
155

36.50%
960

29.90%
811

17.20%
418

4.20%
111

100%
2455

6.30%
218

39.10%
1387

33.00%
1275

17.00%
617

4.50%
146

100%
3643

6.00%
264

38.10%
1140

35.00%
1029

16.90%
338

4.00%
89

100%
2860

8
9 - 10 11 - 12

9.20%
175

39.90%
1016

36.00%
783

11.80%
361

3.10%
81

100%
2416

7.20%
376

42.10%
1018

32.40%
763

14.90%
196

3.40%
46

100%
2399

15.70%

42.40%

31.80%

8.20%

1.90%

100%
32

Overall Math Pilot Summary Data

Grade Number of students and percent falling into each score point Total student N/ %
1928

0 3
771

1
667

2
373

3
81

4
36

40.00%
795

34.60%
800

19.30%
360

4.20%
87

1.90%
58

100%
2100

4
5 6 7 8

37.90%
548

38.10%
513

17.10%
252

4.10%
124

2.80%
44

100%
1481

37.00%
927

34.60%
768

17.00%
269

8.40%
65

3.00%
14

100%
2043

45.40%
896

37.60%
632

13.20%
243

3.20%
62

0.70%
11

100%
1844

48.60%
984

34.30%
791

13.20%
314

3.40%
100

0.60%
51

100%
2240

43.90%
798

35.30%
697

14.00%
186

4.50%
45

2.30%
27

100%
1753

9-10
11-12

45.50%
690

39.80%
602

10.60%
178

2.60%
63

1.50%
9

100%
1542

44.70%

39.00%

11.50%

4.10%

0.60%

100%

33

Key Findings from Phase I Pilot

Students should be earning 3s or 4s to demonstrate grade-level mastery of the standards Preponderance of score points 1 and 2
Incomplete responses Responses hampered by writing skills Students did not show work in mathematics and could not explain why they did what they did

34

Implications for the Classroom

Clearer directions for students so they understand the expectations of a good response
Complete sentences, good grammar and syntax Connections Explanations and rationales

Student self-checklists to assist students in assessing their own responses working on tasks Reinforce instructional recommendations to teachers
Instruction aligned with CCGPS content and rigor Classroom assessments designed with focus on students articulating how they know what they know Lessons and classroom assessments integrate knowledge; thus, address multiple standards and domains
35

Resources to Support the Formative Items/Tasks in the OAS

About the Formative Assessment Items and Tasks in the OAS provides a summary of information the items and development. Formative Instructional Resources Released into OAS is a presentation with details on the items and how they were developed, how to access the items in the OAS, and how to use the items in the classroom.
36

Resources to Support the Formative Items/Tasks in the OAS

Student Self-Checklist for Written Responses to ELA Constructed Response Items
is a rubric that students can use to evaluate whether they have met the requirements of the task provides background information and how to use appropriately and when not to use.

37

Student SelfChecklist for ELA

38

Resources to Support the Formative Items/Tasks in the OAS

Student Self-Checklist for Mathematics Constructed Response Items
is a rubric that students can use to evaluate whether they have met the requirements of the task. provides background information and how to use appropriately and when not to use.

39

Student SelfChecklist for Mathematics

40

Resources to Support the Formative Items/Tasks in the OAS

About the OAS Formative Instructional Resources Item Search Tool provides information about how to use the tool to locate items that are tagged with multiple standards. Item Search Tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. The OAS shows only primary but this will show all standards associated with a single item/task.
41

OAS on GaDOE

http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS.aspx

42

Formative OAS Resources

http://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/OAS-Resources.aspx

43

Where Do You Find the Items?

44

Select the Tests tab

Select Create a new test

Give the test a name and unique identifier; choose any other desired options, then click Submit

Select an Item Level

New CCGPS items are in Level 2

Select a Subject for the test

New CCGPS items are found under Language Arts or Math

Select a Grade level

New CCGPS items are in grades 3-HS

Select a Standard
New CCGPS items are in grades 3-HS

Select a Sub-Standard
New CCGPS items are in grades 3-HS

Then click on Show Items

- OR Use the Advanced Search Method

Copy the unique ID here from the Item Search Tool Then click on Search

Put a check to the left of any and all items to be included in the test

Then click on Add Selected to Test

Make any needed changes or add more items (click Add Items to return to selecting items); when finished, click Create Test

Assign Test to a Class

Click on the Training tab for more information about assigning tests to a class.

56

How Students Respond

Students can respond to multiple choice items regardless of content area within the OAS. Students can respond to constructed-response ELA items within the OAS. Students need to view constructed-response Mathematics items in the OAS but provide written response on paper.
There is no math editor within OAS. Some items require students to show work or draw a graph, number line, or figure.
57

Scoring Open Ended Items in OAS

Scoring Online All open ended ELA items can be scored online Some open ended math items can be scored online Great way to track students proficiency Students can track their own proficiency as well Prepare students for whats to possibly come with PARCC Not Scoring Online A faux student must be created, assigned the test, and take the test to access rubrics and exemplars All open ended ELA items can be scored off line All open ended math items can be scored off line Great idea if computers are not readily available Great for students that have not mastered keyboarding

58

Log in to the OAS

59

Click on Tests Tab; then click Score open ended items.

60

Highlight the test you want to score; then click Select.

61

Check the Select/De-Select All; then click Next.

62

Question Number with Dropdown Feature

Performance Scoring

Show Question & View Sample Paper

Student Recorded Response

63

When the Show Question option is selected the question appears.

64

When the View Sample Paper option is Selected the student exemplars will appear.

65

When the View Sample Paper option is selected the student exemplars will appear

66

This drop down allows for the correct performance score to be given

Here is where the teacher can give commentary/feed back

Click Submit to continue to next item .

67

After you have scored and submitted the last item, this screen will appear. Click the tab labeled Student.

68

Click the View teacher-assigned results.

69

Select the test for which want to see the report, and click Next.

70

Select a student whose report you want to see, then click next.

71

All tests that a student has taken will appear. Click the specific test for which you want to see the results.

72

Results page: this page contains analysis of entire test.

You can click each question and it will display the item and how the student responded or continue to scroll down this page.

This test contained 6 items. 4 open ended that we scored using the 4pt. rubric (1,2,4,6) and 2 multiple choice items (3,5).

73

Student recorded answers with questions

74

 OAS only allows coding by a primary standard code. Many of the constructed response items are coded to multiple standards because content knowledge is not discrete and the Common Core expects students to assimilate content across standards and domains. A tool has been created that will allow teachers to search for a standard.
75

Finding an Item for a Specific Standard

OAS Formative Instructional Resources Item Search Tool

This tool is an Access database that allows teachers to search for items/tasks aligned to certain standards. All items aligned to a standard can be located, whether it be the primary standard or a secondary standard. The Item Search Tool can be downloaded from the GaDOE website and installed on your desktop.
76

How to Use the Search Tool

From the OAS Resources page on the GaDOE website, click Download Item Search Tool Open the zip folder and then the folder named OASFormativeInstructionalResourcesItemSearchTool Select and open the setup.exe file.

77

How to Use the Search Tool

If you do not have Microsoft Access 2007 installed on your computer, Access Run Time 2007 will be automatically installed. This application will allow you to view the database. Step by step instructions for installation are provided in the About the Item Search Tool document on the GaDOE website. A shortcut will be saved to your desktop. Double-click the shortcut to open the database.

78

How to Use the Search Tool

Using the drop down menus select Subject, Grade, and Standard (in that order) and click RUN.

79

How to Use the Search Tool

The report will include all items aligned to the selected standard. Copy and paste each Item ID into Advanced Search in OAS.
Use Ctrl-C to copy

80

Formative Assessment Initiatives

Bringing a Balanced Assessment Focus to the Classroom
First initiative Phase I items released into OAS; Phase II items will be piloted in early 2013 Another initiative to begin pilot in January 2013
Formative Assessment Bank Assessment Literacy Professional Learning

Interim Benchmark Assessments

Third initiative Phase I item development begun with pilot in spring 2013
81

Georgia Department of Education Assessment and Accountability

Melissa Fincher Associate Superintendent Assessment and Accountability 404.651.9405 mfincher@doe.k12.ga.us Dr. Melodee Davis Director Assessment Research and Development 404.657.0312 medavis@doe.k12.ga.us Michael Huneke OAS Manager 404.232.1208 mhuneke@doe.k12.ga.us Dr. Dawn Souter Project Manager. RT3 404.463.6667 dsouter@doe.k12.ga.us Robert Anthony Assessment Specialist, RT3 Formative Item Bank 404.463.6668 ranthony@doe.k12.ga.us Dr. Jan Reyes Assessment Specialist, RT3 Interim Benchmark Assessments 404.463.6665 jreyes@doe.k12.ga.us
82