+

Implementation and Evaluation of Educational Programs

EDUC 830-2

Class 1: Setting the scene

Question 1: What terms/ideas/issues come to mind when you hear the term evaluation or assessment? Think/Pair/Share

Class 1: Setting the scene

Question 2: Imagine a different educational reality. Dont worry about practicalities at this pointif we are going to properly* evaluate schools/students/programs, what should our theory and practice entail. What terms/ideas/issues should be associated with the term evaluation?

Same partner--Think/Pair/Share
* properly = adequately, imaginatively, thoughtfully, respectfully, & fairly

a definition: Evaluation is a systematic process of determining the merit or worth of an object (Joint Committee, 1994). Our object is, in this case, an educational program.

Class 1: Setting the scene

For your consideration:
What

determines how we evaluate educational programs? What shapes what we imagine educational evaluation to be in theory and practice? the story?

Whats

An Historical Look at Evaluation of Educational Programs

The Age of Reform 1792-1900

The Age of Reform 1792-1900

19th centuryrevolutionary social/economic/technological changesmetaphors we (still) live by Farish invents the MARKThe Worlds Most Famous Lazy Teacher?the field of psychometrics is born Boston 1845and it begins

determining the effect of the drill for spellingtest scores serve as means to evaluate educational program

The Age of Reform 1792-1900

creating a science of the socialseeking order/regularity in nature 1850-1900. Developments in GermanyWundt and Fechner; Developments in EnglandGalton roots of American psychologyHall and Thorndikea scientific management view of human behavior Thorndike and SLOBBER. Connectionismthe empirical study of learning; seeking universal laws

The Age of Reform 1792-1900

Core beliefs/Traits:

Nature is orderly; has a pattern that can be identified and measured Rational procedures (=Western Science) can discover these regularities
Positivism. Theoretical ideas about these regularities can be

constructed and the Truth determined

Quantification & standardization support control, prediction

= The origins of a scientific (or perhaps scientistic?) tradition in educational evaluation

THE AGE OF EFFICIENCY AND TESTING 1900-1930

THE AGE OF EFFICIENCY AND TESTING 1900-1930

early 20th centurytime & motion studies at the Bettelheim Steel Plant; the assembly line revolutionizes production Taylorism & the scientific management movement (factories) >> efficiency movement (schools) social values impact schoolssystemization, standardization, efficiency and productivity, norm-referencing Bobbitt, Charters, & Harap

THE AGE OF EFFICIENCY AND TESTING 1900-1930

muck-raking & accountability using test scoresnorm-referenced testing & quantitative analysis of operationally defined (measurable) objectives to evaluate teaching/teachers/schools, like sand in a motorimagination, initiative, innovation?

+ THE TYLERIAN AGE 1930-1945

THE TYLERIAN AGE 1930-1945

whose your daddy?

Tylerthe father of educational evaluation (objectives-based educational evaluation)

educational evaluation = measurement of intended versus actual objectives

operationalism required for evaluationobservable & measurable and observable behaviours

+
THE AGE OF INNOCENCE 1946-1957

THE AGE OF INNOCENCE IGNORANCE 1946-1957

post-war erathe bad: poverty, social despair, rampant racism, consumption and waste, pollutionthe good: expanded educational opportunities

changes in educational evaluation NOT tied to social purpose/empowerment

growth of standardized testingnational educational testing service (US), machine scoring, analysis services, taxonomies of educational objectives, teacher education on test development

THE AGE OF DEVELOPMENT 19581972

THE AGE OF DEVELOPMENT 1958-1972

disillusionment with educational programs (progressivism Hilda, are you there?) andbelieve it or notthe optical scanner (traditional approaches to educational evaluation).

Russian launch of Sputnik 1957implications for education

The ESEA 1964implications for the field of educational evaluation

Bruners Man: A Course of Studyengaging students in the structure of disciplines. Evaluation?

THE AGE OF DEVELOPMENT 1958-1972

seized with a great illnessincreasing concern with educational evaluation needs assessment? diversity of educational context and experience? national and standardized assessment?

evaluation differentiated from testing; evaluation includes measurement of student achievement AND determination of QUALITY of educational programs an epistemological shift? enter deliberation and thick descriptiona practical interest

AND a push back to basicsthe comfort of standardization, measurement, quantification

+ THE AGE OF PROFESSIONALIZATION 1973-1983

+ THE AGE OF PROFESSIONALIZATION 19731983

educational evaluation as a fieldtheoretical/practical developments; journals; university courses increased communication between different ideological/epistemological camps. Alternatives to positivist/psychometric/quantitative approaches to evaluation emerge. Enter phenomenology, ethnography, qualitative inquiry

an enormous amount of chatter (1973-1983)

Some key people:

ScrivenMeta-theory & Value-free Evaluation StakeResponsive Evaluation

HouseDemocratic Deliberative Evaluation

StufflebeamCIPP (context, input, process, product) evaluation LincolnConstructivist Evaluation EisnerConnoisseurship Evaluation

what was everyone chattering about?

previously un(der)valued and, thus, ignored aspects of educational evaluation (needs, values, situational realities, probity, imagination) the educational significance of things not-so easily measured/graphed why educational research makes for a messy scientific experiment little change, lots of chatter

+
THE AGE OF EXPANSION AND INTEGRATION 19832001

THE AGE OF EXPANSION AND INTEGRATION 1983-2001

increasingly diversified field, growth of 1960s/1970s stunted with cutbacks in funding for evaluation continued demand for appropriate evaluation theory/methodology From A Nation At Risk (1983)( an invading army could not do more damage to our country than our schools have already done) to America 2000 (The road to hell is paved with good intentions Eisner, 2002, p.1). 1990sdiscontent with education; seeking universal/standardized solutions (back to 1920s?)

enter assessment

evaluation

1755, from Fr. valuation, from valuer "to find the value of," from - "out" (see ex-) + valuer (see value)

a gentle but firm nudge?

from 1990s assessment increasingly used in educational discourse assessment (in particular authentic assessmentsee Wiggins) seen as a better reflection of the complexity of school life than sanitized testing measures on one hand and more qualitative evaluation procedures of 1960s/70s on the other parallels back to basics movement; call for increased accountability, higher standards, and validity in assessment procedures

+
THE AGE OF IMAGINATION IN EDUCATION
2005--

THE AGE OF IMAGINATION IN EDUCATION 2005-

21st century skillscritical thinking, creativity, imagination? cognitive tools and kinds of understanding. Kieran Egans Imaginative Education. 2001. The Imaginative Education Research Group is formed. 2001-2011. International implementations. MEd Programs (suckers!!). IE Schoolsassociated projects including: LiD, IEE, WSP. Evaluating IE