Evaluation for Education Technologists Weeks 4 Assignment Done: Kirkland Lawrence Submitted to: Dr. J.

Thompson Boise State University Reading of Chapter 8 - Question 1a (1). Does my project represent research or evaluation? How do I make that decision? My project is to evaluate the English program at the New Beginning School. I am definitely doing an evaluation and not a research because the approach to my project entails the gathering of information about the program, analyzing it, and presenting it to the program stakeholders. They can use this information to make decisions whether or not they will improve the program, or if it would be worthwhile to continue investing or supporting the program in the future. On the other hand, the approach to the project could not have been research; because, I am not interesting in placing students in a lab or controlled setting/ condition, neither am I administering any treatment to participants in order to see a cause and effect relationship. Hence, my project is evaluation because it is providing feedback to stakeholders and not necessarily adding to the existing body of knowledge, although that is automatically done when you make a report. (2). Does my project involve sampling? If so, what type of sampling will I employ? My project does not involve any form of sampling random or otherwise. I will use the entire student population enrolled at the New Beginning School. This is because the population is very small 20 students; as a result, there is no need to do any sampling to represent that population. According to Boulmetis, and Dutwin, (2011), if the population is less than 100, use them all. The data for the project will be gathered by administering a survey in the form of questionnaires to the target population. However, if after the survey is administered, and it is found out that all subjects did not participate for some reason or another, the data gathered will be used to represent the whole group. In this case the respondents can be classified as a sample of the initial population. Question 2. Review chapters 1-9 in the Boulmetis & Dutwin text. Then write down your thoughts about each chapter. Chapter 1 What is Evaluation? This chapter deals with the definition of evaluation in two ways: (i) as an art and as science with the important notion that evaluation is a systematic process of collecting and analyzing data to either determine if objectives are met or to make decision. The chapter material also centered

around evaluating efficiency the degree to which a program has been productive in relationship to its resources, effectiveness the degree to which goals have been reached, and impact the degree to which program resulted in change. Kirkpatrick, (1994), highlights four other levels of evaluation that can be used to present data to stakeholders. Chapter 2 Why Evaluate? Chapter 2 deals specifically with why you evaluate. It shows how one procures funding to a program through grants. It also focuses on the benefits and limitations of evaluation. Evaluation is beneficial to both staff and stakeholders as it presents pertinent data to guide the decisionmaking process of stakeholders and at the same time promote discussion/ communication among staff members. One limitation of evaluation is that there may be a staff cut after an evaluation took place as a consequence of the consensus arrived at by stakeholders. However, evaluation is an integral part of most programs and should be planned from the outset. One thing I take away from this chapter is that every program should be evaluated. Chapter 3 Decision Making: Who to Involve, how, and Why? This chapter focuses on several aspects of the evaluation process. Radiating from among them is the Program Cycle, which is cyclical in nature; meaning that the events unfold at each stage and then you move on to the other stage in a systematic fashion and not necessarily in an iterative fashion. Another focus of the chapter is on monitoring and evaluating - actually each one can be used to assist in the development of the other one. Finally, I agree that all stakeholders must be involved in the evaluation process from the onset of the program. Chapter 4 Starting Point: The Evaluators Program Description This chapter focuses primarily on the evaluators program description (EPD) with questions asked about its purpose, components and its development and usefulness. Here the Program Cycle is mentioned once more, with the development of the EPD to address its various stages. The EPD can be used to monitor a programs effectiveness by looking at what is being done and how it is being done. The EPD will ultimately reveals a programs goals and objectives, which allows the evaluator to be aware, record activities and plan to accomplish those goals and objectives. In addition, it reveals what measuring tools are in place to gather data. Essentially, the EPD is designed to answer the evaluation questions. Chapter 5 Choosing an Evaluation Model This chapter mentions several evaluation models and how to employ them in the evaluation process. After reading this chapter, I was in a very comfortable position to choose an evaluation model that suits the program at the New Beginning School. The chapter distinguished between research and evaluation, and also qualitative and quantitative evaluations. The chapter also

establishes seven steps to guide the evaluator during the evaluation process. Additionally, it gives a basic or general evaluation design format that most evaluation models should consist of. Chapter 6 Data Source Chater 6 presents the Logic Model which shows the flow of project. And it reiterates the difference between qualitative data source which gives a descriptive report and quantitative data source which gives a numeric value for the presentation of data. While qualitative data is collected by doing observation, quantitative evaluation is collected using data sources such as tests, counts, measures and instruments. Four levels of data exist: 1. Nominal Data based on only one principle; for example nationality, age, gender, etc. 2. Ordinal Data based on one level principle of level of discourse, but they convey a rank order. 3. Interval Data possessed characteristic of rank order, but they also involve equal interval 4. Ratio Data - possessed characteristics of all three, but has an absolute zero point. This chapter definitely causes me to reflect on a previous Research Methods course that I took when I was pursuing my undergraduate degree. Chapter 7 Data Analysis This chapter is very timely and fitting in the text. It comes at time when data is thrown at the reader from right, left and center. What do you do with these data? This chapter explains it all: analysis of data. However, before chapter 7 delves readers into data analysis, it first gives an overview of some statistical terms such as measures of central tendency which include the mean, mode, and median, and measures of variability which involves range, quartile, and interquartile. The statistical examples used in the chapter are appropriate and self- explanatory. Chapter 8 Is it Evaluation or is it Research? This chapter is very interesting to read. In fact, it puts you on the spot. Are you doing an evaluation or a research project? The characteristics of the two will let you decide which one you are doing. I will be doing a research if I am doing the following: Presenting a hypothesis Collecting data under controlled conditions Providing treatment Controlling variables Experimenting - using experimental designs; example, true experimental design, quasiexperimental design or ex post facto design Adding information to the existing body of knowledge

On the other hand, I will be evaluating if I am doing the following: Learning what is going on in the program Making a case for a new program Making proposal to improve existing program Providing information to stakeholders so that program can change

Chapter 9 - Writing the Evaluation Report After reading chapter 9, I began to develop the real urge of wanting to complete my own evaluation project, especially when I am reflecting on the subject I currently teach. The chapter emphasized that an evaluation report is needed at the end of every program cycle. Analyzed data that are presented in the report will provide stakeholders with feedback that will guide the decision making process concerning the sustenance of a particular program. Knowing your target audience is very critical multiple reports can be presented to several stakeholders with each report tailored to suit the needs of the respective stakeholder. The chapter also outlined the key components of the evaluation report: Cover Page Executive Summary Table of Contents Introduction Methodology Evaluation Result and Findings Summary and Recommendations Appendixes

All of the above bulleted components will be included in my evaluation project for the EDTECH 505 evaluation assignment to be submitted at the end of the semester.