Running head: DOCTORAL COMPS

Doctoral Comprehensive Examination Tonia A. Dousay August 4, 2011

DOCTORAL COMPS

Table of Contents Foundational Process of Instructional Design...............................................................1 Experiential Activity .................................................................................................. 2 Analysis ...................................................................................................................... 3 Performance Analysis............................................................................................. 3 Learner Analysis ..................................................................................................... 7 Training Purpose .................................................................................................... 8 Evaluating the Analysis ......................................................................................... 11 Design........................................................................................................................ 11 Evaluating the Design ...........................................................................................14 Development .............................................................................................................14 Evaluating the Development.................................................................................16 Implementation ........................................................................................................16 Evaluating the Implementation ............................................................................17 Evaluation & Conclusions .........................................................................................18 Theoretical Foundations...............................................................................................19 Cognitive Psychology ............................................................................................... 20 Cognitive Theory of Multimedia Learning............................................................21 Selecting & Designing Media ................................................................................... 22 Learner Interest ....................................................................................................... 25 Research Methodology ................................................................................................ 28 Identifying the Problem ........................................................................................... 28 Significance........................................................................................................... 29 Research Questions.................................................................................................. 30 Research Method ..................................................................................................... 30 Design ................................................................................................................... 30 Setting & Participants............................................................................................31 Design Treatment ..................................................................................................31 Power Analysis...................................................................................................... 33 Measures............................................................................................................... 34 Hypotheses............................................................................................................... 36 Practice Design ............................................................................................................ 38 Training Purpose...................................................................................................... 38 Analysis Summary ................................................................................................... 38 Design Summary...................................................................................................... 39 Content Outline .................................................................................................... 39 Design Document ................................................................................................. 39

DOCTORAL COMPS

ii

List of Figures Number Page Figure 1 Performance assessment: resource gap .......................................................... 5 Figure 2 Performance assessment: motivation gap ...................................................... 6 Figure 3 Performance assessment: knowledge gap ...................................................... 6 Figure 4 General learner analysis.................................................................................. 8 Figure 5 Power analysis output from G*Power........................................................... 34

DOCTORAL COMPS

iii

List of Tables Number Page Table 1 Sample Task Inventory ....................................................................................13 Table 2 Sample Performance Objective .......................................................................13 Table 3 Strategies for Task 1.3......................................................................................16 Table 4 Cognitive Tools for Research Method Selection ............................................ 30 Table 5 Proposed Research Design ............................................................................. 32 Table 6 Proposed SIS Items......................................................................................... 35 Table 7 Design Modifications ...................................................................................... 40

DOCTORAL COMPS

Item #1 Foundations. Write a case study supported by research and theory that explains the foundational process of instructional design. Foundational Process of Instructional Design Defining and creating models of design intended to improve instruction has fascinated practitioners for the last 50 years. Barson (1967) was perhaps the first to introduce the phrase instructional development, defined as the systematic process for improving instruction, as part of a project at Michigan State University. Shortly thereafter, Twelker, Urbach, and Buck (1972) noted that a systematic approach to developing instruction was a popular idea, but cautioned that instructional design methods varied from simple to complex. In an overview of the history of instructional design, Reiser (2001) found that a variety of sets of systematic instructional design procedures had emerged in a relatively short four decades. The reason for this rapid acceptance is perhaps due to the nature of models and their ability to simplify complex realities and apply generic components across multiple contexts (Gustafson & Branch, 2002). Out of hundreds of instructional design models available, Gustafson and Branch (2002) reviewed only 15 different models of instructional design based on historical significance, unique structure, or frequent citation in literature. However, at the root of this survey is the assumption that, models serve as conceptual, management, and communication tools for analyzing, designing, creating, and evaluating guided learning, ranging from broad educational environments to narrow training applications. The progression of analyzing, designing, developing, implementing, and evaluating (ADDIE) forms the basic underlying process that is a distinct component of instructional design regardless of specific model used. With such a rich history that continues to evolve, the framework of ADDIE provides a solid foundation for the instructional design process (Gustafson & Branch, 1997).

DOCTORAL COMPS

Whether by necessity to address the growing field of instructional development or out of a natural progression, an introduction to instructional design has become a part of training in higher education. Richey, Klein, and Tracey (2010) note that instructional design is now an established profession and area of study. Within the field of study, experts often refer to and/or create models as a tool to guide learning in instructional design (Magliaro & Shambaugh, 2006). However, with so many different models of design available, it becomes important for novice designers to understand the foundations of instructional design before seeking to apply the knowledge. At one university in the southeastern United States, the training approach has been to introduce instructional design with an overview of ADDIE followed by an experiential learning activity that requires students to work as a project team for a vetted client with an instructional problem. Through analysis, initiative, and immersion, students are able to quickly assimilate details and assumptions into practice. Students are encouraged to explore established instructional design models and use the method they feel is best suited for their specific application. Experiential Activity During the autumn academic term of 2009, one such student-led team agreed to evaluate the instructional needs of a volunteer program at a community nature center. A total of four students, one undergraduate, one masters student, and two doctoral students, worked cooperatively over the course of eight weeks to analyze the nature centers volunteer training program and make recommendations based on observed gaps. The two doctoral students had professional instructional design experience before taking the course and beginning the project. Under the guidance of the two senior designers, the project team agreed that a generic ADDIE approach to the project would be easier to manage and contribute to the teams understanding

DOCTORAL COMPS

of more specific instructional design models. As such, the following case study examines each phase of the ADDIE process, including the decisions made at each step and justifications for each action. An overview of the project client may be helpful for understanding the context of this case study. The community nature center is a 225-acre property that serves the community as a wildlife sanctuary and an environmental education center. The goal of the community nature centers field studies program is to provide children with opportunities for a variety of positive outdoor learning experiences in a personal, supportive atmosphere. This goal is carried out through the use of volunteer trail guides. This volunteer opportunity is unique in that it requires guides to actively engage themselves beyond their two hours of volunteer work; it calls upon the volunteer to actively learn local flora and fauna, as well as to refine teaching and group management skills for a broad age range. The community nature center provides intensive training sessions twice per year for new guides. If a volunteer guide is unable to participate in a session, they may obtain training through observing walks led by other volunteers, participating in a naturalist-led trail walk, reading the Volunteer Trail Guide Manual (VTGM), and hiking trails during personal free time. Guides-in-training are encouraged to shadow other volunteer guides at least two times before leading groups on their own. After meeting with the client to establish this necessary background information, the team members were comfortable with moving forward with the analysis phase. Analysis Performance Analysis The importance of a good analysis cannot be overstated when teaching or conducting instructional design. Dick, Carey, and Carey (2009) note that identifying the instructional goal is the most critical event in the entire process. Using a

DOCTORAL COMPS

performance technology approach, designers do not bring preconceived notions of what to include in an instructional product. In fact, the designer should acknowledge that the problem might not be instructional at all. Based upon a needs analysis, defined as a discrepancy or gap between the desired state of affairs and the present state of affairs (Burton & P. F. Merrill, 1991), designers can determine what gaps exist and which gaps can be addressed with training (Gagn, Wager, Golas, & Keller, 2005). With respect to the nature center, the results from the volunteer trail guides performance assessment were classified into one of three gaps. A sample size of eight volunteers and seven parents/teachers out of a larger population were surveyed for this assessment. Additionally, further data collection occurred during in-person observation of volunteer trail guides, feedback from a third-party observer, an interview with the community nature center Head Naturalist, and supplemental resource evaluation. The resource gap, illustrated in Figure 1 was comprised of three elements that generally included access to training. With respect to the resource gap, it was clear that a number of steps could be taken to improve the volunteers performance. First, the project team recommended lowering the guide-to-student ratio in order to enhance volunteers ability to interact with visitors and decrease the feeling of being overwhelmed.

DOCTORAL COMPS

Adequate Training

Access to Training Handbook

Guide : Student Ratio

Resource Gap

Figure 1. Performance assessment: resource gap. Six elements were classified in the motivation gap and included issues such as confidence and training completion, as illustrated in Figure 2. Though the observed motivation gap issues could not be directly addressed with training, the project team kept these factors in mind throughout the duration of the project, making suggestions to the stakeholders when appropriate.

DOCTORAL COMPS

Comfortable working with children Complete shadow visits

Confidence adapting to questions Schedule conflicts

Selffamiliarize with trails

Motivation Gap

Sense of responsibility

Figure 2. Performance assessment: motivation gap. The knowledge gap was made up of four major elements that are illustrated in Figure 3. The project team recommended redesigning the volunteer training program to address the identified knowledge gap and producing a new VTGM.

Answer animal, plant, and/or organism questions

Experience working with children

Natural Science background Knowledge Gap

Manage student behavior and discipline issues

Figure 3. Performance assessment: knowledge gap

DOCTORAL COMPS

The findings of the performance analysis were summarized in the analysis summary that was later presented to stakeholders. Learner Analysis The next step in the analysis phase was to assess the current knowledge and skills of the learners. It is important for the designers to consider the characteristics of learners, contexts in which the instruction will occur, and the context in which the skills will be eventually used (Dick et al., 2009). Further, learners have certain qualities that relate to instruction. Characteristics related to instructional design are specifically those that affect information processing (Gagn, et al., 2005). In an overview of instructional design models, (Richey et al., 2010) refer to a number of theorists, including Bloom (1976) and Carroll (1963), to illustrate the importance of examining learners to consider needs, interests, and level of development as they relate to cognitive and affective behavior. Using as much detail as possible to craft a learner profile makes designing and developing the planned instruction an easier task later in the process. The training program focused on the volunteer trail guides, who range in age from 20 to 70 years old. Most volunteer trail guides have worked with students or led instruction in some capacity. A survey of the volunteers revealed that trail guides have experience working with students ranging from ages 4 to 15. While about 50% of trail guides are college graduate students, the remaining half actively work in the fields of education, journalism, environmental advocacy, and restaurant management. Volunteer guiding is a popular avocation for retired professionals, as well. Most trail guides, whether amateur naturalists or graduate students in a scientific field, have a basic knowledge base in environmental science and/or natural history. A general profile of learners appears in Figure 4.

DOCTORAL COMPS

10 male 10 female

Local area

Demographic

Location

Skills Inter-personal Communication Flexiblity

Experience 50% students 50% professionals

Figure 4. General learner analysis. The first important discovery during this analysis was that the age range of the learners is quite wide. Second, the experience level of the learners is extremely varied. Considering these two factors, the team members felt it important to focus on the learners skills for use in designing the instruction. Having a better understanding of the performance gap and learners, the project team could now begin to prescribe a suitable training program. Training Purpose Designers must consider various components of the learning environment when determining instructional goals. Knowing the learners characteristics and having an identified knowledge gap is not enough information to develop quality instructional goals. Generally speaking, Dick et al. (2009) prescribe three steps to selecting instructional goals. In the first step, designers should question whether or not developing the instruction would address the identified instructional problem. Second, goals must be acceptable to the project stakeholders. Third, the designers

DOCTORAL COMPS

must consider the availability of resources in order to develop and implement any prescribed instruction. Keeping these three guidelines in mind, the project team recommended that the purpose of the training program be to provide participants with the knowledge and skills necessary to create positive outdoor learning experiences on the nature center trails. In order to ensure that the purpose of the training program is met, four instructional goals for volunteer trail guides were established. 1. Define the role of environmental science in supplementing student education at the SCNC 2. Identify best practices for working with students of different age groups 3. Manage student behavior based on group dynamics 4. Distinguish between types of living and nonliving components commonly found at the SCNC The goal statements were drafted using guidance from Blooms (1956) Taxonomy of Instructional Objectives. This decision was made because instructional goals are intended to be clear statements of behavior that learners should demonstrate at the conclusion of instruction (Dick et al., 2009). Given that the learner analysis showed a weak understanding in natural sciences, the project team felt that an instructional goal aimed at mastering simple, concrete concepts would be beneficial. To address this issue, the first instructional goal related to environmental science was written at the knowledge level of cognitive domain, which prescribes knowledge of specifics, ways of dealing with specifics, and universals of a field (Krathwohl, 2002). Looking to address the knowledge gap related to working with children, the designers felt that a slightly more complex instructional goal should be drafted. As such, the statement was written at the comprehension level to encourage translation and interpretation (Krathwohl, 2002). The next instructional goal was

DOCTORAL COMPS

10

based upon the learners perceived deficit in knowledge and practice regarding managing student behavior and discipline issues. Volunteers expressed frustration with respect to being able to effectively work with the students in person, and the project team agreed that being able to practice applying concepts related to managing behavior would be the most effective means of addressing the gap. Last, the project team sought to address the knowledge gap related to living and nonliving components of the nature center. This gap was perceived to be the most complex and therefore required a higher element of knowledge transfer and application related to analyzing elements, relationships, and organizational principles, placing it at the analysis level (Krathwohl, 2002). Therefore, all four instructional goals represent a progression of cognitive development that increases in complexity and requirement after achieving each previous goal. In order to carry out these instructional goals, the instructional design team recommended expanding the existing biannual volunteer training program to include an online component. The online materials would focus on group management techniques and best practices for working with children. Further, these materials would include videos, tutorials, and vignette scenarios to present and reinforce the concepts. The face-to-face session would focus on how the community nature center mission supports environmental science programs at local schools; familiarization with the nature center resources, including trails and facilities; and identification of local flora and fauna. Additionally, the face-to-face session would reinforce concepts presented online. Further recommendations included developing special topic seminars offered at periodic intervals throughout the year. These training recommendations were detailed to the stakeholders as two separate options for consideration.

DOCTORAL COMPS

11

Evaluating the Analysis At the conclusion of the analysis phase, the project team drafted an analysis summary for the stakeholders. This document contained narrative details regarding the overall process, performance gap, learner analysis, training purpose, and resource analysis. Additionally, the two recommended training options were presented with cost breakdowns and estimated timelines. Though the project team noted that the recommended training programs did not require any additional management or technology investments, the stakeholders opted not to support the development of an online component or the supplemental videos. Instead, they requested a third action that would effectively redesign the training within the existing constraints and materials. Upon receiving approval from the stakeholders to proceed with the design phase, the project team felt it important to evaluate the analysis phase and adjust the process plan accordingly. Similar to Nieveen's (1997) CASCADE model, the project team wanted to emphasize formative evaluation at each phase in the project process to achieve a satisfactory level of quality. The designers felt that the stakeholders decision to request a previously unproposed option was due to their lack of adequate consideration of the second and third steps as prescribed by Dick et al. (2009) and discussed earlier. The stakeholders did not agree that the project could be completed on time if selecting one of the proposed options. After reaching a compromise with the stakeholders, the project team readjusted the project timeline and cost estimates accordingly and was able to proceed to the design phase. Design Designing for instruction is more than just taking into consideration the purpose of the instruction. Rather than beginning design by asking what will be learned, designers must focus on what students will be doing after they have received

DOCTORAL COMPS

12

instruction (Gagn, et al., 2005). Designers also face a challenge when deciding whether objectives are to be attended at the completion of the training or acquired during the course of study. In order to help organize the design process, a designer can begin with a task analysis or inventory through which they clarify outcomes of instruction and arrange or rearrange components into an instructional sequence. The end result creates a blueprint that helps designers make sure that important parts of the lesson are not ignored and components support one another (Jonassen, Tessmer, & Hannum, 1999). Once the task analysis has been completed, designers then identify instructional strategies and activities to facilitate the prescribed learning. Based upon the previously identified instructional goals, the project team created a complete task inventory. This inventory included all four instructional goals, relevant tasks, subtasks, and prerequisite skills and knowledge. For the purpose of this discussion, the analysis for the first instructional goal appears in Table 1. After compiling a complete task inventory, the project team drafted instructional objectives for each of the tasks and subtasks. Mager's (1997) threecomponent model for writing objective statements was used for this activity. The first element of the objective describes the skill or behavior indicated in the task analysis and what is expected of the learner. The second component addresses the conditions under which the learner will carry out the task. The final segment describes the criteria that are used to evaluate the learner. For example, the objective for Task 1.3 is illustrated in Table 2.

DOCTORAL COMPS

13

Table 1 Sample Task Inventory Instructional Goal Task Subtask Prerequisite Skills & Knowledge 1. Define the role of 1.1. Recognize the environmental interconnection science in between the supplementing [nature center] student and school education at the curriculum SCNC

1.2.1.

1.2. Identify the Define biosphere interconnection of all life 1.2.2. List the Define atmosphere steps in the carbon cycle 1.3. Describe conservation techniques 1.3.1. Explain resource allocation

List the steps in the water cycle

Define ecosystem

Table 2 Sample Performance Objective Component Task Condition Criterion Statement Describe conservation techniques from memory that would impact the SCNC

It bears noting that all objectives were written to a specific performance level of Blooms Taxonomy in an attempt to cover all levels from knowledge to evaluation. As seen in Table 2, the performance objective for task 1.3 was at the comprehension level. Furthermore, each objective included a testing method that instructors could use to measure when the objective had been mastered. In the case of Task 1.3, the testing method was, During group discussion, describe at least one conservation

DOCTORAL COMPS

14

technique that could benefit the SCNC when asked by the instructor. Upon compiling the task inventory and drafting all performance objectives, the designers presented the documentation to the stakeholders for approval. Evaluating the Design Before moving on to development, the designers reviewed the design process and products created as a means of continuing the formative evaluation. From the perspective of the experienced designers, creating a task analysis aligned with a table of performance objectives similar to that displayed in Table 2 effectively created a line of sight. As (Branch, 2009) indicated, line of sight is a practical approach for maintaining an alignment between needs, purpose, goals, objectives, strategies, and assessments throughout the ADDIE process (p. 59). In keeping with the blueprint concept, working through the design process with line of sight in mind helped prevent overlooking any important details and identified missing or duplicate components. Of all the phases completed during the process, the project team felt that design was the most helpful in terms of practical experience. Development Assuming that the purpose of instruction is to support the process of learning, developing instructional materials should be based on learning processes and engaging the learner. Therefore, designers should draft an instructional strategy to describe general components of Gagn et al., (2005) the instructional materials and the procedures to be used in order to foster the desired learning outcomes (Dick et al., 2009). Within the instructional strategy, designers should note that instruction is typically made up of a series of events external to the learner intended to help learners achieve a learning objective. The precise manner in which these events occur generally varies by design, but has been summarized as Gagns Nine Events of Instruction. These events are intended to stimulate information processing and

DOCTORAL COMPS

15

foster the learning process. Table 3 illustrates the strategies as related to the nine events of instruction developed by the project team for Task 1.3. These strategies incorporate the needs identified in the learner analysis as well as performance expectations from the stakeholders. Table 3 Strategies for Task 1.3 Event Strategy Gain Pass around images of the Everglades and regions of the Amazon Rain Attention Forest before man affected them. Ask the group to identify each image. If the group is unable to guess the answer, provide trivia hints accordingly. Once the image has been identified, distribute the current image and explain how a lack of conservation principles and laws resulted in their current state(s). Objective Following the image activity, the instructors prompting statement and subsequent questioning leads into an explanation of the objective. Instructor leads group into nature trails and to identify examples of the carbon and water cycles. Referring to the VTGM, instructor explains how the SCNC came to hold its current status in Athens-Clarke County (ACC) and outlines the conservation techniques that maintain and preserve the park. Using the content as a platform, instructor refers to other park facilities and asks students to name conservation techniques that might be in place in the other locations. Refer to VTGM, Nature Center facilities, personal experience, and online resources. Based upon the ecosystem selected, they are to list at least three conservation techniques that could be in use at the specified location. During both guided and independent practice, instructor reminds students of techniques used at the SCNC. Upon immediate response of questioning, instructor should confirm the answer or provide prompting of further responses if necessary. Instructor individually asks each student to name a conservation technique used at the SCNC.

Prior Knowledge Content

Guided Practice

Individual Practice

Feedback

Assessment

DOCTORAL COMPS

16

Table 3 Strategies for Task 1.3 Event Strategy Closure Instructor asks the group to describe a scenario that could happen to the SCNC if conservation techniques are not followed.

Continuing the development process, the project team drafted a full instructional strategy as recommended by Dick et al., (2009) to support the training program. This strategy included identifying all media to address visual, auditory, and kinesthetic needs of the learners. Together, the described elements herein were presented to the stakeholders in a development summary for their approval before moving on to implementation. Evaluating the Development The project team felt that development was the most difficult and challenging of all the phases in the ADDIE process. Development required substantially more effort than design in that each objective was scrutinized for the instructional strategy. As Gagn et al., (2005) noted, not all events of instruction are appropriate for every lesson. Therefore, the design team evaluated each objective within the context of the events of instruction and drafted a corresponding strategy. Line of sight was again helpful to the team as they created documentation to ensure flow and inclusion of every component from information content to media selection. In the end, the attention to detail proved beneficial as the stakeholders approved the development summary and authorized the team to move forward with implementation. Implementation The implementation phase of instructional design varies among instructional design models, but generally encompasses the same common tasks. Richey et al., (2010) note that during implementation, instructional materials are developed and procedures for installing, maintaining, and periodically repairing the instructional

DOCTORAL COMPS

17

program are specified. As requested, the project team drafted a new VTGM for the volunteers as well as an instructors guide for personnel conducting the semiannual face-to-face training. The new VTGM contained updated graphics and illustrations to replace outdated or ambiguous images. The document was also expanded to include sections that aligned with the tasks identified in the performance analysis and provide learners with the knowledge they would need to fill the perceived gap. Following Diamond's (1989) model of instructional design, the project team included field testing as part of material production. This task was accomplished by working with select volunteers to evaluate sections in the new manual as they were produced. Additional instructional materials included the development of job aids that condensed portions of the VTGM, such as common animal paw prints and leaf types, onto 3x 5 cards for use in the field. The aids were created to reinforce higher level learning objectives that related to living and nonliving components at the nature center. The component of implementation included drafting learner and facilitator plans. The learner plan included learner identification, training schedule, notification description, and tracking requirements. The facilitator plan addressed facilitator identification, scheduling train-the-trainer sessions, and preparation instructions. With all materials complete, the stakeholders signed off on the implementation phase. Evaluating the Implementation Evaluation of the implementation process occurred simultaneously with production. The designers felt that their comprehensive approach to the previous phases of the process made the implementation phase easier to complete and created structured guidelines to use in updating existing materials. As such, the designers found that they had incidentally developed a checklist of corrections and new documentation to be created. By using the Diamond Model recommendation of field

DOCTORAL COMPS

18

testing during production, the designers were able to immediately adjust the content to fit the needs of the learners. Further, it was through this process that the idea to create the job aids came about. In the end, the approach taken by the project team created a high quality product that met with stakeholders expectations. Evaluation & Conclusions The final phase of any process is that of evaluation. Every designer wants assurance that his or her instructional product is valuable (Gagn et al., 2005). It follows then that systematic, summative evaluation is the means by which this assurance can be found. In the case of the nature center training program, the project team provided both level 1 and level 2 evaluation tools. The level 1 instrument was developed to capture learners reaction to the newly improved training, and the level 2 instrument was developed to measure the change in performance of the volunteer guides (Kirkpatrick & Kirkpatrick, 2006). Though the project team would not remain under contract with the nature center to perform the evaluation phase of the project, the stakeholders agreed to use the instruments developed and report the results after one year of implementation. At the time of this paper, the nature center reported to the project leader that they were extremely satisfied with the new training program and had seen a significant improvement in the performance of their volunteers. From the perspective of the two experienced designers, the project served as an excellent reinforcement of the literature and theories related to instructional design. In conclusion, there is much to be said about the vast number of instructional models in existence; however, following the basic ADDIE process remains tantamount in creating instructional materials to fill performance gaps.

DOCTORAL COMPS

19

Item #2 Theory. Explain the psychological theoretical basis for your study. Specifically, explain how the Cognitive Theory of Multimedia Learning (Mayer, Moreno, Sweller, et al) as a design element and Situational Interest (Hidi, Harackiewicz, Linnenbrink-Garcia, et al) in learning environments serve as the two major theoretical underpinnings of your research study. Explain the theories, give a rationale for choosing them, and show how they are related to each other and to your study. Theoretical Foundations Educational technology has been considered as a means for delivering educational content and promoting learner achievement, both of which in turn influence the cycle of research. As technology contributes to instruction, it also creates new and innovative opportunities for research that in turn influences how technology is used in instruction (Salomon & Almog, 1998). Though much of the foundational research in educational technology has been bolstered by the hope that learning is enhanced by the precise combination of medium, student, content, and learning tasks, Clark (1983) argued that only content can influence achievement. As a result of this challenge, it took the field more than a decade to come to terms with how to reconcile the relationship between media and learning. Initially, Kozma (1994) rebutted that perhaps the lack of relationship was because researchers had not yet made one. Later, Cobb (1997) recommended expanding Clarks discourse to include cognitive efficiency. Cobb felt that media selection should capitalize on cognitive processes that maximize learning potential. In what could be considered an affirmation of this recommendation, Clark (2001) later stressed that pedagogy must direct technology if it is to be used effectively in education. While re-evaluating the historical debate, Hastings and Tracey (2005) noted that resolving whether or not the media of 1983 could affect learning is not productive nor does it matter. More

DOCTORAL COMPS

20

significantly, the researchers noted that media capabilities have changed drastically since the original literature review, adding substance to Kozmas position and Cobbs recommendation. While some instructional media, including videos and print, have not changed in the past 30 years, other technological and theoretical advancements have affected strategies for designing, developing, and implementing media. Cognitive Psychology As the use of educational technology has grown, seeking effective applications of media for learning, much of the advancements have been due to the influence of cognitive psychology. Wittrock (1979) asserts that instruction is an art that begins by understanding and diagnosing cognitive and affective processes of learners. In the mid 1980s and through the 1990s, various researchers examined the separate channels humans use for processing visual and auditory information. These studies have proven that though the channels are interconnected, there is a limit to the amount of information that can be processed by each channel at one time (Baddeley, 1986; 1999; Chandler & Sweller, 1991; Paivio, 1986). At the same time, this examination of the cogntive relationships of reading comprehension led to the hypothesis of a generative process that relies upon signals, strategies, and plans to relate events to one another. Thus, Marcus, Cooper, and Sweller (1996) found that pictorial information can help individuals comprehend complex textual information. Mayer (2001; 2002) then synthesized these previous studies to propose that humans actively engage in learning by attending to relevant incoming information, organizing selected information into coherent mental representations, and integrating mental representations with previous knowledge to be stored in longterm memory. Mayer (2005) later described a framework built off of previous studies aimed at maximizing optimal learning potential in multimedia environments (Moreno & Mayer, 1999). Combined, these studies influenced by cognitive

DOCTORAL COMPS

21

psychology have had a distinct impact on future directions for educational media design. Cognitive Theory of Multimedia Learning The culmination of decades of studies on cognitive influences has resulted in a number of implications for the design of learning media. How humans perceive and understand objects and events depends heavily upon appearances and actions as a whole; this is a central tenet of Gestalt theory (Winn, 2004). This influence was systematically organized by Wertheimer (1938) as the Gestalt laws of perceptual organization, and includes design principles such as good figure, figure-ground separation, and continuity. With respect to designing multimedia, Mayer (2005) advises designers to make use of basic multimedia, spatial contiguity, temporal contiguity, modality, redundancy, and coherence principles to reduce extraneous cognitive processing. Each of these principles addresses separate, but related design considerations. First, designers should combine text with content-related images only when learners have low prior knowledge and possess sufficient cognitive abilities to process both the text and pictures. This combination is known as the basic multimedia principle. Second, the spatial contiguity principle recommends presenting written text in close spatial proximity to related images. The temporal contiguity principle takes this concept a step further and suggests presenting spoken text in close temporal proximity to related images. The modality principle proposes the use of spoken words instead of written text for animation. Related to this, the specific redundancy principle clearly states that written text should not duplicate spoken words. Lastly, the coherence principle advises against the use of extraneous words and pictures or unnecessary sound or music. Combined, these principles represent an arsenal of tools to be used by instructional and multimedia designers to capitalize on learners cognitive capabilities to receive and process information. If

DOCTORAL COMPS

22

educational media is to maintain relevance, cognitive load should be a primary concern when designing instruction (Sweller, van Merrinboer, & Paas, 1998). Yet, even though researchers have posited these guidelines, mainstream instructional design practices have yet to fully embrace to psychological theories (Winn, 2004). Perhaps then, a closer look at media design is in order to prescribe updated guidelines and recommendations. Selecting & Designing Media Before designers even begin considering how to design media for educational purposes, instructional design guidelines prescribe what to design. Instructional design is a systematic process that focuses on improving the effectiveness and efficiency of learning and instruction in various environments (Lowe & Schwen, 1975). Regardless of application, past research has shown that a systematic approach to instructional design is both logical and useful (Andrews & Goodson, 1980). Instructional designers using a systematic approach follow prescribed guidelines that impact aspects of the learning environment from instructional goals to media selection. Tosti and Ball (1969) noted that steps leading to design include selecting media to fit the prescribed presentation. When it comes to selecting media, the he importance of this task has not changed over the past 40 years even though processes and models have evolved. Dick et al., (2009) recommend making appropriate decisions regarding media selection after decisions have been made about learning components and content. Additionally, designers are tasked with selecting the best way to package essential instructional methods based on available resources and the cost-effective qualities of media attributes for specific learners and learning goals (Clark, 1983, p. 23)Related to media selection, Principle 3 of Merrill's (2002) First Principles of Instruction establishes a corollary for relevant media that posits, learning is promoted when media play a relevant instructional role and

DOCTORAL COMPS

23

multiple forms of media do not compete for the attention of the learner (p. 48). Although instructional design processes recommend careful consideration for selecting media, it becomes evident that the process is complex. Media selection can be aided by taking into consideration recommended guidelines. Novice and advanced designers should heed Schramm's (1977) caution that no procedure of selecting media applies to every instructional situation. That said, Reiser & Gagn (1982) identified six common factors in media selection; instructional setting, learner characteristics, learning outcomes, events of instruction, physical media attributes, and practical factors of the media. A more recent review of media selection models conducted by Richey et al., (2010) identified five major elements in media selection; content, learner characteristics, instructional strategies, environment, and management. In a quick comparison and contrast between the two lists, many of Reiser and Gagns observations have been combined into broader descriptions. For example, management could encompass physical media attributes and practical factors. However, it is important to note that both lists specifically identify learner characteristics as an element. That the learner is a central component of the two studies conducted 30 years apart denotes a high level of importance in the process. Before addressing how learner characteristics fit within the design framework, it may be helpful to further explore current trends in media selection. As Snyder (2009) points out, improvements in information technology have provided opportunities and challenges for educators to design, develop, and deliver effective instruction. Further support for this observation comes from Richey et al., (2010) as they found that newer media selection models focus on e-learning environments, including web- and computer-based instruction that facilitates any time, anywhere learning. The authors go on to note that these newer media selection models also emphasize

DOCTORAL COMPS

24

learner activity and engagement, components that are dependent upon learner characteristics. Using the multiple resources available, designers should be well equipped to adequately address media selection. However, selecting media for learners is only the beginning and consideration for designing media must occur simultaneously with selecting media. Anglin, Towers, and Levis (1996) posit that the effective use of graphics in designing instruction is an important facet of instructional message design. This may be due to the finding that up to 40 percent of conceptual learning can be attributed to visual experience (Weber, 1922). Media are largely comprised of visual messages and have historically included photographs, drawings, diagrams, maps, and film. McKenzie (2005) has noted that while the medium may not be the message, it is a significant part of the learning experience. Media, and specifically multimedia, can make a significant contribution to curriculum by representing real objects and ideas about reality that may not otherwise be possible (Cohen, 2010). Additionally, using images in instructional materials is effective in supporting learning, because they can help gain a learners attention and help learners interpret and remember the context of illustrated texts (Park & Lim, 2007). Traditionally, textbooks have used images and illustrations sparingly and in a secondary role to conveying content. As learners who are accustomed to multimedia environments become more prevalent, this method will not be able to gain or hold readers attention for very long (Slough & McTigue, 2010). Though advancements in technology have enabled designers to broaden visual messages to include video, animations, and icons (Richey et al., 2010), Baker and Dwyer (2000) caution that not all elements of visuals are equally important for instruction. For instance, color arouses interest and relates to interest, but realistic details may distract learners from the primary task. Perhaps most significantly, Cohen (2010) stresses that multimedia selection and design must consider issues of

DOCTORAL COMPS

25

cognitive load. By considering the instructional attributes of multimedia, a foundation can be created to assess when and how to specific elements in courses. Having guidelines to design media is helpful, but further review of the literature reveals that existing recommendations may not be enough in terms of addressing learning characteristics. Learner Interest As instructional design research continues to evolve and focus on specific efficiencies, the role of the learner is becoming more important. Keller (2010) has observed that research focusing on learner motivation has the potential to impact course and lesson design. Similarly, Kozma (1991; 1994) cited multiple media comparison studies that indicated increased motivation, increasing how long learners were willing to spend interacting with content. This finding coupled with Kellers (1987) ARCS model supports the assertion that increased motivation and time on task increases learning outcomes. Further research has also established a positive link between individual student interest and academic achievement, and the concept of interest is viewed as one of the critical positive emotions in learning contexts (Park & Lim, 2007; Schroff & Vogel, 2010). Similarly, Schraw, Flowerday, and Lehman (2001) noted that interest increases learning and believe that promoting interest increases students intrinsic motivation to learn. These findings also relate to the correlation between positive emotions, such as interest, and cognitive processes, including information processing, decision-making, and creative problem-solving (Isen, Daubman, & Nowicki, 1987; Isen, Johnson, Mertz, & Robinson, 1985; Picard, 1997). Taking into consideration earlier challenges identified with technology-enhanced learning, it appears that specifically designing media to enhance learner interest could lead to better achievement.

DOCTORAL COMPS

26

It is important to note that while not a type of motivation, interest plays a significant role in influencing motivation (Schunk, Pintrich, & Meece, 2008). Further, students interested in a topic may display motivated behaviors, such as choice of the activity, effort, persistence, and achievement. Research exploring the effect of motivation on metacognition has found that when students attempt to complete a course, they are either interested in the content, motivated to attain a goal of importance, or both (Tobias, 2006). Incorporating motivational variables, such as interest, into multimedia design will become an important task if instruction is to provide learners with relevant learning experiences (Fletcher & Tobias, 2005). In a study aimed at making scientific textbook lessons more interesting, Harp and Mayer (1997) found that promoting cognitive interest could be done by adding signals for structural understanding such as summary illustrations with captions. However this finding barely scratches the surface and understanding what interest encompasses would help designers in considering how to incorporate interest into the design process (e.g., media selection and design). Generally speaking, interest can be categorized into one of two subgroups; individual interest and situational interest. Individual, or personal, interest resides within a person, associates positive feelings with a topic or activity, and attributes personal significance to the topic or activity (Hidi & Anderson, 1992; Rathunde, 1993; Renninger, 2000; Renninger, Hidi, & Krapp, 1992; Schiefele, 1991). Conversely, situational interest (SI) emerges as a response to features or effects within an environment (Hidi & Anderson, 1992; Hidi & Baird, 1986; Hidi & Renninger, 2006; Krapp, 2002). Examining situational interest further, there are attention and affective reactions that can be differentiated into triggered-SI and maintained-SI (Hidi & Baird, 1986; Hidi & Harackiewicz, 2000; Hidi & Renninger, 2006; Krapp, 2002; Mitchell, 1993). Where triggered-SI is the initiation or arousal

DOCTORAL COMPS

27

of interest (Hidi, 2001; Hidi & Harackiewicz, 2000; Hidi & Renninger, 2006), maintained-SI is where interest is held and individuals begin to connect with the content (Hidi, 2001; Mitchell, 1993). With respect to instructional situations, attempting to design materials aimed at affecting individual interest is challenging and impractical. However, improving situational interest in learning environments should be a fundamental concern (Park & Lim, 2010). In a review of the literature to explore ways to increase situational interest in the classroom, Schraw, Flowerday, and Lehman (2001), indicated vividness of text as a variable affecting interest. The authors define vividness as segments that stand out because they create suspense, surprise, or are otherwise distinctive (p. 217). Based on previous research, vivid segments are more memorable than less vivid segments, and vividness is related positively to interest and recall (Schraw, Bruning, & Svoboda, 1995). Therefore, learning strategies that take situational interest into consideration when designing instruction have the potential to help students be engaged and focused. However, the effects of triggered-SI can be short-lived if maintained-SI is not adequately considered. As a result of a validity study on the Situational Interest Survey, Linnenbrink-Garcia et al., (2010) found that triggered-SI reflects a and maintained-SI refers to the reaction secondary and post-secondary learners have to the material itself. Based upon these findings, it will be important to continue to examine situational interest across educational settings to further investigate what instructional practices can be designed to promote situational interest. The cumulative effect of these various studies contributes to the importance of learner interest and calls for further examination of how to effectively measure interest. Once research into learner interest is further explored, the potential for improved media design guidelines begins to emerge.

DOCTORAL COMPS

28

Item #4 Methodology. Provide a synthesized review of literature relevant to: A. The problem of your study B. The need for your study C. Theoretical foundations of your study D. The purpose of your study E. Your research questions This synthesis should be written in a style that flows logically. Only the most relevant research literature should be cited and briefly summarized. The synthesis should combine the above elements in an order that could be expected to make sense in the context of a grant proposal for a review panelist. Research Methodology Identifying the Problem The problem that many learners experience in media-rich learning environments is that the information is presented in a design that negatively impacts interest, thus resulting in a decreased intrinsic motivation to continue or complete the lesson. Multiple studies have highlighted the challenge of maintaining learner motivation, self-confidence, and achievement in technology-enhanced courses (Carr, 2000; Meyer, 2003; Wojciechowski & Palmer, 2005). Given that research has shown how positive perceptions may assist in maintaining students interest in content, it may be worthwhile to analyze and address learners perceptions of multimedia (Moreno, Mayer, Spires, & Lester, 2001). To address this problem and propose updated design guidelines, a study is proposed to survey adult learners in a continuing education setting to determine the effect of media design on learners situational interest.

DOCTORAL COMPS

29

Significance Media designers already use design principles based on research related to improving learner cognition. However, Keller (1983; 2010) and Linnenbrink-Garcia et al., (2010) suggest that motivation and interest have been neglected as an influence on understanding and achievement. Further, it is important to address make the learning experience as positive as possible, ensuring that materials are useful and engaging enough to make the learning process desirable (Yu, JannaschPennell, & DiGangi, 2008; Yu, Jannasch-Pennell, DiGangi, & Kaprolet, 2009). The researchers also point out that students exposed to multimedia in instruction report an enhanced motivation to learn the subject matter, regardless of the topic or level of difficulty. Similarly, multimedia presentations that incorporate text, graphics, and animations have been shown to result in increased learner interest (Koeber, 2005; Nowaczyk, Santos, & Patton, 1998; Wekesa, Kiboss, & Ndirangu, 2006; Yaverbaum, Kulkarni, & Wood, 1997). As instructional designers are driven by an increased demand to increase learning opportunities while simultaneously reducing costs without adversely affecting instructional quality, the challenge remains to find the right combination of constructive media (Holden & Westfall, 2010). Thus, research to support design considerations that enhance interest may have an impact on both practice and future research. Based upon the outcome of the proposed study, a number of outcomes could occur. If the proposed hypotheses are contradicted by research findings, then future directions for research may recommend altering instruments attempting to measure situational interest or addressing other aspects of the design environment not studied. If the proposed hypotheses are supported by research results, then improved design principles that address both cognition and situational interest of learners may be recommended for practice and future research. Implications may

DOCTORAL COMPS

30

also include examining the other multimedia design principles for impact on situational interest. Further, supported hypotheses would strengthen the arguments to design for motivation and interest. Research Questions The questions summarized in Table 4 will be used to guide the data collection, summary, and framework for organizing the results of this study. Table 4. Cognitive Tools for Research Method Selection Research Question 1. What are the effects of modality and redundancy design principles on SI-triggered? 2. What are the effects of modality and redundancy design principles on SI-maintained? 3. What is the effect of media designed to affect situational interest on learner outcomes? Research Method Design The design of the proposed study is a two-group posttest-only randomized experiment. The participants will randomly assigned to either a treatment group or a control group. The independent variables are the design principles of modality and redundancy. These specific design principles were selected based upon their inherent related nature as well as previous research findings that even if audio neither hinders nor helps multimedia learning, audio can hold learners interest and help develop positive perceptions of the content presented (Yu et al., 2009). Participants will be unaware of the different conditions. There only dependent variable in the study is situational interest. Data Type Statistical (Descriptive statistics) Statistical (Descriptive statistics) Statistical (Descriptive statistics) Data Source Survey questions 1-4 Survey questions 5-12 Final assessment score

Analysis Procedure 2-way ANOVA 2-way ANOVA Regression/ Correlation

DOCTORAL COMPS

31

Setting & Participants The context for this research study is a continuing education environment. Learners are emergency medical personnel certified at the national and/or state level as Basic Emergency Medical Technicians (EMT-Basic), Intermediate Emergency Medical Technicians (EMT-Intermediate), and Paramedics. The learners are all employed by a private emergency medical service based in the southeastern United States, and are required to complete regularly scheduled continuing education and professional development training. Traditionally, all training is designed and delivered internally by the organization. The researcher will conduct the study over a period of six months, during which time a complete training program comprised of multiple topics is offered to emergency medical personnel. This plan allows for the researcher to fully capture the effects of the design treatment on the learners. All media-based training materials delivered during the program will be redesigned to accommodate the specified treatment groups. Approximately 12 self-paced training topics will be delivered during this timeframe. Learners complete these lessons on their own time and must complete the training within a month of being assigned the topic. Approximately 6 training topics will delivered face-to-face in a traditional classroom setting. Each training topic delivered face-to-face is offered in two sections to accommodate learners schedules. After completion of each redesigned training topic, both self-paced and face-to-face, learners will complete a self-report questionnaire, to measure learners reactions to the presentation as enjoyable and engaging (triggered-SI) and the content as enjoyable and meaningful (maintainedSI). Design Treatment The Control Group will receive existing training materials that have been created by a subject matter expert and training coordinator. The researcher has

DOCTORAL COMPS

32

agreed to redesign multimedia training to be delivered during the study. An instructional designer and multimedia expert will review the redesigned materials to verify adherence to the proposed multimedia design principles. Table 5 summarizes the proposed treatment design. The M Group will receive training materials that have been redesigned to adhere to the modality design principle. The R Group will receive training materials that have been redesigned to adhere to the redundancy design principle. The RM Group will receive training materials that have been redesigned to adhere to both the modality and redundancy design principles. Table 5. Proposed Research Design Present Absent Redundancy Present Absent RM Group M Group R Group Control Group

Modality

Training materials redesigned to address the modality design principle will have text-based content transferred into narration. Existing narration in the original materials that accompanies text-based content will not be altered for the M Group. This leaves the possibility open that design violates the redundancy design principle. Training materials redesigned to address the redundancy principle will eliminate instances where existing narration repeats the text-based content already present. Existing text-based content in the original training materials that is not accompanied by narration will not be transferred into narration for the R Group. This leaves the possibility open that design violates the modality design principle. Training materials redesigned to address both the modality and redundancy design principles will have existing text-based content transferred to narration and remove any existing text-based content that is accompanied by existing narration. This eliminates the possibility that either design principle has been violated. These two design principles were chosen for multiple reasons. First, the two design principles

DOCTORAL COMPS

33

are inter-related as they deal with spoken words versus written text. Second, in a review of the literature, only a few studies were identified that specifically address modality and redundancy beyond the original collective research prescribing and addressing all six of the design principles (Aldalalah & Fong, 2010; Harskamp, Mayer, & Suhre, 2007; Moreno, 2006). Participants will be recruited from the group of learners who are employed at an emergency medical services organization and are required to complete the regularly scheduled continuing education training program to be delivered during the six-month period. Learners will be allowed to opt out of the study at any time during the training program delivery. At the beginning of each training topic, research participants will be randomly assigned into one of the four groups identified in the research design. If possible, an equal number of participants will be assigned to each group. Power Analysis Based on the research questions, the researcher will examine the main effect of each design principle as well as an interaction effect between the two design principles. Using the software program G*Power to input effect size, error, power size, degrees of freedom and number of groups, the researcher has estimated that a sample size of 94 participants will be required for a pilot study. Figure 5 illustrates the power analysis conducted to reach this estimate.

DOCTORAL COMPS

34

Figure 5. Power analysis output from G*Power. The researcher has elected to use a large effect size for the purpose of conducting a pilot study to determine if a more in depth study is necessary. Further, a power of .8 was selected based upon the appropriateness of the value for research in the behavioral sciences (Keppel & Wickens, 2004). Given that there are four proposed groups for the study, the researcher will assign 23-24 participants per group based upon enrollment in each training program. Measures With respect to dependent variables in the proposed study, the researcher will be looking for effects on triggered-SI and maintained-SI. The instrument to measure situational interest has been adapted from the Linnenbrink-Garcia et al., (2010) scale known as the Situational Interest Survey (SIS) for the proposed study

DOCTORAL COMPS

35

context and will have a total of 12 statements to be measured on a 5-point Likert scale, from strongly agree to strongly disagree. Table 6 illustrates the original SIS items and resulting modified instrument statements for the proposed study. The researcher has asked experts, including the original instrument authors, to evaluate the proposed scale statements, in order to address initial validity concerns related to Standard 1.4 from the Standards for Educational and Psychological Testing, which holds the researcher responsible for using the scale in a way that has not been previously validated (AERA, APA, & NCME, 1999). Learners final assessment scores from each training program will also be captured in order to explore indirect effects of situational interest on achievement for further discussion and validation purposes. Table 6 Proposed SIS Items Interest Type SI-triggered SI-triggered Original 1. My math teacher is exciting 2. When we do math, my teacher does things that grab my attention 3. This year, my math class is often entertaining 4. My math class is so exciting its easy to pay attention 5. What we are learning in math class this year is fascinating to me 6. I am excited about what we are learning in math class this year 7. I like what we are learning in math this year Proposed 1. The multimedia presentation was interesting. 2. The multimedia presentation grabbed my attention. 3. The multimedia presentation was often entertaining. 4. The multimedia presentation was so exciting, it was easy to pay attention. 5. What I learned in the multimedia presentation is fascinating to me. 6. I am excited about what I learned in the multimedia presentation. 7. I like what I learned in the multimedia presentation.

SI-triggered

SI-triggered

SI-maintained

SI-maintained

SI-maintained

DOCTORAL COMPS

36

Table 6 Proposed SIS Items Interest Type SI-maintained Original 8. I find the math we do in class this year interesting 9. What we are studying in math class is useful for me to know 10. The things we are studying in math this year are important to me 11. What we are learning in math this year can be applied to real life 12. We are learning valuable things in math class this year Proposed 8. I found the information in the multimedia presentation interesting. 9. What I studied in the multimedia presentation is useful for me to know. 10. The things I studied in the multimedia presentation are important to me. 11. What I learned in the multimedia presentation can be applied to my job. 12. I learned valuable things in the multimedia presentation.

SI-maintained

SI-maintained

SI-maintained

SI-maintained

Hypotheses The researcher possesses a number of expectations regarding the research study outcomes. Specifically, it is postulated that violation of the redundancy design interest will negatively impact SI-triggered. This prediction is based upon the previously discussed phenomenon of triggering as a positive reaction to the way material is presented and the negative cognitive impact inherent to redundancy. If learners are inundated with the same material in audio and text format, they may experience a negative reaction that inhibits motivation to continue the lesson, which may also have a long-term effect on SI-maintained. Further, the researcher does not expect any impact on situational interest with respect to the modality design principle. Although modality has been positively correlated with cognitive influence in learners, mode could be perceived as a personal preference in terms of interest. Still, the researcher feels that evaluating the influence of modality may have an

DOCTORAL COMPS

37

impact on future design principles. In other words, if no impact is observed, then either a modified modality principle may be recommended to designers to attempt to address both cognition and interest or designers may elect to ignore the design principle if seeking to address only learner interest. With respect to the control group, no impact on either SI-triggered or SI-maintained is expected as the original training program design fails to take into consideration any form of motivation design as prescribed by previous research (Hidi, 2001; Hidi & Baird, 1986; Keller, 1983; 2008). As a result of these expectations, a total of nine hypotheses are proposed. A simple way of viewing these hypotheses in terms of the study design is RM>R>M=C in terms of effect on situational interest. With regard to learner achievement, the hypotheses summary is RM=R>M=C. In other words, design considerations that positively affect situational interest will also positively affect learner achievement. 1. H1: There will be no significant effect on situational interest in the Control Group. 2. H2: There will be no effect on situational interest in the M Group. 3. H3: There will be a positive effect on situational interest in the R Group. 4. H4: There will be a positive effect on situational interest in the RM Group. 5. H5: Learners in the RM Group and R Group will have higher achievement scores than learners in the M Group and Control Group.

DOCTORAL COMPS

38

Item #3 Practice. Using the Research and Theory of Richard Mayer, Roxana Moreno, Wolfgang Schnotz, and John Sweller related to the Cognitive Theory of Multimedia Learning Principles of Modality and Redundancy, produce a design document and corresponding artifact that clearly embodies this perspective. Practice Design Training Purpose Emergency Medical Technicians must periodically complete continuing education courses depending upon level of certification, years of service, employing agency or service, and other factors. If an agency or service acquires new equipment, it becomes necessary to train personnel on how to properly use the tools before they are deployed in the field. Similarly, if a service notices an increase in patient complaints or observe improper techniques in the field, refresher training may be offered to remind medics about proper use. This latter reasoning is the premise for the Review of Anatomy and Function of the Upper and Lower Airways In-Service for the Supraglottic Airway Laryngopharangeal Tube training program. Analysis Summary All medics currently employed by the cooperating agency for the previously mentioned study will be required to complete a PowerPoint-based multimedia lesson that has been developed by the assistant director of training. Learners will have 30 days in which to complete the training, and it should take approximately 2 hours for the average learner to complete the lesson. The learning objectives for the lesson, as provided by the assistant training director, are as follows: 1. Name the major components of the upper and lower airways 2. Describe the functions of the upper and lower airways 3. Describe the process of ventilation 4. Describe the process of respiration

DOCTORAL COMPS

39

5. Become familiar with the S.A.L.T. 6. Become proficient in the use of the S.A.L.T. 7. Be familiar with the SMO for the S.A.L.T. The existing PowerPoint file contains 27 slides, including a title slide and references. At the conclusion of the lesson, leaners will be expected to demonstrate proper treatment guidelines on a Transparent Anatomical Manikin (TAM) in order to receive credit for completing the training. Design Summary Content Outline Based upon the original materials, the following content outline is apparent: 1. National EMS Training and Education 2. Respiratory Review and S.A.L.T. In-Service 3. Objectives 4. Terminology 5. Upper and Lower Airways a. Upper Airway Anatomy b. Lower Airway Anatomy 6. Ventilation vs. Respiration 7. Alveolar Function 8. Supraglottic Airway Laryngopharangeal Tube (S.A.L.T.) 9. Airway Management Procedures Design Document The design details that appear in Table 7 include both feedback to the assistant training director and instructions to the researcher for modifications to be made to the original PowerPoint lesson. The original lesson, depicted in thumbnail images within the table, will be provided to the Control Group. Instructions for

DOCTORAL COMPS

40

modifications made based on the different treatment types are identified by R for redundancy, M for modality, and RM for combined redundancy and modality. As previously mentioned, the modality principle states that if animation is used, use spoken text instead of written text and the redundancy principle states that designers should not add written text that duplicates spoken text combined with pictures (Mayer, 2005, p. 65). For the purposes of this redesign, the RM treatment will adhere to both principles, but the R and M treatments will include variations that may violate the related, but untested principle. Note that the designers comments begin with the RM treatment and work down into the individual principles. This combined treatment explanation should be considered when reading the other two comments as it serves as the basis for design. The R and M comments indicate alterations to the RM treatment that will bring the design into alignment for each individual treatment. Table 7 Design Modifications # Original Slide (Control Group) Notes

DOCTORAL COMPS

41

Table 7 Design Modifications # Original Slide (Control Group) Notes RM: Suggest attention-grabbing introduction. Perhaps moving Slide #17 here with audio narration of scenario involving intubating larger patients could capitalize on the first of Gagns Nine Events of Instruction, Gain Attention (Gagn et al., 2005). Multiple pictures should be used or a video created. M: Include image and audio, but present transcript of audio below the image. R: Use only Slide #17 and the narration. No written text. 3 Suggest re-wording the last three objectives to measurable statements that begin with action verbs.

DOCTORAL COMPS

42

Table 7 Design Modifications # Original Slide (Control Group) Notes Use with Slides #5 & #6 for audio/text content, delete existing Slide.

RM: Convert to animation to separate upper from lower and indicate individual components of Upper Airway. Color highlighting to signal active/inactive selection. Audio optional (controlled by user) for pronunciation and definition. M: Audio mandatory; read terms as they appear and brief definition. Text for definition appears during corresponding audio, but fades out. R: Audio mandatory; read terms as they appear and brief definition. No definition text.

DOCTORAL COMPS

43

Table 7 Design Modifications # Original Slide (Control Group) Notes RM: Convert to animation. Color highlighting to signal active/inactive selection. Audio optional (controlled by user) for pronunciation and definition. M: Audio mandatory; read terms as they appear and brief definition. Text for definition appears during corresponding audio, but fades out. R: Audio mandatory; read terms as they appear and brief definition. No text for definition. RM: Convert to animation. Color highlighting to signal active/inactive selection. Audio optional (controlled by user) for pronunciation and definition. M: Audio mandatory; read terms as they appear and brief definition. Text for definition appears during corresponding audio, but fades out. R: Audio mandatory; read terms as they appear and brief definition. No text for definition.

DOCTORAL COMPS

44

Table 7 Design Modifications # Original Slide (Control Group) Notes Use with Slides #7 & #9 for audio/text content, delete existing Slide.

RM: Convert to animation. Color highlighting to signal active/inactive selection. Audio optional (controlled by user) for pronunciation and definition. M: Audio mandatory; read terms as they appear and brief definition. Text for definition appears during corresponding audio, but fades out. R: Audio mandatory; read terms as they appear and brief definition. No text for definition.

DOCTORAL COMPS

45

Table 7 Design Modifications # Original Slide (Control Group) Notes RM: Convert to animation to separate upper from lower and indicate individual components of Lower Airway. Color highlighting to signal active/inactive selection. Audio optional (controlled by user) for pronunciation and definition. M: Audio mandatory; read terms as they appear and brief definition. Text for definition appears during corresponding audio, but fades out. R: Audio mandatory; read terms as they appear and brief definition. No definition text. 11 RM: Convert to animation. Color highlighting to signal active/inactive selection. Audio optional (controlled by user) for pronunciation and definition. M: Audio mandatory; read terms as they appear and brief definition. Text for definition appears during corresponding audio, but fades out. R: Audio mandatory; read terms as they appear and brief definition. No text for definition.

10

DOCTORAL COMPS

46

Table 7 Design Modifications # Original Slide (Control Group) Notes Use with Slides #10 & #11 for audio/text content, delete existing Slide.

12

13

RM: Create animated cycles that illustrate both principles. Audio narration describes the process in steps. No text on screen. M: Text summary for the process appears on screen, but fades as audio advances to next step. R: No text summary for the process; audio only.

DOCTORAL COMPS

47

Table 7 Design Modifications # Original Slide (Control Group) Notes Use with Slide #13 for audio/text content, delete existing Slide.

14

15

RM: Create comparison animation to illustrate statement. Audio narration describes the components and similarities. M: Text summary for components and similarities appears on screen, but fades as audio advances. R: No text summary for the components; audio only.

DOCTORAL COMPS

48

Table 7 Design Modifications # Original Slide (Control Group) Notes RM: Create animated cycles that illustrate both principles. Audio narration describes the process in steps. M: Text summary for components, but fades as audio advances. R: No text summary for the components; audio only.

16

17

Incorporate into attention gaining strategy at beginning of lesson (Slide #2).

DOCTORAL COMPS

49

Table 7 Design Modifications # Original Slide (Control Group) Notes RM: Identify an animated image of the tool that best illustrates the design. Convert text to audio explanation. M: Summary of text appears and fades as audio progresses. R: No text.

18

19

Combine with Slide #18, using text for audio narration.

DOCTORAL COMPS

50

Table 7 Design Modifications # Original Slide (Control Group) Notes Slides #20-23 are best illustrated with an instructional video. The manufacturer has one available on Vimeo or other emergency medical services have produced their own that are available on YouTube. Designer recommends selecting either #1 or #2 and #3. 1. http://www.youtube.com/watch?v =VHaIufNEjXU 2. http://www.youtube.com/watch?v =iNaDGxxOkW0 3. http://www.youtube.com/watch?v =Ep8SFfCzR1I 21

20

DOCTORAL COMPS

51

Table 7 Design Modifications # Original Slide (Control Group) Notes

22

23

DOCTORAL COMPS

52

Table 7 Design Modifications # Original Slide (Control Group) Notes Suggest deleting this slide and not using this video. It does not contain audio or text instructions to accompany the actions depicted, and would be less effective than those suggested above.

24

25

RM: Combine Slides #25 & #26 into one instructional narration with still photographs to illustrate appropriate steps. M: Summary of text appears and fades as audio progresses. R: No text during narration; summary of text appears at conclusion.

DOCTORAL COMPS

53

Table 7 Design Modifications # Original Slide (Control Group) Notes

26

Update sources based upon redesign. 27 Include link to SIS survey.

DOCTORAL COMPS

54

References AERA, APA, & NCME. (1999). Standards for Educational and Psychological Testing. The Standards for Educational and Psychological Testing (2nd ed.). Washington, D.C. American Educational Research Association. Aldalalah, O. M. A. A., & Fong, S. F. (2010). Effects of modality and redundancy principles on the learning and attitude of a computer-based music theory lesson among Jordanian primary pupils. International Education Studies, 3(3), 52-64. Andrews, D. H., & Goodson, L. A. (1980). A comparative analysis of models of instructional design. Journal of Instructional Development. Anglin, G. J., Towers, R. L., & Levis, W. H. (1996). Visual message design and learning: The role of static and dynamic illustrations. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (1st ed., pp. 755-794). New York, NY: Simon & Schuster Macmillan. Baddeley, A. D. (1986). Working memory. Oxford, England: Oxford University Press. Baddeley, A. D. (1999). Human Memory. Boston: Allyn & Bacon. Baker, R., & Dwyer, F. (2000). A meta-analytic assessment of the effect of visualized instruction. International Journal of Instructional Media, 27(4), 41726. Barson, J. (1967). Instructional systems development: A demonstration and evaluation project: Final report. East Lansing, MI. Bloom, B. S. (1956). Taxonomy of Educational Objectives: Handbook I. New York: David McKay Co, Inc. Bloom, B. S. (1976). Human characteristics and school learning. New York: McGrawHill. Branch, R. M. (2009). Instructional Design: The ADDIE Approach. New York: Springer.

DOCTORAL COMPS

55

Burton, J. K., & Merrill, P. F. (1991). Needs assessment: Goals, needs, and priorities. In L. J. Briggs (Ed.), Instructional design: Principles and applications (2nd ed.). Englewood Cliffs, NJ: Educational Technology Publications. Carr, S. (2000). As distance education comes of age, the challenge is keeping the students. Chronicle of Higher Education, A39. Carroll, J. (1963). A model of school learning. The Teachers College Record, 64(8), 723723. Teachers College Record. Chandler, P., & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8(4), 293-332. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445. Clark, R. E. (2001). Learning from media: arguments, analysis, and evidence. Greenwich Conn. Information Age Pub. Cobb, T. (1997). Cognitive efficiency: Toward a revised theory of media. Educational Technology Research and Development, 45(4), 21-35. Cohen, D. E. (2010). The Online Resource Selection Instructional Design Script (ORSIDS) and implications for the widespread diffusion of learning objects. Technology, Instruction, Cognition and Learning, 8, 67-96. Diamond, R. M. (1989). Designing and Improving Courses and Curricula in Higher Education: A Systematic Approach. San Francisco, CA: Jossey-Bass Inc., Publishers. Dick, W., Carey, L., & Carey, J. O. (2009). The Systematic Design of Instruction (7th ed.). Columbus: OH: Pearson Education, Inc. Fletcher, J. D., & Tobias, S. (2005). The multimedia principle. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (p. 117133). New York: Cambridge University Press.

DOCTORAL COMPS

56

Gagn, R. M., Wager, W. W., Golas, K. C., & Keller, J. M. (2005). Principles of Instructional Design (5th ed.). Belmont, CA: Wadsworth Publishing Company. Gustafson, K. L., & Branch, R. M. (1997). Revisioning models of instructional development. Technology Research and Development, 45(3), 73-89. Gustafson, K. L., & Branch, R. M. (2002). Survey of instructional development models (4th ed.). Syracuse, NY: ERIC Clearinghouse on Information & Technology. Harp, S. F., & Mayer, R. E. (1997). The Role of Interest in Learning From Scientific Text and Illustrations : On the Distinction Between Emotional Interest and Cognitive Interest. Journal of Educational Psychology, 89(1), 92-102. Harskamp, E. G., Mayer, R. E., & Suhre, C. (2007). Does the modality principle for multimedia learning apply to science classrooms? Learning and Instruction, 17(5), 465-477. Hastings, N. B., & Tracey, M. W. (2005). Does media affect learning: Where are we now? TechTrends, 49(2), 28-30. Hidi, S. (2001). Interest, reading, and learning: theoretical and practical considerations. Educational Psychology Review, 13(3), 191209. Hidi, S., & Anderson, V. (1992). Situational interest and its impact on reading and expository writing. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 215-238). Hilldale, NJ: Lawrence Erlbaum. Hidi, S., & Baird, W. (1986). Interestingnessa neglected variable in discourse processing. Cognitive Science, 10(2), 179194. Hidi, S., & Harackiewicz, J. M. (2000). Motivating the academically unmotivated: a critical issue for the 21st century. Review of Educational Research, 70, 151179.

DOCTORAL COMPS

57

Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111127. Holden, J. T., & Westfall, P. J. (2010). Developing a blended learning strategy: An instructional media perspective. Journal of Instruction Delivery Systems, 24(1). Isen, A. M., Daubman, K. A., & Nowicki, G. P. (1987). Positive affect facilitates creative problem solving. Journal of Personality and Social Psychology, 52(6), 1122-1131. Isen, A. M., Johnson, M. M. S., Mertz, E., & Robinson, G. F. (1985). The influence of positive affect on the unusualness of word associations. Journal of Personality and Social Psychology, 48(6), 1413-1426. Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task Analysis Methods for Instructional Design. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Ed.), Instructional-design theories and models: An overview of their current status (pp. 383-434). Hillsdale N.J. Lawrence Erlbaum Associates, Publishers. Keller, J. M. (1987). The systematic process of motivational design. Performance and Instruction, 26(10), 1-8. Keller, J. M. (2008). First principles of motivation to learn and e-learning. Distance Education, 29(2), 175-185. Keller, J. M. (2010). Motivational design for learning and performance. Media. Boston, MA: Springer US. Keller, J. M. (2010). Challenges in learner motivation: A holistic, integrative model for research and design on learner motivation. The 11th International Conference on Education Research.

DOCTORAL COMPS

58

Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researchers handbook (4th ed.). Upper Saddle River, NJ: Pearson Education, Inc. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating Training Programs: The Four Levels (3rd ed.). San Francisco, CA: Berrett-Koehler Publishers, Inc. Koeber, C. (2005). Introducing multimedia presentations and a course website to an introductory sociology course: How technology affects student perceptions of teaching effectiveness. Teaching Sociology, 33(3), 285-300. Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61(2), 179-211. doi: 10.2307/1170534. Kozma, R. B. (1994). Will media influence learning: Reframing the debate. Educational Technology Research and Development, 42(2), 719. Krapp, A. (2002). An educational-psychological theory of interest and its relation to SDT. In E. L. Deci & R. M. Ryan (Eds.), The handbook of self-determination research (pp. 405-427). Rochester, NY: Rochester University Press. Krathwohl, D. R. (2002). A revision of Blooms taxonomy. Theory into Practice, 41(4), 212-218. Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., et al. (2010). Measuring situational interest in academic domains. Educational and Psychological Measurement, 70(4), 647. Lowe, A. J., & Schwen, T. M. (1975). The documentation of instructional development. Educational Technology, 15(6), 43-46. Mager, R. F. (1997). Preparing Instructional Objectives: A Critical Tool in the Development of Effective Instruction. Atlanta, GA: The Center for Effective Performance, Inc. Magliaro, S. G., & Shambaugh, N. (2006). Student models of instructional design. Educational Technology Research and Development, 54(1), 83-106.

DOCTORAL COMPS

59

Marcus, N., Cooper, M., & Sweller, J. (1996). Understanding instructions. Journal of Educational Psychology, 88(1), 49-63. Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press. Mayer, R. E. (2002). Multimedia learning. In B. H. Ross (Ed.), (pp. 85-139). San Diego, CA: Academic Press. Mayer, R. E. (Ed.). (2005). The Cambridge Handbook of Multimedia Learning (p. 680). New York, NY: Cambridge University Press. McKenzie, W. (2005). Multiple Intelligences and Instructional Technology. Medical device technology (2nd ed., Vol. 19). Eugene, OR: International Society for Technology in Education. Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59. Meyer, K. A. (2003). The webs impact on student learning. Technical Horizons in Education Journal, 30, 184-187. Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school mathematics classroom. Journal of Educational Psychology, 85, 424 424. Moreno, R. (2006). Does the modality principle hold for different media? A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning, 22(3), 149-158. Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358-368. Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they

DOCTORAL COMPS

60

interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177-213. doi: 10.1207/S1532690XCI1902_02. Nieveen, N. M. (1997). Computer support for curriculum developers: A study on the potential of computer support in the domain of formative curriculum evaluation. University of Twente, Enschede, The Netherlands. Nowaczyk, R. H., Santos, L. T., & Patton, C. (1998). Student perception of multimedia in the undergraduate classroom. International Journal of Instructional Media, 25(4), 367-382. Paivio, A. (1986). Mental representations: a dual coding approach. New York: Oxford University Press. Park, S., & Lim, J. (2007). Promoting positive emotion in multimedia learning using visual illustrations. Journal of Educational Multimedia and Hypermedia, 16(2), 141-162. Picard, R. W. (1997). Affective computing. Cambridge, MA: The MIT Press. Rathunde, K. (1993). The experience of interest: A theoretical and empirical look at its role in adolescent talent development. In M. Maehr & P.R. Pintrich (Eds.), Advances in motivation and achievement (8th ed., Vol. 8, p. 5998). Stamford, CT: JAI Press. Reiser, R. A. (2001). A history of instructional design and technology: Part II. Educational Technology Research and Development, 49(2), 57-67. Reiser, R. A., & Gagn, R. M. (1982). Characteristics of Media Selection Models. Review of Educational Research, 52(4), 499. Renninger, K. A. (2000). Individual interest and its implications for understanding intrinsic motivation. In J. M. Harackiewicz & C. Sansone (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 373-404). San Diego, CA: Academic Press.

DOCTORAL COMPS

61

Renninger, K. A., Hidi, S., & Krapp, A. (1992). The role of interest in learning and development. Hillsdale N.J. Lawrence Erlbaum. Richey, R. C., Klein, J. D., & Tracey, M. W. (2010). The instructional design knowledge base: Theory, research, and practice. New York, NY: Routledge. Salomon, G., & Almog, T. (1998). Educational psychology and technology: A matter of reciprocal relations. The Teachers College Record, 100(2), 222241. Schiefele, U. (1991). Interest, learning, and motivation. Educational Psychologist, 26(3), 299323. Schramm, W. (1977). Big media, little media. Beverly Hills, CA: Sage Publications. Schraw, G., Bruning, R., & Svoboda, C. (1995). Sources of situational interest. Journal of Literacy Research, 27(1), 1-17. Schraw, G., Flowerday, T., & Lehman, S. (2001). Increasing Situational Interest in the Classroom. Educational Psychology Review, 13(3), 211-225. Schroff, R., & Vogel, D. (2010). An investigation on individual students perceptions of interest utilizing a blended learning approach. International Journal on ELearning, 9(2), 279-294. Schunk, D. H., Pintrich, Paul R., & Meece, J. (2008). Motivation in Education: Theory, Research, and Applications (3rd ed.). Columbus: OH: Pearson Education, Inc. Slough, S., & McTigue, E. (2010). Introduction to the Integration of Verbal and Visual Information in Science Texts. Reading Psychology, 31(3), 206-212. Snyder, M. M. (2009). Instructional-design theory to guide the creation of online learning communities for adults. TechTrends, 53(1), 45-57. Sweller, J., Merrinboer, J. J. G. van, & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251296.

DOCTORAL COMPS

62

Tobias, S. (2006). The importance of motivation, metacognition, and help seeking in web-based learning. In H. F. ONeil Jr. & R. S. Perez (Eds.), Web-based learning: Theory, research and practice (p. 203220). Mahwah, NJ: Lawrence Erlbaum Associates. Tosti, D. T., & Ball, J. R. (1969). A Behavioral Approach to Instructional Design and Media Selection. Educational Technology Research and Development, 17(1), 525. Twelker, P. A., Urbach, F. D., & Buck, J. E. (1972). The systematic development of instruction: An overview and basic guide to the literature. Stanford, CA. Weber, J. J. (1922). Comparative effectiveness of some visual aids in seventh grade instruction. Chicago: The Educational Screen, Inc. Wekesa, E., Kiboss, J., & Ndirangu, M. (2006). Improving students understanding and perception of cell theory in school biology using a computer-based instruction simulation program. Journal of Educational Multimedia and Hypermedia, 15(4), 397-410. Wertheimer, M. (1938). Laws of organization in perceptual forms in a source book for Gestalt psychology. London: Routledge and Kegan Paul. Winn, W. (2004). Cognitive perspectives in psychology. In D. Jonassen (Ed.), Handbook of research for educational communications and technology (2nd ed., pp. 79-112). New York, NY: Simon & Schuster Macmillan. Wittrock, M. (1989). Generative Processes of Comprehension. Educational Psychologist, 24(4), 345-376. Wojciechowski, A., & Palmer, L. B. (2005). Individual student characteristics: Can any be predictors of success in online classes? Online Journal of Distance Learning Administration, 8(2).

DOCTORAL COMPS

63

Yaverbaum, G., Kulkarni, M., & Wood, C. (1997). Multimedia projection: An exploratory study of student perceptions regarding interest, organization, and clarity. Journal of Educational Multimedia & Hypermedia, 6(2), 139-153. Yu, C. H., Jannasch-Pennell, A., & DiGangi, S. (2008). A non-technical approach for illustrating item response theory. Journal of Applied Testing Technology, 9(2). Yu, C. H., Jannasch-Pennell, A., DiGangi, S., & Kaprolet, C. (2009). An Exploratory Crossover Study of Learner Perceptions of Use of Audio in Multimedia-Based Tutorials. Journal of Educational Computing Research, 40(1), 23-46.