Research Design 1

Research Design

Effects of the use of Business simulation games on Motivation and Learning Outcomes

Mimi So

Hunter College

EDPS 713

November 23, 2010

 Research Design 2

Introduction

An effort of educators, researchers and practitioners has been the search for effective

interventions to promote motivation and academic achievement. Tao, Cheng and Sun (2009)

found that according to a national survey, of 2005 junior college students, 20.3% of students

rarely participated in class, 29.9% often avoided attending class and 49.7% often did irrelevant

things in class. The authors attributed the cause of student behaviors to lack of student

motivation. While the survey (Tao, Cheng, Sun 2009) shows a decrease in students’

participation in class, a survey of 27 Universities and colleges (Pew Internet and American Life

2002) reveals 70% of college students plays video, computer or online games at least once in a

while and 65% of college students identifies themselves as regular and occasional game players.

Increasing use of computer games for entertainment has become a widespread topic that draws

the attention of educators, researchers and practitioners.

Increased game playing among students has led to researchers’ and educators’ interest in the

effects of video games on student performance. Simulation games attempts to replicate real life

activities in the form of a multimedia interactive tools for learning. Many business and higher

education institutions have started to use Simulation games for training staff and learning

purposes.

Background

Instructional games for training purposes can be dated back to ancient times in war for strategy

use such as the chess game. Recently, many research studies focused on developing instructional

games for educational purposes. A shift in learning from a traditional to a learner-centered

 Research Design 3

model that emphasizes on a more active role of learning (R.Garris, R. Ahlers, J.E.Driskell, 2002)

causes the changes in instructional designs. Many institutions in higher education and

organization begun to adapted computer simulation games to enhance learning and training.

Progress in technology makes digital games a feasible tool for learning (F. Fu, R. Su, S. Yu,

2009). Therefore, there is a need to test the effectiveness of instruction designed games on

student motivation and achievement.

Business Education

Business school curriculum was unable to prepare students adequately to correspond to business

realities ( Lainema, Makonene, 2003) because academic practice in business education are

discussed in a stable and predictable aspects of organizational functions. Business curriculum

teaches students the process and understanding of business functions in a predictable way.

Teaching methods that are solely based on lectures and review of cases neglect the importance of

practice and implementation of business concepts in the real world (Ben-Zvi, 2010). Business

education is also criticized for its content-oriented approach as inappropriate to the changing

environment and unpredictable characteristics of business realities.

Traditional teaching methods or lectures fail to initiate student motivation effectively compare to

problem based learning environment which were offered in most simulation games. A study

taken in Korea comparing the effects of problem based learning and lecture based learning in an

adult health nurse course (Hwang, Kim 2002) found that students in the problem based learning

group gain more knowledge and had higher motivation compared to students in lecture based

learning. Hwang and Kim (2006) attributed the results to lack of attractiveness in traditional

lecture. A literature review for research on effects of games on motivation noted that educational
 Research Design 4

games increase the motivation to learn (Randel, Morris, Wetzel & Whitehill, 1992). While

games are fun, engaging and immersive, research suggested that games can be used as an

educational tool for effective learning and increase motivation ( Tuzun, Yilmaz-Soylu, Karakus,

Inal, &Kizilkaya 2009; Papastergiou 2009; Ben-Zvi 2010; Garris, Ahlers, & Driskell 2002).

Simulation Games

Games in general should be fun, engaging to involve students actively, and create high interest

(Henson, 1982). Games were motivational due to the aspects of fantasy, rules/goals, challenge

and control (R.Garris,et.al 2002). Simulation imitates real-life system or a phenomenon that can

be manipulated through exploration. Simulation should be analytic in nature and contain

elements similar to real-world (Neil, 1975). A simulation game offers learner the opportunity to

practice decision making skills, and strategy planning that requires critical thinking skills and

problem solving skills. It also provides learners with the experience that might not be available

or might be too costly in real-life situations. Simulation teaches specific process within a

specific environment allowing learners to applied and practice concepts in a simulated

environment (Mhaboubian, 2010). Game playing allows learners to be immersed and engaged in

a personal learning experience. Games-based learning environments allow learners to reflect on

their actions, draw conclusions and test their hypothesis by means of a “trial and error” approach

to promote problem solving skills and critical thinking (Vos,van der Meijden & Denessen 2010).

Vos et.al (2010) also stated that games offers learner control of the process, so learner can have

the time to analyze, evaluates and practice concepts the games wants to teach.

Learners engage in activities and challenges where learners activate prior knowledge to construct

new knowledge by means of providing a complex learning environment which can be apply in
 Research Design 5

the real-world ( Tuzun, Yulmaz-Soylu, Karakus, Inal , Kizilkaya, 2009). Many studies and

articles point out that simulations games lead to increase in student achievement and motivation

which was supported by constructivist, a “learning by doing” or “hands-on” approach to learning.

(e.g., Coller, Scott 2009; Garris et.al 2002, Ben-Zvi 2010).

Approaches to Learning

The underlying approach to educational game design is base on constructivism, a learning

approach where learners are trying to make sense of the world around them (Vos, van der

Meijden & Denessen 2010). Business simulation games are developed and designed base on the

constructivist approach to learning. It offers students an opportunity to learn by doing and

engage them in a simulated environment or situations. Constructivist learning emphasizes the

learner’s ability to solve real-life problems. The learner takes an active role in constructing

knowledge through experience. Simulation games provide learner the environment where they

can manipulate, received feedbacks and foster reflective practice.

Knowledge is constructed through the cognitive processing of the collaboration of prior

knowledge with new information. Knowledge is constructed when learners develop a mental

model or presentation while integrating prior knowledge with new information. A learner

received information, retains, analyzes, and form schemas by relating to prior knowledge in long

term memory for evaluating information. Cognitive processing is required for the construction

of knowledge in game play.

Understanding and applying principles of the cognitive load theory in multimedia learning is also

important for educational games to be effective. Cognitive load theory is based on the notion of a
 Research Design 6

limited working memory capacity of 7 plus and minus 2 and an unlimited long-term memory.

Cognitive load theory focuses on complex tasks where instructional control of the cognitive load

is critical to meaningful learning. In order for learning to occur, instructional design should be

developed to decrease extraneous load and intrinsic load while increasing germane load. Plass,

Moreno and Brunken (2010) identified three types of cognitive load that occurs in working

memory essential to learning. Extraneous cognitive load is caused by the inappropriate or

irrelevant information presented in the learning material. The intrinsic load depends on the

interactivity of elements which can be defined in relation to level of expertise. Higher level of

expertise results in low intrinsic load. The third and final type of cognitive load results from

active schema construction processes and beneficial to learning is the germane load.

Moreno (2010) discussed techniques for increasing germane load such as the personalized

principle, and guided principle, methods aimed at engaging learner to exert mental effort in

constructing knowledge. Personalized principle refers to instructions that includes personalized

messages are more likely lead to meaningful learning. The guided principle refers to the

instruction that offers an interactive environment for manipulating learning material and

dialoguing with a pedagogical agent. Base on the guided principle, meaningful learning occurs

when instructions are designed to prompt learners to actively engage in the selection and

organization of information. Educational games attempt to integrate these principles into the

game design for better learning. A study for testing the effects of personalized messages in a

multimedia science lesson proved to enhancing student learning (Moreno, Mayer 2000). Students

exposed to personalized messages in the experiment performed better on transfer and retention.

Simulation games often are role playing games allowing interactions with virtual agents and

affordance of personalized experience.

 Research Design 7

Simulation games maybe designed to include aspects of cognitive load theory for increasing

germane load. Simulation games are personalized through role-playing or having a virtual agent

speaking directly to you. Business simulation games also provide a certain level of guidance

where the virtual agent provides the learner with suggestions and advice. A Virtual Reality

learning Environment (VRLE) identified by Huang, Rauch and Liaw (2010) provides the

interactive environment for immersion as well as offering the simulation a safe and realistic

environment for learners to perform specific task. Immersive environments increase motivation

by providing a strong sense of presence.

Huang, Rauch and Liaw (2010) identified constructivist learning approaches that are offered in a

Virtual reality Learning Environment which are, situated learning, role-playing, problem base

learning and creative learning. In a simulated environment in real time, learners construct

knowledge through interaction with objects and events in a virtual world. Role playing provides

a sense of self in the virtual environment through graphic representatives of character, avatars

and increases student motivation and becomes immersive. In a virtual world, a situation can be

easily created to foster critical thinking skills to solve a specific problem. Learners can explore,

manipulate and experiment with the situation and receives feedback.

Motivation is the willingness of the learner to engage in a task and a key factor on student

achievement. Games fostering intrinsic motivation are a big challenge of every instructional

game design. Individual difference and game genre influence the learners’ engagement is the

game, which can be reflected on learners’ motivational level. A design that is highly engaging

should include role-playing, challenges within the learners’ ability, and include interactive

choices within the game. According to Dickey (2005) interactivity of a game that allows
 Research Design 8

exploration, experimenting and manipulating within a game environment is engaging and leads

to increase in motivation. Game characteristics such as challenge, fantasy and curiosity foster

intrinsic motivation.

The ARCS model of motivation developed by John Keller (1987) identifies four components for

motivating instructions; attention, relevance, confidence and satisfaction. Attention refers to

students’ interest gained or maintained during activities. Relevance is how student perceives the

activities in relation to personal needs. Confidence identified by Keller refers to students’

expectation of winning or completing the task. Satisfaction refers to the reward anticipated. In

order to initiate and maintain motivation during activities, instructional design as well as game

design should include the four components described in the ARC model.

Games designed based on the ARCS model allows for arousal of students’ interest through

integrating game aspects such as role play and stimulate curiosity by posing challenging

questions. Establishes links to learners’ need to increase motivation. A simulated environment

establishes the link and transforms skills practiced in a virtual environment into real-world.

Games often provide feedbacks either through a virtual agent or allow students to see the

outcomes of their choice or action which therefore, increases motivation. Rewards or incentives

promote extrinsic motivation while feelings of accomplishment increase intrinsic motivation.

Students become motivated through the process of learning.

Literature Review

There are mixed results from previous studies regarding the effects of learning games on

motivation and achievement. The gaming approach for learning supported by constructivism is
 Research Design 9

believed to increases motivation and learning outcomes. In a study taken in Turkey, testing the

effects of computer games on primary school student’s achievement and motivation in

geography learning (Tuzun et. Al. 2009) found significantly higher levels of intrinsic motivation

in students who learned in a game based learning environment compared to traditional school

environment. While results indicated a lower extrinsic motivation, students focused less on

grades but became more independent in participating in activities, results reveals a statistically

significant gains in achievement test. Game activities offering exploration and interaction

affordance increases both motivation and learning outcomes compare to traditional teaching.

Another study examines the effects of digital game-based learning in Greek High School science

education on educational effectiveness and student motivation (Papastergiou, 2008). A pre and

post test using the Computer Memory Knowledge test to measure the effects of game learning by

comparing the intervention group with the control group indicates the gaming approach was

more effective in promoting students’ knowledge of computer memory concept and motivational

than the non-gaming approach. Other research also suggested that games are an effective tool

for learning and teaching ( Tao,Cheng, Sun,2009; Ben-Zvi, 2010). Coller and Scott (2009) tested

the effectiveness of using video game to teach a course in mechanical engineering. They

reported that student s taking game-based course were willing to spend twice as much time on

their work outside of class. Gabrielle (2002) examines the effects of technology-mediated

instructional strategies on motivation and performance found that students using technology

mediated instructions has higher grades and has a higher motivation level. This demonstrated

that students are more motivated to learn in a game based learning environment.
 Research Design 10

However, a recent study on the effects of modern mathematic computer games on mathematic

achievement and class motivation (Kebritchi, Hirumi, Bai, 2010) found no significant

improvement in student motivation even if there is a significant improvement in math

achievement. A literature review that compare games to traditional instructional classroom

instructions (Randel, Barbara, Morris, Wetzel &Whitehall 1992) reviewed empirical studies

from 1984 to 1991. A total of 67 Studies reviewed, 28 studies show no difference between games

and traditional instructions; 22 favor games; 5 favor games but remains questionable due to

research design; and 3 favor conventional instructions. Kebritchi, Hirumi and Bai (2009) pointed

out that various literature review related to instructional games indicated mixed results which

requires further research to help educators better understand game effects for implementation in

class instructions. Mixed results found in previous literature posed unclear or weak evidence for

understanding the effects of games on student outcome and motivation.

Research Design

In this study we attempt to measure the effects of simulation games on student achievement and

motivation and to determine whether simulation games can be implemented in educational

instructions. It is hypothesized that game as a component in instructions increases student

motivation and academic achievement. In addition, the study will also examine the effect of

instruction during game play on student motivation and achievement. It is predicted that

students exposed to the game will become immersed by the challenges offers in the game.

Student playing the game will gain knowledge of business functions and processes through

practice in a simulated business environment which leads to academic achievement in class.

 Research Design 11

Simulated environment offered in the game increases student motivation by means of the

confidence that they can learn and apply business concepts.

Method

The research design proposed here attempts to determine whether business simulation games can

be used in an educational setting to promote motivation and learning. The study will be

conducted on College students taking an introductory business course. A pre and post test will be

conducted before and after the implementation to measure student motivation on both the control

and the intervention group. The instructional Material Motivational Survey (IMMS), 36 Likert-

scale survey statements corresponds to the ARCS model of motivation developed by Keller, was

adopted to measure student motivation. Students’ grades will be evaluated for understanding

student achievement.

Sample

Participants will include students a University taking introductory Business classes, age 18-22.

The purpose of selecting this sample group is due to the used of Business Simulation Game in

this study which is suggested for use in Business training and practices.

Materials

The study will employ the use of a stimulation game to test its effects of achievement when

incorporated into the classroom activities. Materials will also include a participant questionnaire

solicited information on participants’ gender, age, grade point average, prior videogame

experience, knowledge of business process or concepts rated in a scale of 1 to 5; 1 for very little

and 5 for very much.

 Research Design 12

Measurement

The ARCS model developed by John Keller identifies four components for motivating

instructions; attention, relevance, confidence and satisfaction. The Instructional Material

Motivational Survey (IMMS) developed by John Keller has been administered in the research

study for measuring the effects of technology-mediated instructional strategies on motivation

(Gabrielle 2002). Instructional Material Motivational Survey was an ARCS model based

motivational measurement instrument, consists of 36 Likert-scale statements. The IMMS is

considered a valid instrument and has a documented reliability coefficient of .96 (Keller 2006).

Students’ academic achievement will be measured based on grades achieved during the semester.

Procedure

The study will take over a period of two semesters. During the first semester, a pre a post test for

measuring motivation will be conducted on students taking introductory business classes. Grades

will also be collected for comparison. For the second semester a pre test and post test will also

be conducted on students taking introductory business classes. During the second semester,

students will need to engage a minimum of one hour per week playing the Business Game

offered in class. Students in the second semester will be divided into two groups; game play

without instruction (GP) and game play with instructions in class (GPI). Students are allowed to

play anytime they want outside of class by simply log-in to the game online. At the end of the

semester, students are to report the number of hours they spend playing the game outside of class.

Again, all students’ grades will be collected for analyzing student achievement. First semester

students will serve as the control group in this experiment whiles the second semester student

will serve as the experimental group (intervention group). A comparison between the two groups
 Research Design 13

in the second semester will also be measured in terms of academic achievement and motivation

to understand whether instruction with game implementation play a role in student achievement

and motivation.

Data Analysis

Data will be analyzed using a repeated measure on each group to understand the transition of

student motivation during the semester in each group. For example, first semester pre-test

(S1Pre) IMMS score will be compared with first semester post-test (S1Post) IMMS score. The

Pre-test and Post-Test IMMS score for the second semester will also be analyzed using a

repeated measure to determine whether there is any increase in student motivation. A one way

ANOVA will be employed to measure the difference in means between the control and

intervention group on motivation. A one way ANOVA will be employed to measure the

difference in the changes in students’ motivation between the group that play the game and

received instruction (GPI) compare to the group that play the game without instructions (GP) to

understand the effects of instructions during game play have on student motivation and

achievement. Student grades will be analyzed using the independent measure ANOVA to

compare students’ achievements among the groups. A Correlation analysis will be conducted to

understand the relationship between time spend playing outside of class on student motivation

and achievement.

Discussion

The purpose of the study is to determine whether business simulation games can be used as an

educational tool in instruction to increase student motivation and achievement. Due to

 Research Design 14

increasing game play among children and adolescents, it is important on an educator’s

perspective to evaluate the effects of game based learning on student motivation and

achievement. Recent changes in education learning also incorporate games in classrooms and

instructions. However, the effectiveness of simulation games on student motivation and

achievement based on previous research is not clear which requires further research.

On a theoretical level, this research adds empirical evidence to the gaming approach theory and

constructivism. Measuring the effects of simulation games allow us to understand and make

connections to how game effects can initiate and sustain student motivation which will lead to

improve in achievement. If the study shows positive results, this study will lead to further

investigation for testing specific aspects of game play on student motivation and achievement.

Game aspects include challenge, fantasy, and control, which each of these aspects might be a key

factor on student motivation and academic achievement.

Reference

Ben-Zvi, T. (2010). The Efficacy of Business Simulation Games in Creating Decision Support

Systems: An Experimental Investigation. Decision Support System, 49, PP.61-69.

 Research Design 15

Fu, F., Sy,R., Yu,S.(2009). EgameFlow: A Scale to Measure Learners’ Enjoyment of E-

Learning Games. Computer and Education, 52, PP.1001-112.

Gabriele, D.M. (2002). The Effects of Technology-Mediated Instructional Strategies on

Motivation, Performance, and Self-directed Learning (Electronic): U.S. Military Academy

Center for Teaching Excellence.

Garris, R., Ahlers, R., Driskell, J.E. (2002). Games, Motivation, and Learning: A Research and

Practice Model. Simulation Gaming, 33(4), PP. 441-467.

Henson, K. (1982). Active Involvement: Simulation Games and Teaching. NASSP Bulletin.

Huang, H., Rauch, U., Liaw, S. (2010). Investigating Learner’s Attitudes Toward Virtual Reality

Learning Environments: Based on Constructivist Approach. Computers and Education, 55,

PP.1171-1182.

Huang,W. (2010). Evaluating Learners’ Motivational and Cognitive Processing in an Online

Game-Base Learning Environment. Computers in Human Behavior (2010).

Doi:10.1016/j.chb.2010.07.021.

Hwang , Kim (2006). A comparison of Problem-Based Learning and Lecture-Based Learning in

an Adult Health Nursing Course. Nurse Education Today 26, PP.315-321.

Kebritchi, M., Hirumi, A. Bai, H., (2010). The Effect of Modern Mathematics Computer Games

on Mathematics Achievement and Class Motivation, 55, PP.427-443.

Kebritchi,M., Hirumi, A., Bai, H. 2010. The Effects of Modern Mathematics Computer Games

on Mathematics achieivement and class Motivation. Computers and Education, 55, PP.427-443.
 Research Design 16

Keller, J. (1987). Development and use of the ARC model of Motivation Design. Journal of

Instructional Development, 10(3), P.2-10.

Lianema, T., Makkonen, P. (2003). Applying Constructivist Approach to Educational Business

Games: Case REALGAME. Simulation and Gaming, 34 (1), PP.131-149.

Mahboubian, M. (2010). Educational Aspects of Business Simulation Software. Procedia Social

and Behavioral Science, 2, PP. 5403-5407.

Moreno.R., & Mayer,R.E. ( 2000). Engaging students in active learning: The case for

personalized multimedia messages. Journal of Educational Psychology, 92 (4), 724-733.

Neil, H. (1975). Simulation: The Motivation Connection. Paper presented at Annual Meeting of

the American Political Science Association. San Francisco, CA(September).

Papastergiou, M. (2009). Digital Game-Based Learning in High School Computer science

Education: Impact on Educational Effectiveness and Student Motivation. Computers and

Education, 52, PP.1-12.

Plass, J., Moreno,R., &Brueken ,R. (2010). Cognitive load theory. New York: Cambridge

University Press.

Randel J., Morris,A., Wetzel , C., Whitehill, B., 1992. The effectiveness of games for

educational purposes: a review of recent research, Simulation and Gaming 23 (3), PP. 261–276

Tafoya, D. (1979). The Motivation Game, Effecting Social Behavior in Small Groups.

Simulation and Games, 10(4), PP.403-418.

 Research Design 17

Tao, Y., Cheng, C., Sun, S. (2009). What Influence College Students to Continue Using Business

Simulation Games) The Taiwan Experience. Computer and Education , 53, PP. 929-939.

Tuzun, H., Yilmaz-Soylu, M., Karkus,T., Inal, Y., Kizilkaya, G. (2009). The Effects of

Computer Games on Primary School Students’ Achievement and Motivation in Geography

Learning. Computers and Education, 52, PP. 68-77.

Vos, N., Meijden, H., Denessen, E. (2011). Effects of Constructing Versus Playing an

Educational Game on Student Motivation and Deep Strategy Use. Computers and Education. 53,

PP. 127-137.

Whitton, N. (2007). Motivation and Computer Game Base Learning. In ICT. Providing Choices

for learners and learning. Proceedings Ascilite Singapore

2007.Http://ascilite.org.au/conferences/singapore07/procs/whitton.pdf.