TOWARDS EMERGENT GAMEPLAY: A FRAMEWORK FOR REALISTIC PSYCHOSOCIAL BEHAVIOUR IN NON PLAYER CHARACTERS

(Spine title: Realistic Psychosocial Behaviour in Non Player Characters) (Thesis format: Monograph)

by

Christine D. Bailey

Graduate Program in Computer Science

A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science

Faculty of Graduate Studies The University of Western Ontario London, Ontario, Canada

Christine D. Bailey 2007

THE UNIVERSITY OF WESTERN ONTARIO FACULTY OF GRADUATE STUDIES

CERTIFICATE OF EXAMINATION

Supervisor

Examiners

Dr. Michael Katchabaw

Dr. Mark Daley

Supervisory Committee

Dr. Robert Mercer

Dr. Nick Dyer-Witheford

The thesis by

Christine Diana Bailey entitled: Towards Emergent Gameplay: A Framework for Realistic Psychosocial Behaviour in Non Player Characters

is accepted in partial fulfillment of the requirements for the degree of Master of Science

Date Chair of the Thesis Examination Board

ii

Abstract

This thesis seeks to develop a framework for emergent psychosocial behaviour in characters in video games. The framework uses concepts behind emergent gameplay to support the mechanics of supporting designer-defined psychological and social concepts, undefined circumstances, and emergence. A prototype has been developed based on the framework. This prototype has been evaluated for realistic emergent behaviour and resource efficiency. The prototype has succeeded in supporting emergent psychosocial behaviour, and is efficient for group sizes up to 10 with no significant increase in resource usage with increased numbers of NPCs. The preliminary work on the framework is encouraging and future work includes integration of the system with a game, user and developer studies for realism and ease of use, and the control of dynamic group formation.

Keywords: realistic behaviour, psychosocial behaviour, non player character behaviour, videogame characters, npc psychology, social behaviour, emotions, personality, emergence, emergent gameplay, stimulus-response systems.

iii

Acknowledgements
I would like to thank several people who aided in the making of this thesis. I would like to thank my supervisor, Dr. Michael Katchabaw, for his assistance, feedback, and the resources he volunteered throughout the development of this work. His unfailing helpfulness and concern kept me on track and pushed me to go the extra mile. Thanks especially for insisting I write a section on qualitative testing with scenarios. I would like to thank Dr. Mark Daley for his early support and feedback on the initial proposal of this work. His critical analysis of the later result also helped me tighten up the final version of my thesis. Without his help my thesis would be far weaker. My thanks to Dr. Bob Mercer for encouraging my early interest in research, as well as for pointing out the things I had overlooked in my thesis. His thoughtful examples on concepts I had not thought of helped give me an idea of the broader scope of this work, and also provided some of the examples in this thesis. Thank you Dr. Nick Dyer-Witheford for providing a different perspective, and in helping me to pinpoint and improve areas in my thesis that were in need of clarification. I would like to thank Harvey Smith for his patience, assistance, and helpful insights into how objects interacted in the Stimulus System. I would also like to thank Randy Smith for his amazingly detailed and comprehensive explanation of how the Act/React system worked. This explanation made me realize that there was much more going on in the system than I had initially suspected, and helped me to improve the end result of my thesis through a more thorough understanding of emergent gameplay. Finally I would like to thank my family for their endless support and encouragement.

iv

Table of Contents
Certificate of Examination...................................................................................................ii Abstract...............................................................................................................................iii Acknowledgements.............................................................................................................iv Table of Contents.................................................................................................................v List of Figures...................................................................................................................viii 1 Introduction..............................................................................................................1 1.1 1.2 1.3 1.4 1.5 1.6 2 Overview......................................................................................................1 Motivation....................................................................................................2 Background..................................................................................................3 Problem Statement.......................................................................................4 Proposed Solution.........................................................................................5 Outline..........................................................................................................7

Related Work............................................................................................................8 2.1 Artificial Intelligence in Games...................................................................8 2.1.1 Frequently Used Techniques..........................................................10 2.1.1.1 Finite State Machines.........................................................10 2.1.1.2 Scripting.............................................................................11 2.1.2 2.2 The Role of the Designer................................................................12

Creating a Better Game: Interactive Storytelling and Social Science........12 2.2.1 The Role of Non Player Characters in Interactive Storytelling......14 2.2.1.1 Current NPC Architectures.................................................15 2.2.2 Psychology, Sociology, and Psychosocial Behaviour....................17 2.2.2.1 Psychology..........................................................................18 2.2.2.1.1 Emotions in Computation...........................19

2.2.2.2 A Link Between Psychology and Social Science: Personality..........................................................................21 2.2.2.3 Social Science.....................................................................23 v

2.2.2.3.1

Faking Social Science with Reputation Systems.......................................................24

2.2.2.3.2 2.2.3 2.2.4 2.3 2.4 3

Other Games with Social Systems.............26

Socially Intelligent Agents.............................................................26 The Relevant Social Research........................................................27

Improvisation in Gameplay: Emergent Gameplay.....................................31 Summary of Literature Review..................................................................34

Design.....................................................................................................................37 3.1 High-Level/Conceptual Design..................................................................37 3.1.1 Elements of a NPC Social Psychology...........................................38 3.1.1.1 Internal States.....................................................................39 3.1.1.2 Social Information..............................................................40 3.1.2 The Mechanics of an Emergent NPC Social Psychology...............41 3.1.2.1 Two Sample Stimulus-Response Systems..........................42 3.1.2.2 An Abstract Stimulus-Response System............................45 3.1.2.3 A Stimulus-Response System for Psychosocial Behaviour...........................................................................46 3.1.3 3.2 Mapping Stimuli to Actions in an NPC Psychology......................50

Software Design.........................................................................................51 3.2.1 Listening to the World Around You: The Observer Design Pattern.............................................................................................52 3.2.2 Architecture of a Psychosocial NPC..............................................53 3.2.2.1 The Relevance Filter...........................................................54 3.2.2.2 The Stimulus Dispatcher....................................................56 3.2.2.3 The Decision-Maker...........................................................56 3.2.2.4 Character State....................................................................57

Prototype Implementation......................................................................................61 4.1 4.2 Data Structures...........................................................................................61 Processes....................................................................................................64

vi

Evaluation...............................................................................................................67 5.1 Scenarios to Examine Realistic Social Behaviour......................................67 5.1.1 5.1.2 5.1.3 5.1.4 5.2 Shannara and the Warlock Lord.....................................................68 A Friend in Need, and a Friend in Deed.........................................71 A Day at Hogwarts.........................................................................73 Summary of Scenario Findings......................................................76

Efficiency and Resource Usage..................................................................77 5.2.1 5.2.2 5.2.3 Methodology...................................................................................77 Results............................................................................................78 Analysis of Results.........................................................................79

5.3 6 7

User and Developer Experience.................................................................80

Conclusions and Future Work................................................................................81 Works Cited............................................................................................................83

Vita.....................................................................................................................................90

vii

List of Figures
Figure 1 - A sample of an NPC psychology.......................................................................38 Figure 2 - The mechanics of Ion Storms Stimulus System...............................................42 Figure 3 - An abstraction of the core concepts behind the mechanics of a stimulusresponse system......................................................................................................45 Figure 4 - Event/Action flow between the game logic and the character...........................52 Figure 5 - The structure of the Observer design pattern.....................................................53 Figure 6 - Architecture of an emergent psychosocial NPC................................................54 Figure 7 - Detailed view of the relevance filter..................................................................55 Figure 8 - Class diagram of an NPC psychology................................................................58 Figure 9 - Three different representations of social ties.....................................................60 Figure 10 - Class diagram of the NPC psychology as used in the prototype......................62 Figure 11 - Screenshot of the prototype being run.............................................................64 Figure 12 - The general layout of individuals and social ties in the scenario from Section 5.1.1........................................................................................................................69 Figure 13 - A screenshot of the prototype running Scenario 5.1.1.....................................71 Figure 14 - A screenshot of the initial emotions at the beginning of running the scenario from Section 5.1.2..................................................................................................72 Figure 15 - A screenshot of scenario 5.1.2 after running for a short period of time..........73 Figure 16 - The general grouping of individuals in Scenario 5.1.3....................................74 Figure 17 - A screenshot of the prototype running Scenario 5.1.3.....................................75

viii

1
1.1 Overview

Introduction

The field of artificial intelligence (AI) in games is a broad, yet demanding area of study. One of the primary areas of interest to AI in games is Non Player Character (NPC) control. Russell and Norvig (2003) define AI as being concerned with thought processes, reasoning, and behaviour as it applies to human performance as well as ideal intelligence, or rationality. AI in games differs from traditional AI in that it is optimized not for the simulation of human performance or rationality, but for entertainment and increased immersion in the game world (Tozour, 2002; Reynolds, 2004). In order to support player immersion, one of the main requirements of AI in games which differs from traditional AI is the necessity for real-time interactive performance (Tozour, 2002; Reynolds, 2004; Brockington, 2003; Young, Riedl, Branly, & Jhala, 2004; McNaughton, Schaeffer, Szafron, Parker, & Redford, 2004) which severely restricts more complex and accurate AI methods. Due to this constraint, realistic NPC behaviours have typically been scripted or hard-coded to reduce computation complexity (McNaughton et al., 2004), however a growing body of research is beginning to recognize the importance of breaking this tradition (Crawford, 2005; Magerko, 2006; Gruenwoldt, Katchabaw, Danton, 2005; Lawson, 2003; Game Informer, 2006a; Game Informer, 2006b; Rizzo, Veloso, Miceli, & Cesta, 1997). Recent advances in hardware, including the use of multiple processors in computers and video game consoles, such as the Xbox 360 and PlayStation 3, may also support more realistic behaviour without hard-coding. One of the more active areas of research in game AI is making more believable characters, also known as Non Player Characters (NPCs) (Funge, 2004; Gruenwoldt, Katchabaw, & Danton, in press; Gruenwoldt, Katchabaw, & Danton, 2005; Prendinger & Ishizuka, 2001; Rizzo, Veloso, Miceli, & Cesta, 1997; Byl, 2004; Lawson, 2003; Game Informer, 2006a; Game Informer, 2006b; Guye-Vuilleme, 2001). This has been investigated through adding simple mental states to characters (Byl, 2004; Funge, 2004),

2 and more complex mental states such as emotions (Funge, 2004; Rizzo, et al., 1997). NPCs can also be implemented to have memory (Funge, 2004), and to be able to forget events (Alt & King, 2002; Grond & Hanson, 1998). More complex believability has been achieved through the creation of reputation systems where NPCs share opinions of the game player based on their behaviour in the game world (Alt, et al., 2002; Brockington, 2003; Electronic Arts, 2005; Grond, et al., 1998; Russel, 2006), and even implementation of social awareness, including the maintenance of interaction histories between NPCs (Tomlinson & Blumberg, 2002a), changes in attitudes and familiarity between NPCs (Prendinger, et al., 2002), changes and propagation of opinions (Cole, 2006), and the design of architectures to enable social identities (Guye-Vuilleme, 2001). As of yet, there has been no known research that has attempted to unify psychosocial behaviour in NPCs so as to include emotions, personality, and individual social relationships. Therefore the focus of this research will be to design a unified psychosocial framework for characters in real-time interactive entertainment simulations, such as video games.

1.2

Motivation
There has been much support for the creation of believable and social agents

(Prendinger & Ishizuka, 2001; Byl, 2004; Rizzo, Veloso, Miceli, & Cesta, 1997; Lawson, 2003; Game Informer, 2006a; Game Informer, 2006b). Some of the reasons for having believable social agents includes that they support the suspension of disbelief required to immerse a player in the game (Byl, 2004), promote realism (Lawson, 2003), create more dramatically interesting game situations (Prendinger, et al., 2001), as well as supporting game designers in providing a compelling game experience, supporting interactivity, player choice, and replayability (Reynolds, 2004). Some games have already begun implementing social systems in games, including Bully (Game Informer, 2006b) and the as yet unreleased Assassins Creed (Game Informer, 2006a). Unfortunately, the developers have not made it public how they have designed and implemented these systems.

1.3

Background
Non Player Character behaviour is often handled using finite state machines to

represent state of mind (Byl, 2004; Funge, 2004), or scripting to hard-code behaviour in each possible game situation (Berger, 2002). The main goal for game AI is to support the designers in providing a compelling game experience, supporting interactivity, player choice, and replayability (Reynolds, 2004). One of the factors in creating a compelling experience is suspending the players disbelief. The AIP (Autonomy Interaction Presence) Cube model (as cited by Byl, 2004) states that the requirements of suspension of disbelief are Autonomy of the NPCs, Interaction between the NPCs and the player, and the NPCs Presence in the game world. Byl (2004) also implicates personality, emotion, self-motivation, the illusion of life (which includes goals, plans, reactions to the environment, human-like restrictions, and many other behaviours which create the appearance of life), change, and social relationships as being important to character believability. Some relevant readings in the area of affective computing and emotion synthesis include Funges (2004) definition of emotion as the sum of past events, Byls (2004) discussion of character emotion in the context of games, and Picards (1995) discussion of shifting emotions, and Picard (1995) and Byls (2004) overview of the field of affective computing. Personality in agents is defined as the agents pattern of behaviours or interactions with its environment (Lawson, 2003). Crawford (2005) outlines one possible personality model for NPCs in games that includes intrinsic variables (i.e. integrity, virtue, intelligence,...), mood variables, relationship variables (beliefs about anothers intrinsic variables), and the readiness to change the previous two variables. Isbister (2006) discusses the social psychology research and discusses the traits of agreeableness and dominance and how they can be used to form many different personalities. Reputation systems such as those in Fable (Alt & King, 2002), Thief: Deadly Shadows (2004), and Ultima Online (Grond & Hanson, 1998) refer to systems that typically manage NPC opinions of the player (Alt, et al., 2002; Electronic Arts, 2005),

4 which are formed immediately and globally among all NPCs upon certain player actions (Alt, et al., 2002; Brockington, 2003; Electronic Arts, 2005; Grond, et al., 1998). Reputation systems do not maintain individual opinions (Alt, et al., 2002; Electronic Arts, 2005), nor opinions about other NPCs, though they may maintain group opinions (Alt, et al., 2002; Grond, et al., 1998; Thief: Deadly Shadows, 2004). Some social science concepts of interest in video game research include an agents roles (Isbister, 2006; Guye-Vuilleme, 2001), cultures and subcultures (Isbister, 2006), norms, values, worldview (Guye-Vuilleme, 2001), and goals (Isbister, 2006). Some papers have attempted to address these (Isbister, 2006; Guye-Vuilleme, 2001). The area of social agents, or Socially Intelligent Agents (SIAs) (Byl, 2004; Prendinger & Ishizuka, 2002) would appear to have much relevance to this research, however many of these social agents do not exhibit realistic social behaviour (Wooldridge, 1992; Jennings, Sycaro, & Wooldridge, 1998; Guye-Vuilleme, 2001; Cesta, Miceli, & Rizzo, 1996; Hogg & Jennings, 2001). Many social agents implement only communicative behaviour that is used in a multi-agent problem-solving context to reduce resource usage and increase efficiency. Some relevant research in this area includes Tomlinson and Blumbergs (2002a) remembered interaction histories between agents, Prendinger, et al.s (2002) change of attitudes and familiarity assessment between agents, Coles (2006) comparison of opinion flow in a multi-agent system to flocking behaviour, and Guye-Vuillemes (2001) high level architecture for social agents. Finally, a particularly interesting approach to problems of providing unique, or emergent, experiences lies in what is often referred to as emergent gameplay (Wooton, 2006; Smith & Smith, 2004). As of yet, emergent gameplay is known to have been used in NPC behaviour only in relatively simple situations and behaviours (Smith, et al., 2004; Wooton, 2006) and not in attempting to implement complex social relationships.

1.4

Problem Statement
This research presents a framework for realistic psychosocial behaviour in NPCs

5 using the concepts behind emergent gameplay. There are several challenges that need to be addressed by this framework. The most important constraint is that the framework must be designed so as to have real-time interactive performance (Young, Rieldl, Branly, & Jhala, 2004; Tozour, 2002; Reynolds, 2004). Memory usage may also have to be constrained within reason (McNaughton, Schaeffer, Szafron, Parker, & Redford, 2004; Alt & King, 2002; Brockington, 2003), and ease of use, time, and effort required from game developers must be considered as this may affect its usability in real world applications (Brockington, 2003). Some of the less quantitative challenges to this work are that the framework must enable the game designer to provide an entertaining and compelling gameplay experience (Tozour, 2002; Reynolds, 2004). It should also be able to support a variety of undefined circumstances that the player may input into the game (Barnes & Hutchens, 2002; Smith & Smith, 2004).

1.5

Proposed Solution
One interesting solution to the problem of developing a framework for realistic

psychosocial behaviour is to use the concepts behind emergence and emergent gameplay to create and maintain a unified psychosocial system. The concept behind emergence is that simple component-level behaviours often result in complex surprising system-level behaviour (Wooton, 2006; Johnson, 2001; Holland, 1998). Emergence, or selforganization, has been studied in disciplines as varied as philosophy, biology, chemistry, physics, ecology, neurology, psychology, and computer science, to name but a few (Johnson, 2001; Holland, 1998). Emergent gameplay focuses on implementing a bottom up view of the game world, in that every (static or non-static) object in the world maintains its own needs and reactions to situations that may arise (Wooton, 2006). By having simple component-level mechanics that interact with each other, overall system behaviour is complex and unpredictable. Other places this concept has been successfully used are in flocking algorithms (Cole, 2006; Wooton, 2006) and board games

6 (Wooton, 2006; Holland, 1998). Chess, for example, only has 24 rules, yet many different and interesting gameplay situations are possible (Wooton, 2006; Holland, 1998). Emergent gameplay has been used in computer games, just as it has been used in board games. Games like SimCity, The Sims (Wooton, 2006), Deus Ex, and Thief: Deadly Shadows (Smith & Smith, 2004) have all used some sort of emergent gameplay, though typically in the form of very simple behaviours. For example, in Thief: Deadly Shadows, emergence is used to define how elements like fire, oil, noise, and guards interact (Smith, et al., 2004). With few exceptions, most developers do not publish how they have (or have not) used emergent gameplay. According to Smith, et al. (2004), Ion Storm used a Stimulus System in order to have objects listen to each other. Emergent systems are not scripted, nor are they rule-based (Wooton, 2006). Emergent systems also do not focus on algorithmic behaviours, but rather very simple stimulus responses at the component level (Wooton, 2006). Note that while the system itself is not rule-based, the responses of the components of the system can be (Holland, 1998). The Mechanics-Dynamics-Aesthetics model (Wooton, 2006; Smith & Smith, 2004) outlines how mechanics of the game (such as rules of the game, what agents can do, and what can physically happen) must first be defined. Dynamics are what occur when these mechanics interact with each other. This step is very important, as emergent gameplay cannot occur without this interaction. Aesthetics refers to the human response to the dynamics. This research presents a unified psychosocial framework, and in doing so uses the concepts developed in emergent gameplay to create and maintain realistic and unpredictable psychosocial behaviour for NPCs in video games. The term realistic is applied here to behaviour that is believable, or convincing, to human observers. Emergence may be used to create this type of behaviour by defining emotional and social concepts as components which react to (or are) stimuli, just as objects such as flame were designed as reactive components in games like Thief: Deadly Shadows and Deus Ex. This framework will support NPCs with emergent psychology and social behaviour, which will allow reactions to undefined circumstances and NPC autonomy. This work will remain

7 efficient in CPU and memory usage to enable use in a realtime interactive application.

1.6

Outline
Chapter 2 will present a review of previous work in this area, beginning with

general artificial intelligence (AI) use in games (2.1), the use of interactive storytelling and the role of social science in creating more realistic AI (2.2), and ending with a discussion of emergent gameplay (2.3). Chapter 3 will present the design of this work (the high-level design in 3.1 and the software design in 3.2). Chapter 4 will outline a prototype that was implemented to evaluate the design, and Chapter 5 will discuss this evaluation. Finally Chapter 6 concludes with final remarks and areas for future work.

Related Work

The focus of this chapter is to review what research has previously been done in the area of realistic Non Player Character (NPC) control, and what areas have yet to be explored. Section 2.1 will discuss artificial intelligence in games, with a focus on how AI in games differs from traditional AI, and what techniques are most frequently used in NPC control. Section 2.2 will start by discussing the importance of realistic and believable NPCs in the context of interactive storytelling and sociology, and then discuss the core concepts behind such an implementation, and the efforts made towards the creation of believable NPCs and agents. The review will be concluded in Section 2.3 with a discussion of a technique called emergent gameplay that has potential applications in this work. Finally the chapter will be concluded in Section 2.4 with a summary of the previous work and the contributions that will be made by the current work.

2.1

Artificial Intelligence in Games

As previously stated, AI in games must support real-time interactive performance.

Non-serious games exist primarily to provide entertainment to the user, and therefore some of the AI present does not need to be accurate (Tozour, 2002). This suggests that some game AI may be handled heuristically rather than algorithmically to support performance. AI in games serves many purposes. Some AI handles strictly NPC, or bot, behaviour (Tozour, 2002; Byl, 2004), while much AI goes into handling game world mechanics (Tozour, 2002; Reynolds, 2004). AI for handling NPC behaviour typically does not need to be as accurate as that supporting game world mechanics. Some examples of these supported mechanics are pathfinding (Tozour, 2002; Reynolds, 2004), complex physics (Reynolds, 2004), game trees, maintenance of rule databases, planning, steering, flocking, group coordination, unit deployment, tactical analysis, contextdependent animation, simulating perception, strategic planning, spatial reasoning,

9 resource allocation, weapon handling, target selection, and situation analysis (Tozour, 2002). Which of these mechanics are needed is entirely dependent on the game in question, and AI specific to different genres differs dramatically (Tozour, 2002). Tozour (2002) states that the development of a generalized AI that will work in any game is impossible due to this difference in specific needs for different genres. Some games have a heavily story- or character-driven focus, such as adventure and role-playing games. These games have a high requirement for Non Player Character AI. For the purpose of this thesis, NPC AI will be focussed on. According to Young, Riedl, Branly, and Jhala (2004) game engines usually handle game AI, and do so in a procedural, highly efficient fashion. This differs from traditional AI interfaces, which are declarative (Young et al., 2004). This conflict may pose problems when trying to integrate more advanced AI techniques into games. Young et al.s (2004) solution to this was to create a runtime plan generator that could be integrated with a game engine and provide a link to declarative AI representations, thus providing game engines with the advanced services that a traditional declarative AI interface would. This runtime plan generator was called Mimesis. Tozour (2002) was of the opinion that Non Player Character AI is more akin to agent design or behaviour modelling than artificial intelligence since NPCs do not have to be intelligent, but rather only convincing or believable. Tozour (2002) stated that context-dependent expertise and believability was preferred in NPCs over general intelligence, since they generally have a very narrow range of required responses and behaviours. Traditionally these NPC behaviours have been scripted or hard-coded to reduce computation complexity (McNaughton, Schaeffer, Szafron, Parker, & Redford, 2004). This also allows the designer to exert a maximum of control when rigidity over events occurring in-game is required. However, some types of games, namely openended games and interactive dramas, such as Grand Theft Auto III (2002) and Faade (2005), may not desire such control over events, and would prefer behaviours that are not hard-coded but are more reactive to the player.

10 2.1.1 Frequently Used Techniques

Artificial intelligence techniques used in games have included finite state machines, expert systems, decision trees, search methods, pathfinding, production systems, planning and scheduling, multi-agent systems, A-Life, flocking, genetic algorithms, neural networks, and fuzzy logic (Tozour, 2002). Finite State Machines (FSMs), Fuzzy State Machines (FuSMs), and Hierarchical State Machines (HSMs) have typically been used as the primary method to represent NPC mind states (Byl, 2004; Tozour, 2002), though expert systems have also been used in bot behaviour to mimic player actions (Tozour, 2002), as may be beneficial in competitive games like First Person Shooters or Real Time Strategies. Board games make heavy use of game trees and search methods (Tozour, 2002), yet brute-force methods are limited for use in modern 3D video games due to the complexity of the game space (Tozour, 2002). Pathfinding techniques are usually restricted to the highly efficient A* method (Tozour, 2002), and are often used in 3D games as well as 2D games where characters must navigate spatial obstacles. Production systems have also been used in maintaining databases of rules in games (Tozour, 2002). The most popular techniques in game development are rule-based for their simplicity, ease of understanding, creation, and debugging (Tozour, 2002). Two of the most frequently used techniques for NPC control have been the use of finite state machines (Byl, 2004; Tozour, 2002; Funge, 2004) and scripting (Berger, 2002; McNaughton, Schaeffer, Szafron, Parker, & Redford, 2004; Barnes & Hutchens, 2002).

2.1.1.1 Finite State Machines

Finite State Machines (FSMs) are used in cases where discrete states and transitions between those states must be determined, such as representing the mental state of an NPC (Funge, 2004; Byl, 2004). Finite state machines are a popular tool in games due to their being simple and understandable to use (Tozour, 2002; Byl, 2004), having a rapid design and implementation (Byl, 2004), having low computation costs (Byl, 2004),

11 and being easy to debug and test due to predictability (Byl, 2004; Tozour, 2002). Some of the disadvantages of using finite state machines are that they can become quite difficult to maintain and understand as the number of possible states in the game increase (Byl, 2004), and are difficult to modify in later stages of development (Young, Riedl, Branly, & Jhala, 2004). They also cannot manage situations where all states and transitions are not known a priori (Byl, 2004; Young et al., 2004). This can present a problem if the game designer wishes to create a game that is very open-ended and allows the game player to have a significant role in shaping the game-world.

2.1.1.2 Scripting

Scripting is hard-coded (McNaughton, Schaeffer, Szafron, Parker, & Redford, 2004), function-like (Barnes & Hutchens, 2002), and interpreted (Berger, 2002). Scripting can be used in games to control a wide range of tasks, including controlling objects or groups of objects (Barnes & Hutchens, 2002), maintaining properties of objects like health levels (Barnes & Hutchens, 2002), controlling conversation trees and dialogue (Berger, 2002; Barnes & Hutchens, 2002), controlling cinematic sequences (Barnes & Hutchens, 2002), camera movement (Barnes & Hutchens, 2002), physics (Barnes & Hutchens, 2002), and AI in general (Barnes & Hutchens, 2002). Scripting can be used to handle nearly all aspects of gameplay (Berger, 2002). Advantages of scripting include reduced computation complexity as compared to other methods (McNaughton et al., 2004) and independence of the scripts from the rest of the game (Barnes & Hutchens, 2002). Because of this independence from the game, changing the scripts does not necessitate recompilation of the entire game (Berger, 2002; Barnes & Hutchens, 2002), and testing of the scripts can be done independent of the game (Barnes & Hutchens, 2002). The designer can tweak scripts without programmer intervention allowing frequent modification and testing of gameplay and object behaviours (Barnes & Hutchens, 2002). This designer tweaking can result in a game that is more reflective of the designers vision (Woodard, 2006). Scripts can even be implemented with a virtual

12 machine to allow for undefined circumstances (Barnes & Hutchens, 2002). Disadvantages of scripting are that scripts are often difficult to create and use by individuals without experience in programming (McNaughton et al., 2004), and that scripts can be difficult and time-consuming to create, test, and orchestrate (as many scripts may run in parallel) (McNaughton et al., 2004).

2.1.2

The Role of the Designer

Reynolds (2004) points out that one of the most important things to consider when discussing AI is the role of the game designer. The game designer is in essence the vision person, and as such, any AI used in game development must be able to support the features the game designer wishes to create. Any AI that does not do this should not be in the game. As such, the game AI must be capable of supporting the designer in making a better game. This may include maximizing interactivity, providing many interesting and meaningful choices to the player, providing tension and excitement through asymmetry, improving the replayability of the game, and in general providing a better experience to the player (Reynolds, 2004). The main goal for game AI is to support the designers in providing a compelling game experience, supporting interactivity, player choice, and replayability (Reynolds, 2004)

2.2

Creating a Better Game: Interactive Storytelling and Social Science

There has been much support for the development of more complex and

believable stories (Crawford, 2005; Magerko, 2006; Game Informer, 2006a; Game Informer, 2006a; Falstein, 2007) and characters in games (Isbister, 2006; Lawson, 2003; Tozour, 2002; Russel, 2006; Reynolds, 2004; Game Informer, 2006a; Game Informer, 2006b; Byl, 2004; Rizzo, Veloso, Miceli, & Cesta, 1997; Falstein, 2007). Some of the reasons for supporting more interesting stories and characters include that they support the

13 suspension of disbelief that is required to immerse a player in the game (Byl, 2004), they promote realism (Lawson, 2003), create more dramatically interesting game situations (Prendinger & Ishizuka, 2001; Bates, Loyall, & Reilly, 1994), enable new forms of gameplay (Cole, 2006), and create a broader appeal for games (Isbister, 2006). One of the ways to support this is interactive storytelling. The term interactive storytelling describes the concept that the player should be able to actively and meaningfully interact with the game story itself (Magerko, 2006; Crawford, 2005). Rather than having a set story written by a writer or game designer, the story would be created by the repercussions of the interactions the player had with the game world. An example game that supports interactive storytelling (also called an interactive drama) is the game Faade (2005). Young, Riedl, Branly, and Jhala (2004) described an automatic plan generator called Mimesys, which would produce novel plan-based NPC and object behaviour at runtime in response to unexpected player input. Young et al. (2004) stated that most other game-related attempts at behaviour generation used FSMs or scripting, leaving behaviour generation in games insensitive to the runtime context. Crawford (2005) pointed out the importance of considering plot writing and character writing when discussing interactive storytelling. Plot writing would place emphasis on allowing meaningful player interaction with the story (or plot), and would be implemented via a director module and drama manager to control how the story would play out in response to player actions (Magerko, 2006). Note that some control (or interventions) may have to be exerted on the story to prevent undesirable experiencebreaking events, or to maintain the story experience desired (Magerko, 2006). An example of such an intervention could occur if the player attempts to kill a story-integral character with a gun, at which point the gun might misfire to prevent such an event (Magerko, 2006). Character writing places emphasis in allowing convincing player interaction with characters in the game (Crawford, 2005). Characters would have to be well-developed (Magerko, 2006), be capable of a wide range of emotions (Magerko, 2006), may have exaggerated personalities to heighten dramatic situations (Crawford, 2005), be capable of

14 communicating with the player (Magerko, 2006), and be able to coordinate with other NPCs (Magerko, 2006). In order to allow this, systems have to be developed to manage emotions, personality traits, social ties, and/or anything else that might make the NPCs more believable. This area of character writing in interactive storytelling appears to share similar goals with the area of creating believable and reactive NPCs. The next section will discuss why believable characters may be desirable in games and what factors promote believability.

2.2.1

The Role of Non Player Characters in Interactive Storytelling

One of the more active areas of research in game AI is making more believable characters (Funge, 2004; Gruenwoldt, Katchabaw, & Danton, in press; Gruenwoldt, Katchabaw, & Danton, 2005; Prendinger & Ishizuka, 2001; Rizzo, Veloso, Miceli, & Cesta, 1997; Byl, 2004; Lawson, 2003; Game Informer, 2006a; Game Informer, 2006b; Guye-Vuilleme & Thalmann, 2001). Reasons for this include creating more interactively interesting games (Crawford, 2005; Magerko, 2006), making games more dramatically interesting (Prendinger & Ishizuka, 2001; Bates, Loyall, & Reilly, 1994), and increasing overall suspension of disbelief (Byl, 2004). Game players have also expressed a desire for more complex and realistic game characters (Lawson, 2003). One of the factors in creating a compelling experience is suspending the players disbelief. The AIP Cube model states that the requirements of suspension of disbelief are Autonomy of the NPCs, Interaction between the NPCs and the player, and the NPCs Presence in the game world (as cited by Byl, 2004). Byl (2004) also implicates personality, emotion, self-motivation, the illusion of life (i.e. non-omniscience and social context), change, and social relationships as being important to character believability. Rizzo, Veloso, Miceli, and Cesta (1997) describe the factors that are important to the creation of believable agents. These factors include the perception that agents can think, feel emotions, have an attitude or disposition towards others around them, have

15 similarities to humans, and have a personality.

2.2.1.1 Current NPC Architectures

According to Travis (as cited by Byl, 2004) the requirements for a generalized Non Player Character architecture would be reusability (to enable use in multiple games); sophisticated AI for action selection and control; emphasis on human limitations, performance and characteristics (i.e. non-omniscience, mistakes occasionally made by NPCs); and the ability to turn off features of the architecture that are not required for the game in development. Byl (2004) adds that such an architecture should consider the repeated tasks of NPC perception and interpretation of environmental input, inference, action selection, and outputted NPC behaviour Non Player Character behaviour is often handled using scripting to hard-code behaviour in each possible game situation (Berger, 2002), or finite state machines to represent state of mind (Byl, 2004; Funge, 2004). A few basic NPC controllers described in Funges (2004) book are reacting controllers, remembering controllers, and communicating controllers. Reacting controllers may be implemented with a few simple mental states to govern reactions (Byl, 2004; Funge, 2004). Remembering (or stateful) controllers have limited memory, and typically all NPCs share the same memory bank (Funge, 2004). Communicating controllers broadcast communications between NPCs (Funge, 2004). More complex mental states such as emotions (Funge, 2004; Rizzo, Veloso, Miceli, & Cesta, 1997) can also be implemented, and explicit forgetting of previously remembered events has also been done (Alt & King, 2002; Grond & Hanson, 1998). Byl (2004) discusses five general agent architectures that are applicable to the use in video games: Reactive, triggering, goal-based, utility-based, and anytime architectures. Reactive agent architectures (also known as reflex, or stimulus-response architectures) are implemented by hard-coding rules. These agents do not hold state, and therefore have no long-term reasoning. Triggering architectures allow a reactive agent to maintain internal

16 state, thus allowing for self-awareness and memory (Byl, 2004). While both reactive and triggering architectures are fast (Byl, 2004), both have the disadvantages of being hardcoded, thus requiring that all possible states are known a priori (Byl, 2004), and having a knowledge base of rules that is not very scalable (Byl, 2004). Goal-based architectures have preferred states (i.e. fed, healthy, safe,...) which define an NPCs goals and desires, and allow the agents to plan in order to achieve their goals. Goal-based architectures have the advantages of allowing NPCs to make non-rigid plans, and allowing new plans to be added to the knowledge base quite easily (Byl, 2004). The disadvantages of a goal-based architecture are that goals are served on a first-come first-served basis, and there is no preference among plans if they will achieve the same goal (Byl, 2004). For example, if an NPC became hungry just before they started getting shot at by an enemy, the NPC would seek food before it would try to seek shelter from the bullets. If the plans available to fulfill the goal of escaping enemy-fire included the option of jumping into a pit of poisonlaced spikes to get away and the option of dodging behind a bunker, the NPC would be just as likely to select the deadly plan (spike-pit) as the safe plan (bunker). Utility-based agents solve this problem by assigning importance ratings to goals dynamically, based on the NPCs current situation. Unfortunately the computation cost of such a calculation can become prohibitive to interactive games as goal-hierarchies become more complex (Byl, 2004). Anytime agent architectures try to resolve this by placing time limits on action selections (Byl, 2004). Disadvantages of this architecture were not discussed in (Byl, 2004). Some hybrid game-specific architectures were discussed in (Byl, 2004), including the Soar architecture of human cognition, the H-CogAff human-like agent architecture, the EXCALIBUR Project, and the being-in-the-world MUD (Multi-User Dungeon) agent architecture. Soar is based on the utility-based agent architecture previously described, and seems mostly concerned with how agents can select and learn new rules. There are three main modules: Chunking (which is responsible for learning new rules), Long Term Memory (which acts as a knowledge base), and Working Memory (which stores the

17 agents goals). Soar also does production matching (to determine which rules are fired), and bases decisions on a preference list (Byl, 2004). So far implementation of Soar in games has been restricted to a version of Quake II deathmatch (Byl, 2004). H-CogAff is based on the anytime agent architecture, and has three layers: the reactive, deliberative, and meta-management layers. The reactive layer is similar to the reactive agent architecture in that it reacts quickly in a stimulus-response manner without relying on states. The deliberative layer is responsible for goal-based planning. The meta-management layer is responsible for self-reflection, self-monitoring, and modifying thought processes. H-CogAff also has a module for defining personalities (Byl, 2004). The EXCALIBUR Projects focus is on generic NPC plan generation for games and is based on the anytime agent architecture. The plan generation takes into account temporal scheduling (making sure actions are still valid by the time they are executed), the NPCs need of environmental resources, and the fact that planning must be done with incomplete knowledge of the game world (Byl, 2004). Finally, the being-in-the-world architecture has two modules: the real-time coping module and the reasoning module. The real-time coping module is responsible for perception, maintaining a knowledge base, rapid stimulus-response behaviours, and breaking plans down into actions based on the current environment state. The reasoning module is responsible for generating those plans to give the real-time coping module. Communication between these levels takes the form of a goal queue (Byl, 2004).

2.2.2

Psychology, Sociology, and Psychosocial Behaviour

In a letter to Gamasutra, Lawson (2003) pointed out that developers are restricting themselves in using cognitive science to model NPCs. He described a far more interesting scenario for games were they to use social science. He described a scenario where a characters level of experience or worthiness in battle would be far less threatening than the potential influence of his social ties. In order to fully understand Lawsons plea, we must first understand what psychology and sociology are, and how

18 they differ from one another.

2.2.2.1 Psychology

Psychology is included in the field of cognitive science (Matlin, 2002), and is defined as being the scientific study of external behaviour and internal thought (Wortman, Loftus, Weaver, & Atkinson, 2000). Cognitive science includes areas from many other fields, including computer science, linguistics, anthropology, neuroscience, and philosophy (Matlin, 2002), and is defined as being the study of the mind (Matlin, 2002). This includes the internal mental representations people have of the external world, but not external behaviours (Matlin, 2002). Cognitive psychology is a subset of psychology, and is concerned with the study of mental processes, such as problem solving, thinking, memory, language, decision making, and learning (Wortman et al., 2000). Cognition can be broadly defined as the process by which information is organized to accomplish a goal. From the previous discussion on agent architectures, it can be noted that many of the architectures are primarily concerned with this area. H-CogAff is also concerned with decision-making, action-selection, and meta-cognition (thinking about ones thoughts, or self-monitoring (Matlin, 2002)). The EXCALIBUR Project is concerned with decisionmaking and planning under certain circumstances that may be present in the game world, and being-in-the-world is likewise concerned mainly with cognitive processes. All of the agent architectures can likewise be considered purely cognitive architectures, with the possible exception of utility-based architectures, including the Soar architecture. The reason for this exception is that utility-based architectures differ from the other architectures in one respect: decision-making due to internal stimuli. According to research done by Damasio (as cited in Wortman, Loftus, Weaver, & Atkinson, 2000) such decisions due to plan preferences cannot be made without emotion. Emotion is defined as being an involuntary response to a challenging situation that involves physiological changes, expressive behaviours, and a state of feeling (Wortman et al., 2000). While this definition may be accurate in the study of real organisms, it is not of much use in

19 attempting to simulate emotion in software. A more useful definition is Funges (2004) definition of emotion as the sum of past events. Damasios studies (as cited in Wortman, Loftus, Weaver, & Atkinson, 2000; Picard, 1995) on individuals with brain damage indicated the importance of emotion in decision-making. With emotions being fully dissociated from cognition, people were capable of reasoning yet were unable to make decisions, even when the decision should have been obvious. When decisions were made, they were selected in an arbitrary fashion. One patient would agonize over whether to have banana pudding or strawberry shortcake for dessert, even though he knew he was severely allergic to strawberries (Wortman et al., 2000). When making a decision he would be just as likely to choose strawberry shortcake as banana pudding. Of the agent architectures that observe agent goals, this very closely resembles the type of behaviour exhibited in the goal-based agent architecture. In the goal-based agent architecture, no plan was considered better than the other (i.e. eat banana pudding versus eat allergy-inducing strawberry shortcake) should they fulfill the same goal (i.e. have dessert). We can now see that cognition can be considered fairly analogous to plan generation, while emotion is analogous to actionselection based on internal stimuli. Both Soar and other utility-based architectures can be seen as having basic emotional simulation because they weigh these internal stimuli in action-selection. Another factor that influences behaviour and is an area of study in psychology is personality. Rizzo, Veloso, Miceli, and Cesta (1997) define personality in agents as the agents pattern of behaviours or interactions with its environment. The next section will discuss what research has been done in the area of simulating emotion. Personality and how it relates to psychology and social science will be covered in Section 2.2.2.2.

2.2.2.1.1

Emotions in Computation

Emotion in computation is primarily studied in the field of affective computing

20 (Picard, 1995; Byl, 2004), which Picard (1995, para. 3) defines as computing that relates to, arises from, or deliberately influences emotions. Affective computing involves attempts to identify, process, or synthesize emotions (Byl, 2004; Picard, 1995). Because the area of Non Player Character control in video games is currently only concerned with synthesis of emotional states, only the latter of these will be discussed. It is noted that in this context only emotional states that will directly affect the characters action selection should be modelled (Funge, 2004; Byl, 2004). It has already been noted how utility-based agent architectures, including Soar, have basic emotional simulation. There are two models of emotion in computation: categorical and dimensional (Tomlinson & Blumberg, 2002a). It has also already been mentioned that Finite State Machines (FSMs) are often used as the primary method to represent NPC mind states (Byl, 2004; Tozour, 2002), yet Funge (2004) notes that this is also the case in modelling emotions in NPCs. This is an example of using the categorical model of emotion in computation (Tomlinson & Blumberg, 2002a). Picard (1995) also notes that this type of categorical model is often used to model emotions, since the differentiation between emotional states is fairly discrete. Transitions between two emotional states may be implemented to have different chances of occurring depending on the states involved - for example, a character should have a higher chance of transitioning from interest to joy than from distress to joy (Picard, 1995). Smith and Ellsworth (as cited by Byl, 2004) have modelled 15 emotions, including happiness, sadness, anger, boredom, challenge, hope, fear, interest, contempt, disgust, frustration, surprise, pride, shame, and guilt. Picard (1995) also discusses rule-based models of emotions that use scripts to model changes of states. In contrast to this, research has also been done in dimensional modelling of emotion in computation, where the emotional range is mapped into a continuous range, for example a float value (Tomlinson & Blumberg, 2002a; Tomlinson & Blumberg, 2002b). Byl (2004) discussed character emotion in video games in great detail. One important aspect that she discussed was that there could be different levels or uses of emotions depending on what is required by the NPC. Byl (2004) described these as

21 emotional behaviours, fast primary emotions, emotional experiences, body-mind interactions, and cognitively-generated emotions. Emotional behaviours are very basic and are largely motivational. An example is Walters tortoise (as cited by Byl, 2004) that used emotions such as a need for power to motivate seeking power sockets. Fast primary emotions are more specific to quick self-preserving reactions, such as dodging enemy-fire, or using health potions. Emotional experience describes the state of selfawareness of ones own emotional state and subjective feelings; Byl (2004) admits this would be difficult to implement in a game, and it could be argued that this would have no relevance in a game due to not having an affect on external behaviour (Funge, 2004; Byl, 2004). Body-mind interactions would model how emotional states affect body states, and vice versa. One implementation of this is Canameros the Abbotts and the Enemy (as cited by Byl, 2004), which is staged in a 2D world where the NPCs have synthetic hormones to influence their actions, including attacking, withdrawing, drinking, eating, playing and resting. Finally cognitively-generated emotions are emotions generated from appraisal of external stimulus. One implementation of this is the Tok agent architecture by the Oz Project (as cited by Byl, 2004; Bates, Loyall, & Reilly, 1994). The Tok agent architecture varies the intensity of emotions depending on the perceived importance of a goal (Byl, 2004). Toks architecture has two modules, Em and Hap, which interact with each other (Bates, Loyall, & Reilly, 1994). Em is responsible for emotions, attitudes, and standards. Hap is responsible for goals and behaviours. Two other architectures that involve emotions are Blumbergs Silas T Dog (as cited by Byl, 2004) and PETEEI (as cited by Byl, 2004).

2.2.2.2 A Link Between Psychology and Social Science: Personality

Personality in agents is defined as the agents pattern of behaviours or interactions with its environment (Rizzo, Veloso, Miceli, & Cesta, 1997). Byl (2004) discusses the three dimensions of personality: physiology, sociology, and psychology. Physiology would most likely not have a bearing in the discussion of personality in software agents

22 since such agents do not have physiology, unless hormones were synthesized, as in Canamaro (as cited by Byl, 2004). Crawford (2005) notes that personality traits that do not affect behaviour should not be modelled, yet specific tendencies (i.e. tendency to shake everyones hand) should likewise not be modelled. Instead modelling should focus on fundamental traits (Crawford, 2005). Isbister (2006) discusses the social psychology research and discusses how different levels (i.e. float values) of the two traits of agreeableness and dominance can be used to form many different personalities. Crawford (2005) outlines one possible personality model for NPCs in games that includes intrinsic variables, mood variables, relationship variables, and the readiness to change the previous two variables (volatility and accordance). Intrinsic variables would include the personality traits of integrity, virtue, power, intelligence, and attraction. Mood variables would include three sliding scales: anger/fear, arousal/revulsion, and joy/sadness. Relationship variables would reflect the NPCs beliefs about anothers intrinsic variables, and would closely model such beliefs as trust (belief in ones integrity), judgement or fear (belief in ones power) and respect (belief in ones intelligence). Finally the last two variables, volatility and accordance, would be the readiness to change mood variables and relationship variables respectively. It should also be noted that Crawfords (2005) model of volatility included three variables: adrenaline (speed of change in the anger/fear mood variable), sensuality (speed of change in the arousal/revulsion mood variable), and manic/depressive (speed of change in the joy/sadness mood variable). Crawford (2005) stated that this model of personality could be implemented as a decision script. Isbister (2006) also discussed the OCEAN (or The Big Five) psychological model of personality, which includes Openness to experience, Conscientiousness, Extroversion, Agreeableness, and Neuroticism.

23 2.2.2.3 Social Science

In Lawsons (2003) letter to Gamasutra, he pointed out the importance of social network modelling concepts such as the modelling of trade networks, the spread of ideas, and conformity, and discussed how social networks could be modelled. In such models there are nodes, ties, and hubs. Nodes would be actors, such as individuals or organizations within a group. Ties are the relationships between nodes, and can be either strong or weak. Hubs would be nodes that have a high number of ties with other nodes. It has already been noted that a combination of the traits of agreeableness and dominance can produce many different kinds of personalities. Isbister (2006) called these two traits the primary social variables, because the perception of agreeableness and dominance in others is often used to determine whether someone is a friend or foe. Isbister (2006) also discussed the social concepts of culture, social roles, power dynamics, and obligations. Cultures are identities that people share and use to determine who is in their group. Cultures can be formed from shared experiences, such as background, goals, profession, ethnicity, gender, beliefs, or interests (Isbister, 2006). Cultures do not have to be static and can shift, blend, or otherwise change over time (Isbister, 2006). Another thing that binds individuals together includes kinship and group obligations (Isbister, 2006). Isbister (2006) also discussed friendly, unfriendly, and neutral social roles with respect to dominance levels. Friendly social roles NPCs may have, from least dominant to most dominant, include minion, rescuee, pet, sidekick, ally, guide, and mentor. Unfriendly social roles, least dominant to most dominant, would include obstacle, enemy, competitor, boss monster, and arch enemy. Finally there are neutral social roles, such as audience, informant, trader, and host (Isbister, 2006). Note that the players role would be roughly equivalent in dominance to the ally role and the competitor role (Isbister, 2006). These dominance differences are similar to the concept of power dynamics that can occur within and between social groups (Isbister, 2006). Guye-Vuilleme and Thalmann (2001) defined a role as a coherent set of behaviours (or plans) that includes the consideration of worldview and can be linked to

24 culture. Worldview is a term that describes the fact that each individual has a different view of the world. It was also noted that multiple cultures may be present in a single individual, and different roles could be active depending on the situation an individual was in, or the active roles of individuals around them (Guye-Vuilleme & Thalmann, 2001). Norms were defined as guides or rules for behaviours in given situations, and may also be associated with roles and culture. Values were defined as personal moral systems, which influence actions with the goal of achieving an ideal abstract state. Such abstract goal states could include being honest, being disciplined, being ruthless, et cetera (Guye-Vuilleme & Thalmann, 2001). These values could also be attached to roles or professions, for example law enforcement (Guye-Vuilleme & Thalmann, 2001). Another interesting point to note is that the social role of the player character must be very carefully considered. Isbister (2006) claims the reason for this is that the ability of the player to relate to their player characters social role may affect their ultimate enjoyment of the game. Social interaction is defined as the process of being influenced by other people, attitudes, social cognition, and the social influence or standing of others (Byl, 2004).

2.2.2.3.1

Faking Social Science with Reputation Systems

Social ties between groups have been attempted through the creation of reputation systems (Alt & King, 2002; Brockington, 2003; Electronic Arts, 2005; Grond & Hanson, 1998; Russel, 2006). Reputation systems such as those in Fable (Russell, 2006), Thief: Deadly Shadows (2004), and Ultima Online (Grond & Hanson, 1998) refer to systems that typically manage NPC opinions of the player (Alt & King, 2002; Electronic Arts, 2005). NPC opinions are formed immediately and globally among all NPCs upon certain player actions (Alt & King, 2002; Brockington, 2003; Electronic Arts, 2005; Grond & Hanson, 1998). Individual opinions are not maintained in reputation systems (Alt & King, 2002; Electronic Arts, 2005), nor are opinions about other NPCs (Alt & King, 2002; Grond & Hanson, 1998; Russell, 2006). Group opinions may be maintained (Alt &

25 King, 2002; Russell, 2006; Brockington, 2003; Grond & Hanson, 1998; Thief: Deadly Shadows, 2004). As an example, Thief: Deadly Shadows (2004) maintains two distinct groups, Pagans and Hammers, and each group can hold a positive or negative opinion about the player. Note that since individual opinions are not maintained, the player cannot be friends with an NPC while being enemies with the group that NPC belongs to. If an NPC that is a member of a group holds an opinion about the player, every member of that group will hold the exact same opinion. Some reputation systems have mechanics to manage whether a player action was witnessed or not, and if the event was not witnessed it will have no effect on the NPC opinions of the player (Alt & King, 2002; Brockington, 2003). Some reputation systems even take into consideration whether witnesses are friendly, neutral, or hostile to the player (Brockington, 2003) or will have different witnesses react differently in accordance with their personality (Russell, 2006). Forgetting, or forgiving, of previous player actions has also been implemented, usually by an event such as the death of the player character (Brockington, 2003) or by the player praying to a shrine (Electronic Arts, 2005), though forgetting due to elapsed time has also been implemented (Grond & Hanson, 1998). Note that this type of system is only faking a social system - while there may be groups of differing opinions and the player has a standing among those groups, there is no influence of individuals on the groups or social ties between individuals. The simulated social ties between groups are static (Cole, 2006; Thief: Deadly Shadows, 2004; Alt & King, 2002). Many of the social science concepts discussed in the previous section, such as roles, social ties, power dynamics, and obligations are not modelled, and any semblance of them is typically hard-coded. It should also be noted that reputation systems have been noted to be much more time consuming and difficult to implement than expected (Brockington, 2003).

26 2.2.2.3.2 Other Games with Social Systems

Some of the latest games concerning social systems are the recently released Bully (Game Informer, 2006b) and the as yet unreleased Assassins Creed (Game Informer, 2006a). Bully has been said to be first and foremost... a game about social relationships (Game Informer, 2006b, p. 53). The player characters standing among the five cliques in the game world directly affect difficulty of missions. Each character has been modelled as an individual, with a name, personality, and clique to which they belong which differs from many open-ended games in which most NPCs are modelled as random people who have no affect on the story and cannot be interacted with on any meaningful level (Game Informer, 2006b). Assassins Creed is a next-gen game in which social norms, crowds, and NPC tolerance levels are used as game obstacles and mechanics. Opinions of the player character are shared by like-minded individuals (Game Informer, 2006a), and directly influence mission difficulty, like in Bully. Individuals within groups have distinct social roles; to kill the biggest enemy or leader in a group of enemies will cause the remaining enemies to retreat, seek help, or surrender (Game Informer, 2006a). While these games have interesting social concepts and gameplay mechanics behind them, the developers have not made it public how these processes were designed or implemented. Assassins Creed uses need-driven NPCs according to Game Informer (2006a), but any other details or processes that may be present were not mentioned. The next section will discuss what has been published in the research on social software characters.

2.2.3

Socially Intelligent Agents

Agents in a game world are autonomous (Barnes & Hutchens, 2002), responsive, pro-active, and flexible (Jennings, Sycaro, & Wooldridge, 1998). This is in contrast with actors that are controlled by a hard-coded script (Barnes & Hutchens, 2002). A single character can have both agent and actor processes within it that are alternated between in

27 certain circumstances (Barnes & Hutchens, 2002). Such a character could be an actor in one situation (i.e. while the player is interacting with the character) and an agent in another situation (i.e. when going about life simulation, pursuing own goals, etc.) (Barnes & Hutchens, 2002). The area of social agents, or Socially Intelligent Agents (SIAs) (Byl, 2004; Prendinger & Ishizuka, 2002) would appear to have much relevance to this research. Prendinger and Ishizuka (2002) define research in Socially Intelligent Agents as being concerned with forming and maintaining social relationships between humans and agents. SIAs have been discussed in the use of studying human and animal sociology (Dautenhahn, 2000; Tomlinson & Blumberg, 2001) However, many of these social agents do not exhibit realistic social behaviour (Wooldridge, 1992; Jennings, Sycaro, & Wooldridge, 1998; Guye-Vuilleme & Thalmann, 2001; Cesta, Miceli, & Rizzo, 1996; Hogg & Jennings, 2001). Guye-Vuilleme and Thalmann (2001) points out that the word social means different things depending on the field in which it is discussed: in the field of graphics, social describes crowd and group control, in the field of agent research it describes agent communication with other agents and humans, and in sociology it describes all human behaviour. Much research in SIAs implements only communicative behaviour that is used in a multi-agent problemsolving context to solve problems, reduce resource usage, and increase efficiency (Hogg & Jennings, 2001; Cesta et al., 1996; Jennings et al., 1998).

2.2.4

The Relevant Social Research

Some relevant research in the area of enabling social awareness in agents includes Tomlinson and Blumbergs (2002a) remembered interaction histories between agents, Prendinger and Ishizukas (2002) change of attitudes and familiarity assessment between agents, Coles (2006) changes and propagation of opinions, and Guye-Vuilleme and Thalmanns (2001) high level architecture to enable social identities for social agents. The main concept behind Tomlinson and Blumbergs (2002a) remembered

28 interaction histories is that previous emotional history with a particular individual affects each new interaction with that individual. This is based on Damasios Somatic Marker Hypothesis (as cited by Tomlinson & Blumberg, 2002a). In this architecture each character has an emotional state, a current object of attention, and an emotional memory for every other character in the simulated world. The essence of this architecture is that each new interaction is influenced by the NPCs current emotion and their remembered emotional memory about the object (or character) of their attention. Every time an interaction ends, the remembered emotional memory is updated to factor in the last interaction (Tomlinson & Blumberg, 2002a). While Tomlinson and Blumberg (2002a) implemented this architecture in a system to study the social behaviours of wolves in a pack, they pointed out that it might be applicable for computer games to improve interaction with virtual characters. Prendinger and Ishizuka (2002) did research into implementing familiarity assessment between characters, and change of attitude. Both of these were dependent on the involved agents emotions (Prendinger & Ishizuka, 2002). Attitude is defined in Wortman, Loftus, Weaver, and Atkinson (2000) as being a disposition towards an object, or a like or dislike. Note that this object of like/dislike can be anything, such as a person, place, event, or idea. Prendinger and Ishizukas (2002) research discussed how changes in familiarity assessment and change of attitude towards a character were affected by that characters actions, previous attitude towards that character, and recency or weight of historical information. Note that decay of previous emotions was not implemented for simplicity (Prendinger & Ishizuka, 2002). It was even considered how conflicting character actions (such as a negative action when all previous interactions were positive) would affect attitude towards them in the form of momentary and essential liking or disliking. Momentary liking/disliking would be integrated into the previous history, while essential liking/disliking would completely replace the previous history (Prendinger & Ishizuka, 2002). The tendency for an agent to have momentary or essential liking/disliking could be influenced by personality traits such as forgiveness, or overall familiarity level with the character involved (Prendinger & Ishizuka, 2002). Familiarity

29 was implemented as a monotonic value, as the assumption was made that characters familiar with each other would never again become unfamiliar (Prendinger & Ishizuka, 2002). This research was implemented using pet characters that would interact with a user, and was thus not implemented in a multi-agent environment. Prendinger and Ishizuka (2002) used the XML-based Multimodal Presentation Markup Language and the SCREAM tool to implement this (Prendinger & Ishizuka, 2002). NPC opinions in games are typically static, but Cole (2006) addressed how changes in opinion could be modelled in a social network in the context of games. Social network analysis, the same concept discussed in Lawson (2003) was used here. Here the nodes (or actors) could be any kind of news source, for example individuals, witnessed events, or a newspaper (Cole, 2006). The strength of ties between nodes was modelled as a connectivity score (or two, as ties could be asymmetric (Byl, 2004)), and hubs were not used (Cole, 2006). Cole (2006) was of the opinion that opinion flow could be modelled as a flock, based on the Boids flocking algorithm. In each individual, their opinion on a given topic could have the forces of alignment, cohesion, and separation acting on it. Alignment would refer to the tendency to agree with the rest of the groups average opinion. Cohesion would be a force moving the NPCs opinion closer to agreement with the opinions of those the NPC respects. Finally separation would be a force moving the NPCs opinion away from agreement with the opinions of those the NPC held in contempt. The field of vision normally used in the Boids algorithm would be replaced with a connectivity score representing the strength of social ties between nodes. Opinion and respect values were dimensionally modelled to range over float values. Cole (2006) also considered the presence of radicals, or individuals who could not be swayed in the same way by group opinion. Ignoring forces under a certain threshold and re-normalizing connectivity scores when connections changed controlled efficiency (Cole, 2006). Memory usage for such an implementation was noted to grow logarithmically with the population, assuming a fixed number of connections (Cole, 2006). It was noted that such an implementation could produce new types of gameplay, an example quest being for the

30 player to find and use sources of influence to sway public opinion (Cole, 2006). Guye-Vuilleme and Thalmann (2001) stated that the requirements of a believable social agent architecture would have to include selection of agent actions being based on the social environment as well as the physical (the very definition of social interaction (Byl, 2004)), and the ability to generate different beliefs for different agents in the same situation (different worldviews). Guye-Vuilleme and Thalmann (2001) discussed how agents could have multiple cultures and roles, and how these may be associated with norms and values. Guye-Vuilleme and Thalmann (2001) defined a role as a coherent set of plans, which would include worldview and would be linked to culture(s). In this architecture, different roles within a single agent could be given weights and priorities, could be associated with one another, and could be switched between depending on the situation. This switching would be handled by maintaining an activation energy value for each role that, once reached, would activate the role. Other factors listed by Guye-Vuilleme and Thalmann (2001) that could influence agent behaviour included norms and values. Norms were defined as rules for behaviour in certain situations (Guye-Vuilleme & Thalmann, 2001). Values were defined as personal moral systems, or the pursuit of an ideal abstract state (i.e. ...because I am honest) (Guye-Vuilleme & Thalmann, 2001). With values in place, agent action selection could factor in these values over chances of success (Guye-Vuilleme & Thalmann, 2001). For example, an NPC may act to save someone (i.e. because I help others) despite their own self-sacrifice in doing so. Guye-Vuilleme and Thalmann (2001) also discussed group belonging, culture, and gender, and how these could affect agent social identities, motivations, and reasoning. Guye-Vuilleme and Thalmanns (2001) agent architecture includes several modules, including a perception module, a cognitive module (which factors in active roles and generates beliefs), a type identifier (which includes an inference engine or rules), a social reasoner (which weighs beliefs and values to select behaviour), and an actions manager (runs action scripts given an action list from the social reasoner). This

31 architecture was implemented using C++, Lisp, and Python. One proposed solution to the problem of integrating personality, mood, and social relationship information is referred to as the Realistic Reaction System (RRS) and involves using graphs to model social ties between individuals, and uses state machines and rule-based systems to implement (Gruenwoldt, Katchabaw, & Danton, in press; Gruenwoldt, Katchabaw, & Danton, 2005). Graph querying is used to determine appropriate behaviours within the context of the social network. Filtering and aggregating of relevant information is used to reduce complexity. Input events are processed by an AI Controller, which does decision-making and action-selection using a state machine. This AI Controller consults a Personality/Mood Filter and the Relationship Manager, which is responsible for filtering and aggregating information from the Relationship Network. The Personality/Mood Filter contains traits such as intelligence, aggressiveness, attentiveness, disposition, and prejudices (Gruenwoldt et al., 2005). Social ties between individuals have attributes that specify the type of relationship (i.e. emotional, familial, business, leadership, etc.), origin of the relationship, the history, regularity, strength, polarity, and validity. Note that relationships cannot be of more than one type (i.e. both business and leadership) (Gruenwoldt et al., in press; Gruenwoldt et al., 2005). This solution has been implemented using Unreal Script.

2.3

Improvisation in Gameplay: Emergent Gameplay

A particularly interesting approach to problems of providing unique, or emergent,

gameplay experiences lies in what is often referred to as emergent gameplay (Wooton, 2006; Smith & Smith, 2004). Mark LeBlanc (2000) defines emergence (or emergent complexity) as properties that cannot be simply inferred from a systems rules. Emergent gameplay (also called emergent behaviour) is a term used to describe the application of this concept to the ability of the game player to contribute to this emergence. Emergent gameplay involves implementation of simple component-level behaviours that interact (Wooton, 2006, Smith & Smith, 2004; Scott, 2002). The concept

32 behind this, and all emergent systems, is that the interaction of these simple componentlevel behaviours often results in complex and surprising system-level behaviour (Wooton, 2006; Johnson, 2001; Holland, 1998). Emergence has been studied in a variety of disciplines, including philosophy, biology, chemistry, physics, ecology, neurology, psychology, and computer science (Johnson, 2001; Holland, 1998). Emergent gameplay focuses on implementing a bottom-up view of the game world, in that every (static or non-static) object in the world maintains its own needs and reactions to situations that may arise (Wooton, 2006). Every object is self-centered and goal-directed (Wooton, 2006). Emergent systems are not scripted, nor are they rule-based (Wooton, 2006). Emergent systems do not focus on algorithmic behaviours, but rather very simple stimulus responses at the component level (Wooton, 2006). While an emergent system itself is not rule-based, it may have components that are (Holland, 1998). One real world example from Smith and Smith (2004) is how global warming results in cooler weather in England. Global warming Ice caps melt Ocean currents shift Less warm air arrives in England Colder England By having simple component-level mechanics that interact with each other, overall system behaviour is complex, robust, and unpredictable (Wooton, 2006; Johnson, 2001; Holland, 1998). Other places this concept has been successfully used are in flocking algorithms (Cole, 2006; Wooton, 2006), card games (LeBlanc, 2000), and board games (Wooton, 2006; LeBlanc, 2000; Holland, 1998). Chess, for example, only has 24 rules, yet many different and interesting gameplay situations are possible (Wooton, 2006; LeBlanc, 2000; Holland, 1998). The Mechanics-Dynamics-Aesthetics model (Wooton, 2006; Smith & Smith, 2004) outlines how mechanics of the game (such as rules of the game, what agents can do, and what can physically happen) must first be defined. Dynamics are what occur when these mechanics interact with each other. This step is very important, as emergent

33 gameplay cannot occur without this interaction. Aesthetics refers to the human response to the dynamics. Emergent gameplay has been used in computer games, just as it has been used in board games. Games like SimCity, The Sims (Wooton, 2006), EverQuest (LeBlanc, 2000), Deus Ex, and Thief: Deadly Shadows (Smith & Smith, 2004) have all used some sort of emergent gameplay. For example, in Ion Storms Thief: Deadly Shadows, emergence is used to define how elements like fire, oil, noise, and guards interact (Smith & Smith, 2004). The Sims uses emergent gameplay in defining how objects can be used (Wooton, 2006). With few exceptions, most developers do not publish how they have (or have not) used emergent gameplay. Smith and Smith (2004) mentions that Ion Storm used a Stimulus System to implement emergent gameplay in order to have objects listen to each other. Emergent gameplay does have its disadvantages. While it can result in interesting system-level behaviour, it is very difficult to completely test such a system (Wooton, 2006). According to Wooton (2006), boundary condition testing works best for systems with emergent gameplay. Typically some scripting is still required in order to control the story experience and story progress (Wooton, 2006; LeBlanc, 2000), as well as to define NPC movement (Wooton, 2006), resulting behaviour (Smith & Smith, 2004), dialog trees, and waypoints (Wooton, 2006). Switching may be done between this scripting control and the emergent gameplay system governing autonomous behaviour, similar to what was earlier described by Barnes and Hutchens (2002). Another thing to note is that the component-level mechanics must be implemented carefully to have the desired affect: a sufficient number of mechanics must be implemented, interactions must be defined between the mechanics, and the mechanics must occur within close enough proximity for interactions to occur during gameplay (Smith & Smith, 2004). Despite these disadvantages, the appeal of what is offered by emergent gameplay is enough to encourage developers to pursue it and often with success (Wooton, 2006; Smith & Smith, 2004; LeBlanc, 2000). Emergent gameplay offers a way of breaking from the limitations of scripting and state machines (Wooton, 2006). This enables

34 improvisational and player-authored gameplay (Smith, 2007; Smith & Smith, 2004), more explorable space within a game (Smith & Smith, 2004; LeBlanc, 2000), non-linear experiences (Wooton, 2006), emergent narrative (LeBlanc, 2000), and surprising and suspenseful emergent challenges (Smith & Smith, 2004; LeBlanc, 2000). Emergent gameplay is highly scalable (Wooton, 2006) resulting in the possibility of larger more complex game worlds. With emergent gameplay, these worlds can be implemented to feel more alive (Smith & Smith, 2004), natural (Wooton, 2006), more dynamic (Wooton, 2006; Smith & Smith, 2004), adaptive (Wooton, 2006), more interactive and responsive (Wooton, 2006; Smith & Smith, 2004), and in such a way that player interaction is not needed to maintain this illusion of a living world (Smith & Smith, 2004). This capability of providing more exploration and improvisation has great potential to increase the appeal of a game to more consumers. Bartle (1996) discussed four player types he was able to discern in his research, including Killers, Socializers, Explorers and Achievers. The addition of emergent gameplay to a game that would typically appeal to Killers or Achievers (such as an action, action-adventure, role playing game, or stealth game) can add appeal to Explorer player types by providing greater opportunity for players to explore and improvise.

2.4

Summary of Literature Review

Many of the papers reviewed had some components of a psychosocial agent

architecture (autonomy, emotion, personality, and realistic social behaviour), but very few had integrated all of the components necessary for realistic believable psychosocial behaviour. This chapter discussed basic agent-control including the use of finite state machines and scripting. It was noted that these presented problems in that they required a priori knowledge and were rigid in the face of unexpected player actions and events. Basic NPC control with reacting, remembering, and communicating NPC control was discussed, noting that none of these satisfied realistic psychosocial behaviour. Reactive,

35 triggering, goal-based, utility-based, and anytime agent architecture were also discussed, and save for utility-based architectures, were determined not to support even basic emotional simulation. Soar, H-CogAff, EXCALIBUR, and being-in-the-world architectures (as cited by Byl, 2004) could support emotional simulation, however none of these architectures appeared to consider social behaviours. Crawfords (2005) personality model discussed relationship variables, yet this only handles NPC beliefs about other NPCs personality traits, and does not model other social features such as social ties or networks. Reputation systems were discussed as a way of simulating social systems such as player standing among group opinion, but were noted to be a coarse and static way of modelling social systems, and are complex to implement. Tomlinson and Blumbergs (2002a) remembered interaction histories does not fully model psychosocial behaviour, particularly with respect to social networks and ties, yet presents a novel and interesting way of handling social moods. Cole (2006) likewise presented an interesting discussion of how opinion flow could be modelled as a flock, though opinion flow is only a small part of social behaviour. However, it may be wondered whether other social concepts, such as gossip, conformity, and the spread of ideas, could be modelled in a similar fashion. Another paper of note was Prendinger and Ishizukas (2002) implementation of change of attitudes and familiarity, which presents some interesting ways of handling some social dynamics in a single-agent architecture. The Realistic Reaction System (Gruenwoldt, Katchabaw, Danton, 2005) was a proposed solution to integrating all the features required for realistic psychosocial behaviour in a multi-agent world with a social network using state machines. However, state machines require that the designer knows every situation that may occur a priori, which may not be possible in games, or may be too complex to exhaustively define. Guye-Vuilleme and Thalmanns (2001) high level architecture, which outlines the handling of social roles, including multiple associated roles, roles being activated by peer roles and situations, and the use of activation energies, provides one of the most interesting attempts to implement social identities in agents. However, it is not known

36 what the performance of such a system would be, or if it would be suitable in the use of real-time interactive games. The current work will support emotion, personality, realistic social behaviour, including group memberships, social roles, and other psychosocial concepts. This unified psychosocial framework will use the concepts behind emergent gameplay on psychological and social concepts, while remaining efficient in CPU and memory usage. The use of emergence will allow the psychosocial framework to realistically and flexibly react to undefined circumstances, as well as maintaining NPC autonomy.

37

Design

The basic design around this framework is based on the core ideas behind emergence discussed in Section 2.3. Recall that emergence focuses on a bottom-up view of the game world, where simple component-level behaviours (or stimulus responses) interact to form complex system-level behaviour. This is done by making every object in the world self-centered, goal-directed, and responsible for its own needs and responses. To extend these ideas into a psychosocial context, simple psychosocial objects must be defined that can react to psychosocial stimuli and other psychosocial objects, and be able to maintain their own attributes, needs, and responses to the dynamic game environment. Such psychosocial objects and stimuli can include emotions, personality traits, Non Player Characters (NPCs), groups, or static objects that might be of import to NPCs (such as possessions). In this way, simple component-level psychosocial behaviours will be defined that can interact in complex and interesting ways. Due to the requirement for NPC decision-making in the face of conflicting needs, and in particular to maintain the appearance of consistent behaviour within any one NPC, this basic model of emergence must be modified somewhat for realistic psychosocial behaviour. Section 3.1 will discuss these modifications and how psychosocial traits within an NPC interact to select appropriate behaviour. Section 3.2 will describe the software design of this framework.

3.1

High-Level/Conceptual Design
When considering the development of emergent psychosocial interactions, we

must consider what factors will influence and motivate NPC behaviour, and in particular social behaviour. The most basic factors influencing behaviour are physiological needs (such as needs for food, water, and safety) (Wortman, Loftus, Weaver, & Atkinson, 2000), however since the focus of this work is on higher level social behaviour, those physiological needs will not be addressed in this work. To see how physiological needs

38 can be addressed, Section 3.1.2.1 discusses how The Sims handled physiological needs in the smart terrain system (McLean-Foreman, 2001). Section 3.1.2.3 has a brief description of how physiological needs can be incorporated into this framework. The following sections outline the elements of an NPC social psychology (3.1.1), the mechanics of an emergent NPC social psychology (3.1.2), and mapping stimuli to actions in an NPC psychology (3.1.3).

3.1.1

Elements of an NPC social psychology

In order to construct a basic social being, many different attributes can be assigned to direct its behaviour. Such attributes may include emotions, personality traits, values, worldview, and culture. In addition, the presence of social ties between social objects must be present.

Figure 1 - A sample of an NPC psychology. Figure 1 depicts a sample set of psychosocial attributes. Note the presence of internal states (a personality model, emotional set, and values), social information (social ties, group memberships, and a social influence modifier which could model social phenomena such as peer pressure), and a representation of the current situation (such as

39 whether a stressor is present or not, whether the current situation is friendly or hostile, etc.). Note that not all of the attributes discussed here may be needed to have convincing psychosocial behaviour, nor is this set of attributes exhaustive. Attributes may be added or modified to specialize the framework toward the needs of a particular game. However it is important to note that some form of emotion and personality, at a minimum, should be defined as attributes of an NPC in order to have realistic behaviour, as without personality every NPC would react alike, and without emotion NPCs would be too rigid in their reactions to stimuli to appear lifelike (see below for a plausible, yet challenging, alternative to this). Other attributes may or may not be required depending on the requirements of the game in question. A social influence modifier is only required when certain social phenomena based on the influence of peers and groups should be modelled. A representation of the situation allows for influences such as stress and panic to play a role. Note that some psychologists insist that personality does not exist and that an individuals behaviour is dependent on the situation in which they are in (Funder, 2001). This is called the person-situation debate, and a representation of the situation in the NPC can allow for a balance of behaviours caused by personality versus situation. Note that the truly adventurous game designer could create an NPC with no personality yet a vivid representation of the situation. The following sections will discuss some of the considerations in developing an NPCs internal state (3.1.1.1) and social information (3.1.1.2).

3.1.1.1 Internal States

An example of some internal states that one might include in an NPC psychology could include a model of emotion, a personality model, and even values and a worldview. Since there are different theoretical models of emotions and personality, and different game worlds may require different values, worldview and cultural constructs, it should be

40 possible for the game designer to define their own trait models. For example, a game designer may wish to use a concise personality model for simplicity, rather than a more comprehensive model. To enable this flexibility, these attributes should be stored as data, instead of hard-coded into the software. Two examples of possible personality models that could be used are the agreeableness/dominance model of personality (Isbister, 2006) and The Big Five personality model (Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) (Isbister, 2006). A comprehensive emotional set is Ekmans universal emotions (Anger, Fear, Disgust, Surprise, Sadness, and Happiness) (Wortman, Loftus, Weaver, & Atkinson, 2000). The models of personality and emotion here are samples, and any model that satisfied the game designers goals could be used instead to define personality and emotion. Other traits that may warrant consideration are predictability, or consistency (acting as a random modifier to behavioural choices), and sociability (affecting whether the NPC in question wishes appeal to groups that are social, or groups that are dyssocial, in nature). Worldview and values could be defined in the form of general goals and preferences. Note that traits could also be modelled to be visible or invisible to other NPCs. Visible traits allow for misunderstandings and deception between NPCs. For example, a shopkeeper may feel angry yet project joy in the presence of customers, or a con-man may project traits which suggest trustworthiness to other NPCs.

3.1.1.2 Social Information

Social aspects of the environment can be handled by having the NPC maintain social ties to other NPCs, group memberships (what groups the character belongs to), and a social influence modifier that could enable social phenomena such as peer pressure, groupthink, social loafing, or mobs. Social ties can include the strength of the relationship between two social objects and their polarity (whether the relationship is

41 amiable or antagonistic). The social ties can also be modelled to be either symmetric or asymmetric (such that two people may feel different about each other, rather than sharing the same feelings mutually). Group memberships can be represented as a list of groups that the NPC belongs to. A social influence modifier could be defined as a function to modify the influence of internal states. For example, as the number of people around our character increases, our character may modify his behaviours to conform to the groups behaviours. This section has outlined the elements we may wish to include in our character to implement believable social behaviour. The next section will discuss the mechanics of developing a psychosocial system that is emergent.

3.1.2

The Mechanics of an Emergent NPC Social Psychology

In order to enable emergence in an NPC psychology, it is necessary to understand how emergent systems operate. Emergent systems use simple component-level behaviours that can interact to form complex system-level behaviour (Wooton, 2006; Smith & Smith, 2004). These component-level behaviours operate as stimulus-responses, in that every agent in the world monitors its own perceptions of stimuli and are responsible for responding to those stimuli (Wooton, 2006). Two sample stimulus-response systems (Ion Storms Stimulus System and The Sims object-oriented approach) will be discussed in the next section. Ion Storms Stimulus System was used to enable emergent gameplay by allowing objects to react to stimuli caused by other objects in the world. Characters in The Sims maintained their own physiological needs and used a stimulus-response system to detect stimulus-broadcasting objects that would meet those needs. Note that Ion Storms Stimulus System dealt only with concrete physical objects, such as candles, oil puddles, and guards while The Sims object-oriented approach dealt only with physiological needs. Following the discussion of the two sample stimulus-response systems, an abstract view of a stimulus-response system will be reviewed, culminating in the design of the psychosocial stimulus-response

42 system.

3.1.2.1 Two Sample Stimulus-Response Systems

In order to understand how a stimulus-response system operates to produce appropriate responses to the environment, it is useful to see how the mechanics of an existing system work. The Stimulus System used by Ion Storm is just such a system that was used to enable emergent gameplay (Figure 2). This system was also known as Act/React where it originated at Looking Glass Studios (Smith, 2007). Smith and Smith (2004) described the Stimulus System as a channel, or message passing system (Smith, 2007), through which objects could listen to each other. In this way, a candle could be associated with a Fire Stimulus, which could be defined to interact with an oil puddle in a particular way (i.e. set fire to the oil puddle).

Figure 2 - The mechanics of Ion Storms Stimulus System (Smith & Smith, 2004). The Stimulus System acted as a message passing system for objects and events (Smith, 2007). The messages contained data about the stimulus, such as the stimulus type, propagation type, magnitude, and fall off radius. Stimulus type described what type of stimulus the source object or event produced (i.e. Fire, Piercing, Concussion,...). Propagation type described the method by which the stimulus was passed on to another

43 object, such as by contact or radius. Magnitude described the magnitude of the stimulus (i.e. was this a small, medium, or large explosion?). Finally, the fall off radius would apply when the magnitude of the stimulus decreased with the distance from the point of contact (such as the concussion magnitude from an explosion) (Smith, 2007). Objects in the message passing system were equipped with listeners that would listen for specific types of stimulus (Smith, 2007). These listeners could trigger behaviours upon detection of those stimuli, which could in turn cause more stimuli in the environment. This allowed for feedback loops within the system, contributing to emergence. These triggered behaviours could also involve the transformation of one object into another object that may listen for different stimuli. For example, an object unlit torch that listens for fire stimuli would, upon detection of a fire stimulus, turn into the object lit torch, which would not listen for a fire stimulus (Smith, 2007). Just as the stimulus type of the message could trigger certain behaviours, the other data in the message could also be associated with particular behaviours. For example, an explosion would have a concussion magnitude that would affect the amount of damage and physical impulse dealt to a nearby object (Smith, 2007). The Sims is another game that enables emergence. Like the Stimulus System used by Ion Storm, The Sims uses an object-oriented approach where objects maintain their own needs and behavioural decisions (Woodcock, 2000). Objects broadcast the stimuli they provide in the form of advertisements (McLean-Foreman, 2001; Woodcock, 2000), and the affect of these advertisements decreases with distance (McLean-Foreman, 2001), similar to the Stimulus Systems fall off radius of area affects. Unlike the Stimulus System however, only certain objects in The Sims listen for other objects. These listening objects are the characters in the world, known as Sims. Other objects, such as refrigerators, television sets, and toilets only broadcast advertisements, but do not listen for them. Sims also maintain eight dynamic physiological and mental needs (hunger, hygiene, bladder, comfort, energy, social, fun, and room) that are based on Maslows hierarchy of needs (McLean-Foreman, 2001). These needs are dynamic because the degree of each need changes over time (for

44 example, the need for sleep increases with the amount of time spent awake since sleeping last). Objects in the environment advertise the needs they satisfy, as well as how to use them (McLean-Foreman, 2001; Woodcock, 2000). For example, a refrigerator advertises hunger, a toilet advertises bladder, and a phone advertises social and fun (McLean-Foreman, 2001), and all of these objects advertise how they can be used to satisfy those needs (Woodcock, 2000). This is called smart terrain (Woodcock, 2000). When the Sims detect these advertisements around them, they consult the degree of each of their needs to determine which advertisement would satisfy their most pressing need and carry out the behaviour required to use that object. Unlike the Stimulus System described above, the advertisements in The Sims carry data on how the Sim should behave in order to use that object, while the Sims themselves carry very little information about how they are to interact with other objects. For example, a refrigerator would advertise the need it satisfies (hunger), as well as a sequence of instructions for how the Sim can use it (the Sim must first walk up to it, stand in front of it, face it, then run an animation depicting the interaction, etc.) (McLean-Foreman, 2001). Note that Sims can also form simple relationships to the objects in their environment by repeated use (McLeanForeman, 2001); this relationship data is included in the objects advertisement (i.e. if a Sim has slept in a particular bed in their house multiple times, they will prefer to sleep in it over any other bed). Note that this relationship data is very simple as it holds nothing more than the fact that the Sim will have a preference towards using that object if it is nearby. Both the Stimulus System and the smart terrain system have mechanisms that enable emergent gameplay. By having these systems not directly link behaviours of objects to other object behaviours, but rather provide each object with the capability to execute its own behaviours in response to more general stimuli, emergent experiences can occur. The next section will describe the details of an abstract view of a stimulusresponse system.

45 3.1.2.2 An Abstract Stimulus-Response System

A stimulus-response system operates by having objects create and listen for stimuli. By defining a general channel that allows only certain objects to interact with each other through a stimulus that is created and responded to, we have an easy to use mechanism for defining emergent interactions (see Figure 3).

Figure 3 - An abstraction of the core concepts behind the mechanics of a stimulusresponse system. Note that a Reactor Object can also be an Actor Object if its responses create a stimulus. The object or event that begins the interaction by creating a stimulus can be referred to as the interaction Actor. We will refer to the object that detects and responds to the stimulus as the interaction Reactor (which may then become an Actor through its response). The stimulus acts as a message that is broadcast from the Actor and is received by the Reactors in the area. Reactors listen for messages with particular stimulus types, since each object will react to different stimuli. For example, in the Stimulus System, an unlit torch would only react to messages with a stimulus type of fire, but not messages with stimulus types of piercing, concussion, or water. In the smart terrain system, a Sim who is very hungry will only listen to advertisements of type hunger(McLean-Foreman, 2001), while ignoring other message types. Note that a Reactor Objects response could

46 in turn create another stimulus, thus also being an Actor Object. The stimulus messages can also hold data about the attributes of the associated stimulus. In the Stimulus System, these other attributes included the magnitude of the stimulus, the propagation type, and the fall off radius of the stimulus affect (Smith, 2007). In the smart terrain system, these attributes included instructions for how to use the object, relationship data, as well as the fall off radius of the affect (Woodcock, 2000). These attributes can be mapped onto different behaviours in the Reactor. Note that Reactors do not have to share the same response to a given stimulus. By applying some modifications, this stimulus-response abstraction can be specialized towards psychosocial concepts.

3.1.2.3 A Stimulus-Response System for Psychosocial Behaviour

For an NPC to react in a realistic fashion, they should have the ability to react to the actions of other NPCs, seeing NPCs or objects (i.e. seeing an enemy upon turning a street corner), as well as other events or occurrences in the world, such as natural disasters or accidents. All of these antecedents could be considered the Actors that cause some sort of stimuli. Since these events are to be affected and be reacted to in a psychosocial context, the stimuli created by these events would be psychosocial in nature, such as emotional stimuli (i.e. a witnessed attack may cause an emotional stimulus of fear) or a stimulus that causes a perceived change in situation (i.e. the same witnessed attack may cause the NPC to feel the situation has become hostile). Psychosocial stimuli are created by Actors as messages and are broadcast to the social objects in the area, similar to the abstract stimulus-response system described in the previous section. These messages hold data about the psychosocial stimulus, such as stimulus type, magnitude, propagation, and fall off, as discussed in the sample stimulusresponse systems in Section 3.1.2.1. However, these data are modified to reflect the fact that the stimuli are now psychosocial. Recall that the data in previous stimulus-response systems described physical (or

47 physiological) stimuli and the physical attributes of those stimuli. These attributes included stimulus type (i.e. Fire, Piercing, and Concussion stimuli; hunger, thirst, and fun stimuli), and could also include attributes like magnitude of the affect, the propagation method (how is the stimulus is detected, i.e. fire only causes an affect by contact), and a fall off radius (how does the effect decrease with distance). Since the psychosocial stimulus-response system is not dealing with physical or physiological stimuli, these attributes must be modified somewhat. Stimulus type would refer to emotional and social stimuli instead, such as happiness, sadness, anger, hostile environment, and groupthink. Magnitude of the stimulus would likewise describe the magnitude of the psychosocial effect of the stimulus (i.e. was this a highly emotional event?). Propagation would describe whether the psychosocial stimulus only affects the people directly involved in the event (such as two people greeting each other), if it affects the people witnessing it at a distance as well (such as two people attacking each other). Other types of social stimuli can also be modelled using these stimulus messages, such as gossip and the spread of information. Social stimuli over long distances (such as gossip through a telephone) could be sent as a direct message from one person to another. Note that a social fall off radius could be used as a way of simulating social phenomena. By defining the psychosocial stimulus to propagate by radius and defining a fall off rate, an entire group of people can be affected by the behaviour of a few. This may be useful in modelling social behaviour such as riots or group-induced panics. Note that while this may not be an accurate model, it can simulate behaviour that appears to be life-like. Another attribute that messages could hold is a reference to the Actor object which started the psychosocial stimuli, which may be useful in allowing NPCs to react directly to that Actor. This, or another similar mechanism, appears to be used in the smart terrain system, as Sims may have to seek the source of advertisements (one possible alternative mechanism the smart terrain system may use is in including directions to the source object in the advertisement; note that this will only work if the source is an object that will not move before the advertisement data can be used).

48 The Actors can be anything that broadcasts psychosocial stimuli, including NPCs, objects, and events. The psychosocial stimuli are broadcast to all other social objects in the nearby area. The social objects that are receiving these broadcasts are the interaction Reactors as described in Section 3.1.2.2. These Reactor Objects can be any social object that can react, including NPCs or other social beings, such as groups. Note that nonresponsive social objects (such as possessions) will not react to psychosocial stimuli in any way and therefore it would not be desirable to include these objects as Reactors. It may sometimes be desirable to have more general emergent gameplay, by having a stimulus-response system that processes both physical stimuli, and psychosocial stimuli simultaneously (i.e. to allow a game with physical emergent gameplay such as Thief: Deadly Shadows (2004) to have emergent social behaviour as well). In these cases nonsocially-reactive objects (like candles) may be treated as Reactors that are only listening for physical, not psychosocial stimuli. Note that in the rest of the thesis, the terms reactive psychosocial object and NPC (or character) will be used interchangeably for conciseness and readability. In the discussion of the sample and abstract stimulus-response systems, Reactors listened for particular stimulus types. Thus an unlit torch would listen for stimulus messages with a stimulus type of fire, but would not listen for stimulus types of piercing, concussion, or water. A hungry Sim would listen for stimulus types of hunger but ignore other stimuli. This listening mechanism works because these Reactors were physical objects that react only to some types of stimuli, while ignoring other types of stimuli. Social objects, however, will in general react to, or at least detect, psychosocial stimuli of all types. People tend to react to all emotions, any social situation or stressors they find themselves in, and participate in (or at least detect and make a decision in response to) social phenomena that are occurring around them. In cases where these reactions do not take place, it is because the person (or social object) has gone through some decision-making process to decide how (or how not) to react. This process will require the detection and processing of the stimulus, regardless of its type. Therefore, social objects must listen for all types of psychosocial stimuli. However, social objects

49 will typically ignore events that are not relevant to them (i.e. a discussion between two strangers). Therefore social objects can listen for psychosocial stimuli with particular relevance types. In the case where one person attacks another person (we can call them Adam and Bill), for example, the psychosocial stimulus caused by this event could be tagged as being relevant to Adam, Bill, friends of Adam, friends of Bill, enemies of Adam, and enemies of Bill. This can be compared to the way a fire arrow in the previous sample stimulus-response system could be tagged as having the stimulus types of fire and piercing. If Bill was a member of a group or faction, this relevance list may also include the group name concerned, such that other members of the group would take notice of the event (i.e. if Bill was a member of the group banditos, the relevance list would now include banditos). We may wish other events to be noticed by everyone in the area, for example a bomb going off in the middle of a town square. The psychosocial stimuli from such an event could have a relevance of all. In order for these relevances to have an effect, social objects must be listening to events that are relevant to them. Every social object should be listening for messages with a relevance to all, and all members of the banditos should be listening for messages with a relevance to banditos, et cetera. This method of tagging group relevances of an event can also be used to fix unexpected behaviours on the part of the system: for example, if Terry witnessed the fight between Adam and Bill and did not know either of them, our system as is would not notify him of this event. However, if Terrys traits, values, and worldview suggested that he was a paladin, we may expect him to try to break up the fight. In this case we can fix this odd discrepancy in his behaviour by tagging him as belonging to the group paladins (or alternatively, meddlers), and tagging the event as being of interest to paladins (or meddlers). Terry would then notice the event and could choose to attempt to break up the fight. By tagging the event as being of interest to a more broad group of meddlers instead of the rather exclusive group paladins, we could then make the event of interest to both Terry the Paladin, as well as to Nighthag the Hellcat (who may then chose a very un-paladin-like response).

50 By defining a psychosocial mechanism in the way described in this section, NPCs and other social objects can react to psychosocial stimuli in realistic as well as unexpected and novel ways to their environment. This can result in chain reactions as reactions to psychosocial stimuli create behaviours or events that can in turn cause more psychosocial stimuli that can be reacted to. This would achieve the goal of having emergent social behaviour in NPCs. By allowing the game player to participate in this mechanism through their actions in the game world, this will enable meaningful decision-making and emergent gameplay.

3.1.3

Mapping Stimuli to Actions in an NPC Psychology

Each NPC has a decision-making process to map input from the environment to an output behaviour. The designer should have the ability to define how NPCs should react to things in the environment. This should be done in a way that will not necessitate explicitly stating every individual NPCs behaviours separately. To do this, a common set of behaviours should be defined. In order to define this common set of behaviours such that it can be applied over several different NPCs without making specific changes for each NPC, these behaviours can be defined by social role in relation to the NPC in question. For example, Adam may be Barts best friend, but Corys worst enemy. Bart and Cory should therefore treat Adam differently from each other, so defining common behaviours towards the psychosocial object named Adam in response to an event would not result in accurate behaviour. Instead we can define behaviours by social roles, so that behaviours are linked to these roles instead of specific psychosocial objects. This way, when Bart experiences an event x that causes him to seek his best friend, he will seek Adam, but if Cory experiences the event x, he will seek his best friend, instead of sharing Barts behaviour. These social roles may include terms like friend, pet, home, or possession. It has already been discussed how social ties between psychosocial objects can include strength and polarity of the relationship. These same ties can be tagged with these social roles. Note that a single social tie could be tagged with

51 several social roles. For example, if Corys wife Debbie is also his best friend, the social tie between them should be tagged with the two social roles spouse and best friend. By using social roles to define behaviours, a common set of behaviours can be used by all NPCs to react in the game world. When a stimulus from the environment is sensed by an NPC, it will refer to a mapping set in data by the game designer. This mapping will associate an environmental stimulus to a set of appropriate general behaviours such as those discussed above. Another designer-defined mapping will associate these behaviours to the traits that influence them. For example, the trait of extraversion will be positively correlated with behaviours involving seeking or interacting with other people. Introversion will be negatively correlated with those behaviours, such that an NPC with high introversion should be less likely to select those behaviours. This mapping can be used in decisionmaking by taking the set of appropriate behaviours from the stimulus-behaviour set mapping, and choosing from those the most likely behaviour or behaviours to be committed given the NPCs traits and the trait-behaviour mapping. Section 3.1 discussed the high-level concepts behind the emergent psychosocial framework. The next section will discuss the software design of this framework.

3.2

Software Design
The mechanism explained in the previous sections can be implemented by

defining a set of behaviours or events that can be reacted to, a set of stimuli that the events create, and a set of social objects that can react to the stimuli. Objects can detect events around them by using a simple event listener, such as the Observer design. Note that events are passed to the NPCs from the game engine, and actions of the NPCs would be sent to the game engine where they would be processed and executed (Figure 4). Therefore the NPC logic need not be concerned with how its actions are carried out in the game world.

52

Figure 4 - Event/Action flow between the game logic and the character.

3.2.1

Listening to the World Around You: The Observer Design Pattern

In order to have NPCs perceive the world around them, a simple event listener can be implemented within the NPC to detect events that have been broadcast in the game world. A well-known example of just such an event listener comes in the form of the Observer design pattern (Gamma, Helm, Johnson, & Vlissides, 1995) (Figure 5).

53

Figure 5 - The structure of the Observer design pattern (Gamma, Helm, Johnson, & Vlissides, 1995). The Observer pattern works by having a subject notify observers of its state changes (Gamma, Helm, Johnson, & Vlissides, 1995). Observers must subscribe to a subjects event notifications to get these updates. When a subject changes its state, it publishes these changes to all of its subscribers. Note that this is essentially a publish/subscribe architecture. By using this mechanism, social objects and events in the emergent psychosocial system can notify other social objects of the stimuli they are causing. The observers of those stimuli can then decide whether or not to act on that information. The subjects would be social objects and events (Actors). Observers would be all the social objects in the world that react to the stimuli (Reactors). Observers would subscribe to subject updates when entering within a certain radius, or physical proximity, of the subject. This area can be defined to be larger to have social objects be more observant of the environment around them, or smaller to conserve the computation resources necessary for this message passing.

3.2.2

Architecture of a Psychosocial NPC

The architecture of a psychosocial NPC involves the input of a stimulus and the output of a behaviour. Some stimuli may be filtered by relevance and thus not cause any

54 resulting output behaviour.

Figure 6 - Architecture of an emergent psychosocial NPC. In order to make an appropriate response, the stimulus information must pass through a Relevance Filter, a Stimulus Dispatcher, and a Decision-Maker (see Figure 6). These processes will use the character state as needed. The following sections describe the processes in detail.

3.2.2.1 The Relevance Filter

When the stimulus is perceived by the NPC, the Relevance Filter processes it. The Relevance Filter uses the stimulus attributes to determine whether or not the NPC should process this stimulus. The Relevance Filter can have several stages to determine what is relevant. The stages described in this thesis include a group membership filter, a propagation method check, and a magnitude calculation of the stimulus affect (see Figure 7).

55

Figure 7 - Detailed view of the relevance filter.

The group membership filter consults the stimulus relevance field and compares this against the groups that the NPC belongs to. The NPC Character State holds the NPCs group membership information. If the stimulus is not relevant to any of the groups that the NPC is a member of, then the stimulus is ignored and will not be processed by any of the other processes in the NPC architecture, otherwise it will be passed on to the next stage. Note that group memberships can include cultures or subcultures (i.e. I am a biker), their relationships to other people (i.e. I am a friend of Jim), or any other type of information that may be relevant to the psychosocial mesages they should pay attention to (i.e. I am allergic to cats). The propagation method check simply checks to see if the NPC is within sensing distance of the stimulus. For example, if the stimulus propagation method is by contact (the NPC can only sense the stimulus if they are within touching distance of the source of the stimulus), then this check will determine if the NPC is within that distance and if it is not, the stimulus will be ignored. If the NPC is affected by the propagation method, the stimulus will proceed to the next stage. The magnitude calculation of the stimulus affect is only needed if the propagation method is by radius and has a fall off. For example, if the affect of the stimulus decreased by a factor of two for every foot in distance between the source of the stimulus and the sensing NPC, then the magnitude calculation would determine how far the NPC is from

56 the stimulus source, what the original stimulus magnitude was (from the stimulus message), and then use this data to calculate how much of the stimulus affect remains for the NPC to sense. Note that in the event that several types of stimulus are being processed (i.e. if physical, physiological, and psychosocial were being considered together), the Relevance Filter can have additional checks to process these stimulus types in order of priority. For example, someone who is starving may not pay as much attention to psychosocial stimuli, but will instead pay attention to messages that are relevant to getting food. This priority given to different messages would be dependent on the state of the NPC. Once the Relevance Filter determines that a stimulus is in fact relevant to the NPC and what its remaining affect is, the stimulus information and affect is passed on to the Stimulus Dispatcher.

3.2.2.2 The Stimulus Dispatcher

The Stimulus Dispatcher is responsible for determining the type of psychosocial stimulus (i.e. whether the stimulus is hostile, friendly, anger-inducing, depressing,...), and determining what set of possible behavioural reactions and mood modifications would be appropriate in response to this type of stimulus. The designer would set this information in data as described in Section 3.1.3. For example, the designer may decide that a set of possible behavioural reactions to a joy-inducing stimulus includes smiling, hugging, dancing, skipping, and singing, while the set of possible mood modifications to the same stimulus include increasing overall joy and agreeableness levels, and decreasing overall anger and sadness levels. This set of possible responses will then be passed on to the Decision-Maker.

3.2.2.3 Decision-Maker

The Decision-Maker will consult the character state in order to determine how the

57 character should react, given the appropriate set of possible behavioural reactions and mood modifications in response to the given stimulus type. The character state includes information about the NPCs personality, current mood, social ties, worldview, values, group memberships, social influences, and situational context. The Decision-Maker must decide first whether or not the character will react to the psychosocial stimulus, given its character state, and second what its reaction will be given its character state. This reaction will be in the form of mood modifications and/or output behaviours. This will be handled in the form of a trait to behaviour mapping as described in Section 3.1.3. Note that these associations can be expressed as a correlation value in between -1.0 to 1.0 to indicate what reactions are most strongly associated with which traits.

3.2.2.4 Character State

The character state refers to the set of attributes that describe the character as a psychological and social being. This can include information about internal states (traits and moods), social ties, group memberships, and social influences (as described in Section 3.1.1). Note that depending on the complexity of the social being these attributes may be set to null (for example, we may not expect a chicken to have group memberships and social influences, even though it is a social being with responses to stimuli; though it might be questionable whether it could detect some types of psychosocial stimuli, such as the psychosocial stimulus caused by one person being rude to another person). See Figure 8 for a sample class diagram of an NPC psychology.

58

Figure 8 - Class diagram of an NPC psychology. For details on the design of the SocialTie class, see Figure 9. Both personality traits and emotional traits can be defined as being data structures consisting of an identifying name (i.e. joy, agreeableness), and a degree. Not shown in Figure 8 are depictions of traits such as moral values, beliefs, and worldview, however all of these traits can also be modelled with an identifying name and degree. These traits can be abstracted into a Trait superclass with subclasses such as PersonalityTrait and Emotion classes. A container for each of these object types can be used as well, such as a Personality class which holds multiple PersonalityTrait objects, and an EmotionalRange class to hold the Emotion objects. Note that emotions can be represented as either categorical or dimensional (Tomlinson & Blumberg, 2002a; see Section 2.2.2.1.1). This behaviour can be defined through the attributes of the EmotionalRange class. These container objects could also be abstracted to a more general container class, or InternalState class. To enable the flexibility for game developers to change trait definitions, trait information should be read in from data. Social ties can be either symmetric or asymmetric. Symmetric social ties (see Figure 9a) would have a pointer to each social object involved in the relationship, and a

59 polarity and strength of the relation (a high negative number indicating a very hostile relationship for example, a high positive number indicating a very friendly relationship, with 0 indicating a neutral relationship). The social tie can also denote the type of social roles held by the social objects in the relationship. For example, two NPCs in a social tie can be labelled friends, enemies, or neighbours. In this setup, there would be either zero or one social tie per pair of social objects (zero if the social objects had no relationship to one another). Asymmetric social ties could be represented as described above, with modifications. A simple modification would be to allow zero, one, or two social ties per pair of social objects, with no other change in the social tie data structure (see Figure 9b). Another modification would be to continue to allow only zero or one social tie between two social objects, but for each social tie to have two strength and polarity representations, representing the feelings of each individual in the relationship (see Figure 9c).

60

Figure 9 - Three different representations of social ties.

Group memberships can be represented as a list of groups that the NPC belongs to. A social influence modifier could influence a function to modify the influence of internal states. For example, as the number of other NPCs in the radius of our NPC increases, the current NPCs behaviours may change, for example the NPC may become less dominant (this would be similar to what occurs in groupthink or peer pressure). A simple prototype of this design was implemented as a proof of concept. The next chapter will describe the data structures and mechanisms used in this prototype.

61

Prototype Implementation

A prototype was made as a proof of concept in the development of this work, implementing the basics behind the emergent psychosocial framework. Microsoft Visual Studio 6 was used to develop the prototype on the Microsoft Windows XP platform. To promote portability of the code, the prototype was developed in C++ using OpenGL to render a simple graphical representation of social objects, the social ties between them, and their moods. A limited character state was implemented including a categorical representation of emotion, a simple personality model, symmetric social ties, and a small behaviour/stimulus set. These elements provide sufficient proof of concept for initial validation and testing. Other aspects of character state such as representations of values, worldview, social influence, and group membership were left for future work. While it was previously highlighted that traits could be read in from data to enable flexibility for the game designer, the prototype did not use a data-driven design, but instead had the traits hard-coded to simplify initial development. Psychosocial stimuli and the events causing them were implemented in the form of message passing from one Non Player Character (NPC) to another. These interactions started when NPCs sharing a social tie (amiable or antagonistic) entered within a certain radius of each other. Section 4.1 will describe the data structures used in the prototype. Section 4.2 will describe the underlying processes as outlined in Section 3.2.

4.1

Data Structures
Several data structures are used in this prototype. An NPC data structure (called

Npc) holds all the attributes of an NPC, including their emotions, personality, and social ties. Data structures are present to hold all applicable emotions (EmotionalRange and Emotion classes), personality (Personality and PersonalityTrait classes), and social ties (held as a vector of SocialTie objects). Refer to Figure 10 for a class diagram of this architecture.

62 Npc : This data structure holds an EmotionalRange, Personality, and a vector of SocialTies.

EmotionalRange : This class holds one Emotion object for each emotion the NPC can experience. In this prototype the emotions of anger, sadness, joy, fear, and disgust are used. The EmotionalRange class also acts as a wrapper for the Emotion objects, in the event of emotional changes to the NPC.

Figure 10 - Class diagram of the NPC psychology as used in the prototype. Emotion : These objects consist of a string to indicate an identifying name of the emotion, and a float representing the degree of the emotion ranging in value from 0 (indicating that the emotion is not felt at all) to 1.0 (that emotion is felt very strongly). The most dominant emotion is reflected in the colour of the NPC (see Figure 11). If no emotions are felt strongly, the NPC appears black. The NPC

63 appears red if the dominant emotion is anger, blue if it is sadness, yellow if the dominant emotion is joy, green if disgust, and orange if fear. The degree to which the dominant emotion is felt is also be reflected in the colour. If the NPC is feeling the emotion very strongly, the colour appears very saturated. If the NPC is not feeling the emotion strongly, the colour appears less saturated.

Personality : This class holds PersonalityTraits for the NPC, and acts as a wrapper for changes to the PersonalityTraits.

PersonalityTrait : These objects consist of a string to indicate the name of the personality trait, and a float representing the degree of the trait possessed by the NPC ranging from -1.0 (indicating a much less than average degree of the trait) to 1.0 (indicating a much higher than average degree of the trait). For example, a value of -1.0 in the agreeableness trait would indicate that the person is very disagreeable, perhaps on the point of being hostile, while a value of 0.0 would indicate that the individual is of average agreeableness, and a value of 1.0 would indicate that the person is much more agreeable than average. In the prototype the personality traits of agreeableness and dominance are used.

SocialTie : Social ties are modelled in the SocialTie class by having a pointer to each NPC in the relationship and having a float strength value ranging from -1.0 (strong dislike of each other) to 1.0 (strong liking of each other). This is a symmetrical representation of a social tie. Labelling SocialTies with social roles has been left to future work. Npcs hold their SocialTies in a vector. When new SocialTies are created, they are added to the end of the vector. Queries are done as a linear search. SocialTies between two Npcs are displayed graphically as lines between them (see Figure 11). If the SocialTie is amiable, the line is displayed in green. If the SocialTie is antagonistic, the line is displayed in red. If the SocialTie is neutral, it appears black.

64

Figure 11 - Screenshot of the prototype running a scenario with three NPCs who know and like each other. Positive social ties are shown in green. Negative social ties would be shown in red (not displayed here). One character is happy (displayed in yellow), one is sad (in blue), and one is slightly angry (in dark red). Several buttons allow control of the run, including pause, step (which plays out the run for a set period and then pauses), and play (to begin the run, as well as toggling off the step and pause features).

4.2

Processes
The prototype processes follow the general flow of the design as described in

Chapter 3. In order for stimuli to be broadcast and detected in the environment, message passing is used in the functions sendMsg() and recvMsg(). These functions take a destination Npc pointer and a stimulus as parameters. Four basic actions/stimulus types are provided, being labelled hug,greet, snub, and attack. These stimuli are very basic and were chosen for their value in acting as a proof of concept, not in providing a

65 comprehensive stimulus set. The stimulus types should be more general than this when used in a game. In the prototype the sendMsg() function is called directly when NPCs met, but in a broadcasting situation would be called from a notify function in the Npc class as described in the Observer design pattern described in Section 3.2.1. This would allow all NPCs in the area to know of the events occurring. The recvMsg() function could likewise be called from an Npc update function as described in Section 3.2.1. In the prototype NPCs are added to each others subscriber list when entering within each others immediate radius. This list is held as a linked list of Npc pointers to each subscriber. In the psychosocial NPC architecture described in Section 3.2.2 the first process when stimuli was detected by the character was the Relevance Filter. This included filters for group membership, propagation method, and a magnitude calculation based on a fall off radius. In the prototype group memberships are not implemented, and there is no fall off of the affect with distance. This leaves the propagation method, which is handled only by radius. Essentially, when NPCs enter within each others radius, they are able to sense each other and thus send stimuli to each other. The second process in the NPC architecture was the Stimulus Dispatcher. This is handled in the prototype recvMsg() function that checks what the message (stimulus) type is and dispatches it to the appropriate handler function. The Decision-Maker is embodied in the handler functions for each message type. At this point it consults the relevant personality traits and moods (note that not all traits will need to be checked for every stimulus type) and picks an appropriate output behaviour or modification to mood. In the prototype, output behaviours take the form of messages being sent back to the source, and mood modifications are just increases or decreases in the NPCs various moods. Note that a change in the dominant mood results in a change of the displayed NPCs colour. These decisions are hard-coded as the prototype was meant only to evaluate whether such a system could work in practice. The output behaviour or mood modification is dependent on the personality traits and mood of the NPC. For example, consider a situation in which an NPC named Alexis

66 was sent an attack message from another NPC named George. If Alexis was less dominant than George, his fear would increase, otherwise his anger would increase. If Alexis was highly agreeable, this attack from George would also cause an increase in Alexis sadness. Alexis resulting dominant mood would then be used to pick an appropriate action towards George. For example, if Alexis dominant mood was anger, and his traits suggested he was disagreeable, he may send an attack to George in response to his mood and traits.

67

Evaluation

The goal of the framework design was to come up with a unified framework for realistic psychosocial Non Player Character (NPC) behaviour in video games. To determine if this goal has been met we must evaluate whether the system allows characters to have realistic psychosocial behaviour. Other goals included the ability to support a variety of undefined circumstances that may occur in the game world, and ease of use, time, and effort required from game developers. It must also be determined whether a system from this framework could run in a realtime interactive video game, by determining whether resource usage is reasonable for such an application. Section 5.1 will discuss a few social scenarios that were run with the emergent psychosocial system prototype to determine whether social and emotional behaviour was realistically modelled. Section 5.2 will evaluate the efficiency and resource usage of the prototype. Section 5.3 discusses future work including evaluating developer and player experience with the system.

5.1

Scenarios to Examine Realistic Social Behaviour

In order to gain an impression of whether realistic emotional and social behaviour

occurred in the prototype, several cases were run with different psychosocial scenarios to observe any emergent behaviour. The personality traits used were agreeableness and dominance, as described in Isbister (2006). The emotions used were anger, sadness, joy, fear, and disgust, based on Ekmans universal emotions (Workman, Loftus, Weaver, & Atkinson, 2000). NPCs had the ability to detect and react to other NPCs by greeting, hugging, snubbing, or attacking. These actions acted as the psychosocial stimuli. Stimuli were reacted to based on the emotions and personality traits of the reacting NPC with those same actions. The scenarios in the following sections included having a single angry individual dissenting in a group of happy individuals (in Section 5.1.1), a single sad individual in a

68 group of supportive individuals as well as having one happy individual in a group of unhappy individuals (5.1.2), and having individuals in two groups who disliked each other (5.1.3). The following sections will describe these scenarios in more detail and the resulting behaviour from the test runs.

5.1.1

Shannara and the Warlock Lord

The first scenario was based on determining what behaviours would result if a single angry individual was in a group of joyful individuals. To test this, a scenario was setup in which an individual, was set to be very angry, very dominant and disagreeable, and had a strong dislike for all of the other individuals in the world. The other individuals in the world were all set to have joy set to 1.0, and all other emotions set to 0.0. These individuals had varying personalities, and varying connection strengths between each other, but were, for the most part, on positive terms with one another. See Figure 12 for a general layout and description of the individuals and social ties in this scenario.

69

Figure 12 - The general layout of the individuals and social ties in the scenario from Section 5.1.1. Bronas personality has been set to have very high disagreeableness and dominance. Bronas emotions have been set with very high anger and disgust, and very low joy, fear, or sadness. All other individuals have varying personalities and high levels of joy, and low levels of other emotions. Social ties are random within the happy group, and are on the most part friendly (with the exception of Flick and Menions weakly antagonistic connection). Social ties between Brona and everyone else are strongly antagonistic.

During the run, Brona remained angry regardless of who was around him, and would tend to attack anyone who was around. NPCs who were agreeable and less dominant than Brona (i.e. Shea, Flick, and Hendel) were noted to become sad when around Brona. Individuals who were not agreeable and less dominant than Brona (i.e. Keltset) became fearful when around Brona. Those just as dominant as Brona became angry when around him (i.e. Alannon and Panamon). Note that when two NPCs met who shared a friendly social tie, they would become happy again. Typically attacks were

70 initiated by Brona, however characters with dominance levels comparable to Brona would attack him. Those with lower dominance levels tended to snub Brona in response to his anger. An interesting observation that was not expected was that if an individual came into proximity with two other individuals who were disliked, and one of those individuals had a higher dominance, the individual became fearful. This was noted when Flick and Menion (who have a weakly antagonistic relationship) came into proximity of each other and Brona (who has a strongly antagonistic relationship with both of them). Both Flick and Menion became fearful in this case. They would only become fearful if both were present with Brona, otherwise would become sad if around Brona alone. If Menion and Flick were together without Brona around, each would remain happy and would greet each other (not having an antagonistic enough relationship for it to affect their otherwise good mood). Some of these behaviours would change depending on who else was around them. See Figure 13 for a screenshot and description of the prototype running this scenario.

71

Figure 13 - A screenshot of the prototype having run Scenario 5.1.1 for a short period of time. Red lines denote antagonistic relationships and green lines denote friendly relationships. Individuals are displayed as triangles of the colour denoting their primary emotion (yellow is joy, red is angry, blue is sad, orange is fearful, and green is disgust). Saturation of emotion colours shows how strongly that emotion is felt. Note the emotions of the individuals seen from top to bottom, left to right: Menion is happy, Shea is indifferent, Hendel is indifferent, Brona is angry, Alannon is happy, Flick is sad, Keltset is fearful, Panamon is happy, and Balinor is sad. 5.1.2 A Friend in Need, and a Friend in Deed

This scenario was run twice with a slight modification to the emotions. The individuals and social ties were as described in the scenario from Section 5.1.1, excluding the individual Brona. In the first scenario in this section, all the NPCs had joy set to very high except for Shea, who was set to have very high sadness. The second scenario had

72 only Shea feeling joy, while everyone else was experiencing various random (non-happy) emotions. In the first case, Shea was quickly cheered when he entered the proximity of any other character. Other characters would greet or hug him depending on their personalities, which would result in an increase in Sheas joy. Every character was then happy, and no change in emotions occurred after this steady state was reached.

Figure 14 - The initial emotions at the beginning of running the prototype with the second scenario in Section 5.1.2. Note the emotions of the characters from top to bottom, left to right: Panamon is angry, Shea is happy, Balinor is fearful, Hendel is disgusted, Flick is sad, Keltset is angry, Menion is angry, and Alannon is fearful. In the second scenario, joy was gradually spread from Shea through to the other characters. Figure 14 shows the initial emotions for all the characters at the start of running the prototype with this scenario. Figure 15 shows how Sheas happiness has

73 begun to spread to the other characters a few moments later.

Figure 15 - The same scenario a few moments later. Note that joy has spread from Shea to Panamon, Balinor, Hendel, and Flick. The characters furthest from Shea have not yet been affected.

5.1.3

A Day at Hogwarts

Finally the last scenario run was based on what might happen when two distinct groups were in conflict with each other. While groups and group memberships were not implemented in this prototype, the social ties were setup such that the individuals had positive connections only with certain individuals in the world, and negative connections with other individuals in the world (who likewise had positive connections within their group, and negative connections with others).

74

Figure 16 - The general grouping of the individuals in Scenario 5.1.3. Note that all members of the group Slytherin have strongly antagonistic connections to Harry, Ron, and Hermione, have weakly antagonistic connections to Neville, and have no social connections to Seamus. All members of the Gryffindors have moderate to strong friendly connections to each other, and all members of the Slytherins have moderate to strong friendly connections to each other.

The NPCs in this case were named Harry, Ron, Hermione, Neville, and Seamus (belonging to the hypothetical group Gryffindor), and Malfoy, Crabbe, and Goyle (belonging to the group Slytherin). See Figure 16 for details about the connections. Individuals had varying personalities defined on the agreeableness and dominance dimensions. Each individual also had a set of random starting emotions defined.

75

Figure 17 - A screenshot of the prototype having run Scenario 5.1.3 for a short period of time. Note that NPCs who are in the presence of those they like appear happy, and those who are around those they dislike are either sad or angry. Those who are angry have high dominance levels, and those who are sad have low dominance levels. The emotional states of the individuals, from top to bottom and left to right are: Seamus is happy, Harry is happy, Malfoy is angry, Neville is sad, Goyle is happy, Crabbe is angry, Hermione is happy, as is Ron. When this scenario was run, NPCs became happy when in the presence of friends. When around enemies, the NPCs would become either angry or disgusted (if they were as dominant or more dominant than those around them), or sad (if they were less dominant and agreeable). These responses were due to attacks or snubs, and were modulated by the receiving NPCs personality type and current mood. Fear was also observed in a less dominant (and disagreeable) character when attacked by a more dominant character. Note that some of these behaviours were affected by stimuli present only when those

76 surrounding them felt certain emotions. For example, Hermione felt disgust after Goyle sent a snub, but Goyle would send a snub only when he was not angry. When he was angry he would send an attack, to which Hermione would respond with anger, rather than disgust. Hermiones emotional and behavioural response to Goyles presence depended on his actions, which were influenced by his emotions. See Figure 17 for a screenshot of the prototype being run with this scenario.

5.1.4

Summary of Scenario Findings

The scenarios run showed that many unexpected yet appropriate behaviours emerged from the social systems. Tweaking may have been desirable in some cases where responses were not ideal (i.e. some NPCs may have been expected to feel fear instead of sadness in response to attacks), but the flexibility of the framework (being datadriven) would make tweaking of inappropriate behavioural responses relatively easy. Another observation is that the spread of emotions occurs quickly, and in some cases may reach a steady state (such as when everyone is feeling joy). Due to these occurrences, it may be desirable to limit the amount of emotional change that can occur within a short period of time when the system is used in a game. It would also be necessary to avoid steady states such as the one described above, perhaps by adding in random mood modifications, or by making people react to the steady emotion with another emotion. For example, in the first scenario from Section 5.1.2, once utopian joy was reached by all NPCs, every NPC reacted with more joy, creating an infinite positive feedback loop. By placing more antagonists in the world who would react to this joy (or to the presence of their own antagonists) with either disgust or anger, this steady state will never be reached. The observations from these scenarios show that this framework is capable of producing realistic and emergent psychosocial behaviour. The next section will determine whether this system can run efficiently.

77

5.2

Efficiency and Resource Usage

To determine whether the system could run efficiently in a realtime application,

Microsoft Spy++ was used to evaluate the resource usage of the prototype. This will be discussed in the next section.

5.2.1

Methodology

Microsoft Spy++ (2002) was used to evaluate resource usage of the prototype. The measured resources include CPU time, virtual bytes used, peak virtual bytes used, working set, and peak working set. CPU time is the total cumulative time the CPU spent on the process and is user time combined with privileged time (which were also measured) (Microsoft Corporation, 2002a). Virtual bytes refers to the amount of virtual address space used by the process (Microsoft Corporation, 2002b). Peak virtual bytes is the maximum virtual address space used at any moment (Microsoft Corporation, 2002b). Working set is the set of memory pages touched recently by the process, and peak working set is the maximum number of pages in the working set at any moment (Microsoft Corporation, 2002b). The experimental parameters that varied in this study are the number of NPCs active in the system environment and the number of social ties between them. In order to measure the worst-case scenario, with each value for the number of NPCs, the number of social ties was set at the maximum such that every character had a social tie to every other character. These were used as experimental parameters since the system should become more resource intensive as the number of NPCs and social ties increase. Each NPC will require memory to hold its traits and social ties, and will require CPU to compute what its next action will be. No other parameters are present that would affect the resource usage. Graphics were disabled during performance monitoring to control for resource usage due to graphical demands. In a game with such social ties, every NPC and social tie should not be active at all

78 moments in time. Modern video games typically restrict resource usage by processing only objects and events that appear on-screen. Some games have the game world divided into areas, and anything that is not within the area occupied by the game players avatar will not be processed. For this reason every NPC and social tie in the game will not have to be active at all moments in time. Furthermore, it is highly unlikely (and also socially unmanageable) for there to be large numbers of individuals interacting with each other all at once. In real social situations, such groups would tend to fragment into smaller, more manageable groups. Performance and resource usage measurements for a group of interacting individuals has therefore been limited to 10 characters. It is unlikely that more than 10 characters will appear on the screen at once (assuming NPCs are the size of the average character in most graphical story-based games on market), and furthermore that all 10 characters will know each other and be engaging in social interactions with all other characters at once. Further research will explore how social groups form to interact with each other, and how the number of members in such a group is limited to remain socially maintainable. For cases in which a large number of individuals will appear on-screen, behaviour can be switched from the psychosocial system controller to a more general crowd-controller. These measurements were taken for cases in which there were zero NPCs (to get the overhead of the prototype without any characters or interactions), and for every number of NPCs up to 10. The social ties between NPCs were set to its worst case such that every NPC knew every other NPC. For each number of NPCs, the program was run 10 times with resource measurements taken. These measurements were then averaged to get the average resource usage per number of NPCs. These measurements were taken on a Pentium 4 running Windows XP Professional at 2.00 GHz with 640 MB of RAM.

5.2.2

Results

It was found that the amount of CPU time required for the system peaked at the

79 beginning of the program, and this peak never went above 0.07 seconds of CPU time. User time likewise peaked at the beginning of the program and never exceeded 0.05 seconds. Total CPU time and total user time measured after the initial peaks were so small as to be immeasurable by Microsoft Spy++. Virtual bytes and peak virtual bytes used by the psychosocial system, regardless of the number of NPCs remained at 7376 Kb and 8272 Kb respectively. The working set size never exceeded 960 Kb, and the peak working set never exceeded 980 Kb. No significant increase in these values with the increase in the number of NPCs was detected by Microsoft Spy++.

5.2.3

Analysis of Results

Results of the measurements of resource usage were very positive. The prototype runs efficiently for small group sizes with immeasurable increase in CPU usage for each NPC added. Any increase in CPU usage that did occur for each NPC was so small that it could not be detected given the granularity of the measurement tools. Memory usage likewise did not increase significantly with the increase in number of NPCs. Note that due to the nature of the problem, as the number of NPCs and social ties grows, there will be an exponential increase in complexity if the social tie graph is fully connected. This will make large scale psychosocial emergence prohibitive. Constraints will have to be put on the operation of this system, for example by limiting the social radius of NPCs (so they can only interact with those within a certain radius of themselves), by restricting the number of people engaged in social interactions on screen, by dividing large interacting social groups into smaller disconnected subgroups, and using more general crowd-control in cases where there are many NPCs in one area. Future work will focus on exploring how many NPCs and social ties can be supported before resource usage becomes prohibitive.

80

5.3

User and Developer Experience

A rigorous user study to determine if the psychosocial behaviour and interactions

implemented are convincing and realistic is left to future work. This may involve the integration of the emergent psychosocial system with a game. Impressions from the simple prototype implemented in this work were achieved by the spread of colours through the NPCs: each colour represented a different emotion. It was observed that as NPCs came within proximity of other NPCs, their emotions did change appropriately without designer intervention or explicit interaction rules. This suggests that the system is capable of supporting a variety of circumstances that were not defined by the designer. A formal evaluation of the ease of use of the system by developers is also left to future work. In using the prototype to evaluate the design framework, some impressions as to the ease of use of the system and possible improvements can be noted. In order to define NPCs and their social systems, information was entered in data defining each NPCs attributes and starting emotions, with another file defining social ties. Social ties were entered per character, with each new field under each character defining a character identifier of the acquaintance and a connection value. Ease of NPC creation with varying personalities was therefore relatively simple. It was found that social tie creation, however, could have been implemented in a simpler fashion by reading straight from a matrix (each row and column representing each NPC), rather than with the field format used in the prototype implementation. This type of field-style entry required defining every social tie twice, once for each NPC. By reading from a matrix, or by using another format for social ties, the representation of social tie data would not necessitate duplication of entered data (particularly in the case where social ties are symmetric), making the entry of this data much easier.

81

Conclusions and Future Work

In conclusion, the preliminary work on the framework design and experimental prototype for the emergent psychosocial system is very positive. The system succeeds in achieving the goals of enabling emergent psychosocial behaviour and interactions even in the presence of undefined circumstances, and the resource usage measurement results are encouraging. The system proposed supports designer-defined emotion, personality, and social behaviour. Social concepts such as group memberships, social roles, and values are supported. The use of emergence has allowed the flexibility to react to undefined circumstances that might be created by the player, while allowing player-authored experience and Non Player Character autonomy. Future work would include implementing more traits and social concepts, such as social roles, group memberships, social influences and peer pressure, values, and/or representations of individual worldviews. Implementing the data-driven element of the design would aid in this. Dynamic social ties will be implemented by allowing the decision-maker to modify them. An area of interest is determining whether an accurate simulation could be created with this framework using more accurate models of behaviour. While this emergent psychosocial framework allows character development, it does not handle machine learning, or the assessment of the environmental response to actions. Future work in integrating these fields could be used for more accurate social simulations. The integration of this system in a fully functional game engine would be another area for future work. This would allow the measurement of performance when used in a graphical realtime interactive game including the processing of other game logic. An integration with a functional game engine would also allow user studies to determine if the character behaviour and responses are convincing and realistic, and also to determine whether this system results in a better overall gameplay experience. A user study to evaluate usability of the design framework for developers would also be useful in

82 determining the frameworks practical value in real world applications. And finally, an area that needs to be explored is determining how to realistically fragment populations into smaller groups of interacting individuals, and how to control the formation and dispersion of these groups in a realistic and realtime fashion.

83

Works Cited

Alt, G., & King, K. (2002). A dynamic reputation system based on event knowledge. In S. Rabin (Ed.), AI Game Programming Wisdom (pp. 426-435). Hingham, MA: Charles River Media.

Barnes, J., & Hutchens, J. (2002). Scripting for undefined circumstances. In S. Rabin (Ed.), AI game programming wisdom (pp. 530-540). Hingham, MA: Charles River Media.

Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs [Electronic version]. Journal of MUD Research, 1(1).

Bates, J., Loyall, A.B., & Reilly, W.S. (1994). An architecture for action, emotion, and social behavior [Electronic version]. Artificial Social Systems Selected Papers from the Fourth European Workshop on Modeling Autonomous Agents in a Multi-Agent World, MAAMAW-92. Retrieved January 2, 2007, from http://citeseer.ist.psu.edu/bates94architecture.html.

Berger, L. (2002). Scripting: overview and code generation. In S. Rabin (Ed.), AI Game Programming Wisdom (pp. 505-510). Hingham, MA: Charles River Media.

Brockington, M. (2003). Building a reputation system: hatred, forgiveness, and surrender in Neverwinter Nights. In T. Alexander (Ed.), Massively Multiplayer Game Development (pp. 454-463). Hingham, MA: Charles River Media.

Byl, P.B.D. (2004). Programming believable characters in games. Hingham, Massachusetts: Charles River Media, Inc.

84 Cesta, A., Miceli, M., & Rizzo, P. (1996). Help under risky conditions: robustness of the social attitude and system performance. Proceedings of the First International Conference on Multi--Agent Systems, 18-25.

Cole, S. (2006). Modeling opinion flow in humans using Boids algorithm and social network analysis. Retrieved January 2, 2007, from http://gamasutra.com/features/20060928/cole_01.shtml.

Crawford, C. (2005). Chris Crawford on interactive storytelling. Berkeley, CA: New Riders.

Dautenhahn, K. (2000). Socially intelligent agents and the primate social brain - towards a science of social minds. In K. Dautenhahn (Ed.), Socially Intelligent Agents: The Human in the Loop (Papers from the 2000 AAAI Fall Symposium) (pp. 35-51). North Falmouth, MA: AAAI Press.

Electronic Arts, Inc. (2005). Reputation, karma and fame. Retrieved January 2, 2007, from http://guide.uo.com/combat_5.html.

Faade. (2005). Procedural Arts. Available for download, from http://www.interactivestory.net.

Falstein, N. (2007). Game shui: do, dont do. Game Developer, 14(1), 42.

Funder, D.C. (2001). The Personality Puzzle. New York, NY: W. W. Norton & Company.

Funge, J.D. (2004). Artificial intelligence for computer games. Wellesley, MA: A K Peters.

85 Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-Oriented Software. Boston, MA: Addison-Wesley.

Game Informer. (2006a). Assassins Creed. June 2006, Issue 158. Vol XVI: No 6.

Game Informer.(2006b). Bully. September 2006, Issue 161, vol XVI: No 9.

Grand Theft Auto III. (2002). Rockstar Games, Inc. New York, NY: Take-Two Interactive Software.

Grond, G., & Hanson, B. (1998). Ultima Online reputation system FAQ. Retrieved January 2, 2007, from http://update.uo.com/repfaq.

Gruenwoldt, L., Katchabaw, M., & Danton, S. (in press). Achieving realistic reactions in modern video games. In Castell, S., & Jenson, J. (Eds.), Worlds in Play. Peter Lang Press.

Gruenwoldt, L., Katchabaw, M., & Danton, S. (2005). Creating reactive non player character artificial intelligence in modern video games. Game-On-NA 2005: 1st International North American Conference on Intelligent Games and Simulation. (pp. 10-17). Montreal: EUROSIS-ETI.

Guye-Vuilleme, A., & Thalmann, D. (2001). A high-level architecture for believable social agents. VR Journal, 5, 95-106.

Hogg, L.M.J., & Jennings, N.R. (2001). Socially intelligent reasoning for autonomous agents. IEEE Transactions on Systems, Man, and CyberneticsPart A: Systems and Humans, 31(5), 381-393.

86 Holland, J. H. (1998). Emergence: From chaos to order. Reading, MA: Addison-Wesley.

Isbister, K. (2006). Better game characters by design. Boston, MA: Morgan Kauffman.

Jennings, N.R., Sycaro, K., & Wooldridge, M. (1998). A roadmap of agent research and development. Autonomous Agents and Multi-Agent Systems, 1, 275-306.

Johnson, S. (2001). Emergence: The connected lives of ants, brains, cities, and software. New York, NY: Scribner.

Lawson, G. (2003). Stop relying on cognitive science in game design - use social science. Retrieved January 2, 2007, from http://www.gamasutra.com/phpbin/letter_display.php?letter_id=647.

LeBlanc, M. (2000). Formal design tools: Emergent complexity, emergent narrative. [Electronic version]. GDC, 2000. Retrieved August 4, 2007, from http://algorithmancy.8kindsoffun.com/gdc2000.ppt.

Magerko, B. (2006). Intelligent story direction in the interactive drama architecture. In S. Rabin (Ed.), AI Game Programming Wisdom 3 (pp. 583-596). Boston, MA: Charles River Media.

Matlin, M.W. (2002). Cognition. Toronto, ON: Thomson Learning.

McLean-Foreman, J. (2001). An interview with Will Wright. Retrieved March 12, 2007, from http://www.gamasutra.com/features/20010501/wright_01.htm.

McNaughton, M., Schaeffer, J., Szafron, D., Parker, D., & Redford, J. (2004). Code generation for AI scripting in computer role-playing games. American Association

87 for Artificial Intelligence, 129-133.

Microsoft Corporation. (2002a). Process Properties General Tab (Windows NT). In Microsoft Spy++ Help. Seattle, WA: Microsoft Corporation.

Microsoft Corporation. (2002b). Process Properties Memory Tab (Windows NT). In Microsoft Spy++ Help. Seattle, WA: Microsoft Corporation.

Microsoft Spy++. (2002). Microsoft Corporation. Seattle, WA: Microsoft Corporation.

Picard, R.W. (1995). Affective computing. Media Laboratory, Perceptual Computing TR 321, MIT Media Lab.

Prendinger, H., & Ishizuka, M. (2001). Social role awareness in animated agents. International Conference on Autonomous Agents, 270-277.

Prendinger, H., & Ishizuka, M. (2002). Evolving social relationships with animate characters. Proceedings of the AISB-02 Symposium on Animating Expressive Characters for Social Interactions, 73-78.

Reynolds, B. (2004). How AI enables designers. Retrieved January 2, 2007, from http://www.gamasutra.com/php-bin/news_index.php?story=11577.

Rizzo, P., Veloso, M., Miceli, M., & Cesta, A. (1997). Personality-driven social behavior in believable agents. Proceedings of the AAAI Fall Symposium on Socially Intelligent Agents, 8-10.

Russel, A. (2006). Opinion systems. In S. Rabin (Ed.), AI game programming wisdom 3 (pp. 531-554). Boston, MA: Charles River Media.

88 Russell, S., & Norvig, P. (2003). Aritificial intelligence: A modern approach. Upper Saddle River, NJ: Prentice Hall.

Scott, B. (2002). Architecting a game AI. In S. Rabin (Ed.), AI Game Programming Wisdom (pp. 285-289). Hingham, MA: Charles River Media.

Smith, H., & Smith, R. (2004). Practical techniques for implementing emergent gameplay: would the real emergent gameplay please stand up? [Electronic version]. GDC, 2004. Retrieved January 2, 2007, from http://www.planetdeusex.com/witchboy/GDC04_Emergence/sld001.htm.

Smith, R. (2007). RE: [Fwd: Emergent Gameplay presentation]. Email to Christine Bailey. Date emailed: 3 Aug, 2007.

Thief: Deadly Shadows. (2004). Ion Storm. San Francisco, CA: Eidos Interactive.

Tomlinson, B., & Blumberg, B. (2001). Social behavior, emotion and learning in a pack of virtual wolves. Paper presented at AAAI Fall Symposium, 2001.

Tomlinson, B., & Blumberg, B. (2002a). Using emotional memories to form synthetic social relationships. Retrieved January 2, 2007, from http://citeseer.ist.psu.edu/rd/19718822%2C537507%2C1%2C0.25%2CDownload/ http%3AqSqqSqbadger.www.media.mit.eduqSqpeopleqSqbadgerqSqPublications qSqAgents2002Final.pdf.

Tomlinson, B., & Blumberg, B. (2002b). Synthetic social relationships in animated virtual characters. Proceedings of the seventh international conference on simulation of adaptive behavior on from animals to animats, 401-402.

89 Tozour, P. (2002). The evolution of game AI. In S. Rabin (Ed.), AI game programming wisdom (pp. 3-15). Hingham, MA: Charles River Media.

Woodard, C. (2006). GDC: God of War: How the left and right brain learned to love one another. Retrieved March 31, 2006, from http://www.gamasutra.com/features/20060327/woodard_01.shtml.

Woodcock, S. (2000). Game AI: The state of the industry. Retrieved March 11, 2007, from http://www.gamasutra.com/features/20001101/woodcock_01.htm.

Wooldridge, M.J. (1992). The logical modeling of computational multiagent systems. Thesis. Retrieved January 2, 2007, from http://citeseer.ist.psu.edu/wooldridge92logical.html.

Wooton, B. (2006). Designing for emergence. In S. Rabin (Ed.), AI game programming wisdom 3 (pp. 45-54). Boston, MA: Charles River Media.

Wortman, C.B., Loftus, E.F., Weaver, C., & Atkinson, M.L. (2000). Psychology. Toronto, ON: McGraw-Hill Ryerson.

Young, R.M., Riedl, M.O., Branly, M., & Jhala, A. (2004). An architecture for integrating plan-based behavior generation with interactive game environments. Journal of Game Development, 1(1), 51-70.

90 VITA

Name:

Christine D. Bailey

Post-secondary Education and Degrees:

University of Western Ontario London, Ontario, Canada 2000-2005 B.Sc.

Honours and Awards:

Western Scholarship of Excellence 2000-2001

Aiming for the Top Tuition Scholarship 2000-2004

Nortel Networks Scholarship of Distinction 2002-2004

Faculty of Science Honour Roll 2003, 2004

NSERC Undergraduate Student Research Award (USRA) 2003, 2004, 2005

Ontario Graduate Scholarship (OGS) 2006-2007

Related Work Experience:

Teaching Assistant The University of Western Ontario 2005-2006

91 Research Assistant The University of Western Ontario 2002-2005

Publications: Bailey, C., & Katchabaw, M. (2005). An experimental testbed to enable auto-dynamic difficulty in modern video games. Proceedings of the 2005 GameOn North America Conference, 18-22.