GENETIC ARTIFICIAL INTELLIGENCE

TECHNICAL REPORT
Vivek Deshpande GPA1

CONTENTS
Technical Report Introduction Summary Understanding Genetics Implementation of evolution theory in Computer science Evolutionary Algorithms Genetic Algorithms Reproduction Mutation Fitness Function Limitations of current approach Viability of genetic algorithms Convergence towards a non-optimum solution Time-consumption Implementation of GA in game development invAIders Tower of reus STRATLOC Remodeling Genetic Algorithm My Implementation Extending the current Prototype Conclusion References 1 3 3 4 5 5 5 6 7 8 9 9 9 9 10 10 11 12 13 14 17 17 18

Vivek Deshpande GPA1 2013-2014 Page 2

INTRODUCTION

In this technical report, I will be explaining the basic concepts of genetics and genetic algorithms. I will be noting down the current implementations of genetic algorithms and the limitations they face. I will be taking a look at some games that implement evolutionary algorithms1 and the problems seen in the AI I will then be remodeling the algorithm to over-come these limitations. I will break down my implementation of this model of genetic algorithms and I will be noting down the problems I faced while developing the prototype and how I overcame them. Finally, I will explain how my technical research will benefit me and how it can be used to further improve a game.

SUMMARY

My research is on genetic algorithms and specifically, their implementation in artificial intelligence used in game development. Genetic algorithms are generally used to mimic natural selection and evolution in many fields so as to acquire useful solutions to search and optimizing problems. My interest in genetic algorithms is because of the current AI in games today. All of them are built upon rigid logic with the AI having no adaptive properties. I agree that rigid logic is sometimes useful for games where a player can only do some actions and not have the freedom to do everything that is possible in real life. However, I feel that we have come to a point in game development where immersion is seen by many developers as a USP; but instead of realistic immersion2, we only have games that give secondary immersive properties like lighting, graphics, SFX etc. My goal is to be part of a team that introduces actual immersion inside a game. But since the game in itself will be as realistic as possible, we also need the artificial intelligence to be realistic too; which means that bots will change their characteristics and are adaptive towards the players actions. The implementation of genetic algorithms is my first step towards understanding evolutionary algorithms as a whole and their benefits to a game developer in designing the perfect artificial intelligence.

Evolutionary algorithm is a parent category which includes all algorithms that are based on natural occurrences like genetic algorithms, neural networks etc. 2 Realistic immersion for me is being able to do any action possible by a human being if he were in the same situation.
1

Vivek Deshpande GPA1 2013-2014 Page 3

UNDERSTANDING GENETICS

Understanding the basic concepts of genetics and evolution theory is extremely important to understand genetic algorithms. I will be explaining them in brief below. Genetics is a field of study in biology which explains the concepts of genomes and inheritance through natural selection. It is the process of inheritance of personal traits from one generation to next by recombination of DNA. Each cell in a living organism consists of a number of proteins, carbohydrates and a specific type of protein called chromosomes. A chromosome is an organized structure of DNA and RNA proteins. According to Wikipedia3, DNA (Deoxyribonucleic acid) is a molecule that encodes the genetic instructions used in the development and functioning of all known living organisms and many viruses. DNA is a nucleic acid; alongside proteins and carbohydrates, nucleic acids compose the three major macromolecules essential for all known forms of life. A gene is a sequence of DNA that is required of each cell is a body to perform a certain function. Hence, each gene decides the size, shape and function of each cell. So, in laymans terms, every living beings traits on the planet are defined by the combination of these DNA molecules. The structure of these molecules remains constant; however because of inheritance of chromosomes from parents to offspring, the behavior of each gene will emulate either of its ancestors traits. If traits are only being passed down through each generation, there will not be any actual advance in a species ability to survive. Mutation is the process where genes mutate in a child and because of which it introduces new characteristics in the pool which can be harmful or beneficial to the communitys survival. The ultimate test of good genes in a species is the predator. Evolutionary theory states that a predator is another species that may or may not kill their prey prior to feeding on them, but the act of predation often results in the death of its prey and the eventual absorption of the prey's tissue through consumption. This means that for a species to survive, it needs to defeat its predator through some action; it can be elusive, stealthy or be powerful enough to drive off the predator. Hence, in every generation, a certain set of individuals are able to not be preyed upon. These individuals carry the necessary characteristics so that its next generations survival is ensured. The individuals that fall to the predator had genes that were bad for the communitys survival.

Literal Definition : http://en.wikipedia.org/wiki/DNA

Vivek Deshpande GPA1 2013-2014 Page 4

IMPLEMENTATION OF EVOLUTION THEORY IN COMPUTER SCIENCE

EVOLUTIONARY ALGORITHMS
Evolutionary algorithms are inspired by evolution and uses biological mechanisms like reproduction, mutation, re-population and selection to find optimal solutions to a problem. Genetic algorithms are a subset of evolutionary algorithms that follow a generic guideline regarding the structure of the algorithm. Evolutionary algorithms also contain evolution strategy and neuro-evolution techniques. All of these algorithms are widely used in the world today with genetic algorithm being most popular simply because of its flexibility in structure.

GENETIC ALGORITHMS
All genetic algorithms are either derived from or explicitly follow the following model.

Vivek Deshpande GPA1 2013-2014 Page 5

In the initial setup phase, we begin by identifying a problem. The problem is re-interpreted so that it is possible to develop a programmable solution for it. This solution is a set of behaviors or traits which in the right circumstances is able to fix the problem. Generally, each solution has a finite number of traits; however, some implementations can have an infinite number of traits. The initial population is randomly generated and then fed into the program. The most common method of defining a solutions traits is to represent each trait in a bit-wise format. For the purpose of this report, let us have a look at a simple problem. Using only the numbers 4, 5,6,7,8 and 9 and operators + and -, find a sequence that will equate to 25. Now this problem already defines its traits for us. They are 4, 5, 6, 7, 8, 9, + and -, so in total, we have 8 traits. If we are to represent these traits in a bit-wise format, we can do so using three digit strings.

STRING 000 001 010 011 100 101 110 111

TRAIT Number 4 Number 5 Number 6 Number 7 Number 8 Number 9 Operator + Operator -

A quick solution that we can think of ourselves would be 4+5+9+7. Using the encoding table above, our solution would be represented by the string 000110001110101110011. Depending upon the complexity of the problem, we can either randomly generate a population with a fixed number of strings or have it completely random by generating any number of strings. Note that in this case it is possible to have nonsensical strings containing too many operators or too many numbers. To avoid this, we add a filter after the random generation which eliminates such type of solutions.

REPRODUCTION

After we have our initial population, we start the reproduction process. In the general model, we select any two parent solutions; apply a cross-over technique to generate a child solution. There are multiple crossover techniques which are all viable, but the uniform cross-over and zipper cross-over techniques are the most popular.

Vivek Deshpande GPA1 2013-2014 Page 6

From Parent A

From Parent B

The general model of genetic algorithms will not mix bits between separate strings. This way the developers can ensure that the traits of the parents will be consistent and will appear again in the child solution. However, because of the flexible nature of genetic algorithms, if required we can have cross-over between two strings.

MUTATION
A mutation function in a genetic algorithm will change the bits of a child solution (after cross-over). The rate at which child population mutates can be controlled by the engineer by setting a threshold value which triggers once there are a certain number of un-successful generations. It can also be designed to introduce mutation randomly based of a specific percentage. This method is most popular as it we ensure at least a portion of the child population undergo mutation and hence have some unique traits to them.

Before Mutation

After Mutation

Vivek Deshpande GPA1 2013-2014 Page 7

FITNESS FUNCTION
Once the entire child population is generated, a fitness function is run on each individual. A fitness function determines the fitness score of each child solution by running each solution into the problem. It then checks how much of the problem was the solution able to solve and assigns a score accordingly. A fitness function in a genetic algorithm removes the unfit individuals from reach the reproduction stage. This ensures that we obtain the best genes possible going forward. For our example stated earlier since the problem is simple, our fitness function can also be simple. I believe a fitness score which is equal to the absolute value of the difference between the target number and the solution number would be ideal. This way a solution with the fitness score 0 would be our ideal solution. The fitness function is the most important process in a genetic algorithm. If the fitness function is poorly designed, the algorithm would be sub-optimal as either every generation would not come close to the perfect solution or possible solutions would get removed by the fitness function. For complex problems, a fitness function may be an entire project.

Good Fitness Score Next Gen Parent

Good Fitness Score Next Gen Parent

Bad Fitness Score - Discarded

Bad Fitness Score - Discarded

Vivek Deshpande GPA1 2013-2014 Page 8

LIMITATIONS OF CURRENT APPROACH

Even though the genetic algorithm is very flexible, it still has some limitations. These limitations are the reason that other evolutionary algorithms like neural networks are preferred over genetic algorithms.

VIABILITY OF GENETIC ALGORITHMS

Genetic algorithms are generally computationally expensive. A fitness function for a complex, multidimensional problem takes generally hours of high-end processing time. Sometimes, it is not viable to use genetic algorithms for such problems as the cost of such a project overbears on the efficient solution that it provides.

CONVERGENCE TOWARDS A NON-OPTIMUM SOLUTION

Fitness functions are designed according to the problem that we are facing. However, since most problems are much more complex than our example, they are broken down in sub-problems and GA (genetic algorithm) is applied to all of them. However, this way we sometimes lose focus on the big-picture and design fitness functions which will give us an optimum solution to a sub-problem. But once this solution is integrated into the bigger parent, we realize that it wasnt that optimal at all. Because of this, fitness functions need to be precise and also need to consider the entire problem.

TIME-CONSUMPTION
As mentioned before, fitness functions on huge problems are computationally expensive. To check the fitness score for an entire generation, it takes an abnormally large amount of processing time. Also, because of incorrect logic in the mutation, cross-over or fitness functions, it is entirely possible to scrap days of work and start from beginning. Sometimes, the fitness function in itself wastes time as developers redo it because of bad initial design.

Vivek Deshpande GPA1 2013-2014 Page 9

IMPLEMENTATION OF GA IN GAME DEVELOPMENT

In the current video game market, there are not a lot of games that actually deploy genetic or evolutionary algorithms to create a better AI Most work in this field still falls under the score of R&D. Unfortunately, until someone designs the perfect algorithm that can adapt the AI to any circumstances, genetic algorithms are going to remain obscured. Here are some of the projects that use genetic algorithms for AI

INVAIDERS
InvAIders was designed by Michael Martin as a genetic AI test. His goal was the same as mine, which is to design an AI that can adapt endlessly to the players actions. For performing this experiment, Michael designed a shoot-em-up genre with invaders style gameplay. He released the game on Xbox Live and it was considered to be a moderate success for a one-man development team. InvAIders pits the player against generations of enemy AI. When the player beats a generation, the genetic algorithm ranks the AI and uses them to create a new generation of enemy to fight the player. Through successive generations, the AI is able to learn every tactic that the player can potentially deploy. blog post4 Although, this technique can be considered by some as trial-and-error, it is still effective. The describes the entire development process and the pros and cons of the technique too.

InvAIders screenshot

http://gamasutra.com/blogs/MichaelMartin/20110830/8325/Using_a_Genetic_Algorithm_to_Create_Adaptive_Enem y_AI.php

Vivek Deshpande GPA1 2013-2014 Page 10

TOWER OF REUS
Tower of Reus was developed by a team at DeVry University in Irving, Texas as a part of their semester paper. The game is a tower-defense type with genetic algorithm applied to an AI which places towers along the enemies path. The enemies in each wave are fixed and over a period of successive generations, the algorithm is able to generate an AI which can withstand the wave until the end. Tower of Reus was interesting during my research because they implemented the most basic model of genetic algorithm quite well. As it can be seen in the video, each AI is given a base logic of where to NOT place towers and then the process is started. The USP of the game is that the player creates maps and then lets the AI try out and beat the map. If the map is beaten, the player is given the chance to play his own map and have a better score than the AI. Its an interesting twist of the classic tower defense genre.

Tower Of Reus Screenshot

Vivek Deshpande GPA1 2013-2014 Page 11

STRATLOC
While researching the topic of genetic algorithms, I found this5 blog post to be a godsend. H. Chase Stevens was a part of the team that won the University of Edinburgh 2013 OOP Contest. Their project called StratLoc is a turn-based strategy game played on a hexagonal board, a la Civilization (sans a great deal of complexity). The objective of the game is to eliminate your opponents by capturing their cities. The game is played on a randomly generated map featuring plains, hills, mountains and water tiles (both of which are impassable), metal tiles, and four starting cities (one for each player). From cities, players can build military units and workers; players must also control at least one city to avoid losing the game. Cities are permanent fixtures which can be captured by opposing players. Stevens, in his blog post, perfectly describes the entire development process of the AI in their game. A genetic algorithm is used to pit 4 AI in the same population group against each other in the game, with the winner given the highest fitness score. The game is a perfect example of the benefits of using a genetic algorithm to model an AI. Instead of defining a rigid logic, he let the AI define its own logic by having all possible behaviors of the AI as part of its genome. Based on its genome the AI will perform actions in the current state that it is in. And since all AI are playing against each other, at the end of the game, the best AI will have the best genomes and is carried forward towards the next generation. At the end of the process, we have an AI that is extremely difficult to beat.

StratLoc Screenshot

http://www.chasestevens.com/blog/read_post.php?time=1356663953

Vivek Deshpande GPA1 2013-2014 Page 12

REMODELING GENETIC ALGORITHM

For my implementation of genetic algorithms I decided to not use them to completely model my AI. Instead of having the algorithm let the AI do whatever it wants, I gave every AI a base logic. I used the genetic algorithm to moderate the values used in my logic. This technique is called evolutionary programming6 and is one of the four major evolutionary paradigms. The basic EP method involves 3 steps (Repeat until a threshold for iteration is exceeded or an adequate solution is obtained): 1. Choose an initial POPULATION of trial solutions at random. The number of solutions in a population is highly relevant to the speed of optimization, but no definite answers are available as to how many solutions are appropriate (other than >1) and how many solutions are just wasteful. 2. Each solution is replicated into a new population. Each of these offspring solutions are mutated according to a distribution of MUTATION types, ranging from minor to extreme with a continuum of mutation types between. The severity of mutation is judged on the basis of the functional change imposed on the parents. 3. Each offspring solution is assessed by computing its fitness. Typically, a stochastic tournament is held to determine N solutions to be retained for the population of solutions, although this is occasionally performed deterministically. There is no requirement that the population size be held constant, however, or that only a single offspring be generated from each parent.

Generally EP does not have crossover as a genetic operator. However, for the sake of experiment, I decided to include crossover before mutating a child solution. This way, I could adhere to the basic model of a genetic algorithm and still perform EP on my project.

http://en.wikipedia.org/wiki/Evolutionary_programming

Vivek Deshpande GPA1 2013-2014 Page 13

MY IMPLEMENTATION
Before starting implementation, I had to think a lot about the genre of the game in which I was going to implement it. Since turn-based strategy and tower defense games require a complex base which I didnt have at the beginning. So for the sake of simplicity, I decided to develop upon a 2D platformer base which we had made for our class assignment.

My 2D Platformer

Initially, in my project the 2D platformer had platforms of varying width placed at a random width and height from the last platform. The player was a cube that had to traverse this level which didnt have an end. I changed the varying width to be a constant width and slightly altered the randomness in the separation of the platforms so that they werent too far away from each other. One of the major mistakes I made during my implementation was to do too much at once. I added small rectangles that would move in the negative x direction and imitate flying enemies. I also had the platforms move vertically on their own and added vertical obstacles on each platform which would be placed at any possible position. After designing this endless level I started designing my AI. Unfortunately because of the complexity of the game (where the player had to dodge enemies, jump precisely and negotiate the obstacles), the rigid AI

Vivek Deshpande GPA1 2013-2014 Page 14

logic wasnt working like I thought it would. I wasted a lot of time in tweaking the AI logic until I realized my mistake. Then I decided to scrap all the extra elements I added in the platformer leaving only platforms which didnt move on their own. My primary logic for the AI in this level is below:

AI Logic

Vivek Deshpande GPA1 2013-2014 Page 15

The values for Walk Acceleration, Jump Acceleration, Maximum X velocity, Maximum Y velocity, Minimum X distance from next platform, Minimum Y distance from next platform were parameters that the EP algorithm would solve for me. After running the algorithm for one whole day, it came upon the perfect solution in its 30th generation. I tested the solution exhaustively until I concluded that I can continue further and added obstacles at the center of each platform. I had to change the AI logic so that it would jump over the obstacle.

New AI Logic

Vivek Deshpande GPA1 2013-2014 Page 16

The fitness function for this new AI was designed so that if current solution distance is greater than 20*(generation count+1) then advance it to the next generation else cull it. Also because some AI were not even able to get past the first obstacle, I added in a destruction timer on each so that after 30 seconds the AI would self-destruct (even if it was able to get past the first obstacle) and the next AI would start the level again. This proved to be good since I was able to calculate how much variation there was in a parent and a child solution. As the results indicated, there wasnt a noticeable difference.

EXTENDING THE CURRENT PROTOTYPE

From the beginning my goal has been to develop adaptive AI. Using the techniques learnt in this technical research I feel that designing an adaptive AI would be possible. Currently I was using EP to alter AI logic values; I believe if it would be possible to encode all AI actions into a bit string, it would also be possible to use bit-swapping crossover techniques so that in any situation our AI will be able to respond with any action from its list. Once it performs this action, we can assess it with our fitness function and then allow the AI with good fitness scores to go ahead. Although this theory looks promising it can only be tested once I can apply it to a project which needs its use. But since most games limit the players actions, I feel using this technique would be a waste of time. Instead if a project which had real immersion and which allowed the player to perform any action that a human being can perform, an AI with this technique would be the perfect catalyst for a good game.

CONCLUSION
Genetic algorithms are efficient; there is no doubt about that. For relatively simple solutions which require a lot of grunt work, it is viable to setup a GA system which does the work for you. GA is also the base for almost all of the new optimized evolutionary algorithms as well. However, for complex problems it is far more cost-efficient to use new evolutionary algorithms as they are designed to be better as the problem gets bigger and complex too. In game development, evolutionary algorithms can be widely used. Currently the only projects using evolutionary algorithms for artificial intelligence are small indie companies or one man teams doing so for experimental reasons. I believe if a game is designed for an adaptive AI which uses evolutionary techniques it would revolutionize the game development industry.

Vivek Deshpande GPA1 2013-2014 Page 17

REFERENCES
1. Wikipedia (Genetic algorithms, Evolutionary algorithms, Evolutionary Programming) a. http://en.wikipedia.org/wiki/Genetic_algorithm b. http://en.wikipedia.org/wiki/Evolutionary_algorithm c. http://en.wikipedia.org/wiki/Evolutionary_programming 2. Genetic algorithm tutorial http://www.ai-junkie.com/ga/intro/gat1.html 3. What are Evolutionary Algorithms (EAs)? http://www.cs.cmu.edu/Groups/AI/html/faqs/ai/genetic/part2/faq-doc-1.html 4. H. Chase Stevens Blog http://www.chasestevens.com/blog/read_post.php?time=1356663953 5. Tower of Reus a. Main Page http://www.cameronferguson.net/projects/games/towers-of-reus b. Gameplay https://www.youtube.com/watch?v=rHhpyisO0vw 6. Using a Genetic Algorithm to Create Adaptive Enemy AI (InvAIders) http://gamasutra.com/blogs/MichaelMartin/20110830/8325/Using_a_Genetic_Algorithm_to_Crea te_Adaptive_Enemy_AI.php 7. Genetic Algorithms and Evolutionary Computation http://www.talkorigins.org/faqs/genalg/genalg.html

Vivek Deshpande GPA1 2013-2014 Page 18