Objectives

Intelligent Agents You should

! be able to provide a definition of a rational
agent.
! be able to compare and contrast various
agents including reflex, goal-based, and
utility-based agents.
Dr. Richard J. Povinelli ! be able to classify the environment in
which a particular agent operates.

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 1  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 2

Agents and environments Vacuum-cleaner world

Agents include humans, robots, softbots,

thermostats, …. Percepts: location and contents, e.g.,
The agent function maps from percept [A,Dirty]
histories to actions:
! f : P* -> A Actions: Left, Right, Suck, NoOp
The agent program runs on the physical
architecture to produce f
 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 3  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 4

A vacuum-cleaner agent Rationality

Percept sequence Action Fixed performance measure evaluates the
[A,Clean] Right environment sequence
[A,Dirty] Suck ! one point per square cleaned up in time T?
[B,Clean] Left ! one point per clean square per time step, minus one per
[B,Dirty] Suck move?
[A,Clean], [A,Clean] Right ! penalize for > k dirty squares?
[A,Clean], [A,Dirty] Suck
A rational agent chooses whichever action maximizes
function Reflex-Vacuum-Agent([location, status]) returns an action the expected value of the performance measure
if status = Dirty then return Suck given the percept sequence to date
else if location = A then return Right Rational ≠ omniscient
else if location = B then return Left
Rational ≠ clairvoyant
What is the right function? Rational ≠ successful
Can it be implemented in a small agent program?
Rational ⇒ exploration, learning, autonomy
 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 5  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 6

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 1

 CAT – PEAS for an Automatic Taxi
PEAS
To design a rational agent, we must specify the task Describe the
environment ! Preformance measures
Consider, e.g., the task of designing an automated taxi: ! Environment
Actuators
Performance measure !

! Sensors
! What we measure an agents against.
Environment For an automatic taxi.
! The world in which the agent operates. Work with a partner for 5 minutes.
Actuators
! How the agent can modify its environment
Sensors
! How the agent can sense its environment.

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 7  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 8

PEAS for an Automatic Taxi Agent functions and programs

Performance measures An agent is completely specified by the agent function mapping
percept sequences to actions
!
In principle, one can supply each possible sequence to see what
Environment it does. Obviously, a lookup table would usually be immense.
! One agent function (or a small equivalence class) is rational
Aim: find a way to implement the rational agent function
concisely
Actuators
An agent program takes a single percept as input, keeps
!
internal state:
Sensors
function SKELETON-AGENT(percept) returns action
! static: memory, the agent’s memory of the world
memory ← UPDATE-MEMORY (memory, percept)
action ← CHOOSE-BEST-ACTION (memory)
memory ← UPDATE-MEMORY (memory, action)
return action
AIMA Slides © Stuart Russell and Peter Norvig, 1998 Chapter 2 AIMA Slides © Stuart Russell and Peter Norvig, 1998 Chapter 2
 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 9  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 10

Agent types Simple Reflex Agent

Four basic types in order of increasing
generality
simple reflex agents
reflex agents with state
goal-based agents
utility-based agents

AIMA Slides © Stuart Russell and Peter Norvig, 1998 Chapter 2

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 11  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 12

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 2

 Reflex Agents with State Goal-based Agents

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 13  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 14

Utility-based Agents Environment Types

Accessible vs. hidden
Deterministic vs. stochastic
Episodic vs. nonepisodic
Static vs. dynamic
Discrete vs. continuous

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 15  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 16

CAT – Environment Types Environment Types Example

With a partner fill in the following Solitaire Backgammon Internet Taxi
shopping
table. You have 5 minutes
Observable??
Solitaire Backgammon Internet Taxi
shopping
Deterministic??
Accessible??
Deterministic?? Episodic??

Episodic?? Static??
Static??
Discrete??
Discrete??

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 17  Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 18

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 3

 AlMA code
The code for each topic is divided into four
directories:
! agents: code defining agent types and programs
! algorithms: code for the methods used by the agent
programs
! environments: code defining environment types,
simulations
! domains: problem types and instances for input to
algorithms
! Often run algorithms on domains rather than agents
in environments.

AIMA Slides © Stuart Russell and Peter Norvig, 1998 Chapter 2

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 19

 Copyright Richard J. Povinelli rev 1.1, 8/28/2003 Page 4