Sunteți pe pagina 1din 31

AFOSR

COGNITIVE MODELING AND ROBUST


DECISION MAKING
14 March 2011

Dr. Willard Larkin for Dr. Jun Zhang


Program Manager
AFOSR/RSL
Distribution A: Approved for public release; distribution is unlimited. 88ABW-2011-0784
1
PORTFOLIO OVERVIEW

Program Manager: Jun Zhang, Ph.D. (Briefing by Willard Larkin)

BRIEF DESCRIPTION OF PORTFOLIO:

• Experimentsand modeling of high-order cognitive processes


for human performance on complex tasks
• Computational principles for symbiosis of mixed human-
machine systems with allocating and coordinating requirements

SUB-AREAS IN PORTFOLIO:

2313/B: Mathematical Modeling of Cognition and Decision


2311/H: Human System Interface and Robust Decision Making

2
Program Components, Goals, and
Strategy

Mathematical Modeling of Cognition and Decision


(Subtask 2313/B)
Advance mathematical & computational models of human cognition,
especially reasoning, planning, problem solving and decision making.
• (Includes Sec. of Defense PBD709 Topic, “Information Fusion and Decision
Science,” emphasizing math foundations of machine learning)

Human-System Interface
(Subtask 2311/H)
Advance research on mixed human - machine systems to aid inference,
communication, prediction, planning, scheduling, and decision making.
• Includes “Robust Decision Making” Discovery Challenge Thrust (DCT)

Primary Strategy:
• Forge useful connections among experts in math, computation,
neuroscience, and cognitive behavior.
• Seek algorithms for adaptive intelligence inspired by brain science 3
TechHorizons
Priority Key Technology Areas
(COGNITIVE MODELING AND ROBUST DECISION-MAKING, RSL)

• Autonomous systems • Spectral mutability


• Autonomous reasoning and learning • Dynamic spectrum access
• Resilient autonomy • Quantum key distribution
• Complex adaptive systems • Multi-scale simulation technologies
• V&V for complex adaptive systems • Coupled multi-physics simulations
• Collaborative/cooperative control • Embedded diagnostics
• Autonomous mission planning • Decision support tools
• Cold-atom INS • Automated software generation
• Chip-scale atomic clocks • Sensor-based processing
• Ad hoc networks • Behavior prediction and anticipation
• Polymorphic networks • Cognitive modeling
• Agile networks • Cognitive performance augmentation
• Laser communications • Human-machine interfaces
• Frequency-agile RF systems
4
Examples of Algorithms for
Cognition and Decision

(1) Information Accumulation: Seek optimal trades between speed


and accuracy for decisions under time pressure.

(2) Reinforcement Learning : Balance short- and long-term goals


in executing sequential, continuous tasks.

(3) Categorization & Classification: Generalize from past examples


to future encounters by optimally regulating complexity and
data-fitting performance.

(4) Causal Reasoning and Bayesian inference: Seek optimal


fusion of prior knowledge with new evidence for reasoning and
prediction under uncertainty, ambiguity, and risk.

5
All Projects – 6 Clusters
(For program review purposes only)

1. Mathematical Foundation of Decision Under Uncertainty


2. Neural Basis of Decision Making Under Time Pressure
3. Optimal Planning/Control via Reinforcement Learning
4. Robust Classification and Prediction (includes Machine
Learning Sub-program)
5. Memory, Context, and Causal Reasoning
6. Vision, Communication, and Autonomous Systems

Goal: Supporting human-in-the-loop in various cognitive domains


6
Recent Transitions

To IARPA and US Army: Mathematical implementations of Cultural


Consensus Theory to improve forecasting technology and information
aggregation for analysts. IARPA Program: “Integrated Cognitive-Neuroscience
Architectures for Understanding Sensemaking” (ICArUS) Program (Dr. W.
Batchelder, U.C. Irvine.)

To HRL Laboratories: Techniques for spike-time encoding and


decoding of information in analog signals. (Dr. Aurel Lazar, Columbia Univ.
POC: Peter Petrie, Malibu CA.)

To AFRL/RH: Techniques for implementing generative and persistent


models for information sampling and learning. (Dr. B. Love, U. Texas, Austin..
POCs: S. Douglas and C. Myers, AFRL/RH.)

To AFRL/Rome Lab: An AFRL-RI funded project with G. Kreiman


(Harvard): Implementing kernel machine techniques for modeling feedback in
vision systems. (Dr. T. Poggio (MIT). POCs: Todd Howlett and Yuri Luzanov.

7
CLUSTER one

Mathematical Foundations of Decision Making


and Reasoning Under Uncertainty
1. Narens (UC Irvine): New axioms of subjective probability.

2. Chichilnisky (Columbia): Topological methods for unexpected events with


catastrophic risk (“Black Swan” theory).

3. Halpern (Cornell): Theory for games with deficient awareness.

4. Schweickert (Purdue): Conditional independence and selective influence.

5. Luce (UC Irvine): Conjoint structures for multi-modal sensory scaling.

6. *Bringsjord (RPI): Fusing analogical and deductive reasoning.

Scientific Goal: Develop formal foundations for cognitive systems.


8
Foundations of Subjectively Rational
Decisions (Louis Narens, UCI)

Scientific Challenge: SUBJECTIVE RATIONALITY:


• Emotions and contexts affect
human decisions.
• Design a decision theory as close
as possible to rational decision
theory, but allow for such effects.

New Axioms & Theorems for


a Calculus of Subjective
Rationality:

Generalizes notion of a finitely


additive probability measure on an
Event Space X with a Pseudo-
Complement Such that:

Coherence: assumed in the subjective,


not the objective domain
Intuitively: is iis the largest event
that “seems like” the complement of A 9
CLUSTER two
Neural Basis of Decisions under Time Pressure
1. McClelland (Stanford): MURI on neural basis of decision making
with Ditterich (UC Davis): multi-alternative perceptual decisions.

2. Pouget (Rochester): optimal cue integration for decision making.

3. *Lee (UC Irvine): adaptive threshold for speed-accuracy tradeoff.

4. *Ratcliff (OSU): change detection and performance monitoring.

5. T. Zhang (Johns Hopkins): hippocampal spatial memory.

6. Chua (Berkeley): memristor models for neurons and synapses.

Former AFOSR Program Manager Jerome Busemeyer started this theme. It has
now become main-stream.

10
McClelland MURI (2007)
Neural Basis of Decision Making

GOAL: Build a lattice of alternative models, motivated by


neurological data, for decision-making under time pressure;
test them in human tasks.

A GENERALIZED DRIFT-DIFFUSION MODEL: yY1


1 yY2
2

“LEAKY COMPETING ACCUMULATORS”

• Proposes accumulators of noisy I 1 I 2


evidence, y1, y2, with leakage, and
mutual inhibition:

EVIDENCE LEVEL
dy1/dt = I1-gy1–bf(y2)+x1 Y1
dy2/dt = I2-gy2–bf(y1)+x2
f(y) = [y]+
Y2
• In time controlled tasks, choose
response 1 if y1-y2 > 0
Time (sec.)
11
McClelland MURI (2007)
Neural Basis of Decision Making

Leakage Dominates Inhibition Dominates


Alternative formulations
for “Accumulators of
Competing Evidence”
can be distinguished by
qualitative data
signatures.

d (t )   R (t )  R (t ) 

d (t )  kS (1  et ) 12
How does reward for fast decisions
affect how information is integrated ?
Three Alternative Hypotheses:
1. Reward acts as an information input from the reward
cue onset to the end of the integration period

2. Reward influences the state of the information


accumulators before the stimulus onset

3. Reward introduces a fixed offset in the decision


variable.
13
Reward Bias Signatures Plotted against
Decision Latency

Data for 4 Individuals

REWARD BIAS
UNDERESTIMATES
OPTIMAL BIAS;
BOTH DECLINE
TO A NON-ZERO
PLATEAU

Data lead to a clear choice among the three alternative models


14
Qualitative Data Signatures Rule Out H1
and H3 (McClelland, cont.)
“REWARD MIMICS “REWARD PRE- “REWARD SETS A
INFORMATION INPUT’” CONFIGURES THE FIXED OFFSET”
ACCUMULATORS”

X X

15
CLUSTER three

Achieving Optimal Planning and Control via


Reinforcement Learning
1. Barto (U Mass): domain competency and skill learning.
2. Tenenbaum (MIT): causal modeling of representation learning.
3. Love (UT Austin): feature selection and information sampling.
4. Jones (Colorado): kernel-based representation for RL.
5. Mahadevan (U Mass): graph-theoretic methods for RL.
6. Qian (Columbia): attention and sensori-motor control.

GOAL: Build on past success of the reinforcement learning algorithm


(which cuts across optimization, machine learning, animal learning, and
neuroscience), push new frontiers , e.g., intrinsic motivation, hierarchical
planning, feature selection, attention.
16
Automated Representation Discovery in
Sequential Decision Problems

To avoid human trial-and-error, discover how to


Scientific Challenge: generate novel representations of complex tasks,
such as Markov Decision Problems (MDPs), and
other stochastic problems, to facilitate their solution.

A Graph-Theoretic Approach for Task Hierarchies:

Sridhar Mahadevan
UMass Amherst

Osentoski and Mahadevan, A Recursive Basis Function A State Graph of a


Decomposition for a “Taxi” Problem “Taxi” Subtask
17
AAMAS 2010
Automated Basis Function Construction
Speeds Reinforcement Learning

Convergence Results Comparing Four


Graph Laplacian Approaches
“GRIDWORLD”

264 Basis functions


Steps to Goal

from a State graph

260 Basis functions


from a joint State-
Action graph
Number of Learning Iterations in
a 4-Room “Gridworld” Problem

18
S. Osentoski and S. Mahadevan, UMass, Amherst
CLUSTER four

Robust Classification and Prediction


1. Zhu (UW Madison): unsupervised and semi-supervised learning

2. Wang (Wright State): semi-supervised learning for structured prediction

3. Nosofsky (Indiana): rule-based categorization

4. Batchelder (UC Irvine): aggregation, cultural consensus model

Includes Sub-Program in
Mathematical Foundations of
Machine Learning
(from PBD 709 funds)

19
Mathematical Development of
Cultural Consensus Theory
The Problem:
• Informants or respondents share culture-specific
knowledge, assumptions and beliefs

• They respond to questionnaire items, but we do not


know which answers, if any, represent their special
shared knowledge -- Nor do we know, a priori,
anyone’s “cultural competency,” response biases, or
Wm. Batchelder the “cultural saliency” of the questions we ask.
UC Irvine
• This problem inverts the usual psychometric testing
situation in which the answer key is already known.

The Scientific Challenge:


Develop mathematical models of the response process from which
“culture -specific answers” and informant characteristics can be
inferred to identify belief structures and to forecast behaviors.
20
Algorithms transitioned to IARPA and Army
Information Fusion and Decision Science
(PBD709 Topic from SECDEF)

Mathematical Foundations of Machine Learning


1. Lafferty (CMU): statistical learning for structured high dimensional data

2. Poggio (MIT): hierarchical kernels for visual object recognition

3. Xu (Syracuse): kernel learning and refinement in RKHS

4. Bertsekas (MIT): large-scale convex optimization and approximate DP

5. T. Zhang (Rutgers): multi-stage convex relaxation in machine learning

6. *Casazza (U Missouri): frames theory and compressive sensing

GOAL: Investigate fundamental mathematical concepts invoked in


machine learning; extend them to the domain of cognitive information
processing.
21
Frames and Quantitative Redundancy
P.Casazza (Missouri), M. Fickus (AFIT)

Scientific Challenge:
Develop new mathematics for dimension-
reduction problems encountered in pattern
recognition and information fusion.

Approach:
FRAME THEORY -- a rigorous infrastructure for
Prof. Casazza study of redundant linear measurements –
developed mainly by Peter Casazza

Recent Publications:
P.G. Casazza, M. Fickus, D.G. Mixon and J.C. Tremain, The Bourgain-
Tzafriri conjecture and concrete constructions of non-pavable
projections, to appear in Operators and Matrices.

P.G. Casazza, M. Fickus, D.G. Mixon, Y. Wang and Z.


Zhou, Constructing tight fusion frames, to appear in
Applied Computational Harmonic Analysis. AFIT MISSOURI PRINCETON
22
A Joint Initiative with AFRL/RH and
George Mason University

A Center of Excellence in Neuro-


Ergonomics, Technology and Cognition
Started 15 July 2011
Convened 14 September 2010 at George Mason University

Goal:
Stimulate productive new collaborations with
scientists in AFRL’s Human Effectiveness
Directorate on a broad range of problems in
cognitive science
Prof. Raja
Parasuraman

GENETIC DETERMINANTS, NEUROADAPTIVE TRAINING, TRUST IN AUTOMATION


NEUROIMAGING, ATTENTION, MULTI-TASKING, MEMORY, SPATIAL COGNITION 23
3D Spatial Visualization
(Glenn Gunzelmann AFRL 711HPW/RH)

TASK: Imagine steps Up, Down, Left,


or Right, one by one, along a random
path in a 3D grid. Detect each return to
your prior positions.

Scientific Questions:
• What are the constraints and limits of
mental navigation?
• Do individuals differ in this ability?
• How and why is the ability fragile?
Grid path is held in • Can a computational model (ACT-R)
“The Mind’s Eye” account for the phenomena?

24
Path Visualization Performance:
Depends on No. of Steps between Re-Visits

PERFORMANCE FOR 7 X 7 GRID • Individual performance varies


widely – some people are
100
extraordinarily competent.
Percent Correct

90 • The ACT-R model for one size


grid correctly predicts data for
80 other sizes.
• Increasing path length boosts
70 associative interference, but
this has a very minor role in the
60
lag effects shown here.
3 7 11 15 19
Number of Path Segments (Lag) • The ACT-R activation decay
Separating Return Visits rate parameter best explains
individual differences.

Gunzelmann, et al. 711HPW AFRL/RH 25


CLUSTER six

Novel Approaches to Vision, Communication,


& Autonomous Systems
1. Pizlo (Purdue): robotic navigation with symmetric-based shape recognition
2. Tyler (SKERI): surface representation as intermediate vision
3. Yu (Indiana): development of symbolic processing in communication
4. Damasio/Yau/ (USC/Geometric Informatics): Ricci-flow method for aligning
brain imaging data
5. Myung (Ohio State): Cognitive Modeling Repository
6. Lazar (Columbia): Spike-time encoding & reconstruction of visual displays

NOVEL APPROACHES TO: Topics such as intermediate representation in vision,


emergence of symbolic representation for communication, analysis of brain imaging
data, cognitive and biomimetic approaches to machine intelligence.
26
Spike-Time Encoding and Decoding of
Visual Scenes (Aurel Lazar, Columbia U.)

INITIAL STAGES OF VISION


GENERATE NEURAL SPIKE TRAINS

Scientific Challenge:
Invent a computational
engine to encode visual
scenes and to reconstruct
them from spike-timing data

SUBSEQUENT STAGES
ASSUMED INVERTIBLE

27
Analog Signal Recovery from a Model
Bipolar Cell & Spike-Timing Analysis

Bipolar Neuron model


converts analog signal
into a series of “On” ON ---
and “Off” spikes at
threshold crossings.
OFF --

Bipolar cell model


with thresholding
Signal Recovery
and feedback
28
Aurel A. Lazar, Columbia University
Video Stream Recovery from Spiking
Neurons– Using Hodgkin-Huxley Model
Progress:
ORIGINAL RECONSTRUCTED
• First demonstration
of natural scene
recovery from
spiking neuron
models based upon
an architecture that
includes visual
receptive fields and
neural circuits with
feedback.
Reconstruction minimizes a 3-component cost
• Scalable decoding function that prevents over-fitting to noise,
algorithms were measures reconstruction error, and regulates the
demonstrated on a trade between smoothness and match to data,
parallel computing
platform.
Aurel A. Lazar, Columbia University
Transitions to HRL Labs and others 29
SUMMARY:
Transformational Impacts & Opportunities
4

• Program represents a new level of mathematically


sophisticated cognitive science:
• New cohort of interdisciplinary theorists and experimentalists
• Embraces wide-ranging ideas and non-traditional expertise
• Supports multiple collaborations between Universities and AFRL

• Responds to 6 high-priority topics in AF TechHorizons

• Coherent investments in the development of


computational algorithms for adaptive intelligence

• Several research areas with extraordinary promise:


• Novel math frameworks to characterize risk, uncertainty, & subjectivity
• Autonomous methods to optimize complex reinforcement learning
• Robust enhancements of classification, recognition, and reasoning
30
Questions?

Thank you for your attention


Willard Larkin, Program Manager, AFOSR/RSL
703-696-7793

Willard.Larkin@afosr.af.mil

SPECIAL THANKS TO LT. IAN PRUDHOMME


FOR HELP WITH THIS PRESENTATION
31

S-ar putea să vă placă și