Sunteți pe pagina 1din 4

Course name: Games, Distributed Decision Making, and Learning in

Networks and Systems


Instructor: Prof. Mihaela van der Schaar (Electrical Engineering, UCLA)

Quarter: Winter 2009

Brief Description:

Future wireless/wired networks and distributed computing systems will support a wide variety of
heterogeneous applications and services, with different and dynamic traffic and workload
characteristics, utilities and quality of service requirements. Efficient allocation of network and
system resources will be one of the most crucial factors in such dynamic and heterogeneous
environments. The interaction among the various agents (applications, machines, tasks, users etc.)
can be modeled as a game, in which these autonomous agents can strategically interact with each
other in order to acquire the dynamically available network or system resources and improve their
achievable performance. Importantly, the interactions that are of interest in engineered systems,
from communications and networking to computer systems, take place among agents having
limited and often asymmetric information about each other and the environment, as well as
different abilities to process this information. Hence, the information-decentralization and the
heterogeneity of agents will play a key role in shaping the resulting inter-agent interactions and,
consequently, the efficiency of the resulting communications, networking and computer systems.

In this course, we discuss how different cooperative and non-cooperative games among
agents can be constructed to model, analyze, optimize, and shape the emerging interactions
among users in different networks and system settings. We also discuss how strategic agents can
successfully compete with each other for the limited and time-varying resources, by optimizing
their decision process and learning from their past interaction with other agents. To determine
their optimal actions in these distributed, informationally-decentralized environments, agents will
need to learn and model directly or implicitly the other agents’ responses to their actions. Hence,
this course will also discuss in detail several existing multi-agent learning techniques that can be
successfully deployed in multi-agent systems. Finally, the outcome of various interactions among
self-interested agents with various amounts of knowledge will be analyzed both in terms of
dynamics and steady state equilibrium(s), and we will present how new multi-user
communication mechanisms can be synthesized, which achieve new measures of optimality,
rationality, and fairness for networks and systems commonly studied in electrical and computer
engineering ranging from wireless networks, cognitive radio networks, and peer-to-peer networks
to multi-core and grid computers.

In this course, we will draw from the fields of networking, communications, game
theory, multi-agent learning and distributed systems. A unique and distinguishing feature
of this course as compared to other game theory or multi-agent learning courses is the
extent of networking, communications, and distributed systems specific modeling in
developing the game theoretic framework driving the learning and optimization of
strategies and actions available to the interacting agents in the studied electrical and
computer engineering systems. This is in contrast to much of the courses existing on game
theoretic methods in economics, which assume simplistic utility functions and system
settings and more concerned with describing the end result of a game using equilibrium
concepts, rather than providing efficient methods to achieve these equilibria in the studied
multi-agent systems.
Intended audience: Communication, Networking, Signal Processing, Computer Engineering and
Systems Graduate Students

Topics included (each bullet corresponds to roughly one week of class):

1. Multi-agent interactions in communications, networks and computing systems – a


motivation (0.5 week)
• Existing multi-user and system optimizations, and solutions – limitations
• Why are game-theoretic and economics principles useful? – A brief motivation
• Illustrative examples
o Multi-user wireless communications: CSMA, TDMA
o Multi-user networking: TCP
o Peer-to-peer networking: Bit-Torrent
o Multi-tasking over multi-core processors
• Defining realistic utility-resource functions
• Pareto efficiency (optimality)
• Brief discussion about fairness
2. Non-collaborative game theory and its importance in different networks, communication
and computing systems (2 – 2.5 weeks)
• Games in normal form
• Analyzing games: from Pareto optimality to equilibrium
o Defining best response and Nash equilibrium
o Finding Nash equilibria
o How to reach Nash equilibria (best response dynamics and water-filling
algorithm)?
o Generic inefficiency of Nash equilibria
• More advanced solution concepts for normal-form games
o Dominant strategies, Dominated strategies
o Correlated equilibrium
o Stackelberg equilibrium
o Conjectural equilibrium
o Other equilibria
• Extensive-form games
• Illustrative examples – Multi-user Communication Games
3. More advanced representations, beyond normal and extensive forms (2 weeks)
• Repeated games
• Stochastic games
• Markov Decision Processes
• Bayesian Games
• Congestion games
• Illustrative example – Multi-user resource allocations in wireless LANs
• Illustrative example – Dynamic resource management in cognitive radio
networks
• Comparisons of different solutions in terms of complexity, performance and
informational requirements for dynamic networks and systems
4. Multi-agent learning and teaching (1.5 week)
• What to learn, when to learn, how to learn?
• Knowledge function, value of learning and value of information
• Fictitious play
• Reinforcement learning
• No-regret learning
• Evolutionary learning and other large-population models
• The interaction between learning and teaching
• Illustrative examples in wireless systems, computer networks and grid computers
• Learning in cognitive radio networks
5. Protocols for strategic Agents: Mechanism design (1.5 week)
• Mechanism Design Principles
• Efficient mechanisms
• Applications to resource allocations in networks and systems
• Auctions – requirements, design, principles
• Applications to cognitive radios
6. Teams of selfish agents: an introduction to coalitional game theory (1 week)
• Bargaining Theory
• Shapley Value
• The core
• Illustrative examples – Peer to peer networks, Resource Brokerage, Multi-task
optimization in multimedia systems
7. Knowledge, beliefs and communication among agents (0.5 week)
8. Focus on applications: current applications and solutions based on economics concepts to
multi-user wireless communications in current networks, cognitive radio networks,
Internet 2, P2P systems, and processing in grid-computers (1 week)

Course format: 3 hours/week lecture, 1 hour/week discussions and presentations

Material used:
Tutorial Articles from Various Magazines and Journals
Also, some materials from several books will also be provided.
Note: this class will require the students to independently study a certain amount of related
literature.

Grading:
• 20% - homeworks (3 homeworks consisting of several theoretical, review and hands-on
problems)
• 10% - project proposal detailing your proposed project, related literature, initial problem
formulation etc. (up to 4 pages)
• 10% - one in-class presentation and discussions of 20 minutes + report (up to 3 pages)
discussing in-depth one existing journal paper related to your project and also specify in your
presentation how the presented paper is related to your project and how your project will be
different from that paper
• 20% - in-class mid-term exam testing the thorough understanding of the principles, theory
and algorithms discussed in this course
• 39% - final project report and presentation (students will work individually or in groups of 2
students for the final project)
• 1% - final survey completion
Final Project

Students are expected to complete a project during the course of the quarter. The project can be
entirely analytical (i.e. no simulations involved) or can involve some programming (using C, C++
or Matlab). A detailed project proposal has to be submitted. The proposal cannot be more then
four pages (single space, double column, Time Roman 10 pt) in length including figures and
references. It has to be submitted in pdf format. Based on the project proposal, the instructor will
provide you thorough feedback and guidance on how to improve the project based on this plan.

Guidelines for the project

Four classes of problems can be chosen this year for your project: 1) Multi-user communication
in slotted Aloha or contention-based protocols, 2) Multi-user communication in cognitive radio
networks; 3) Multi-task distributed systems; 4) Cooperation and competition in peer-to-peer
networks. Note that you do not need to have specific a priori knowledge in any of these areas.
The instructor will provide you with basic material necessary to get acquainted with these multi-
agent systems.

1. Choose the problem setting you want to work on ASAP. Select papers you will base your
project on (i.e. related literature).
2. Think carefully about the game itself. Precisely define the strategy space, the constraints,
the utility, and the interactions between players. Do you need to modify the theoretical
model to make it implementable? Additional assumptions, restrictions?
3. Experiment with extensions to the material presented in class. Consider different utilities,
run the game repeatedly in a tournament fashion. Ask questions that might be difficult to
solve analytically but are of practical interest. For example, would learning the behavior
of the other players help? If yes, what would be an interesting learning algorithm?
4. By thinking through your design carefully, keep the length of your code to a minimum.
You have little time, so do something simple and elegant which conveys the essence of
your game, and gives some insights/intuition verifying or going beyond the theoretical
results. Keep in mind that a simple program that works has more of an impact than a
sophisticated program that does not perform well.
5. Note that your written and oral presentation of the project is very important!

Some important dates:

January 22 - Homework 1 due


February 3 - Homework 2 due
February 10 - Mid-term
February 12 - Project proposal due
February 26 - Homework 3 due
March 17 – Final project presentation and project report
In class presentations will take place typically on Thursdays.
Important note: Please notice that the mid-term as well as the project proposal are due in the
same week. This requires you to plan ahead. Please start working on your project proposal as
soon as you can.

Office hours:
Mondays 1-2pm
Tuesdays and Thursdays 6-6.30pm

S-ar putea să vă placă și