Tit for tat is an English saying meaning "equivalent retaliation".

It is also a highly effective strategy

in game theory for the iterated prisoner's dilemma. It was first introduced by Anatol Rapoport in Robert
Axelrod's two tournaments, held around 1980. An agent using this strategy will initially cooperate, then
respond in kind to an opponent's previous action. If the opponent previously was cooperative, the agent is
cooperative. If not, the agent is not. This is similar tosuperrationality and reciprocal altruism in biology.

Contents
[hide]

• 1 Overview

• 2 Example of play

• 3 Implications

• 4 Problems

• 5 Tit for two tats

• 6 Real world use

o 6.1 Peer-to-peer file sharing

o 6.2 Explaining Reciprocal Altruism in Animal

Communities

o 6.3 War

• 7 Popular culture

• 8 See also

• 9 References

• 10 External links

[edit]Overview

This strategy is dependent on four conditions, which have allowed it to become the most successful
strategy for the iterated prisoner's dilemma:[1]

1. Unless provoked, the agent will always cooperate

2. If provoked, the agent will retaliate
3. The agent is quick to forgive
4. The agent must have a good chance of competing against the opponent more than once.
In the last condition, the definition of "good chance" depends on the payoff matrix of the prisoner's
dilemma. The important thing is that the competition continues long enough for repeated punishment and
forgiveness to generate a long-term payoff higher than the possible loss from cooperating initially.

A fifth condition applies to make the competition meaningful: if an agent knows that the next play will be
the last, it should naturally defect for a higher score. Similarly if it knows that the next two plays will be the
last, it should defect twice, and so on. Therefore the number of competitions must not be known in
advance to the agents.

Generally, in game theory, effectiveness of a strategy is measured under the assumption that each player
cares only about him or herself. (Thus, the game-theory measure of effectiveness is impractical in
many real life situations where players do have a vested interest in, or an altruistic compassion towards,
other players.) Furthermore, game-theory effectiveness is usually measured under the assumption of
perfect communication, where it is assumed that a player never misinterprets the intention of other
players. By this game-theory definition of effectiveness tit for tat was superior to a variety of alternative
strategies, winning in several annual automated tournaments against (generally far more complex)
strategies created by teams of computer scientists, economists, and psychologists. Some game theorists
informally believe the strategy to be optimal, although no proof is presented.

In more recent competitions tit for tat was not the most effective strategy, even under the game-theory
definition of effectiveness. However, tit for tat would have been the most effective strategy if the average
performance of each competing team were compared. The team which recently won over a pure tit for tat
team outperformed it with some of their algorithms because they submitted multiple algorithms which
would recognize each other and assume a master and slave relationship (one algorithm would "sacrifice"
itself and obtain a very poor result for the other algorithm to be able to outperform tit for tat on an
individual basis, but not as a pair or group). This "group" victory illustrates one of the important limitations
of the Prisoner's Dilemma in representing social reality, namely, that it does not include any natural
equivalent for friendship or alliances. The advantage of tit for tat thus pertains only to a Hobbesian world
of so-called rational solutions (with perfect communication), not to a world in which humans are inherently
social.[citation needed] However, that this winning solution does not work effectively against groups of agents
running tit for tat illustrates the strengths of tit for tat when employed in a team (that the team does better
overall, and all the agents on the team do well individually, when every agent cooperates).

[edit]Example of play Cooperate Defect

Assume there are four agents: two use the tit-for-tat strategy, and Cooperate 3, 3 0, 5
two are "defectors" who will simply try to maximize their own
Defect 5, 0 1, 1
winnings by always giving evidence against the other. Assume that
each player faces the other three over a series of six games. If one Prisoner's dilemma example
player gives evidence against a player who does not, the former gains 5 points and the latter nets 0. If
both refrain from giving evidence, both gain 3 points. If both give evidence against each other, both gain 1
point.

When a tit-for-tat agent faces off against a defector, the former refrains from giving evidence in the first
game while the defector does the opposite, gaining the control 5 points. In the remaining 5 games, both
players give evidence against each other, netting 1 point each game. The defector scores a total of 10,
and the tit-for-tat agent scores 5.

When the tit-for-tat agents face off against each other, each refrains from giving evidence in all six games.
Both agents win 3 points per game, for a total of 18 points each.

When the defectors face off, each gives evidence against the other in all six games. Both defectors win 1
point per game, for a total of 6 points each.

Each tit-for-tat agent scores a total of 18 points, over the six matches. Each defector scores only 6 points.

Despite the fact that the tit-for-tat agents never won a match and the defectors never lost a match, the tit-
for-tat strategy still came out ahead, because the final score is not determined by the number of match
wins, but the total points score. Simply put, the tit-for-tat agents gained more points tying with each other
than they lost to the defectors.

The more tit-for-tat agents that there are in the described game, the more advantageous it is to use the tit-
for-tat strategy. The fewer tit-for-tat agents that there are in the described game, the less advantageous it
is to use the tit-for-tat strategy.

[edit]Implications

The success of the tit for tat strategy, which is largely cooperative despite that its name emphasizes an
adversarial nature, took many by surprise. In successive competitions various teams produced complex
strategies which attempted to "cheat" in a variety of cunning ways, but tit for tat eventually prevailed in
every competition.

This result may give insight into how groups of animals (and particularly human societies) have come to
live in largely (or entirely) cooperative societies, rather than the individualistic "red in tooth and claw" way
that might be expected from individuals engaged in a Hobbesian state of nature. This, and particularly its
application to human society and politics, is the subject of Robert Axelrod's book The Evolution of
Cooperation.

[edit]Problems

While Axelrod has empirically shown that the strategy is optimal in some cases, two agents playing tit for
tat remain vulnerable. A one-time, single-bit error in either player's interpretation of events can lead to an
unending "death spiral". In this symmetric situation, each side perceives itself as preferring to cooperate,
if only the other side would. But each is forced by the strategy into repeatedly punishing an opponent who
continues to attack despite being punished in every game cycle. Both sides come to think of themselves
as innocent and acting in self-defense, and their opponent as either evil or too stupid to learn to
cooperate.

This situation frequently arises in real world conflicts, ranging from schoolyard fights to civil and regional
wars. Tit for two tats could be used to avoid this problem[2]

"Tit for tat with forgiveness" is sometimes superior. When the opponent defects, the player will
occasionally cooperate on the next move anyway. This allows for recovery from getting trapped in a cycle
of defections. The exact probability that a player will respond with cooperation depends on the line-up of
opponents.

The reason for these issues is that tit for tat is not a subgame perfect equilibrium.[3] If one agent defects
and the opponent cooperates, then both agents will end up alternating cooperate and defect, yielding a
lower payoff than if both agents were to continually cooperate. While this subgame is not directly
reachable by two agents playing tit for tat strategies, a strategy must be a Nash equilibrium in all
subgames to be subgame perfect. Further, this subgame may be reached if any noise is allowed in the
agents' signaling. A subgame perfect variant of tit for tat known as "contrite tit for tat" may be created by
employing a basic reputation mechanism.[4]

[edit]Tit for two tats

Tit for two tats is similar to tit for tat in that it is nice, retaliating, forgiving and non-envious, the only
difference between the two being how nice the strategy is.

In a tit for tat strategy once an opponent defects, the tit for tat player immediately responds by defecting
on the next move. This has the unfortunate consequence of causing two retaliatory strategies to
continuously defect against one another resulting in a poor outcome for both players. A tit for two tats
player will let the first defection go unchallenged as a means to avoid the "death spiral" of the previous
example. If the opponent defects twice in a row, the tit for two tats player will respond by defecting.

This strategy was put forward by Robert Axelrod during his second round of computer simulations
at RAND. After analyzing the results of the first experiment he determined that had a participant entered
the tit for two tats strategy it would have emerged with a higher cumulative score than any other program.
As a result he himself entered it with high expectations in the second tournament. Unfortunately due to
the more aggressive nature of the programs entered in the second round, tit for two tats did significantly
worse (in the game-theory sense) than did tit for tat, due to aggressive strategies being able to take
advantage of its highly forgiving nature.[5]
[edit]Real world use
[edit]Peer-to-peer file sharing
See also: BitTorrent_(protocol)#Downloading torrents and sharing files

BitTorrent peers use tit-for-tat strategy to optimize their download speed.[6] More specifically, most
BitTorrent peers use a variant of Tit for two Tats which is called optimistic unchoking in BitTorrent
terminology. BitTorrent peers have a limited number of upload slots to allocate to other peers.
Consequently, when a peer's upload bandwidth is saturated, it will use a tit-for-tat strategy. Cooperation is
achieved when upload bandwidth is exchanged for download bandwidth. Therefore, when a peer is not
uploading in return to our own peer uploading, the BitTorrent program will choke the connection with the
uncooperative peer and allocate this upload slot to a hopefully more cooperating peer. Optimistic
unchoking corresponds very strongly to always cooperating on the first move in prisoner’s dilemma.
Periodically, a peer will allocate an upload slot to a randomly chosen uncooperative peer (unchoke). This
is called optimistic unchoking. This behavior allows searching for more cooperating peers and gives a
second chance to previously non-cooperating peers. The optimal threshold values of this strategy are still
the subject of research.

[edit]Explaining Reciprocal Altruism in Animal Communities

Studies in the prosocial behaviour of animals, have led many ethologists and evolutionary psychologists
to apply tit-for-tat strategies to explain why altruism evolves in many animal communities. Evolutionary
game theory, derived from the mathematical theories formalised byVon
Neumann and Morgenstern (1953), was first devised by Maynard Smith(1972) and explored further in bird
behaviour by Robert Hinde. Their application of game theory to the evolution of animal strategies
launched an entirely new way of analysing animal behaviour.

Reciprocal altruism works in animal communities where the cost to the benefactor in any transaction of
food, mating rights, nesting or territory is less than the gains to the beneficiary. The theory also holds that
the act of altruism should be reciprocated if the balance of needs reverse. Mechanisms to identify and
punish "cheaters" who fail to reciprocate, in effect a form of tit for tat, is an important mechanism to
regulate reciprocal altruism.

[edit]War

The tit for tat strategy has been detected by analysts in the spontaneous non-violent behaviour, called
"live and let live" that arose during trench warfare in the First World War. Troops dug in only a few
hundred feet from each other would evolve an unspoken understanding. If a sniper killed a soldier on one
side, the other could expect an equal retaliation. Conversely, if no one was killed for a time, the other side
would acknowledge this implied "truce" and act accordingly. This created a "separate peace" between the
trenches.[7]
[edit]Popular culture
The tit for tat strategy was employed in an episode of Numb3rs, where FBI agents were interrogating and
attempting to obtain information from an inmate on death row. The strategy was working, but the FBI
would not implement a "tit for two tats".

This approach to interactions can be seen as a parallel to the eye for an eye approach from Judeo-
Christian-Islamic tradition, where the penalty for taking someone's eye is to lose one's own.