Iterated Prisoner's Dilemma
Java Simulation Classes

Introduction to the Classical Iterated Prisoner's Dilemma

A formal model for cooperation

Let two artificial rational agents have the choice between two moves:

• Cooperation, let us note C
• Defection, let us note D

They play one against the other, in a synchronous manner, so that they do not know what the other will play. They get a score according to the situation of the move:

• They both cooperate and then get both the Reward for cooperation payoff, let evaluate it to R points;
• They both defect and then get both the selfish Punishment payoff, let it be P points;
• One chooses to defect while the other chooses to cooperate, then the one who has defected gets the Temptation payoff, let it be T points, and the one who has cooperated gets the Sucker's score, let it be S points.

The classical choice of value for payoff is (row player payoff are given first):

	Cooperate	Defect
Cooperate	R = 3 R = 3	S = 0 T = 5
Defect	T = 5 S = 0	P = 1 P = 1

To have a dilemma, temptation must pay more than cooperation, which must pay more than punishment, which must be better than to be the sucker. This is formalised by:

T > R > P > S

Since the one-shot version of the Prisoner's Dilemma is not very interesting (the most rational choice is to defect), the game is repeated an unknown number of times.
The game is said iterated.
The final score of a payer is the sum of all its moves score.
Since none player knows when the game will be ended, it is possible to study each agent's strategy, to look, for instance, how each player tries to put cooperation in the game.

In order to favour cooperation, i.e. common interest at the expens of selfish interest, this inequation is respected:

2 R > T + S

With this restriction, strategies have no advantage in alternatively cooperate and defect. To study the behavior of strategies, two kinds of computation can be done.

• The first one is a simple round-robin tournament, in which each strategy meets all other strategies. Its final score is then the sum of all scores done in each confrontation. At the end, the strategy's strength measurement is given by its rank in the tournament.
• The second one is a simulated ecological evolution, in which at the beginning there is a fixed population including the same quantity of each strategy. A round-robin tournament is made and then the population of bad strategies is decreased whereas good strategies obtain new elements. The simulation is repeated until the population has been stabilised (the population does not change anymore). This is this way that nasty strategies, those who take the initiative of the first defection, have been discovered to be not very stable, because they are invaded by kind ones.

The basic strategies available in java-prison

all_c

Always cooperates. [c]*

all_d

Always defects. [d]*

tit_for_tat

The tit_for_tat strategy was introduced by Anatole Rapoport. It begins to cooperate, and then play what its opponent played in the last move.

spiteful

It cooperates until the opponent has defected, after that move it always defects.

soft_majo

Plays opponent's majority move, if equal then cooperates. First move is considered to be equality.

per_ddc

Plays periodically : [d,d,c]*

per_ccd

Plays periodically : [c,c,d]*

mistrust

Defects, then plays opponent's move.

per_cd

Plays periodically [c,d].

pavlov

The win-stay/lose-shift strategy was introduced by Martin Nowak and Karl Sigmund. It cooperates if and only if both players opted for the same choice in the previous move.

tf2t

Cooperates except if opponent has defected two consecutive times.

hard_tft

Cooperates except if opponent has defected at least one time in the two previous move.

slow_tft

Plays [c,c], then if opponent plays two consecutive time the same move plays its move.

hard_majo

Plays opponent's majority move, if equal then defects. First move is considered to be equality.

random

Cooperates with probability 1/2.

The java prison libray

Installation

1. Unpack the archive ;
2. Add the prison.jar file to your CLASSPATH environment variable, see your Java Virtal Machine (probably the Sun Java Development Kit's one) documentation. For instance on UNIX under c-shell, suppose you have moved the prison.jar file into the /home/foo directory, then you will just have to say :
   

   setenv CLASSPATH ${CLASSPATH}:/home/foo/prison.jar

Description and use

fr.lifl.prison

In this package are defined the main classes used to make the simulations of Iterated Prisoner's Dilemma games.

fr.lifl.prison.strategies

In this package are defined some classes, extending a class of the previous package called Player, in order to define some generic kind of players (strategies), as well as a class implementing an array of some basic strategies, which were described in the first section of this document.

fr.lifl.prison.util

In this package are defined some useful general utility classes which may help in making the results of Iterated Prisoner's Dilemma simulations, among other things, more human readable. It includes graphic representation of data sets, as well as matrix manipulations and quicksort algorithm. A special class called Util is used to make some links between classes of this package and classes of the fr.lifl.prison one.

The full API documentation of those packages and their classes is available in the doc directory.

Three applets demonstrating the use of java-prison, and their java sources are available in the applets directory. They are demonstrating the simulation of:

1. A match between two strategies in the applets/match directory;
2. A tournament between some user choosen strategies, in the applets/tournament directory;
3. An ecological evolution involving user defined population of user choosen strategies, in the applets/evolution directory.

In order to demonstrate the localization capabilities of the java-prison library on each one of those three previous applets, french prepared html file calling the previous three applets are also available:

1. applets/match/index.fr.html
2. applets/tournament/index.fr.html
3. applets/evolution/index.fr.html

Authors and contacts

The PRISON project is a project from the SMAC team of the LIFL at the University of Science and Technology of Lille, FRANCE.

The main interest of the SMAC team is the study of Multi Agents Systems and Cooperation. The PRISON project is the cooperation side study of the project, but uses some Multi Agents technics too. The main goal of the project being the use of computer science tools in order to collect informations on the way cooperation between autonomous agents could arise and be maintaned.

Some of the keywords of the project are : (Iterated) Prisoner's Dilemma, Cooperation, Artificial Life, Evolution, Game Theory, Multi-Agents Systems

You can contact us for any bug reports, improvement ideas, informations request, comments, if you need help in using the java-prison classes, or if you are interested by our work on the Multi Agent Systems and Cooperation. Here are the way to contact us:

• You can read the PRISON project world wide web site at : http://www.lifl.fr/IPD
  On this site you will find a presentation of the Classical Iterated Prisoner's Dilemma (CIPD), SMAC published papers as well as a bibliography about the prisoner's dilemma, simulation softwares, available for downloading as well as for online testing, and some links of close interest.
• You can join us by e-mail at : prison@lifl.fr
• We could be joined by snail mail at this address :
  

  Philippe Mathieu - PRISON project headmaster Laboratoire d'Informatique Fondamentale de Lille Université des Sciences et Technologies de Lille UFR IEEA - Bât M3 59655 Villeneuve d'Ascq Cedex FRANCE

• We could also be joined by phone or fax at those numbers :
  

  Phone	Fax
  P. Mathieu : +33 3 20 43 45 04 B. Beaufils : +33 3 20 43 69 02	+33 3 20 43 65 66

Sponsors

• Le Laboratoire d'Informatique Fondamentale de Lille
• L'Université des Sciences et Technologies de Lille
• Le Centre National de la Recherche Scientifique
• Le contrat plan état/région Ganymede