During last Conference on Evolutionary Computation a competition, celebrating the 20th anniversary of Robert Axelrod The evolution of Cooperation book publication, has been organized by a team of people headed by Graham Kendall from University of Nottingham, UK.
Partial results of those competition have been made available through the web site of the competition.
As some other people having participated to this competition we have studied those results in order to be able to understand how they have been produced. We thus noticed that those results have absolutely no signification due to the way the competition have been produced.
One of the participants,
from University of Southampton, UK, have however pretented its team have
finally beated the
tit_for_tat strategy. Even more its team published
and began to spread those results.
A new competition based on approximatively the same rules and methods is planned to be done for another conference next year.
As others, we feel that it was important to react to such non scientific behavior, by replying to the call for particiption for the next competition with the mail reproduced here. If you participated to the first competition (at CEC'2004) and agree with us, do not hesitate to send us a note so that you could be added to the list of signatories.
Following a mail sent by Elpida Tzafestas just after the CEC'2004 Iterated Prisoner's Dilemma competition results publication and after some public announcement made those last week by Southampton University, relayed in web media such as Wired and Slashdot and some others blogs, we think it is very important for us to react in 3 short points about this competition: * the first prisoner's dilemma competition at CEC'2004 has been made without any scientific spirit * results made by those public announcements are definitively not new information and are known since a long time * new rules established for the new prisoner's dilemma competition does not fix problems introduced by the first competition Let us detail those 3 points : * The first competition has been made without any scientific approach or spirit. As it has been suggested by Elpida in its reaction no computation methods are normalised. Even more serious, no results may be accessible and thus reproducible before the end of 2005 while serious doubts have been emitted on this first competition results. Some participants have proposed more than one strategy, while for almost all other (and in any case the vast majority of participants) it was implicit that only one could be proposed by players. Some players have even proposed several times the SAME strategy, while it is well known that it completely changes the meaning of such a tournament (see for instance Axelrod's original book). Just for information, and using only given data (that is name of strategy, and of participant and obtained scores) : 26 strategies have been used more than once (23 used 2 times, 2 used 3 times, and one used 4 times) and approximatively 9 of the 43 participants proposed more than 1 strategy with the winner proposing more than 107 strategies on the 223 finally used. If people had only submitted one strategy this competition would have counted only 43 participants which is small. For instance, more than 10 years ago, we organised an open competition, in association with the french edition of the Scientific American, which involved more than 90 differents readers. * Results and their interpretation publicly announced by the University of Southampton which has been spread, and are still spreading, are definitively no news. Defeating tit_for_tat, or "beating it" as it is often written, in the classical iterated prisoner's dilemma has been done since a long time by different people (including us). The quality of tit_for_tat has been call into question in a lot of not so recent references (see for instance  and ). We even have studied the idea of master/slaves strategies, today presented as NEW result, for instance in . Almost the same quantity of references on the iterated prisoner's dilemma with noise may be found (with another difficulty in the definition of what noise should be). We can ask ourselves if people making those announcement and those spreading those *new results* have study a little bit the bibliography on the subject (see for instance here and here). * New rules established for the next competition do not fix problems introduced by the first competition : + only one competition is changed to accept only one strategy by participants, which means that almost nothing will changed in the results interpretation. + this is still a one-shot tournament. Results are not averaged on differents runs of simulations. We all know problems which may arised in such situation due for instance to random number generator uses. + there is still no normalisation in the computation of score, which means that all strategies are not treated in the same way as it is done in all serious game theory simulations. For all those reasons we symbolically refuse to participe to, and ask for the substraction of our proposition from, the CIG'05 prisoner's dilemma competition. We publish this statement on our web site and invite all people agreeing with it to co-sign it by sending us an email asking to do so. We are, however, ready to evaluate any strategies which will be send to us. We have published methods and implemented all the tools necessary to evaluate the *quality* of a strategy. Some of them are even available since a long time on our project web site http://www.lifl.fr/ipd. Pr Philippe MATHIEU Pr Jean-Paul DELAHAYE Dr Bruno BEAUFILS References :  "A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game" by Martin Nowak and Karl Sigmund in Nature volume 364 in 1993  "Our Meeting with Gradual, A Good Strategy for the Iterated Prisoner's Dilemma" by Bruno Beaufils, Jean-Paul Delahaye and Philippe Mathieu in the Proceedings of the Artificial Life V conference in 1996.  "L'altruisme perfectionné" by Jean-Paul Delahaye and Philippe Mathieu in Pour La Science (French Edition of Scientific American, volume 187 in 1993.