Name: _________________________________________

## CSC 107       Fall 2001

### Lab 3: Prisoner's Dilemma

In this lab, you will perform numerous simulations of the repeated Prisoner's Dilemma. In addition to experimenting with basic strategies such as coop, defect, and titForTat, you will define your own strategies in the form of JavaScript functions. These functions can then be incorporated into an existing Web page to allow for experimentation and comparison with the other strategies.

### The Prisoner's Dilemma

Recall the basic rules of the repeated Prisoner's Dilemma, as described in Douglas Hofstadter's paper:

For each round of play, two players independently choose to cooperate or defect.
• if both players cooperate, then both receive 3 points
• if both players defect, then both receive 1 point
• if one cooperates and one defects, then the defector receives 5 points
The goal for each player is to accumulate as many points as they can.

In his paper, Hofstadter describes several basic strategies for playing the repeated Prisoner's Dilemma. The coop strategy always cooperates, the defect strategy always defects, while the randomness strategy cooperates or defects at random. The titForTat strategy cooperates in the first round, and then mimics whatever the opponent did in the previous round. It is interesting to note that the defect strategy will never "lose" to an opponent, meaning that it will never accumulate fewer points than any opponent. Similarly, coop and titForTat will never "win" against an opponent. However, the number of points accumulated in "winning" and "losing" may vary greatly.

EXERCISE 1:    For each of the following pairings, predict the outcome if the two strategies were to compete in a repeated Prisoner's Dilemma. In particular, how many points would each strategy accumulate (assuming 50 rounds)?

coop vs. coop defect vs. defect titForTat vs. titForTat coop vs. defect coop vs. titForTat defect vs. titForTat

Which of the strategies would tend to do best against randomness? Which would do worst? Explain.

EXERCISE 2:    The Web page www.creighton.edu/~csc107/Labs/dilemma.html contains JavaScript code for simulating the repeated Prisoner's Dilemma. You are able to select which strategies you want to compete, and then click on a button to see the simulation. The button labeled "Click for next round" will display one round at a time, while the button labeled "Click for complete simulation" will complete all 50 rounds.

### Prisoner's Dilemma Tournaments

What makes the Prisoner's Dilemma interesting is that locally optimal behavior is not necessarily globally optimal. Consider a round-robin tournament between defect and two copies of titForTat (assuming 50 rounds per contest). Even though defect outscores each opponent in head-to-head contests, it only earns 54 points in each, for a total of 108. When one copy of titForTat plays the other, however, they both earn 150 points. Adding this to the 49 points earned against defect, each copy of titForTat earns a total of 199 points. Thus, defect wins all of its battles, but loses the war!

A Prisoner's Dilemma tournament can be described nicely in ecological terms. The defect and coop strategies, because of their aggressive and passive natures, can be characterized as hawks and doves. The titForTat strategy, which is passive by nature but will become aggressive when provoked, may be characterized as a sparrow. According to this analogy, the fact that defect outscores titForTat in a head-to-head contest shows that a lone hawk dominates a lone sparrow. However, the tournament shows that two sparrows support each other and can fight off a lone hawk. Using this type of analogy, the Prisoner's Dilemma has been used by researchers as a way of explaining how cooperative behavior can evolve without consciousness. In fact, studies done on the hunting patterns of lions and the defensive behaviors of fish have supported the Prisoner's Dilemma model of cooperation.

The results of a round-robin tournament can be displayed neatly in a table. The above tournament between a hawk (defect) and two sparrows (titForTat) would appear as follows. The entries across a row contain the points earned by a strategy playing against each of the other strategies. For example, the hawk earns 54 points when it goes against sparrow1, so there is a 54 in the table at the row labeled hawk and column labeled sparrow1. Conversely, sparrow1 only earns 49 points against the hawk, so there is a 49 in the table at the row labeled sparrow1 and column labeled hawk.

 hawk sparrow1 sparrow2 TOTAL hawk 54 54 108 sparrow1 49 150 199 sparrow2 49 150 199

EXERCISE 3:    In the table below, show the results of a round-robin tournament between a hawk, a dove, and two sparrows. How does the addition of the dove affect the dynamics between the hawk and the sparrows compared to the example above?

 hawk dove sparrow1 sparrow2 TOTAL hawk dove sparrow1 sparrow2

EXERCISE 4:    In the table below, show the results of a round-robin tournament between a hawk, a dove, and three sparrows. How does the additional sparrow affect the results when compared to the previous exercise?

 hawk dove sparrow1 sparrow2 sparrow3 TOTAL hawk dove sparrow1 sparrow2 sparrow3

### Defining Strategies as Functions

The dilemma.html page includes numerous HTML elements for controlling and displaying the results of the repeated Prisoner's Dilemma. These include text boxes for displaying the histories (past moves) of the two strategies and their scores, as well as buttons for controlling the simulation. There are also select boxes that allow the user to select from the different strategies. A select box is defined using the tags <select> and </select>. Within those tags, the different options from which the user can select are listed using option tags. For example, a select box for picking a Prisoner's Dilemma strategy is as follows:

<select name="strategy1"> <option value="coop"> coop <option value="defect"> defect <option value="randomness"> randomness <option value="titForTat"> titForTat <option value="titFor2Tats"> titFor2Tats </select> The actual strategies are defined as JavaScript functions in a separate file named strategies.js, which is included by dilemma.html. These functions all have the same form: they have two inputs representing the histories of the simulation so far, and return either "C" (for cooperate) or "D" (for defect). The histories are represented as sequences of "C"s and "D"s corresponding to the past moves by each player. That is, the first parameter will contain the sequence of choices ("C"s and "D"s) made by that same player in the past, while the second parameter will be the sequence of choices made by the opponent. For example, the randomness strategy is implemented as the following function: function randomness(myHistory, yourHistory) // Assumes: myHistory and yourHistory represent the histories of the players // Returns: either "C" or "D", randomly chosen { return randomOneOf(["C", "D""]); } The actual simulation of the repeated Prisoner's Dilemma is performed by a function called repeatDilemma. This function is called when the user clicks on a button. At that time, the names of the strategies selected in the select boxes are accessed, and the corresponding functions are passed to the repeatDilemma function as arguments. The fact that functions can be passed as arguments to other functions just like any other piece of data is a very handy feature of JavaScript. In fact, JavaScript is said to have first-class functions since functions are treated the same as any other type of data.

Before continuing with the following exercises, make copies of both dilemma.html and strategies.js in your local directory. You can copy dilemma.html by loading the page and then selecting "Save As" under the "File" menu. To access the source for strategies.js, load the page www.creighton.edu/~csc107/Labs/strategies.html and then save the code as before.

EXERCISE 5:    The following function defines a new (somewhat bizarre) Prisoner's Dilemma strategy called reverseTat. The function does the opposite of titForTat mimicking the opponent's last move in reverse. If the opponent cooperated the last time, the reverseTat strategy will defect. If the opponent defected, then the strategy would cooperate. The function works by calling the titForTat function with the current histories, and then returns the opposite of what titForTat returned.

function reverseTat(myHistory, yourHistory) // Assumes: myHistory and yourHistory represent the histories of the players // Returns: the opposite of what the opponent did last { if (titForTat(myHistory, yourHistory) == "C") { return "D"; } else { return "C"; } }

What would the reverseTat strategy do in the first round? Why?

Cut-and-paste a copy of the reverseTat function into your copy of strategies.js. Then, add a new option to each of the select boxes in your dilemma.html page. Once you have done this, how this new strategy performs relative to titForTat. That is, for each of the following opponents, does reverseTat accumulate fewer or more points than titForTat would?

reverseTat vs. coop reverseTat vs. defect reverseTat vs. randomness

Consider a round-robin tournament such as the ones you conducted in Exercises 3 and 4. Would the presence of reverseTat strategies be able to ward off the hawk (defect) and protect the dove (coop). If so, how many copies of reverseTat would be needed. If not, why not?

EXERCISE 6:    Define a new Prisoner's Dilemma strategy called randomTat in your copy of strategies.js. The function should automatically defect 10% of the time. For the other 90% of the time, it should behave the same as titForTat (mimicking the opponent's move on the previous round).

Add a new option to each of the select boxes on the page, then use the page to determine how this new strategy performs relative to titForTat. That is, for each of the following opponents, does randomTat accumulate fewer or more points than titForTat would? Note: there is some variability due to the random component of the randomTat function.

randomTat vs. coop randomTat vs. defect randomTat vs. randomness

Consider a round-robin tournament such as the ones you conducted in Exercises 3 and 4. Would the presence of randomTat strategies be able to ward off the hawk (defect) and protect the dove (coop). If so, how many copies of randomTat would be needed. If not, why not? Again: there is some variability due to the random component of the randomTat function.

Hand in a printout of your modified strategies.js and dilemma.html documents, attached to these sheets.