|This is a file in the archives of the Stanford Encyclopedia of Philosophy.|
Stanford Encyclopedia of Philosophy
Evolutionary game theory was first developed by R. A. Fisher [see The Genetic Theory of Natural Selection (1930)] in his attempt to explain the approximate equality of the sex ratio in mammals. The puzzle Fisher faced was this: why is it that the sex ratio is approximately equal in many species where the majority of males never mate? In these species, the non-mating males would seem to be excess baggage carried around by the rest of the population, having no real use. Fisher realized that if we measure individual fitness in terms of the expected number of grandchildren, then individual fitness depends on the distribution of males and females in the population. When there is a greater number of females in the population, males have a higher individual fitness; when there are more males in the population, females have a higher individual fitness. Fisher pointed out that, in such a situation, the evolutionary dynamics lead to the sex ratio becoming fixed at equal numbers of males and females. The fact that individual fitness depends upon the relative frequency of males and females in the population introduces a strategic element into evolutions.
Fisher's argument can be understood game theoretically, but he did not state it in those terms. In 1961, R. C. Lewontin made the first explicit application of game theory to evolutionary biology in "Evolution and the Theory of Games" (not to be confused with the Maynard Smith work of the same name). In 1972, Maynard Smith defined the concept of an evolutionarily stable strategy (hereafter ESS) in the article "Game Theory and the Evolution of Fighting." However, it was the publication of "The Logic of Animal Conflict," by Maynard Smith and Price in 1973 that introduced the concept of an ESS into widespread circulation. In 1982, Maynard Smith's seminal text Evolution and the Theory of Games appeared, followed shortly thereafter by Robert Axelrod's famous work The Evolution of Cooperation in 1984. Since then, there has been a veritable explosion of interest by economists and social scientists in evolutionary game theory (see the bibliography below).
There are two approaches to evolutionary game theory. The first approach derives from the work of Maynard Smith and Price and employs the concept of an evolutionarily stable strategy as the principal tool of analysis. The second approach constructs an explicit model of the process by which the frequency of strategies change in the population and studies properties of the evolutionary dynamics within that model.
As an example of the first approach, consider the problem of the Hawk-Dove game, analyzed by Maynard Smith and Price in "The Logic of Animal Conflict." In this game, two individuals compete for a resource of a fixed value V. (In biological contexts, the value V of the resource corresponds to an increase in the Darwinian fitness of the individual who obtains the resource; in a cultural context, the value V of the resource would need to be given an alternate interpretation more appropriate to the specific model at hand.) Each individual follows exactly one of two strategies described below:
Hawk Initiate aggressive behaviour, not stopping until injured or until one's opponent backs down. Dove Retreat immediately if one's opponent initiates aggressive behaviour.
If we assume that (1) whenever two individuals both initiate aggressive behaviour, conflict eventually results and the two individuals are equally likely to be injured, (2) the cost of the conflict reduces individual fitness by some constant value C, (3) when a Hawk meets a Dove, the Dove immediately retreats and the Hawk obtains the resource, and (4) when two Doves meet the resource is shared equally between them, the fitness payoffs for the Hawk-Dove game can be summarized according to the following matrix:
Hawk Dove Hawk ½(V - C) V Dove 0 V/2
Figure 1: The Hawk-Dove Game
(The payoffs listed in the matrix are for that of a player using the strategy in the appropriate row, playing against someone using the strategy in the appropriate column. For example, if you play the strategy Hawk against an opponent who plays the strategy Dove, your payoff is V; if you play the strategy Dove against an opponent who plays the strategy Hawk, your payoff is 0.)
In order for a strategy to be evolutionarily stable, it must have the property that if almost every member of the population follows it, no mutant (that is, an individual who adopts a novel strategy) can successfully invade. This idea can be given a precise characterization as follows: Let F(s1,s2) denote the change in fitness for an individual following strategy s1 against an opponent following strategy s2, and let F(s) denote the total fitness of an individual following strategy s; furthermore, suppose that each individual in the population has an initial fitness of F0. If is an evolutionarily stable strategy and a mutant attempting to invade the population, then
F() = F0 + (1-p)F(,) + pF(,)
F() = F0 + (1-p)F(,) + pF(,)
where p is the proportion of the population following the mutant strategy .
Since is evolutionarily stable, the fitness of an individual following must be greater than the fitness of an individual following (otherwise the mutant following would be able to invade), and so F() > F(). Now, as p is very close to 0, this requires that either that
F(,) > F(,)or that
F(,) = F(,) and F(,) > F(,)(This is the definition of an ESS that Maynard Smith and Price give.) In other words, what this means is that a strategy is an ESS if one of two conditions holds: (1) does better playing against than any mutant does playing against , or (2) some mutant does just as well playing against as , but does better playing against the mutant than the mutant does.
Given this characterization of an evolutionarily stable strategy, one can readily confirm that, for the Hawk-Dove game, the strategy Dove is not evolutionarily stable because a pure population of Doves can be invaded by a Hawk mutant. If the value V of the resource is greater than the cost C of injury (so that it is worth risking injury in order to obtain the resource), then the strategy Hawk is evolutionarily stable. In the case where the value of the resource is less than the cost of injury, there is no evolutionarily stable strategy if individuals are restricted to following pure strategies, although there is an evolutionarily stable strategy if players may use mixed strategies.
As an example of the second approach, consider the well-known Prisoner's Dilemma. In this game, individuals choose one of two strategies, typically called "Cooperate" and "Defect." Here is the general form of the payoff matrix for the prisoner's dilemma:
where T > R > P > S and T > R > P > S. (This form does not require that the payoffs for each player be symmetric, only that the proper ordering of the payoffs obtains.) In what follows, it will be assumed that the payoffs for the Prisoner's Dilemma are the same for everyone in the population.
Cooperate Defect Cooperate (R,R) (S,T) Defect (T,S) (P,P)
Figure 2: Payoff Matrix for the Prisoner's Dilemma.
Payoffs listed as (row, column).
How will a population of individuals that repeatedly plays the Prisoner's Dilemma evolve? We cannot answer that question without introducing a few assumptions concerning the nature of the population. First, let us assume that the population is quite large. In this case, we can represent the state of the population by simply keeping track of what proportion follow the strategies Cooperate and Defect. Let pc and pd denote these proportions. Furthermore, let us denote the average fitness of cooperators and defectors by WC and WD, respectively, and let denote the average fitness of the entire population. The values of WC, WD, and can be expressed in terms of the population proportions and payoff values as follows:
WD = F0 + pcF(C,C) + pdF(C,D)Second, let us assume that the proportion of the population following the strategies Cooperate and Defect in the next generation is related to the proportion of the population following the strategies Cooperate and Defect in the current generation according to the rule:
WD = F0 + pcF(D,C) + pdF(D,D)
= pcWC + pdWD
We can rewrite these expressions in the following form:
If we assume that the change in the strategy frequency from one generation to the next are small, these difference equations may be approximated by the differential equations:
These equations were offered by Taylor and Jonker (1978) and Zeeman (1979) to provide continuous dynamics for evolutionary game theory and are known as the replicator dynamics.
The replicator dynamics may be used to model a population of individuals playing the Prisoner's Dilemma. For the Prisoner's Dilemma, the expected fitness of Cooperating and Defecting are:
WC = F0 + pcF(C,C) + pdF(C,D) = F0 + pcR + pdS
Since T > R and P > S, it follows that WD > WC and hence WD > > WC. This means that
WD = F0 + pcF(D,C) + pdF(D,D) = F0 + pcT + pdP.
Since the strategy frequencies for Defect and Cooperate in the next generation are given by
respectively, we see that over time the proportion of the population choosing the strategy Cooperate eventually becomes extinct. Figure 3 illustrates one way of representing the replicator dynamical model of the prisoner's dilemma, known as a state-space diagram.
Figure 3: The Replicator Dynamical Model of the Prisoner's Dilemma
We interpret this diagram as follows: the leftmost point represents the state of the population where everyone defects, the rightmost point represents the state where everyone cooperates, and intermediate points represent states where some proportion of the population defects and the remainder cooperates. (One maps states of the population onto points in the diagram by mapping the state when N% of the population defects onto the point of the line N% of the way to the leftmost point.) Arrows on the line represent the evolutionary trajectory followed by the population over time. The open circle at the rightmost point indicates that the state where everybody cooperates is an unstable equilibrium, in the sense that if a small portion of the population deviates from the strategy Cooperate, then the evolutionary dynamics will drive the population away from that equilibrium. The solid circle at the leftmost point indicates that the state where everybody Defects is a stable equilibrium, in the sense that if a small portion of the population deviates from the strategy Defect, then the evolutionary dynamics will drive the population back to the original equilibrium state.
At this point, one may see little difference between the two approaches to evolutionary game theory. One can confirm that, for the Prisoner's Dilemma, the state where everybody defects is the only ESS. Since this state is the only stable equilibrium under the replicator dynamics, the two notions fit together quite neatly: the only stable equilibrium under the replicator dynamics occurs when everyone in the population follows the only ESS. In general, though, the relationship between ESSs and stable states of the replicator dynamics is more complex than this example suggests. Taylor and Jonker (1978), as well as Zeeman (1979), establish conditions under which one may infer the existence of a stable state under the replicator dynamics given an evolutionarily stable strategy. Roughly, if only two pure strategies exist, then given a (possibly mixed) evolutionarily stable strategy, the corresponding state of the population is a stable state under the replicator dynamics. (If the evolutionarily stable strategy is a mixed strategy S, the corresponding state of the population is the state in which the proportion of the population following the first strategy equals the probability assigned to the first strategy by S, and the remainder follow the second strategy.) However, this can fail to be true if more than two pure strategies exist.
The connection between ESSs and stable states under an evolutionary dynamical model is weakened further if we do not model the dynamics by the replicator dynamics. For example, suppose we use a local interaction model in which each individual plays the prisoner's dilemma with his or her neighbors. Nowak and May (1992, 1993), using a spatial model in which local interactions occur between individuals occupying neighboring nodes on a square lattice, show that stable population states for the prisoner's dilemma depend upon the specific form of the payoff matrix.
When the payoff matrix for the population has the values T = 2.8, R = 1.1, P = 0.1, and S = 0, the evolutionary dynamics of the local interaction model agree with those of the replicator dynamics, and lead to a state where each individual follows the strategy Defect--which is, as noted before, the only evolutionarily stable strategy in the prisoner's dilemma. The figure below illustrates how rapidly one such population converges to a state where everyone defects.
However, when the payoff matrix has values of T = 1.2, R = 1.1, P = 0.1, and S = 0, the evolutionary dynamics carry the population to a stable cycle oscillating between two states. In this cycle cooperators and defectors coexist, with some regions containing "blinkers" oscillating between defectors and cooperators (as seen in generation 19 and 20).
Generation 1 Generation 2 Generation 3 Generation 4 Generation 5 Generation 6
Figure 4: Prisoner's Dilemma: All Defect
[view a movie of this model]
Notice that with these particular settings of payoff values, the evolutionary dynamics of the local interaction model differ significantly from those of the replicator dynamics. Under these payoffs, the stable states have no corresponding analogue in either the replicator dynamics nor in the analysis of evolutionarily stable strategies.
Generation 1 Generation 2 Generation 19 Generation 20
Figure 5: Prisoner's Dilemma: Cooperate
[view a movie of this model]
A phenomenon of greater interest occurs when we choose payoff values of T = 1.61, R = 1.01, P = 0.01, and S = 0. Here, the dynamics of local interaction lead to a world constantly in flux: under these values regions occupied predominantly by Cooperators may be successfully invaded by Defectors, and regions occupied predominantly by Defectors may be successfully invaded by Cooperators. In this model, there is no "stable strategy" in the traditional dynamical sense.
Generation 1 Generation 3 Generation 5 Generation 7 Generation 9 Generation 11 Generation 13 Generation 15
Figure 6: Prisoner's Dilemma: Chaotic
[view a movie of this model]
These models demonstrate that, although numerous cases exist in which both approaches to evolutionary game theory arrive at the same conclusion regarding which strategies one would expect to find present in a population, there are enough differences in the outcomes of the two modes of analysis to justify the development of each program.
Yet a difficulty arises with the use of Nash equilibrium as a solution concept for games: if we restrict players to using pure strategies, not every game has a Nash equilbrium. The game "Matching Pennies" illustrates this problem.
Heads Tails Heads (0,1) (1,0) Tails (1,0) (0,1)
Figure 7: Payoff matrix for the game of Matching Pennies
(Row wins if the two coins do not match, whereas Column wins if the two coins match).
While it is true that every noncooperative game in which players may use mixed strategies has a Nash equilibrium, some have questioned the significance of this for real agents. If it seems appropriate to require rational agents to adopt only pure strategies (perhaps because the cost of implementing a mixed strategy runs too high), then the game theorist must admit that certain games lack solutions.
A more significant problem with invoking the Nash equilibrium as the appropriate solution concept arises because games exist which have multiple Nash equilibria (see Section 2.5, Solution Concepts and Equilibria, in the entry on game theory). When there are several different Nash equilibria, how is a rational agent to decide which of the several equilibria is the "right one" to settle upon? Attempts to resolve this problem have produced a number of possible refinements to the concept of a Nash equilibrium, each refinement having some intuitive purchase. Unfortunately, so many refinements of the notion of a Nash equilibrium have been developed that, in many games which have multiple Nash equilibria, each equilibrium could be justified by some refinement present in the literature. The problem has thus shifted from choosing among multiple Nash equilibria to choosing among the various refinements. Some (see Samuelson (1997), Evolutionary Games and Equilibrium Selection) hope that further development of evolutionary game theory can be of service in addressing this issue.
Numerous results from experimental economics have shown that these strong rationality assumptions do not describe the behavior of real human subjects. Humans are rarely (if ever) the hyperrational agents described by traditional game theory. For example, it is not uncommon for people, in experimental situations, to indicate that they prefer A to B, B to C, and C to A. These "failures of the transitivity of preference" would not occur if people had a well-defined consistent set of preferences. Furthermore, experiments with a class of games known as a "beauty pageant" show, quite dramatically, the failure of common knowledge assumptions typically invoked to solve games.6] Since evolutionary game theory successfully explains the predominance of certain behaviors of insects and animals, where strong rationality assumptions clearly fail, this suggests that rationality is not as central to game theoretic analyses as previously thought. The hope, then, is that evolutionary game theory may meet with greater success in describing and predicting the choices of human subjects, since it is better equipped to handle the appropriate weaker rationality assumptions.
We repeat most emphatically that our theory is thoroughly static. A dynamic theory would unquestionably be more complete and therefore preferable. But there is ample evidence from other branches of science that it is futile to try to build one as long as the static side is not thoroughly understood. (Von Neumann and Morgenstern, 1953, p. 44)The theory of evolution is a dynamical theory, and the second approach to evolutionary game theory sketched above explicitly models the dynamics present in interactions among individuals in the population. Since the traditional theory of games lacks an explicit treatment of the dynamics of rational deliberation, evolutionary game theory can be seen, in part, as filling an important lacuna of traditional game theory.
One may seek to capture some of the dynamics of the decision-making process in traditional game theory by modeling the game in its extensive form, rather than its normal form. However, for most games of reasonable complexity (and hence interest), the extensive form of the game quickly becomes unmanageable. Moreover, even in the extensive form of a game, traditional game theory represents an individual's strategy as a specification of what choice that individual would make at each information set in the game. A selection of strategy, then, corresponds to a selection, prior to game play, of what that individual will do at any possible stage of the game. This representation of strategy selection clearly presupposes hyperrational players and fails to represent the process by which one player observes his opponent's behavior, learns from these observations, and makes the best move in response to what he has learned (as one might expect, for there is no need to model learning in hyperrational individuals). The inability to model the dynamical element of game play in traditional game theory, and the extent to which evolutionary game theory naturally incorporates dynamical considerations, reveals an important virtue of evolutionary game theory.
As noted previously, evolutionary game theoretic models may often be given both a biological and a cultural evolutionary interpretation. In the biological interpretation, the numeric quantities which play a role analogous to "utility" in traditional game theory correspond to the fitness (typically Darwinian fitness) of individuals. How does one interpret "fitness" in the cultural evolutionary interpretation?
In many cases, fitness in cultural evolutionary interpretations of evolutionary game theoretic models directly measures some objective quantity of which it can be safely assumed that (1) individuals always want more rather than less and (2) interpersonal comparisons are meaningful. Depending on the particular problem modeled, money, slices of cake, or amount of land would be appropriate cultural evolutionary interpretations of fitness. Requiring that fitness in cultural evolutionary game theoretic models conform to this interpretative constraint severely limits the kinds of problems that one can address. A more useful cultural evolutionary framework would provide a more general theory which did not require that individual fitness be a linear (or strictly increasing) function of the amount of some real quantity, like amount of food.
In traditional game theory, a strategy's fitness was measured by the expected utility it had for the individual in question. Yet evolutionary game theory seeks to describe individuals of limited rationality (commonly known as "boundedly rational" individuals), and the utility theory employed in traditional game theory assumes highly rational individuals. Consequently, the utility theory used in traditional game theory cannot simply be carried over to evolutionary game theory. One must develop an alternate theory of utility/fitness, one compatible with the bounded rationality of individuals, that is sufficient to define a utility measure adequate for the application of evolutionary game theory to cultural evolution.
Another question facing evolutionary game theoretic explanations of social phenomena concerns the kind of explanation it seeks to give. Depending on the type of explanation it seeks to provide, are evolutionary game theoretic explanations of social phenomena irrelevant or mere vehicles for the promulgation of pre-existing values and biases? To understand this question, recognize that one must ask whether evolutionary game theoretic explanations target the etiology of the phenomenon in question, the persistence of the phenomenon, or various aspects of the normativity attached to the phenomenon. The latter two questions seem deeply connected, for population members typically enforce social behaviors and rules having normative force by sanctions placed on those failing to comply with the relevant norm; and the presence of sanctions, if suitably strong, explains the persistence of the norm. The question regarding a phenomenon's etiology, on the other hand, can be considered independent of the latter questions.
If one wishes to explain how some currently existing social phenomenon came to be, it is unclear why approaching it from the point of view of evolutionary game theory would be particularily illuminating. The etiology of any phenomenon is a unique historical event and, as such, can only be discovered empirically, relying on the work of sociologists, anthropologists, archaeologists, and the like. Although an evolutionary game theoretic model may exclude certain historical sequences as possible histories (since one may be able to show that the cultural evolutionary dynamics preclude one sequence from generating the phenomenon in question), it seems unlikely that an evolutionary game theoretic model would indicate a unique historical sequence suffices to bring about the phenomenon. An empirical inquiry would then still need to be conducted to rule out the extraneous historical sequences admitted by the model, which raises the question of what, if anything, was gained by the construction of an evolutionary game theoretic model in the intermediate stage. Moreover, even if an evolutionary game theoretic model indicated that a single historical sequence was capable of producing a given social phenomenon, there remains the important question of why we ought to take this result seriously. One may point out that since nearly any result can be produced by a model by suitable adjusting of the dynamics and initial conditions, all that the evolutionary game theorist has done is provide one such model. Additional work needs to be done to show that the underlying assumptions of the model (both the cultural evolutionary dynamics and the initial conditions) are empirically supported. Again, one may wonder what has been gained by the evolutionary model--would it not have been just as easy to determine the cultural dynamics and initial conditions beforehand, constructing the model afterwards? If so, it would seem that the contributions made by evolutionary game theory in this context simply are a proper part of the parent social science--sociology, anthropology, economics, and so on. If so, then there is nothing particular about evolutionary game theory employed in the explanation, and this means that, contrary to appearances, evolutionary game theory is really irrelevant to the given explanation.
If evolutionary game theoretic models do not explain the etiology of a social phenomenon, presumably they explain the persistence of the phenomenon or the normativity attached to it. Yet we rarely need an evolutionary game theoretic model to identify a particular social phenomenon as stable or persistent as that can be done by observation of present conditions and examination of the historical records; hence the charge of irrelevancy is raised again. Moreover, most of the evolutionary game theoretic models developed to date have provided the crudest approximations of the real cultural dynamics driving the social phenomenon in question. One may well wonder why, in these cases, we should take seriously the stability analysis given by the model; answering this question would require one engage in an empirical study as previously discussed, ultimately leading to the charge of irrelevance again.
This criticism seems less serious than the charge of irrelevancy.
Cultural evolutionary game theoretic explanations of norms need not
"smuggle in" normative claims in order to draw normative conclusions.
The theory already contains, in its core, a proper subtheory having
normative content--namely a theory of rational choice in which
boundedly rational agents act in order to maximize, as best as they
can, their own self-interest. One may challenge the suitability of
this as a foundation for the normative content of certain claims, but
this is a different criticism from the above charge. Although
cultural evolutionary game theoretic models do act as vehicles for
promulgating certain values, they wear those minimal value
commitments on their sleeve. Evolutionary explanations of social
norms have the virtue of making their value commitments explicit and
also of showing how other normative commitments (such as fair
division in certain bargaining situations, or cooperation in the
prisoner's dilemma) may be derived from the principled action of
boundedly rational, self-interested agents.
|J. McKenzie Alexander