I want to present a new frame here of "selection games." I think this will help inform all the talk of surrogation and in-bounds vs out-of-bounds play. We'll build up from the most simple kind of selection game—an agent selecting a non-agentic object—and then scale up to more complex institutional interactions.
John is selecting among rocks to build a wall. There are certain criteria he uses to make his selection, criteria which involve the rock’s apparent shape, size, and kind. The rock has no perception of John’s selection, nor any interest in whether or not it is selected. It is unable to change itself or its appearance in any way to increase or decrease its chances of selection, even if it were aware and interested in the selection process. There is only a static, one-way perceptual relationship between John and the rock.
Another, similar example, with a dashed arrow added to signify the one-way perceptual relationship. A tree has some perceptual awareness of whether it has been selected (via being chopped down), and it has an obvious interest in not being chopped down, but it cannot adaptively alter its appearance to John in order to pre-emptively escape his selection. If John is searching for maple trees, his objects of selection cannot feign being oak trees.
However, though an individual maple tree lacks the intelligence and bodily agency to adaptively alter its appearance, maple trees in general are governed by evolutionary dynamics, which in the long-term simulate intelligent response. John inevitably will use certain markers of mapledom, for instance, the shape of a tree’s leaves, or the patterns on its bark. Maple trees that have irregular leave shapes will out-survive maple trees with regular leaf shapes, and over sufficient generations, John will be faced with a forest of maples which has functionally countered his selection criteria, and no longer possess the marker or indicator he used for identifying the best trees for his fence. At that point, the relationship can be diagrammed as…
…where the solid gray arrow signifies selection-countering behavior by the tree. Of course, while this is a situation in which (to anthropomorphize evolution) the tree is “trying” to avoid John’s selection, many selection games involve trying to be selected (e.g. for some preferential treatment, as in mating, preservation, workplace promotion, etc). To say that these moves are “selection-countering” is to illustrate that they are adversarial—that the tree and John have different interests. John, as the selecting party, is (broadly) always interested in discerning the pragmatic truth about his object of selection. He wishes to identify all trees that will work well with his fence-building project, and to avoid wasting time chopping down trees that will work poorly. His interest lies in clarity of vision. The maple tree’s interest, on the other hand—again, to anthropomorphize the evolutionary process, which has no desire per se—is to avoid being chopped down.
We can see this very clearly when we get to inter-agentic selection games, where the object of selection simultaneously 1) has a stake in being selected, and 2) is able to alter its probability of being selected.
Here, the lion assesses John in a selection game for lunch. He’ll use certain perceptual markers, such as John’s size, physical distance, and gait as proxies to the caloric value of John as a meal, whether John can be easily caught, whether John is healthy, etc. If John is malnourished or injured, it will behoove him to expend energy toward creating the opposite impression—putting on a show of vitality, or feigning aggression, rather than lying weakly in the grasses. John, as a full-bodied agent, is able—unlike the rock or maple tree—to fully play out his side of the selection game: he is perceptually aware of being engaged in a selection game; his is able, roughly, to model the criteria by which selections are made; and he is able to adaptively alter the probability of his being selected or not selected. And, of course, he has an active and non-trivial stake in whether he is selected.
The above diagram, however, is a naive model of how selection games play out. It neglects certain key dynamics: the lion has some ostensible awareness that John does not wish to be selected, and that John can pick up on the selection game; it may not approach John directly but instead sneak up on his flank, precluding John’s ability to counter his selection by preventing John’s awareness of being engaged in a selection game in the first place. In other words, just as an agent who is the object of a selection game is incentivized to strategically alter his appearance to the selector, the selector is incentivized to strategically alter his appearance to the object. Thus:
When these games meet the human social world, they quickly become complex. Frequently, they are not two-player but three-player: whereas, in the examples of the maple tree or the lion, there is a clear, automatically allocated, intrinsic payoff as the result of the selection game—the lion gains a meal; John gains or loses his life—in human social games, a judging party makes a subjective judgment as to the game’s winner, and the allocation of payoffs is the result of social custom rather than “natural” or physical fact. And in these scenarios, naive models of perception and judgment, which fail to acknowledge the recursive, adversarial nature of selection games, fall short of adequately modeling their dynamics.
Here is a naive model of how basketball is played:
The players mutually model one another and strategically preempt one anothers’ moves. If the offensive player in possession of the ball appears to be moving one direction toward the basket, the defending player will attempt to cut off his path. Knowing this, the offensive player may feign left, in order to manipulate his opponent into moving left preemptively, thus clearing a rightward path toward the basket. The defensive player takes this possibility of deception into account by watching the offensive player’s hips (instead of his gaze, shoulders, or ball) for cues to his future path, since there is an inherent physical connection between one’s ability to move and the position of one’s hips which is not as_ _true of gaze or shoulder position. Meanwhile, a referee observes both players’ actions to determine their legality.
Why this naive model is wrong should be immediately obvious given what we know about the players. They have an active stake in whether a referee interprets (or “selects”) their action as legal or illegal. They have a roughly accurate model of the bases on which selection decisions are made, and the ability to alter the probability of being selected as the fouling or fouled party. Thus, the real game can at least be modeled as:
Here, players strategically self-represent in order to improve the selection odds in their own favor. When an opponent knocks into them, they may exaggerate the contact, falling onto the court or flailing wildly, a practice known as flopping. When they initiate contact with a player, they may attempt to hide it from the referee’s line of sight. We see similar dynamics of strategic self-representation to third-party judges across social selection games, from job applications and college admissions to dating, draft-dodging, and law-breaking.
Insofar as we can simplify the goal of the referee, or the hiring board, or a military doctor, as access to truth—a “clear vision” or accurate assessment relative to their own pragmatic goals, for instance identifying civilians who are mentally and physically sound enough for military duty—then we have to understand the objects of their assessment as constantly seeking to subvert the assessment procedure, either by obscuring vision, or preventing false impressions, etc.
Crucially, this requires a model of the basis on which selection decisions are made. Typically, there are “public” appearances and “private” realities. Whether a recruit is mentally sound, or a candidate fundamentally honest; whether a bright red snake is in fact poisonous; whether a used car is a lemon—these cannot be known with certainty in advance, if at all. Instead, there are certain public markers or measurements—such as John’s gait, to the lion, or a recruit’s heart rate, to the military doctor, which will suggest the more inaccessible—but more important—private quality of “health.” Knowledge of these measures or markers, and the way they are interpreted by the selecting party, is critical in selection objects’ ability to influence selection decisions.
If an evaluator has an active stake in the accuracy of his choice—which may, for instance, be more true of a manager hiring for his own team than a hiring process outsourced to an independent contractor, or a sports referee with tenure—then, as in the case of John and the lion, the selection game becomes properly, mutually adversarial. The savvy evaluator, rather than subscribing to some naive belief in neutral perception, actively disguises or misrepresent his own selection games in order to subvert the evaluated objects’ own attempts at manipulating the selection process:
Unfortunately, a desire—especially among bureaucracies—for decision-making to be ritualized and routine, rather than dynamic, recursive, and adaptive, leads to a simplistic, static attitude toward assessment. Evaluative markers and metrics tend to be generically, naively, and transparently implemented. (From here on out, I’ll use “metric” to mean any measurement which is used as the basis of selection decisions.) This makes them “gameable.” The gaming of metrics is sometimes referred to as “Goodhart’s Law”—that any measure which becomes a target ceases to be a good measure. And yet, it is precisely our lack of understanding of selection games which leads us to see this dynamic as a conditional law—_if _a measure becomes a target—rather than an obvious functioning of metrics. That is, any measurement that exerts selection pressure—that is a metric—will be undermined or adversarially manipulated by the objects of selection it seeks to evaluate. This is true even in situations where the objects of selection lack awareness of, and ability to manipulate, the selection game—so long as evolution is present to “counter” metrics and markers at the population level.
- 1 replies
Part 2: scaling selection up to institutions
To illustrate these selection dynamics, we can take a look at the opening scene of The Wire’s Season 5. Detectives bring in two suspects for questioning; one is suspected of murder, the other of being an accomplice, or at least witness, to the killing:
The basic structure of the selection game is thus: The suspects wish to escape a legal conviction, and preferably, a court appearance. The detectives, roughly, want to identify and earn a confession from the true killer. (This is a problematic summarization of the detectives’ motives, as we’ll see shortly.)
The detectives first tell the suspected killer that his friend and accomplice is cooperating with the investigation, that they’ve provided him with a full McDonald’s lunch he’s been so helpful. They then parade the friend past the interrogation room with a happy meal in-hand—the friend, of course, has no idea he is being used as a ploy, and is confused why the detectives have been so friendly with the food. But the detectives are attempting to strategically mis-represent the game state in order to provoke a confession, and using the McDonald’s meal as a confirming metonym to reinforce their (mis)representation. Regardless of whether or not he committed the murder, the suspect’s interest is in self-representing himself as innocent, or not worth pursuing legally—that no evidence can stick to him. The detectives, meanwhile, are trying to manipulate his assessment of the situation so that he commits a game-forfeiting blunder. The situation is not so different from John, up a tree after being chased by a lion, seeing the lion wander off. He now believes the coast is clear, and comes down; meanwhile, the lion has snuck around back, and pounces. Actions are always based in perceptions, and by manipulating perceptions, one can manipulate opponent behavior to the opponent’s disadvantage (and the manipulator’s advantage).
Next, the detectives bring the suspect into a room with a Xerox machine, and tell him that it is a lie detector. Again, they are manipulating the game state. A sergeant pretends to be a “professor” in charge of administering the lie detector, which is “never wrong.” The false identity of the sergeant works in part because the sergeant is wearing suspenders, which act as a metonym that again “confirms” his identity. The detectives then pull a scam on the suspect which has the Xerox print out the word “false” when the suspect is asked, and responds to the question as to, whether he committed the murder. At this point, convinced he is beaten, he breaks down and confesses. The real selection game they are playing here has been obfuscated for _another _sort of selection game. The suspect believes his confession is immaterial because the detectives have already proved his guilt, through the Xerox machine, and secured a confession from his accomplice. The countering strategy which would prevent his being “selected” for imprisonment—the silence which would prevent his being convicted sans confession or material evidence—is made to appear unavailable to him, so he will not use it to counter the detectives’ attempt to fish out “the truth.”
We can get now to the problematics of saying that the detectives simply wish to identify (or “select”) the actual murderer. In cases such as John and the lion, the game’s reward is allocated “automatically,” and is “intrinsic” to the game. No third party’s assessment mediates the competition between John and the lion, and the cost of being eaten, or of eating, is bound innately to the game being played. However, in most of human social and institutional life, payoffs are socially mediated. There may be some intrinsic reward, for a detective, to catching “the right guy”—a satisfaction which exists regardless of whether anyone else knows, regardless of whether he receives a financial bonus for his good detective work. But the larger incentive structure he is embedded within looks like socially mediated selection games up and down. The relationship between the detective and the suspect is much like the relationship between the detective’s supervising officer and the detective: each are involved in a layer of selection game in which they hope to “look good” with respect to the evaluation (be it competent, in the case of the detective, or innocent, in the case of the suspect). The same is true of the detective’s supervisor; even at top levels, the very most senior officials are still answerable to Congress, or to shareholders, or to the public. It is selection games all the way up and down. The detective desires promotions, bonuses, the respective of his colleagues, etc—and also to avoid being fired or prosecuted for his actions. And because the “truth” of his quality as a detective can never be known with certainty, only “testified” too via publicly available signs—just as the innocence or guilt of a suspect can never be known with certainty, lost to the unvisitable past—his is a game of appearances. The detectives who appear competent will (to simplify) be promoted just as the suspects who appear innocent will be let off. Non-natural selection games are ruled by appearances, because appearances are all that are available to the agents in charge of judgment.