Python实现囚徒困境：Tit for Tat未达最优的问题排查

问题背景 #

用Python实现囚徒困境锦标赛后，发现背叛类策略（如always_defect、random_choice_defect）得分最高，而公认的Tit for Tat策略表现不如预期，与Veritasium视频中的结论不符，需要排查代码问题。

代码与运行结果 #

实现代码 #

import random
from colorama import Fore, Style
import numpy as np
# Define the actions
COOPERATE = 'cooperate'
DEFECT = 'defect'

# Define the strategies
def always_cooperate(history):
    return COOPERATE

def always_defect(history):
    return DEFECT

def random_choice_cooperate(history):
    return COOPERATE if random.random() < 0.75 else DEFECT

def random_choice_defect(history):
    return COOPERATE if random.random() < 0.25 else DEFECT

def random_choice_neutral(history):
    return COOPERATE if random.random() < 0.5 else DEFECT

def tit_for_tat(history):
    if not history:  # If it's the first round, cooperate
        return COOPERATE
    opponent_last_move = history[-1][1]  # Get the opponent's last move
    return opponent_last_move  # Mimic the opponent's last move

def tat_for_tit(history):
    if not history:  # If it's the first round, cooperate
        return DEFECT
    opponent_last_move = history[-1][1]  # Get the opponent's last move
    return opponent_last_move  # Mimic the opponent's last move

def tit_for_two_tats(history):
    if len(history) < 2:  # If it's the first or second round, cooperate
        return COOPERATE
    opponent_last_two_moves = history[-2:]  # Get the opponent's last two moves
    if all(move[1] == DEFECT for move in opponent_last_two_moves):  # If the opponent defected in the last two rounds
        return DEFECT
    return COOPERATE

# Define the payoff matrix
payoff_matrix = {
    (COOPERATE, COOPERATE): (3, 3),
    (COOPERATE, DEFECT): (0, 5),
    (DEFECT, COOPERATE): (5, 0),
    (DEFECT, DEFECT): (1, 1)
}


# Define the players
players = [always_cooperate, always_defect, random_choice_defect, tit_for_tat, tit_for_two_tats, random_choice_cooperate, tat_for_tit, random_choice_neutral]

# Assign a unique color to each player
player_colors = {
    'always_cooperate': Fore.GREEN,
    'always_defect': Fore.RED,
    'tit_for_tat': Fore.BLUE,
    'random_choice_cooperate': Fore.MAGENTA,
    'random_choice_defect': Fore.LIGHTRED_EX,
    'tat_for_tit': Fore.LIGHTYELLOW_EX,
    'random_choice_neutral': Fore.WHITE,
    'tit_for_two_tats': Fore.LIGHTBLACK_EX,
}


def tournament(players, rounds=100):
    total_scores = {player.__name__: 0 for player in players}
    for i in range(len(players)):
        for j in range(i+1, len(players)):
            player1 = players[i]
            player2 = players[j]
            history1 = []
            history2 = []
            match_scores = {player1.__name__: 0, player2.__name__: 0}
            # print(f"\n{player1.__name__} vs {player2.__name__}")
            for round in range(rounds):
                move1 = player1(history1)
                move2 = player2(history2)
                score1, score2 = payoff_matrix[(move1, move2)]
                match_scores[player1.__name__] += score1
                match_scores[player2.__name__] += score2
                total_scores[player1.__name__] += score1
                total_scores[player2.__name__] += score2
                history1.append((move1, move2))
                history2.append((move2, move1))
            # print(f"{player1.__name__} moves: {''.join([Fore.GREEN+'O'+Style.RESET_ALL if move[0]==COOPERATE else Fore.RED+'X'+Style.RESET_ALL for move in history1])}")
            # print(f"{player2.__name__} moves: {''.join([Fore.GREEN+'O'+Style.RESET_ALL if move[0]==COOPERATE else Fore.RED+'X'+Style.RESET_ALL for move in history2])}")
            # print(f"Match scores: {player1.__name__} {match_scores[player1.__name__]}, {player2.__name__} {match_scores[player2.__name__]}")  
    sorted_scores = sorted(total_scores.items(), key=lambda item: item[1], reverse=True)
    return sorted_scores

# Run the tournament
# for player, score in tournament(players):
#     print(f'\nFinal score: {player}: {score}')

num_tournaments = 1000
results = {player.__name__: [] for player in players}

for _ in range(num_tournaments):
    for player, score in tournament(players):
        results[player].append(score)

# Calculate the median score for each player and store them in a list of tuples
medians = [(player, np.median(scores)) for player, scores in results.items()]

# Sort the list of tuples based on the median score
sorted_medians = sorted(medians, key=lambda x: x[1])

num_players = len(sorted_medians)

# Print the sorted median scores with gradient color
for i, (player, median_score) in enumerate(sorted_medians):
    # Calculate the ratio of green and red based on the player's position
    green_ratio = i / (num_players - 1)
    red_ratio = 1 - green_ratio

    # Calculate the green and red components of the color
    green = int(green_ratio * 255)
    red = int(red_ratio * 255)

    # Create the color code
    color_code = f'\033[38;2;{red};{green};0m'
    player_color = player_colors.get(player, Fore.RESET)
    # Print the player name and median score with the color
    print(f'{player_color}{player}: {median_score} coins')

运行结果 #

always_cooperate: 1347.024 coins
random_choice_cooperate: 1535.651 coins
tit_for_two_tats: 1561.442 coins
tit_for_tat: 1609.444 coins
tat_for_tit: 1619.43 coins
random_choice_neutral: 1663.855 coins
always_defect: 1711.764 coins
random_choice_defect: 1726.992 coins

问题原因分析 #

1. 锦标赛对战规则不符合原版设定
   原版Axelrod锦标赛中，每个策略会与**所有策略（包括自身）**对战，但当前代码仅让策略两两配对对战一次（j从i+1开始），跳过了策略与自身的对战，也没有让策略和每个对手双向对战（比如A vs B和B vs A）。Tit for Tat与自身对战时会全程合作，每轮拿3分，这部分得分缺失会大幅拉低其总排名。

2. 策略池背叛倾向策略占比过高
   当前策略池中，背叛倾向的策略（always_defect、random_choice_defect、tat_for_tit、random_choice_neutral）占比达50%，而合作型策略仅4个。在背叛策略密集的环境中，Tit for Tat会被频繁背叛，只能在与合作策略对战时拿到高分，而背叛策略能从所有合作策略身上获取最高单轮分（5分），自然总分更高。

3. tat_for_tit策略存在注释与代码不一致的错误
   该策略注释标注“首回合合作”，但代码实际返回DEFECT，属于首步背叛的恶意策略，会触发更多背叛循环，进一步压缩Tit for Tat的生存空间。

修复方案 #

1. 修正锦标赛对战逻辑 #

修改tournament函数的循环，让每个策略与所有策略（包括自身）对战，还原原版锦标赛规则：

def tournament(players, rounds=100):
    total_scores = {player.__name__: 0 for player in players}
    # 每个策略和所有策略（含自身）对战
    for i in range(len(players)):
        for j in range(len(players)):
            player1 = players[i]
            player2 = players[j]
            history1 = []
            history2 = []
            match_scores = {player1.__name__: 0, player2.__name__: 0}
            for round in range(rounds):
                move1 = player1(history1)
                move2 = player2(history2)
                score1, score2 = payoff_matrix[(move1, move2)]
                match_scores[player1.__name__] += score1
                match_scores[player2.__name__] += score2
                total_scores[player1.__name__] += score1
                total_scores[player2.__name__] += score2
                history1.append((move1, move2))
                history2.append((move2, move1))
    sorted_scores = sorted(total_scores.items(), key=lambda item: item[1], reverse=True)
    return sorted_scores

2. 修正tat_for_tit策略的注释或代码 #

如果是想实现“首步合作的反向Tit for Tat”，修正代码：

def tat_for_tit(history):
    if not history:  # If it's the first round, cooperate
        return COOPERATE
    opponent_last_move = history[-1][1]
    # 反向模仿：对手合作则背叛，对手背叛则合作
    return DEFECT if opponent_last_move == COOPERATE else COOPERATE

如果确实是首步背叛的模仿策略，修正注释：

def tat_for_tit(history):
    if not history:  # If it's the first round, defect
        return DEFECT
    opponent_last_move = history[-1][1]  # Get the opponent's last move
    return opponent_last_move  # Mimic the opponent's last move

3. 调整策略池组成 #

增加更多合作倾向的策略（如grim_trigger：首次被背叛后永久背叛），或调整现有策略比例，让策略池更贴近原版锦标赛的多样性，减少背叛策略的占比。

修复后效果 #

修改对战逻辑后，Tit for Tat会获得与自身对战的满分加成，同时在与合作策略对战时保持全合作得分，与背叛策略对战时仅在首回合吃亏，后续以背叛回应，总分会显著提升，大概率成为排名前列的策略。

内容的提问来源于stack exchange，提问作者danielassayag