如何优化discord.py命令以稳定处理千名以上服务器成员

我使用discord.py库为自有自营的Discord服务器开发了定制机器人，该服务器是美国某零售品牌的非官方社区，核心成员以品牌在职员工为主。服务器配置@U.S. Employee身份组做员工身份识别，通过校验的用户可访问员工专属频道。每年我们会统一发起在职状态核验，核验通过的用户会被授予@Verified - 2022身份组。

我开发了名为prune_unverified的管理命令，核心校验逻辑如下：

• 未持有@U.S. Employee或@Canada Employee身份组的用户，不做任何操作
• 持有上述任意员工身份组、同时持有@Verified - 2022身份组的用户，不做任何操作
• 持有上述任意员工身份组、但未持有@Verified - 2022身份组的用户，移除其所有现有身份组，替换为@Former Employee身份组

代码在小规模测试中可零报错运行，执行逻辑完全符合预期。但每年正式运行该命令时，符合「持有员工身份组但未完成验证、需调整为前员工身份组」条件的成员规模超过1000人：命令启动后处理不到200名成员就会中止运行，不会按设计发送执行完成的回复消息，但机器人进程本身不会崩溃，仅该命令功能失效，重试执行、重启机器人均无法恢复该功能，机器人其余功能可全程正常运行。

部署环境信息：

• 机器人源码托管在私有GitHub仓库，关联开启自动部署、运行于heroku-20栈的Heroku应用
• 当前应用slug大小为62.9MiB（总配额为500MiB），使用售价7美元/实例/月的hobby dyno运行

原始问题代码 #

@bot.command(name = "prune_unverified", help = "prune the unverified employees", enabled = True)
@commands.has_role("Owner")
async def prune_unverified(context):
    await context.message.delete()
    guild = discord.utils.get(bot.guilds, name = GUILD)

    ServerMembers     = [i for i in guild.members]
    VerifiedEmployees = []
    PrunedUsers       = []

    EmployeeRoleUS = guild.get_role(EMPLOYEE_ROLE)
    EmployeeRoleCA = guild.get_role(EMPLOYEE_CA_ROLE)
    VerifiedRole   = guild.get_role(EMPLOYEE_VERIFIED_ROLE)

    for user in ServerMembers:
        if (EmployeeRoleUS or EmployeeRoleCA) in user.roles:
            if VerifiedRole in user.roles:
                VerifiedEmployees.append(user)
            else:
                PrunedUsers.append(user)
        else:
            continue

    for user in PrunedUsers:
        await guild.get_member(user.id).edit(roles = []) # Remove all roles
        await guild.get_member(user.id).add_roles(guild.get_role(EMPLOYEE_FORMER_ROLE)) # Award the former role
        await asyncio.sleep(15)
        # pass # Adding a pass here because we are just testing

    # create a .csv file of pruned users
    with open("pruned_users.csv", mode = "w") as pu_file:
        pu_writer = csv.writer(pu_file, delimiter = ",")
        pu_writer.writerow(["Server Nickname", "Username", "ID"])
        for user in PrunedUsers:
            pu_writer.writerow([f"{user.nick}", f"{user.name}#{user.discriminator}", f"{user.id}"])

    # create a .csv file of verified users
    with open("verified_users.csv", mode = "w") as vu_file:
        vu_writer = csv.writer(vu_file, delimiter = ",")
        vu_writer.writerow(["Server Nickname", "Username", "ID"])
        for user in VerifiedEmployees:
            vu_writer.writerow([f"{user.nick}", f"{user.name}#{user.discriminator}", f"{user.id}"])

    embed = discord.Embed(
        description = f":crossed_swords: **`{len(PrunedUsers)}` users were pruned by <@{context.author.id}>.**"
                      + f"\n:shield: **There are `{len(VerifiedEmployees)}` users who completed re-verification.**"
                      + f"\n\n**Here is a `.csv` file of all the users that were pruned:**",
        color = discord.Color.blue()
    )
    await context.send(embed = embed)
    await context.send(file = discord.File(r"pruned_users.csv")) # upload the .csv file we created
    await context.send(file = discord.File(r"verified_users.csv"))
    await log_command(context, "prune_unverified")

原代码存在3个核心问题，共同导致大规模执行时中断：

1. 角色判断逻辑错误：原写法(EmployeeRoleUS or EmployeeRoleCA) in user.roles存在Python语法逻辑漏洞，or运算会直接返回第一个非空对象，等价于仅判断用户是否持有美国员工角色，加拿大员工角色的校验完全失效
2. 成员数据不全：直接读取guild.members获取的是discord.py本地缓存的成员列表，若未开启特权成员意图，大服务器下缓存的成员数量远低于实际总人数，遍历过程中遇到缓存缺失的对象时会触发静默异常中断当前协程，不会导致整个机器人进程崩溃
3. 执行超时：原逻辑每处理1个用户就强制休眠15秒，处理1000个用户仅休眠时长累计就超过4小时，远超Heroku hobby dyno的执行阈值，会被平台强制中断当前协程执行，与故障现象完全吻合

更新时间：2022年6月26日周日 美国东部时间约下午4:13
调整代码逻辑后，命令总执行时长约24分钟，全程运行无异常。
核心修复点：

• 修正角色判断逻辑，分别校验两个员工角色的持有状态
• 改用async for user in context.guild.fetch_members(limit=None)主动通过API拉取全量成员数据，不依赖本地缓存
• 移除冗余的guild.get_member(user.id)重复调用，直接使用遍历得到的成员对象执行操作
• 删除不必要的15秒强制休眠，依赖discord.py内置的API限速机制控制请求频率，大幅提升执行效率
• 直接从context.guild获取当前服务器对象，不需要通过bot.guilds遍历查找，减少冗余操作

修复后代码：

@bot.command(name = "prune_unverified", help = "prune the unverified employees", enabled = True)
@commands.has_role("Owner")
async def prune_unverified(context):
    await context.message.delete()

    VerifiedEmployees = []
    PrunedUsers       = []

    EmployeeRoleUS = context.guild.get_role(EMPLOYEE_ROLE)
    EmployeeRoleCA = context.guild.get_role(EMPLOYEE_CA_ROLE)
    VerifiedRole   = context.guild.get_role(EMPLOYEE_VERIFIED_ROLE)

    # Begin new code here
    async for user in context.guild.fetch_members(limit = None):
        if (EmployeeRoleUS in user.roles) or (EmployeeRoleCA in user.roles):
            if VerifiedRole in user.roles:
                VerifiedEmployees.append(user)
            else:
                PrunedUsers.append(user)
        else:
            continue
    # End new code here

    for user in PrunedUsers:
        #await user.edit(roles = []) # Remove all roles (Commented out for testing)
        await user.add_roles(context.guild.get_role(990703166041501806)) # Award the former role

    # create a .csv file of pruned users
    with open("pruned_users.csv", mode = "w") as pu_file:
        pu_writer = csv.writer(pu_file, delimiter = ",")
        pu_writer.writerow(["Server Nickname", "Username", "ID"])
        for user in PrunedUsers:
            pu_writer.writerow([f"{user.nick}", f"{user.name}#{user.discriminator}", f"{user.id}"])

    # create a .csv file of verified users
    with open("verified_users.csv", mode = "w") as vu_file:
        vu_writer = csv.writer(vu_file, delimiter = ",")
        vu_writer.writerow(["Server Nickname", "Username", "ID"])
        for user in VerifiedEmployees:
            vu_writer.writerow([f"{user.nick}", f"{user.name}#{user.discriminator}", f"{user.id}"])

    embed = discord.Embed(
        description = f":crossed_swords: **`{len(PrunedUsers)}` users were pruned by <@{context.author.id}>.**"
                      + f"\n:shield: **There are `{len(VerifiedEmployees)}` users who completed re-verification.**"
                      + f"\n\n**Here is a `.csv` file of all the users that were pruned:**",
        color = discord.Color.blue()
    )
    await context.send(embed = embed)
    await context.send(file = discord.File(r"pruned_users.csv")) # upload the .csv file we created
    await context.send(file = discord.File(r"verified_users.csv"))
    await log_command(context, "prune_unverified")

内容的提问来源于stack exchange，提问作者Iron_Legion