如何确保Python中httpx异步请求以并行方式执行？

你的当前代码是串行执行的——在循环里每次await client.get(url)都会等待当前请求完成后才发起下一个，总耗时等于所有请求时间的总和，完全没用到异步的并行能力。

要实现真正的并行，需要用asyncio.gather()批量调度所有异步请求，具体修改方案如下：

1. 把单个用户的请求逻辑抽成独立的异步函数
2. 提前创建所有请求任务，再一次性并行执行

修改后的完整代码：

import asyncio
import time

import httpx

async def fetch_bio(client, username):
    url = f"https://hn.algolia.com/api/v1/users/{username}"
    response = await client.get(url)
    data = response.json()
    print('.')
    return data['about']

async def main():
    t0 = time.time()

    usernames = [
        "author",
        "abtinf",
        "TheCoelacanth",
        "tomcam",
        "chauhankiran",
        "ulizzle",
        "ulizzle",
        "ulizzle",
        "cratermoon",
        "Aeolun",
        "ulizzle",
        "firexcy",
        "kazinator",
        "blacksoil",
        "lucakiebel",
        "ozim",
        "tomcam",
        "jstummbillig",
        "tomcam",
        "johnchristopher",
        "Tade0",
        "lallysingh",
        "paulddraper",
        "WilTimSon",
        "gumby",
        "kristopolous",
        "zemo",
        "aschearer",
        "why-el",
        "Osiris",
        "mdaniel",
        "ianbutler",
        "vinaypai",
        "samtho",
        "chazeon",
        "taeric",
        "yellowapple",
        "Kye",
    ]

    headers = {"User-Agent": "curl/7.72.0"}
    async with httpx.AsyncClient(headers=headers) as client:
        # 创建所有请求任务
        tasks = [fetch_bio(client, username) for username in usernames]
        # 并行执行所有任务并获取结果
        bios = await asyncio.gather(*tasks)

    t1 = time.time()
    total = t1 - t0
    print(bios)
    print(f"Total time: {total} seconds") # 耗时会大幅降低，接近单个请求的最长时间

asyncio.run(main())

关键说明： #

• asyncio.gather(*tasks)会同时调度所有传入的异步任务，等待全部完成后返回结果列表，结果顺序和任务列表的顺序完全一致。
• httpx的AsyncClient默认支持并发连接（默认上限100），足够处理你当前的用户列表规模。

内容的提问来源于stack exchange，提问作者Harry Moreno