如何高效利用Cloud Run并发？Python批量请求优化咨询

方案一：使用线程池实现并发请求 #

Python的concurrent.futures.ThreadPoolExecutor可创建线程池，让多个请求在不同线程中同时发送，无需等待前一个请求完成。

示例代码：

import requests
from concurrent.futures import ThreadPoolExecutor

def send_request(checkin, checkout):
    todo = { 
        "hotelurl_json": hotel_dict,
        "checkin": checkin,
        "checkout": checkout
    }
    try:
        response = requests.post(api_url, json=todo)
        print(f"请求完成，状态码: {response.status_code}")
    except Exception as e:
        print(f"请求失败: {str(e)}")

# 根据Cloud Run并发能力调整max_workers数值，比如设置为10或更高
with ThreadPoolExecutor(max_workers=10) as executor:
    for checkin, checkout in zip(checkin_list, checkout_list):
        executor.submit(send_request, checkin, checkout)

方案二：使用异步HTTP客户端（aiohttp） #

通过asyncio配合aiohttp实现异步IO，效率更高，适合大量请求场景。

先安装依赖：

pip install aiohttp

示例代码：

import asyncio
import aiohttp

async def send_request(session, checkin, checkout):
    todo = { 
        "hotelurl_json": hotel_dict,
        "checkin": checkin,
        "checkout": checkout
    }
    try:
        async with session.post(api_url, json=todo) as response:
            print(f"请求完成，状态码: {response.status}")
    except Exception as e:
        print(f"请求失败: {str(e)}")

async def main():
    async with aiohttp.ClientSession() as session:
        tasks = [send_request(session, checkin, checkout) for checkin, checkout in zip(checkin_list, checkout_list)]
        await asyncio.gather(*tasks)

if __name__ == "__main__":
    asyncio.run(main())

注意事项 #

• 调整并发数：根据Cloud Run的最大实例数等配置，设置合适的并发请求数，避免触发限流。
• 错误处理：保留异常捕获逻辑，防止单个请求失败影响整体任务执行。
• 响应处理：若无需处理响应，可简化代码仅发送请求，进一步提升发送速率。

内容的提问来源于stack exchange，提问作者saurav