如何在pika的basic_consume中实现异步消息回调
需求可行性说明
完全可实现,你当前已经在使用pika提供的AsyncioConnection,它本身就可以和你已有的asyncio事件循环无缝集成,适配异步回调、httpx异步调用的需求。
现有代码存在的问题
你当前的写法有几个不符合pika异步API规范的点:
AsyncioConnection的初始化、通道创建、交换机/队列声明等操作都是异步IO操作,不能在同步的__init__方法里直接调用channel.start_consuming()是阻塞调用,不能直接在asyncio场景下使用,会卡住整个事件循环- 异步回调不能直接传给
basic_consume,需要用pika异步适配的方式调度
修正后的实现代码
import asyncio from typing import Callable, Any from pika import PlainCredentials, ConnectionParameters from pika.adapters.asyncio_connection import AsyncioConnection from pika.channel import Channel import logging logger = logging.getLogger(__name__) # 此处替换为你自己的settings导入路径 from your_module import settings class AsyncConsumer: connection: AsyncioConnection | None = None channel: Channel | None = None routing_key: str message_callback: Callable[[bytes], Any] def __init__(self, routing_key: str, message_callback: Callable[[bytes], Any]) -> None: self.routing_key = routing_key self.message_callback = message_callback async def connect(self) -> None: """异步创建连接、通道、声明交换机""" credentials = PlainCredentials( settings.mqtt_vhost_user, settings.mqtt_vhost_password ) parameters = ConnectionParameters( settings.mqtt_host, settings.mqtt_port, settings.mqtt_vhost, credentials ) # 异步创建连接 self.connection = await AsyncioConnection.create(parameters) # 异步创建通道 self.channel = await self.connection.channel() # 异步声明交换机 await self.channel.exchange_declare( exchange=settings.mqtt_exchange, exchange_type=settings.mqtt_exchange_type, passive=False, durable=True, auto_delete=False, internal=False ) async def close(self) -> None: """异步关闭通道和连接""" if self.channel: await self.channel.close() if self.connection: await self.connection.close() async def consume(self) -> None: """启动消费逻辑""" if not self.connection or not self.channel: await self.connect() logger.info(f"Started consumer for {self.routing_key}") # 异步声明队列 await self.channel.queue_declare( queue=self.routing_key, passive=False, durable=True, exclusive=False, auto_delete=False ) # 异步绑定队列 await self.channel.queue_bind( queue=self.routing_key, exchange=settings.mqtt_exchange, routing_key=self.routing_key ) # 异步消息处理回调 async def on_message(channel: Channel, method, properties, body: bytes): # 这里可以直接await异步回调,包括httpx异步调用 await self.message_callback(body) await channel.basic_ack(delivery_tag=method.delivery_tag) # 注册消费回调 await self.channel.basic_consume( queue=self.routing_key, on_message_callback=on_message ) # 异步启动消费,不会阻塞事件循环 await self.channel.start_consuming()
使用示例
如果你已经有运行中的asyncio事件循环,直接把消费者任务加入循环即可:
# 你的异步消息处理回调,可直接用httpx异步调用内部服务 async def my_message_handler(body: bytes): import httpx async with httpx.AsyncClient() as client: resp = await client.post("http://你的内部服务地址/api", json={"data": body.decode()}) print(f"调用内部服务返回状态码:{resp.status_code}") async def main(): consumer = AsyncConsumer(routing_key="你的路由key", message_callback=my_message_handler) await consumer.consume() if __name__ == "__main__": # 如果你已有运行中的事件循环,直接把main任务加入现有循环即可 asyncio.run(main())
注意事项
- 消息处理过程中建议添加异常捕获逻辑,按需决定是重试还是投递到死信队列,避免未ack的消息一直堆积
- 如果需要并发处理多个消息,可以调用
await self.channel.basic_qos(prefetch_count=你要的并发数)调整预取数量,控制同一时间推送的未ack消息上限
内容的提问来源于stack exchange,提问作者muuvmuuv
相关产品推荐
相关产品推荐

