无法连接GCP PubSub Emulator问题排查求助

问题背景 #

已在本地部署GCP PubSub模拟器，用于运行指向它的Python脚本。模拟器启动状态正常，但出现以下三类问题：

• 使用curl可正常操作模拟器，但模拟器会输出HTTP版本检测警告；
• 通过Postman请求时返回500错误，提示RST_STREAM closed stream. HTTP/2 error code: PROTOCOL_ERROR，模拟器日志显示Connection头相关协议错误；
• 使用Python客户端时，请求从未到达服务器（无日志输出），脚本仅输出正确环境变量但无RPC响应。

模拟器启动日志 #

➜ gcloud beta emulators pubsub start --project=tokyo-rain-123 --host-port=127.0.0.1:8085
Executing: /Users/pabloruiz/Downloads/google-cloud-sdk/platform/pubsub-emulator/bin/cloud-pubsub-emulator --host=127.0.0.1 --port=8085
[pubsub] This is the Google Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
[pubsub] Nov 17, 2022 3:32:04 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: IAM integration is disabled. IAM policy methods and ACL checks are not supported
[pubsub] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[pubsub] SLF4J: Defaulting to no-operation (NOP) logger implementation
[pubsub] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[pubsub] Nov 17, 2022 3:32:04 PM com.google.cloud.pubsub.testing.v1.Main main
[pubsub] INFO: Server started, listening on 8085

curl操作结果 #

➜ curl -X PUT http://localhost:8085/v1/projects/tokyo-rain-123/topics/topic-data1
{
  "name": "projects/tokyo-rain-123/topics/topic-data1"
}

模拟器警告日志 #

[pubsub] INFO: Server started, listening on 8085
[pubsub] Nov 17, 2022 4:00:57 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected non-HTTP/2 connection.
[pubsub] Nov 17, 2022 4:00:57 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected HTTP/2 connection.

Postman响应 #

{
    "error": {
        "code": 500,
        "message": "RST_STREAM closed stream. HTTP/2 error code: PROTOCOL_ERROR",
        "status": "INTERNAL"
    }
}

模拟器错误日志 #

[pubsub] WARNING: Metadata key is 'Connection', which should not be used. That is used by HTTP/1 for connection-specific headers which are not to be forwarded. There is probably an HTTP/1 conversion bug. Simply removing the Connection header is not enough; you should remove all headers it references as well. See RFC 7230 section 6.1
[pubsub] java.lang.RuntimeException: exception to show backtrace
[pubsub]    at io.grpc.Metadata$Key.validateName(Metadata.java:740)
[pubsub]    at io.grpc.Metadata$Key.<init>(Metadata.java:762)
[pubsub]    at io.grpc.Metadata$Key.<init>(Metadata.java:671)
[pubsub]    at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:971)
[pubsub]    at io.grpc.Metadata$AsciiKey.<init>(Metadata.java:966)
[pubsub]    at io.grpc.Metadata$Key.of(Metadata.java:708)
[pubsub]    at io.grpc.Metadata$Key.of(Metadata.java:704)
[pubsub]    at io.gapi.emulators.grpc.HttpAdapter$StubMethodHandler.handle(HttpAdapter.java:537)
[pubsub]    at io.gapi.emulators.grpc.HttpAdapter$UnaryMethodHandler.handle(HttpAdapter.java:572)
[pubsub]    at io.gapi.emulators.grpc.HttpAdapter.handleRequest(HttpAdapter.java:195)
[pubsub]    at io.gapi.emulators.netty.HttpHandler.channelRead(HttpHandler.java:52)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[pubsub]    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[pubsub]    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[pubsub]    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
[pubsub]    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
[pubsub]    at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
[pubsub]    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
[pubsub]    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
[pubsub]    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
[pubsub]    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
[pubsub]    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[pubsub]    at java.base/java.lang.Thread.run(Thread.java:1589)
[pubsub]
[pubsub] Nov 17, 2022 4:02:20 PM io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener shouldIgnoreHeadersOrDataFrame
[pubsub] INFO: [id: 0xde374c44, L:/127.0.0.1:56005 - R:/127.0.0.1:56628] ignoring DATA frame for stream RST_STREAM sent.
[pubsub] Nov 17, 2022 4:02:20 PM io.gapi.emulators.netty.HttpHandler$1 onError
[pubsub] INFO: Exception when handling request: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: PROTOCOL_ERROR

Python脚本 #

import logging
import os
from dotenv import load_dotenv
from google.cloud import pubsub_v1

load_dotenv()
logger = logging.getLogger(__name__)

HOST, PROJECT_ID = os.environ.get("PUBSUB_EMULATOR_HOST"), os.environ.get("PUBSUB_PROJECT_ID")
print(f"Host: {HOST}, Project ID: {PROJECT_ID}")

publisher = pubsub_v1.PublisherClient()

def list_topics(project_id):
    project_path = f"projects/{project_id}"
    all_topics = list(publisher.list_topics(
        request={"project": project_path},
        timeout=5
    ))
    print(f"Found {len(all_topics)}")
    return all_topics


logger.info("Listing topics for project:")
topics = list_topics(project_id=PROJECT_ID)
for topic in topics:
    logger.info(topics)

Python脚本输出 #

Host: [::1]:8085, Project ID: tokyo-rain-123

问题原因分析 #

1. curl操作的HTTP版本警告 #

PubSub模拟器同时支持HTTP/1.1和HTTP/2协议，curl默认使用HTTP/1.1发起请求，模拟器检测到非HTTP/2连接后会输出Detected non-HTTP/2 connection的INFO级日志，这属于正常提示，不影响功能。

2. Postman的500协议错误 #

Postman默认会在请求中携带Connection等HTTP/1.x专属头字段，但模拟器的gRPC-HTTP适配器会严格校验元数据键名，Connection头不符合gRPC元数据规范，导致适配器抛出异常，返回HTTP/2协议错误。本质是Postman的HTTP/2请求兼容处理存在问题，错误携带了HTTP/1.x的连接专属头。

3. Python客户端无响应 #

从脚本输出可见，PUBSUB_EMULATOR_HOST被设置为[::1]:8085（IPv6本地回环地址），但模拟器启动时绑定的是IPv4地址127.0.0.1:8085，两者地址族不匹配，导致Python客户端的gRPC请求无法连接到模拟器。此外，脚本未配置日志级别，logger.info的输出可能未被打印，掩盖了连接超时的错误信息。

内容的提问来源于stack exchange，提问作者Pablo Ruiz Ruiz