GRPC服务POST请求被Istio NettyServerHandler拒绝如何解决
根因分析
该报错的核心原因是Istio错误将HTTP JSON请求通过HTTP/2协议直接转发到了gRPC原生服务端口,而非对接的gRPC-Gateway HTTP端口:
- GET、DELETE请求没有请求负载,gRPC Netty服务端处理这类无体请求时不会走到请求体解析逻辑,因此不会触发空指针异常,表面表现为请求正常
- POST请求携带请求体,Netty服务端按照gRPC HTTP/2帧规则解析普通JSON payload,内部预期的gRPC元数据对象为空,直接抛出空指针导致请求被拦截
解决方案
- 修正K8s Service端口协议标识:Istio默认通过端口名前缀/
appProtocol字段识别端口协议,如果你的端口对外提供HTTP JSON服务,需要将端口名改为http-<自定义后缀>,或者显式设置appProtocol: http,避免Istio将流量识别为gRPC协议强制用HTTP/2转发 - 拆分服务端口:如果需要同时对外提供原生gRPC服务和HTTP JSON接口,配置两个独立端口:一个端口协议标识为
grpc给gRPC原生客户端调用,另一个标识为http给gRPC-Gateway处理HTTP请求,两类端口不要复用 - 检查EnvoyFilter自定义规则:如果你的集群配置过强制HTTP2升级的自定义EnvoyFilter,需要排除对应HTTP服务端口的匹配规则,避免规则误命中
- 验证gRPC-Gateway配置:确认你的gRPC-Gateway组件已经正确注册对应POST方法的路由,且服务启动时启用了HTTP/1.1支持
报错日志回顾
2021-11-15 15:39:05.794 io.netty.handler.codec.http2.Http2Exception$StreamException: at io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:172) at io.grpc.netty.NettyServerHandler.newStreamException(NettyServerHandler.java:791) at io.grpc.netty.NettyServerHandler.onDataRead(NettyServerHandler.java:489) at io.grpc.netty.NettyServerHandler.access$900(NettyServerHandler.java:103) at io.grpc.netty.NettyServerHandler$FrameListener.onDataRead(NettyServerHandler.java:814) at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onDataRead(DefaultHttp2ConnectionDecoder.java:307) at io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onDataRead(Http2InboundFrameLogger.java:48) at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readDataFrame(DefaultHttp2FrameReader.java:415) at io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:250) at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:159) at io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41) at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:173) at io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:378) at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.NullPointerException at io.grpc.netty.NettyServerHandler.onDataRead(NettyServerHandler.java:480) ... 30 more 2021-11-15 15:39:05.793 WARNING: Stream Error 2021-11-15 15:39:05.792 Nov 15, 2021 7:39:05 AM io.grpc.netty.NettyServerHandler onStreamError 2021-11-15 15:39:05.792 2021-11-15 15:39:05.791 java.lang.NullPointerException at io.grpc.netty.NettyServerHandler.onDataRead(NettyServerHandler.java:480) at io.grpc.netty.NettyServerHandler.access$900(NettyServerHandler.java:103) at io.grpc.netty.NettyServerHandler$FrameListener.onDataRead(NettyServerHandler.java:814) at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onDataRead(DefaultHttp2ConnectionDecoder.java:307) at io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onDataRead(Http2InboundFrameLogger.java:48) at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readDataFrame(DefaultHttp2FrameReader.java:415) at io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:250) at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:159) at io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41) at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:173) at io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:378) at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) 2021-11-15 15:39:05.705 WARNING: Exception in onDataRead() 2021-11-15 15:39:05.704 Nov 15, 2021 7:39:05 AM io.grpc.netty.NettyServerHandler onDataRead
内容的提问来源于stack exchange,提问作者Prashanth Viswanathan
相关产品推荐
相关产品推荐

