Undertow HTTP2请求线程耗尽：线程阻塞于awaitReadable/Writable操作

异常现象 #

• 5个可用实例中，随机单个实例每日偶尔出现数次异常：工作线程阻塞，耗尽全部工作线程后服务无法响应健康检查并触发重启
• 异常仅出现在POST/PUT/PATCH请求场景，阻塞的请求会持续数十分钟后以400或500状态码失败
• 阻塞线程增长迅速，数分钟内即可耗尽线程池，其余4个实例运行完全正常

技术环境 #

• Spring Boot版本：3.1.4 / 3.0.11
• 架构：Spring Cloud Gateway(Netty) 通过HTTP2 SSL与带REST端点的Spring服务(Undertow)通信

读阻塞堆栈 #

java.lang.Throwable: null
    at java.base@17.0.2/java.lang.Object.wait(Native Method)
    at java.base@17.0.2/java.lang.Object.wait(Object.java:338)
    at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.awaitReadable(AbstractFramedStreamSourceChannel.java:354)
    at org.xnio.conduits.StreamSourceChannelWrappingConduit.awaitReadable(StreamSourceChannelWrappingConduit.java:71)
    at org.xnio.conduits.ConduitStreamSourceChannel.awaitReadable(ConduitStreamSourceChannel.java:151)
    at io.undertow.channels.DetachableStreamSourceChannel.awaitReadable(DetachableStreamSourceChannel.java:77)
    at io.undertow.server.HttpServerExchange$ReadDispatchChannel.awaitReadable(HttpServerExchange.java:2305)
    at org.xnio.channels.Channels.readBlocking(Channels.java:345)
    at io.undertow.servlet.spec.ServletInputStreamImpl.readIntoBuffer(ServletInputStreamImpl.java:201)
    at io.undertow.servlet.spec.ServletInputStreamImpl.read(ServletInputStreamImpl.java:176)
    at io.undertow.servlet.spec.ServletInputStreamImpl.read(ServletInputStreamImpl.java:162)
    at io.undertow.servlet.spec.ServletInputStreamImpl.read(ServletInputStreamImpl.java:153)
    at java.base@17.0.2/java.io.FilterInputStream.read(FilterInputStream.java:82)
    at java.base@17.0.2/java.io.PushbackInputStream.read(PushbackInputStream.java:135)
    at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodArgumentResolver$EmptyBodyCheckingHttpInputMessage.<init>(AbstractMessageConverterMethodArgumentResolver.java:331)
    at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodArgumentResolver.readWithMessageConverters(AbstractMessageConverterMethodArgumentResolver.java:172)
    at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.readWithMessageConverters(RequestResponseBodyMethodProcessor.java:163)
    at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.resolveArgument(RequestResponseBodyMethodProcessor.java:136)
    at org.springframework.web.method.support.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:122)
    at org.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:179)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:146)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:884)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:974)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1011)

写阻塞堆栈 #

java.lang.Throwable: null
    at java.lang.Object.wait(Object.java)
    at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.awaitWritable(AbstractFramedStreamSinkChannel.java:329)
    at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.awaitWritable(AbstractFramedStreamSinkChannel.java:313)
    at org.xnio.conduits.StreamSinkChannelWrappingConduit.awaitWritable(StreamSinkChannelWrappingConduit.java:99)
    at org.xnio.conduits.ConduitStreamSinkChannel.awaitWritable(ConduitStreamSinkChannel.java:134)
    at io.undertow.channels.DetachableStreamSinkChannel.awaitWritable(DetachableStreamSinkChannel.java:87)
    at io.undertow.server.HttpServerExchange$WriteDispatchChannel.awaitWritable(HttpServerExchange.java:2141)
    // ... 183 frames truncated

• 已添加Undertow StuckThreadDetector捕获到上述阻塞线程堆栈
• 发现有类似问题案例，但未明确给出修复方式
• 计划通过配置BlockingReadTimeoutHandler和BlockingWriteTimeoutHandler实现请求快速失败，避免线程池耗尽
• 仍需进一步排查该随机异常的根本原因

内容的提问来源于stack exchange，提问作者Andriy Yarish