Spring WebClient调用支付网关遇SslClosedEngineException问题求助

我们在Spring Boot应用中使用WebFlux自带的WebClient调用外部支付网关，技术栈为spring-boot-starter-parent 2.5.3、spring-boot-starter-webflux 2.5.6。即使在测试环境负载极低的情况下，API调用偶尔会失败，抛出错误：

The connection observed an error io.netty.handler.ssl.SslClosedEngineException: SSLEngine closed already

错误堆栈 #

[id:31e016c9-2, L:/x.xx.xxx.xx:56499 - R:xxx.payu.in/xy.xyy.xyyy.xyy:443] The connection observed an error
io.netty.handler.ssl.SslClosedEngineException: SSLEngine closed already
    at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:861)
    at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:800)
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:781)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
    at io.netty.handler.logging.LoggingHandler.flush(LoggingHandler.java:304)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:531)
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125)
    at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:356)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:765)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790)
    at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:758)
    at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:808)
    at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1025)
    at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:306)
    at reactor.netty.http.HttpOperations.lambda$send$0(HttpOperations.java:128)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:125)
    at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.onNext(ScopePassingSpanSubscriber.java:90)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:127)
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:169)
    at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.request(ScopePassingSpanSubscriber.java:76)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110)
    at org.springframework.cloud.sleuth.instrument.reactor.ScopePassingSpanSubscriber.onSubscribe(ScopePassingSpanSubscriber.java:69)
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96)
    at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
    at reactor.core.publisher.Mono.subscribe(Mono.java:4338)
    at reactor.core.publisher.FluxConcatIterable$ConcatIterableSubscriber.onComplete(FluxConcatIterable.java:147)
    at reactor.core.publisher.FluxConcatIterable.subscribe(FluxConcatIterable.java:60)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110)
    at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157)
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398)
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110)
    at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
    at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
    at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:424)
    at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:654)
    at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.run(DefaultPooledConnectionProvider.java:287)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Unknown Source)

当前WebClient配置 #

@Bean
public WebClient apiClient() {
    HttpClient httpClient = HttpClient.create()
        .wiretap("reactor.netty.http.client.HttpClient", LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL)
        .responseTimeout(Duration.of(5, ChronoUnit.SECONDS));

    return WebClient.builder()
        .clientConnector(new ReactorClientHttpConnector(httpClient))
        .build();
}

依赖树信息 #

+- org.springframework.boot:spring-boot-starter-webflux:jar:2.5.6:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-json:jar:2.5.3:compile
[INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.12.4:compile
[INFO] |  |  +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.12.4:compile
[INFO] |  |  \- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.12.4:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-reactor-netty:jar:2.5.3:compile
[INFO] |  |  \- io.projectreactor.netty:reactor-netty-http:jar:1.0.9:compile
[INFO] |  |     +- io.netty:netty-codec-http2:jar:4.1.66.Final:compile
[INFO] |  |     +- io.netty:netty-resolver-dns:jar:4.1.66.Final:compile
[INFO] |  |     |  \- io.netty:netty-codec-dns:jar:4.1.66.Final:compile
[INFO] |  |     +- io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.66.Final:compile
[INFO] |  |     |  \- io.netty:netty-transport-native-unix-common:jar:4.1.66.Final:compile
[INFO] |  |     +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.66.Final:compile
[INFO] |  |     \- io.projectreactor.netty:reactor-netty-core:jar:1.0.9:compile
[INFO] |  |        \- io.netty:netty-handler-proxy:jar:4.1.66.Final:compile
[INFO] |  |           \- io.netty:netty-codec-socks:jar:4.1.66.Final:compile
[INFO] |  +- org.springframework:spring-web:jar:5.3.9:compile
[INFO] |  |  \- org.springframework:spring-beans:jar:5.3.9:compile
[INFO] |  \- org.springframework:spring-webflux:jar:5.3.9:compile

解决方案 #

1. 优化连接池配置，清理无效连接 #

默认连接池可能复用已被服务端关闭的连接，导致SSL引擎异常。显式配置连接池的空闲超时、存活时间，及时清理无效连接：

@Bean
public WebClient apiClient() {
    HttpClient httpClient = HttpClient.create()
        .wiretap("reactor.netty.http.client.HttpClient", LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL)
        .responseTimeout(Duration.of(5, ChronoUnit.SECONDS))
        // 配置固定大小连接池
        .poolResources(PoolResources.fixed("payu-pool", 10))
        .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 3000)
        // 启用长连接并配置超时
        .keepAlive(true)
        .idleTimeout(Duration.of(30, ChronoUnit.SECONDS))
        .maxIdleTime(Duration.of(60, ChronoUnit.SECONDS));

    return WebClient.builder()
        .clientConnector(new ReactorClientHttpConnector(httpClient))
        .build();
}

2. 升级底层依赖版本 #

当前使用的Reactor Netty和Netty版本存在部分SSL连接相关的已知BUG，升级到Spring Boot 2.5.x系列兼容的最新补丁版本：
在pom.xml中添加版本指定：

<properties>
    <reactor-netty.version>1.0.17</reactor-netty.version>
    <netty.version>4.1.77.Final</netty.version>
</properties>

3. 添加针对性重试机制 #

针对偶发的连接错误，添加重试策略保障调用可靠性：

// 在WebClient调用时配置重试
webClient.post()
    .uri("支付网关接口地址")
    .bodyValue(requestBody)
    .retrieve()
    .bodyToMono(Response.class)
    .retryWhen(Retry.backoff(3, Duration.ofMillis(500))
        .filter(throwable -> throwable instanceof SslClosedEngineException || throwable instanceof IOException));

4. 临时禁用连接复用（应急方案） #

如果上述方案无效，可临时禁用连接池，每次调用创建新连接（会增加性能开销，仅作临时修复）：

@Bean
public WebClient apiClient() {
    HttpClient httpClient = HttpClient.create()
        .wiretap("reactor.netty.http.client.HttpClient", LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL)
        .responseTimeout(Duration.of(5, ChronoUnit.SECONDS))
        // 禁用连接池
        .disablePool();

    return WebClient.builder()
        .clientConnector(new ReactorClientHttpConnector(httpClient))
        .build();
}

内容的提问来源于stack exchange，提问作者Anmol Vijaywargiya