Tomcat优雅停机时强制关闭长连接问题及优化诉求

在使用Embedded Tomcat 9.0.68的Spring Boot应用中，前端部署了非HTTP感知的负载均衡器。缩容应用时，即便配置了server.shutdown=graceful让Tomcat优先完成停机再关闭Bean等组件，仍出现报错。测试发现，Tomcat收到Spring发送的停机信号时，会立即关闭所有已建立的keep-alive连接，存在竞态条件，可能导致客户端已发送的请求丢失。

期望的优雅停机流程：

运维环境视角 #

• 用户通知运维环境执行缩容操作
• 运维环境停止向待缩容实例发送新连接，但无法干预已建立的连接
• 运维环境向待缩容实例发送SIGTERM信号并等待终止

Tomcat（应用）收到SIGTERM信号后 #

• 立即停止接受新连接
• 对停机前建立的连接，返回响应时添加Connection: close头，发送后立即关闭连接
• 若所有连接以此方式关闭则完成停机；若存在无活动的keep-alive连接无法返回该头，则等待配置的keepAliveTimeout时长（客户端已通过keepAlive头获知该超时）后，合法关闭剩余长连接，再通知Spring Boot完成后续停机流程，最终告知运维环境（如Kubernetes）

这种方式可确保所有场景下的干净停机，停机时长受keepAliveTimeout限制，询问是否有办法实现或模拟该行为？

可以通过自定义Tomcat连接器配置和扩展Spring Boot的停机逻辑实现需求，具体步骤如下：

1. 自定义Tomcat连接器，控制连接关闭逻辑 #

创建自定义TomcatConnectorCustomizer，在停机时调整连接器行为：

import org.apache.catalina.connector.Connector;
import org.springframework.boot.web.embedded.tomcat.TomcatConnectorCustomizer;
import org.springframework.context.ApplicationListener;
import org.springframework.context.event.ContextClosedEvent;
import java.util.concurrent.Executor;
import java.util.concurrent.TimeUnit;

public class GracefulShutdownTomcatConnectorCustomizer implements TomcatConnectorCustomizer, ApplicationListener<ContextClosedEvent> {

    private volatile Connector connector;
    private ShutdownConnectionCloseValve shutdownValve;

    @Override
    public void customize(Connector connector) {
        this.connector = connector;
    }

    public void setShutdownValve(ShutdownConnectionCloseValve shutdownValve) {
        this.shutdownValve = shutdownValve;
    }

    @Override
    public void onApplicationEvent(ContextClosedEvent event) {
        // 标记进入停机状态，让响应添加Connection: close头
        shutdownValve.setShuttingDown(true);
        // 立即停止接受新连接
        connector.pause();

        Executor executor = connector.getProtocolHandler().getExecutor();
        if (executor instanceof org.apache.tomcat.util.threads.ThreadPoolExecutor) {
            try {
                org.apache.tomcat.util.threads.ThreadPoolExecutor tomcatExecutor = (org.apache.tomcat.util.threads.ThreadPoolExecutor) executor;
                // 关闭线程池，不再接收新任务
                tomcatExecutor.shutdown();
                // 获取Tomcat配置的keepAliveTimeout，默认20000ms
                long keepAliveTimeout = connector.getProtocolHandler().getKeepAliveTimeout();
                // 等待空闲连接自然超时关闭
                if (!tomcatExecutor.awaitTermination(keepAliveTimeout, TimeUnit.MILLISECONDS)) {
                    // 超时后强制关闭剩余线程
                    tomcatExecutor.shutdownNow();
                }
            } catch (InterruptedException ex) {
                Thread.currentThread().interrupt();
            }
        }
    }
}

2. 给响应添加Connection: close头 #

自定义Tomcat Valve，在停机状态下自动给所有响应添加关闭头：

import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase;
import javax.servlet.ServletException;
import java.io.IOException;

public class ShutdownConnectionCloseValve extends ValveBase {

    private volatile boolean isShuttingDown = false;

    public void setShuttingDown(boolean shuttingDown) {
        this.isShuttingDown = shuttingDown;
    }

    @Override
    public void invoke(Request request, Response response) throws IOException, ServletException {
        if (isShuttingDown) {
            response.setHeader("Connection", "close");
        }
        getNext().invoke(request, response);
    }
}

3. 整合到Spring Boot应用 #

在配置类中注册自定义组件：

import org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class TomcatGracefulShutdownConfig {

    @Bean
    public TomcatServletWebServerFactory tomcatServletWebServerFactory() {
        TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();
        GracefulShutdownTomcatConnectorCustomizer customizer = new GracefulShutdownTomcatConnectorCustomizer();
        ShutdownConnectionCloseValve shutdownValve = new ShutdownConnectionCloseValve();

        factory.addConnectorCustomizers(customizer);
        factory.addContextValves(shutdownValve);
        customizer.setShutdownValve(shutdownValve);

        return factory;
    }
}

4. 配合Spring Boot优雅停机配置 #

在application.properties中配置：

server.shutdown=graceful
# 该值需大于等于Tomcat的keepAliveTimeout，确保有足够时间处理连接关闭
spring.lifecycle.timeout-per-shutdown-phase=30s

补充说明 #

• 流程触发后，Tomcat会先停止接受新连接，对正在处理的请求正常响应并添加Connection: close头，空闲keep-alive连接会等待keepAliveTimeout后自然关闭，之后再执行Spring Bean销毁等后续流程。
• 若使用Kubernetes，需调整terminationGracePeriodSeconds，使其大于spring.lifecycle.timeout-per-shutdown-phase的值，避免Kubernetes提前强制杀死Pod。

内容的提问来源于stack exchange，提问作者mv123