Spring Boot应用在Kubernetes预发布环境启动后立即关闭求助

我的Spring Boot应用在本地及AWS Kubernetes开发环境运行正常，但部署到预发布（PP）环境后，启动完成后立即关闭。以下是相关配置与日志：

pom.xml #

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.3</version>

        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.xyz</groupId>
    <artifactId>my-app</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>my-app</name>
    <description>My App System</description>
    <properties>
        <java.version>11</java.version>
        <skipTests>true</skipTests>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.flywaydb</groupId>
            <artifactId>flyway-core</artifactId>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
            <version>2.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bootstrap</artifactId>
            <version>3.1.3</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-sts</artifactId>
            <version>1.12.263</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

主类 #

@SpringBootApplication
public class MyApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
    
    @Bean
    public RestTemplate getRestTemplate() {     
        return new RestTemplate();
    }

}

启动日志 #

2022-11-30 13:10:53.112  INFO 1 --- [           main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 13:11:03.604  WARN 1 --- [           main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2022-11-30 13:11:11.305  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 16 endpoint(s) beneath base path '/actuator'
2022-11-30 13:11:11.816  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2022-11-30 13:11:12.110  INFO 1 --- [           main] c.a.f.MyApplication                      : Started MyApplication in 96.297 seconds (JVM running for 104.706)
2022-11-30 13:11:12.513  INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 13:11:12.607  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2022-11-30 13:11:12.797  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

更新：DEBUG日志 #

Application availability state ReadinessState changed to REFUSING_TRAFFIC


2022-11-30 14:45:14.420  INFO 1 --- [           main] c.a.f.MyApplication                      : Started FMyApplication in 98.611 seconds (JVM running for 107.522)
2022-11-30 14:45:14.512 DEBUG 1 --- [ionShutdownHook] o.s.b.a.ApplicationAvailabilityBean      : Application availability state ReadinessState changed to REFUSING_TRAFFIC
2022-11-30 14:45:14.514 DEBUG 1 --- [ionShutdownHook] ConfigServletWebServerApplicationContext : Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@5733f295, started on Wed Nov 30 14:44:06 GMT 2022, parent: org.springframework.context.annotation.AnnotationConfigApplicationContext@78ffe6dc
2022-11-30 14:45:14.516 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 2147483647
2022-11-30 14:45:14.601 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Bean 'webServerGracefulShutdown' completed its stop procedure
2022-11-30 14:45:14.601 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Stopping beans in phase 2147483646
2022-11-30 14:45:14.616 DEBUG 1 --- [           main] o.s.b.a.ApplicationAvailabilityBean      : Application availability state LivenessState changed to CORRECT
2022-11-30 14:45:14.715 DEBUG 1 --- [           main] o.s.b.a.ApplicationAvailabilityBean      : Application availability state ReadinessState changed from REFUSING_TRAFFIC to ACCEPTING_TRAFFIC
2022-11-30 14:45:14.716 DEBUG 1 --- [ionShutdownHook] o.s.c.support.DefaultLifecycleProcessor  : Bean 'webServerStartStop' completed its stop procedure
2022-11-30 14:45:14.812 DEBUG 1 --- [ionShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
2022-11-30 14:45:14.904  INFO 1 --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.hibernate.internal.SessionFactoryImpl  : HHH000031: Closing
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.h.type.spi.TypeConfiguration$Scope     : Un-scoping TypeConfiguration [org.hibernate.type.spi.TypeConfiguration$Scope@6534185f] from SessionFactory [org.hibernate.internal.SessionFactoryImpl@d499c13]
2022-11-30 14:45:14.904 DEBUG 1 --- [ionShutdownHook] o.h.s.i.AbstractServiceRegistryImpl      : Implicitly destroying ServiceRegistry on de-registration of all child ServiceRegistries
2022-11-30 14:45:14.915 DEBUG 1 --- [ionShutdownHook] o.h.b.r.i.BootstrapServiceRegistryImpl   : Implicitly destroying Boot-strap registry on de-registration of all child ServiceRegistries
2022-11-30 14:45:14.917  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2022-11-30 14:45:14.917 DEBUG 1 --- [ionShutdownHook] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before shutdown stats (total=10, active=0, idle=10, waiting=0)
2022-11-30 14:45:15.009 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@14a049f9: (connection evicted)
2022-11-30 14:45:15.011 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@82010d7: (connection evicted)
2022-11-30 14:45:15.013 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@47daf598: (connection evicted)
2022-11-30 14:45:15.018 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@4ff4f31c: (connection evicted)
2022-11-30 14:45:15.021 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@76d717a6: (connection evicted)
2022-11-30 14:45:15.104 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@37134f98: (connection evicted)
2022-11-30 14:45:15.106 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@27f50ad7: (connection evicted)
2022-11-30 14:45:15.109 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@4ff16126: (connection evicted)
2022-11-30 14:45:15.111 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@1d53559: (connection evicted)
2022-11-30 14:45:15.113 DEBUG 1 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection org.postgresql.jdbc.PgConnection@67077c93: (connection evicted)
2022-11-30 14:45:15.201 DEBUG 1 --- [ionShutdownHook] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After shutdown stats (total=0, active=0, idle=0, waiting=0)
2022-11-30 14:45:15.201  INFO 1 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

排查方向与解决步骤 #

从日志看，应用已完成启动（Tomcat就绪、Spring上下文初始化完成），但随即触发ShutdownHook执行关闭流程，核心排查点如下：

1. Kubernetes健康检查配置问题

   • 检查Pod的livenessProbe和readinessProbe配置：应用启动耗时约96秒，若健康检查的initialDelaySeconds设置过小（比如小于100秒），会导致K8s判定应用未就绪，直接终止Pod触发关闭。
   • 确认健康检查路径是否正确：比如是否配置了/actuator/health，端口是否为8080，避免因路径错误导致检查失败。
   • 执行kubectl describe pod <pod-name>查看Events字段，确认是否有健康检查失败、Pod被驱逐的记录。

2. AWS参数存储权限或配置缺失

   • 应用依赖AWS Parameter Store，需确认PP环境的Pod IAM角色是否有访问对应参数的权限，是否存在参数缺失情况。可开启AWS SDK的DEBUG日志，排查是否有权限拒绝或参数加载失败的信息。

3. 资源限制不足

   • 对比开发环境与PP环境的Pod资源配置（resources.requests和resources.limits），若PP环境分配的CPU/内存不足，可能导致应用因资源耗尽被K8s终止。查看Pod的资源使用情况，确认是否有OOM（内存溢出）事件记录。

4. 外部终止信号触发

   • 日志中的ShutdownHook触发符合收到SIGTERM信号的特征，需排查PP环境中是否有自动部署脚本、Pod调度策略导致Pod被提前终止，比如滚动更新配置不合理、节点资源紧张导致Pod被驱逐。

内容的提问来源于stack exchange，提问作者Som