Spring Boot归档应用使用Transfer Manager后无法自动shutdown问题排查

我基于Spring Boot开发了一个归档应用，配置spring.main.web-application-type: none，原本启动后会从数据库拉取数据，通过PutObject上传至S3，之后自动关闭应用，日志显示Hikari连接池、JPA实体管理器工厂正常关闭，进程退出码为0。

近期为优化1-2百万级文件的S3上传性能，改用Transfer Manager。现在应用执行流程为：从数据库取数据→生成文件→通过Transfer Manager批量上传至S3，流程执行完成后日志停留在“Archiver process completed with status SUCCESS”，应用无法自动关闭。

排查后怀疑是生成文件所用的PrintWriter未正常关闭，但已使用try-with-resources语法，也尝试过手动调用close()方法，排除该原因。

文件生成代码 #

public void storeFile(KeyEntity keyEntity, List<JsonObject> jsonObjects, String archiveDate) {

        String fileName= getFileName(archiveDate,keyEntity);
        String path = ArchiveProperties.getFileUploadPath();
        Path dir= Paths.get(path);

        if(!Files.exists(dir)){
            try {
                Files.createDirectories(dir);
            } catch (IOException e) {
                log.error("Exception occurred while creating directory {}",ArchiveProperties.getFileUploadPath());
                throw new RuntimeException(e);
            }
        }
        Path file = Paths.get(path,fileName);
        try(PrintWriter printWriter = new PrintWriter(new FileWriter(file.toFile()))) {
            printWriter.print(jsonObjects);
        } catch (IOException e) {
            log.error("Exception occurred while writing into directory {} for filename {}", ArchiveProperties.getFileUploadPath(), fileName);
            throw new RuntimeException(e);
        }
    }

Transfer Manager批量上传代码 #

public Boolean bulkUpload(String path, S3Properties s3Properties,String filePrefix){

        log.info("S3ObjectStore:bulkUpload :: Initialing bulk upload for path {}",path);

        AmazonS3Client s3 = S3Configuration.getS3Client(provider,s3Properties);
        TransferManager transferManager = TransferManagerBuilder.standard().withS3Client(s3).build();
        try {

            MultipleFileUpload multipleFileUpload = transferManager.uploadDirectory(s3Properties.getBucket(),filePrefix,new File(path),false);
            showTransferProgress(multipleFileUpload);
            multipleFileUpload.waitForCompletion();

        }
        catch ( Exception e) {
            log.error("S3ObjectStore:bulkUpload :: Exception occurred while uploading directory to s3",e);
            return Boolean.FALSE;
        }finally {
            transferManager.shutdownNow();
          
        }
        return Boolean.TRUE;
    }

AmazonS3Client创建代码 #

public static AmazonS3Client getS3Client(final IdaS3ClientKeysProvider provider, final S3Properties s3Properties){

        log.info("Creating AmazonS3Client for bulk upload ..............");
        IdaS3ClientKeysProvider idaS3ClientKeysProvider = provider;
        //Set up the connection configuration
        ClientConfiguration config = new ClientConfiguration().withRetryPolicy(ClientConfiguration.DEFAULT_RETRY_POLICY);
        //DEFAULT_RETRY_POLICY=3 times retry for failure files
        config.setProtocol(Protocol.HTTPS);
        config.setSignerOverride("S3SignerType");
        //Set up the AWS credentials
        BasicAWSCredentials credentials = new BasicAWSCredentials(idaS3ClientKeysProvider.getS3Keys().getS3AccessKey(),idaS3ClientKeysProvider.getS3Keys().getS3SecretKey());
        //Set up AWS Client
        AmazonS3Client client = new AmazonS3Client(credentials,config);
        client.setS3ClientOptions(S3ClientOptions.builder().setPathStyleAccess(true).build());
        client.setEndpoint(s3Properties.getDataplaneEndpoint());
        log.info("Inside AmazonS3Client :: access key :{}, secret key :{}",provider.getS3Keys().getS3AccessKey(),provider.getS3Keys().getS3SecretKey());
        return client;
    }

解决方案 #

1. 显式关闭AmazonS3Client #

TransferManager默认不会关闭传入的S3Client，其内部后台线程可能阻止JVM退出。在bulkUpload方法的finally块中添加S3Client关闭逻辑：

finally {
    transferManager.shutdownNow();
    s3.shutdown(); // 显式关闭S3Client，释放后台线程资源
}

2. 自定义TransferManager线程池为守护线程 #

TransferManager默认线程池中的非守护线程会维持JVM运行。可以自定义线程池，将线程设置为守护线程：

// 在bulkUpload方法中创建TransferManager时修改
ExecutorService executor = Executors.newFixedThreadPool(10, r -> {
    Thread t = new Thread(r);
    t.setDaemon(true); // 守护线程会随JVM退出自动终止
    return t;
});

TransferManager transferManager = TransferManagerBuilder.standard()
        .withS3Client(s3)
        .withExecutorFactory(() -> executor)
        .build();

3. 主动触发Spring上下文关闭 #

如果应用通过CommandLineRunner或ApplicationRunner执行归档逻辑，任务完成后主动触发上下文关闭，确保所有资源释放：

@Component
public class ArchiveRunner implements CommandLineRunner {

    @Autowired
    private ApplicationContext context;
    @Autowired
    private ArchiveService archiveService;

    @Override
    public void run(String... args) throws Exception {
        boolean success = archiveService.executeArchive();
        int exitCode = success ? 0 : 1;
        // 主动触发上下文关闭并退出进程
        SpringApplication.exit(context, () -> exitCode);
        System.exit(exitCode);
    }
}

4. 排查残留线程（可选） #

若以上方法无效，使用jstack命令导出线程栈，定位未终止的线程：

jstack <进程ID> > thread_dump.txt

分析线程栈中是否存在AWS相关的非守护线程，进一步针对性处理。

内容的提问来源于stack exchange，提问作者Shaggy