自定义ExecutorService在AWS S3 TransferManager中未生效问题排查

使用AWS SDK v2（版本software.amazon.awssdk:s3:2.29.27）的S3TransferManager时，已将带自定义线程工厂的ExecutorService配置到TransferManager，但日志显示上传/下载任务仍使用默认线程名（如Thread-0），而非自定义名称（如CustomPool-1），以下是问题分析与解决方案：

代码与日志 #

配置代码 #

ExecutorService executorService = Executors.newFixedThreadPool(10, new CustomThreadFactory("CustomPool"));

SdkAsyncHttpClient httpClient = NettyNioAsyncHttpClient.builder()
.maxConcurrency(50)
.connectionTimeout(Duration.ofSeconds(10))
.writeTimeout(Duration.ofSeconds(30))
.build();

S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
.httpClient(httpClient)
.region(Region.US_EAST_2)
.credentialsProvider(StaticCredentialsProvider.create(
AwsBasicCredentials.create(accessKeyId, secretAccessKey)))
.build();

S3TransferManager transferManager = S3TransferManager.builder()
.s3Client(s3AsyncClient)
.executor(executorService)
.build();

// File upload task
UploadFileRequest uploadFileRequest = UploadFileRequest.builder()
.source(Paths.get(fullPath))
.addTransferListener(LoggingTransferListener.create())
.putObjectRequest(PutObjectRequest.builder()
.bucket(bucketName)
.key(keyFileName)
.build())
.build();

transferManager.uploadFile(uploadFileRequest).completionFuture().thenRun(() -> {
logger.info("Upload completed for: " + keyFileName);
});

运行日志 #

[main] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer initiated...
[main] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |                    | 0.0%
[main] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer initiated...
[Thread-0] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |====================| 100.0%
[aws-java-sdk-NettyEventLoop-0-5] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - |====================| 100.0%
[sdk-async-response-1-0] INFO com.test.tucm.awssdkv2.demo.S3MultiThreadedUploadTestWithHttpClient - downloaded completed for: download.js
[sdk-async-response-1-0] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer complete!
[sdk-async-response-1-1] INFO com.test.tucm.awssdkv2.demo.S3MultiThreadedUploadTestWithHttpClient - Upload completed for: download.js
[sdk-async-response-1-1] INFO software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener - Transfer complete!

问题原因 #

这既不是代码配置错误，也不是SDK Bug，而是对S3TransferManager的executor参数作用范围理解偏差：

• TransferManager顶层配置的executor仅用于自身的协调任务，比如分块上传的分片调度、任务状态跟踪等，不负责执行实际的文件IO或网络请求。
• 实际任务的执行由两个独立线程池处理：
  1. Netty事件循环线程池：对应日志中的aws-java-sdk-NettyEventLoop-*，由NettyNioAsyncHttpClient管理，负责网络IO操作。
  2. 文件IO线程池：对应日志中的Thread-0，这是TransferManager内部用于读取本地文件内容的线程池，默认使用JDK的通用线程池，不会复用顶层配置的executor。

解决方案 #

1. 让文件IO线程使用自定义线程名 #

在构建UploadFileRequest时，通过fileUploadConfiguration()指定自定义ExecutorService：

UploadFileRequest uploadFileRequest = UploadFileRequest.builder()
    .source(Paths.get(fullPath))
    .addTransferListener(LoggingTransferListener.create())
    .putObjectRequest(PutObjectRequest.builder()
        .bucket(bucketName)
        .key(keyFileName)
        .build())
    .fileUploadConfiguration(config -> config.executor(executorService)) // 配置文件IO线程池
    .build();

2. 修改Netty事件循环线程名（可选） #

如果需要统一网络IO线程的名称，在构建NettyNioAsyncHttpClient时配置自定义线程工厂：

SdkAsyncHttpClient httpClient = NettyNioAsyncHttpClient.builder()
    .maxConcurrency(50)
    .connectionTimeout(Duration.ofSeconds(10))
    .writeTimeout(Duration.ofSeconds(30))
    .eventLoopGroup(new NioEventLoopGroup(10, new CustomThreadFactory("CustomNettyPool")))
    .build();

内容的提问来源于stack exchange，提问作者Tucm