如何使用AWS Java SDK清空Amazon S3存储桶

前置依赖 #

首先需要引入AWS SDK for Java 2.x的S3依赖，Maven配置示例如下：

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>s3</artifactId>
    <version>2.20.0</version> <!-- 可替换为最新稳定版 -->
</dependency>

通用实现步骤 #

• 初始化S3客户端实例，默认会自动加载环境变量、本地AWS配置文件、运行环境IAM角色中的凭证信息，无需硬编码密钥
• 遍历存储桶内所有对象，S3的列表接口默认单次最多返回1000个对象，需要处理分页逻辑拉取全量对象
• 批量执行删除操作，单次删除请求最多支持删除1000个对象，减少API调用次数
• 如果存储桶开启了版本控制，需要额外遍历删除所有版本的对象以及删除标记，否则无法完全清空存储桶

代码示例 #

1. 无版本控制存储桶清空代码 #

import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
import java.util.ArrayList;
import java.util.List;

public class ClearS3Bucket {
    public static void clearNonVersionedBucket(String bucketName, Region region) {
        try (S3Client s3Client = S3Client.builder().region(region).build()) {
            ListObjectsV2Request listRequest = ListObjectsV2Request.builder().bucket(bucketName).build();
            ListObjectsV2Response listResponse;
            do {
                listResponse = s3Client.listObjectsV2(listRequest);
                if (listResponse.hasContents()) {
                    List<ObjectIdentifier> objectsToDelete = new ArrayList<>();
                    listResponse.contents().forEach(s3Object -> 
                        objectsToDelete.add(ObjectIdentifier.builder().key(s3Object.key()).build())
                    );
                    DeleteObjectsRequest deleteRequest = DeleteObjectsRequest.builder()
                        .bucket(bucketName)
                        .delete(Delete.builder().objects(objectsToDelete).build())
                        .build();
                    s3Client.deleteObjects(deleteRequest);
                }
                listRequest = ListObjectsV2Request.builder()
                    .bucket(bucketName)
                    .continuationToken(listResponse.nextContinuationToken())
                    .build();
            } while (listResponse.isTruncated());
        }
    }
}

2. 开启版本控制的存储桶清空代码 #

public static void clearVersionedBucket(String bucketName, Region region) {
    try (S3Client s3Client = S3Client.builder().region(region).build()) {
        ListObjectVersionsRequest listVersionsRequest = ListObjectVersionsRequest.builder().bucket(bucketName).build();
        ListObjectVersionsResponse listVersionsResponse;
        do {
            listVersionsResponse = s3Client.listObjectVersions(listVersionsRequest);
            List<ObjectIdentifier> objectsToDelete = new ArrayList<>();
            // 添加所有版本对象
            if (listVersionsResponse.hasVersions()) {
                listVersionsResponse.versions().forEach(version -> 
                    objectsToDelete.add(ObjectIdentifier.builder()
                        .key(version.key())
                        .versionId(version.versionId())
                        .build())
                );
            }
            // 添加所有删除标记
            if (listVersionsResponse.hasDeleteMarkers()) {
                listVersionsResponse.deleteMarkers().forEach(marker -> 
                    objectsToDelete.add(ObjectIdentifier.builder()
                        .key(marker.key())
                        .versionId(marker.versionId())
                        .build())
                );
            }
            if (!objectsToDelete.isEmpty()) {
                DeleteObjectsRequest deleteRequest = DeleteObjectsRequest.builder()
                    .bucket(bucketName)
                    .delete(Delete.builder().objects(objectsToDelete).build())
                    .build();
                s3Client.deleteObjects(deleteRequest);
            }
            listVersionsRequest = ListObjectVersionsRequest.builder()
                .bucket(bucketName)
                .keyMarker(listVersionsResponse.nextKeyMarker())
                .versionIdMarker(listVersionsResponse.nextVersionIdMarker())
                .build();
        } while (listVersionsResponse.isTruncated());
    }
}

注意事项 #

清空操作属于不可逆操作，执行前请确认存储桶内所有数据均可删除；如果存储桶内对象量级超过100万，建议优先使用S3生命周期规则设置对象到期自动删除，或使用S3批量操作任务完成清空，避免本地Java程序长时间运行消耗资源。

内容的提问来源于stack exchange，提问作者Anit Aggarwal