如何使用AWS S3API将同存储桶根目录的TXT文件移动至文件夹

Hey there! Let's figure out how to move all those root-level TXT files in your S3 bucket to a specified subfolder. The s3api copy-object command you tried won't work directly because it doesn't support wildcard patterns like *.txt. Here are a few practical, reliable methods to get this done:

Method 1: Use aws s3 mv (Simplest Approach) #

The aws s3 mv command is perfect here—it essentially copies files to the new location and deletes the originals, which is exactly what "moving" means in S3 (since objects are immutable).

If you want to move all TXT files in the bucket (including those in subdirectories) to the folder/ subfolder, use this command:

aws s3 mv s3://bucket-1/ s3://bucket-1/folder/ --exclude "*" --include "*.txt" --recursive

If you only want to move root-level TXT files (ignoring those in subdirectories), add a filter to exclude files with slashes in their path:

aws s3 mv s3://bucket-1/ s3://bucket-1/folder/ --exclude "*/*" --include "*.txt"

• --exclude "*/*" skips any file that's in a subdirectory
• No need for --recursive here since we're only targeting the root directory

Pro Tip: Preview First #

Add the --dry-run flag to see what would be moved without actually making changes:

aws s3 mv s3://bucket-1/ s3://bucket-1/folder/ --exclude "*/*" --include "*.txt" --dry-run

Method 2: Bash Loop for Precise Control (Root-Only Files) #

If you prefer more granular control, use a bash loop to list root-level TXT files and move them one by one:

# List all root-level objects, filter for TXT files, then move each
aws s3 ls s3://bucket-1/ | grep "\.txt$" | awk '{print $4}' | while read -r filename; do
  aws s3 mv "s3://bucket-1/$filename" "s3://bucket-1/folder/$filename"
done

• aws s3 ls s3://bucket-1/ lists only root-level objects (no --recursive flag)
• grep "\.txt$" filters for files ending in .txt
• awk '{print $4}' extracts the filename from the ls output
• The loop moves each file to the target subfolder

Method 3: Use s3api with jq (For Advanced Workflows) #

If you need to integrate this into a script that uses the low-level S3 API, combine s3api list-objects-v2 with jq to filter and move files:

# List root-level TXT files, then copy and delete each
aws s3api list-objects-v2 --bucket bucket-1 --delimiter "/" | jq -r '.Contents[] | select(.Key | test("\.txt$")) | .Key' | while read -r key; do
  # Copy the file to the subfolder
  aws s3api copy-object --bucket bucket-1 --copy-source "bucket-1/$key" --key "folder/$key"
  # Delete the original file
  aws s3api delete-object --bucket bucket-1 --key "$key"
done

• --delimiter "/" ensures we only get root-level objects
• jq filters the results to only include TXT files
• We explicitly copy then delete (mimicking the mv command's behavior)

Important Notes #

• Permissions: Make sure your IAM user/role has the necessary permissions: s3:GetObject, s3:PutObject, and s3:DeleteObject on the bucket.
• Large Datasets: If you have tens of thousands of files, the loop methods may take time. For very large batches, consider using AWS Batch or Lambda, but the CLI methods work fine for most use cases.
• Error Handling: For production scripts, add error checking (e.g., if [ $? -eq 0 ]; then ...) to handle any failed moves.

内容的提问来源于stack exchange，提问作者Andres Urrego Angel