AWS S3默认上传隐藏文件问题及IAM权限配置咨询

Hey there! Let's break down this problem and fix it step by step. First, a quick clarification: the aws s3api put-object command itself doesn't automatically upload hidden files—it only uploads exactly the file you specify in the --body parameter. That said, if you're seeing hidden files end up in your bucket, it's likely from using bulk operations like aws s3 sync or aws s3 cp with wildcards, or accidentally pointing put-object at a hidden file. Here's how to handle it:

Fix 1: Stop Hidden Files from Being Uploaded in Bulk Operations #

If you're using sync or cp for bulk file transfers, these commands default to including hidden files (like Windows' Thumbs.db or Unix-style dot-prefixed files). Use these parameters to exclude them:

• Exclude all dot-prefixed hidden files:

  aws s3 sync ./your-local-folder s3://<bucket-name>/poc --exclude ".*"
  

• Exclude system-specific hidden files too:
  Add extra rules for Windows-specific hidden files like Thumbs.db:

  aws s3 sync ./your-local-folder s3://<bucket-name>/poc --exclude ".*" --exclude "Thumbs.db"
  

• Precisely include only the files you want:
  If you only need to upload specific file types (e.g., .txt), lock it down by excluding everything first, then including your target files:

  aws s3 sync ./your-local-folder s3://<bucket-name>/poc --exclude "*" --include "*.txt" --exclude ".*"
  

Fix 2: Avoid Accidental Uploads with put-object #

Since put-object targets a single file, double-check the path in your --body parameter. Make sure you're not pointing to a hidden file (e.g., .test.txt instead of test.txt). On Windows, you can confirm non-hidden files by running dir /a-d in Command Prompt to verify the file you're uploading doesn't have the hidden attribute.

Fix 3: Block Hidden File Uploads at the Bucket Level (Safety Net) #

If you want to prevent any user from uploading hidden files to this bucket entirely, add a bucket policy that denies PutObject requests for keys starting with a dot:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket-name>/*",
      "Condition": {
        "StringLike": {
          "s3:key": ".*"
        }
      }
    }
  ]
}

Note: If you need to upload legitimate hidden files (like .env for an application), adjust the Condition to exclude specific paths or filenames (e.g., "s3:key": "!poc/.env").

Fix 4: Clean Up Existing Hidden Files in S3 #

If hidden files are already in your bucket, delete them with this command:

# Delete all dot-prefixed hidden files in the entire bucket
aws s3 rm s3://<bucket-name> --recursive --include ".*"

Or target a specific folder (like your poc directory):

aws s3 rm s3://<bucket-name>/poc/ --recursive --include ".*"

内容的提问来源于stack exchange，提问作者Andres Urrego Angel