使用CURL和实例配置文件向S3传文件时遇InvalidAccessKeyId错误

我尝试在EC2实例上通过Instance Profile，用bash/curl向S3存储桶上传文件，代码如下：

instance_profile=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/`
aws_access_key_id=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep AccessKeyId | cut -d':' -f2 | sed 's/[^0-9A-Z]*//g'`
aws_secret_access_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep SecretAccessKey | cut -d':' -f2 | sed 's/[^0-9A-Za-z/+=]*//g'
token=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | sed -n '/Token/{p;}' | cut -f4 -d'"'
file="test_file.txt"
bucket="MM-test-s3-bucket"
filepath="/${bucket}/${path}/${file}"
contentType="application/x-compressed-tar"
dateValue=`date -R`
signature_string="PUT

${contentType}
${dateValue}
${filepath}"
signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${aws_secret_access_key} -binary | base64`

curl -X PUT -T "${file}" -H "Host: ${bucket}.s3.amazonaws.com" -H "Date: ${dateValue}" -H "Content-Type: ${contentType}" -H "Authorization: AWS ${aws_access_key_id}:${signature_hash}" https://${bucket}.s3.amazonaws.com/${file}

但收到错误：

InvalidAccessKeyIdThe AWS Access Key Id you provided does not exist in our records.

以下是排查和解决方法：

问题根源与修复步骤 #

1. Instance Profile变量存在冗余字符 #

原命令未使用-s参数抑制curl的进度输出，且未清理换行符，导致instance_profile变量可能包含多余的换行或空格，后续请求元数据时会出错。

修复：

# 静默获取Instance Profile名称并清理换行
instance_profile=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ | tr -d '\n')

2. 手动解析JSON易出错 #

原代码用grep/cut/sed提取密钥，这种方式对JSON格式的变化容错性极低，很可能提取到错误的Key值。建议用jq工具（需提前安装：sudo yum install jq或sudo apt install jq）解析JSON：

修复：

# 获取完整的临时凭证JSON
aws_creds=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile})
# 用jq提取字段，确保准确性
aws_access_key_id=$(echo "$aws_creds" | jq -r '.AccessKeyId')
aws_secret_access_key=$(echo "$aws_creds" | jq -r '.SecretAccessKey')
token=$(echo "$aws_creds" | jq -r '.Token')

3. 临时凭证缺少必要的请求头 #

Instance Profile提供的是临时凭证，必须在S3请求中添加X-Amz-Security-Token头，原代码提取了token但未使用，这会导致AWS拒绝验证凭证。

4. 签名字符串路径错误 #

原代码中filepath包含存储桶名称，且${path}变量未定义（会变成空字符串），导致签名字符串中的资源路径不符合S3要求：

• 签名字符串中的路径应为/${bucket}/${object-key}（完整资源路径），而非带虚拟主机的格式；若${path}未定义，需删除该变量避免路径出错。

5. 修复后的完整代码 #

# 安装jq（如果未安装）
# sudo yum install -y jq || sudo apt install -y jq

# 获取Instance Profile名称
instance_profile=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ | tr -d '\n')

# 获取临时凭证
aws_creds=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile})
aws_access_key_id=$(echo "$aws_creds" | jq -r '.AccessKeyId')
aws_secret_access_key=$(echo "$aws_creds" | jq -r '.SecretAccessKey')
token=$(echo "$aws_creds" | jq -r '.Token')

# 配置上传参数
file="test_file.txt"
bucket="MM-test-s3-bucket"
object_key="${file}"  # 若需存子目录，改为"subdir/${file}"
contentType="application/x-compressed-tar"
dateValue=$(date -R)

# 构建正确的签名字符串
signature_string="PUT

${contentType}
${dateValue}
/${bucket}/${object_key}"

# 生成签名
signature_hash=$(echo -en "${signature_string}" | openssl sha1 -hmac "${aws_secret_access_key}" -binary | base64)

# 执行上传，添加Security-Token头
curl -X PUT -T "${file}" \
  -H "Host: ${bucket}.s3.amazonaws.com" \
  -H "Date: ${dateValue}" \
  -H "Content-Type: ${contentType}" \
  -H "Authorization: AWS ${aws_access_key_id}:${signature_hash}" \
  -H "X-Amz-Security-Token: ${token}" \
  "https://${bucket}.s3.amazonaws.com/${object_key}"

额外验证步骤 #

• 确认EC2实例已正确关联Instance Profile，且该Profile拥有s3:PutObject权限（检查IAM角色的信任策略和权限策略）。
• 测试临时凭证是否有效：执行echo "$aws_creds"查看是否返回包含AccessKeyId、SecretAccessKey、Token的JSON，且Status为Active。

内容的提问来源于stack exchange，提问作者Muneeb