使用Curl从AWS S3下载Landsat 8时遭遇403访问拒绝错误

开发者阿泽

2026-7-16

问题：AWS S3 usgs-landsat存储桶下载Landsat影像返回403 AccessDenied

我写了一段bash脚本，本应支持付费用户从AWS S3的usgs-landsat存储桶下载Landsat 8影像，脚本如下：

#!/usr/bin/env bash


#!/bin/bash

# Path to your test file
req_path="/collection02/level-2/standard/oli-tirs/2023/233/245/LC08_L2SP_233245_20230510_20230518_02_T2/LC08_L2SP_233245_20230510_20230518_02_T2_SR_B1.TIF"

# We need the current date to calculate the signature and also to pass to S3/GCS
curr_date=`date -R`

# This is the name of your S3/GCS bucket
bucket_name="usgs-landsat"
string_to_sign="GET\n\n\n${curr_date}\n/${bucket_name}${req_path}"

# Your secret (this one works fine when I used aws cli command)
secret="qc2Kc...."

# Your S3 key (this one works fine when I used aws cli command)
s3_key="AKIAY..."

# We will now calculate the signature to be sent as a header.
signature=$(echo -en "${string_to_sign}" | openssl sha1 -hmac "${secret}" -binary | base64)

# That's all we need. Now we can make the request as follows.


# S3
curl -v -H "Host: ${bucket_name}.s3.amazonaws.com" \
        -H "Date: $curr_date" \
        -H "Authorization: AWS ${s3_key}:${signature}" \
         "https://${bucket_name}.s3.amazonaws.com${req_path}"

运行后返回403错误：

< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 07 Jul 2023 01:00:40 GMT
< Server: AmazonS3
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
<?xml version="1.0" encoding="UTF-8"?>
* Connection #0 to host usgs-landsat.s3.amazonaws.com left intact
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>9AST0DFVKK34KFW8</RequestId><HostId>dFyJ9kCQeeWVEnB8WoXbHpZhsloRmqm1Mc3yY7WP90ZWhrOk02J7Vxe1ci9/3bvu59soxAYx6pU=</HostId></Error>

注：脚本中使用的密钥在AWS CLI中可正常使用，寻求解决该403访问拒绝问题的方法。

解决方法

修正签名字符串的换行格式：AWS S3 V2签名要求待签字符串的换行必须是真实的换行符，而非\n转义字符。脚本中用"GET\n\n\n${curr_date}\n/${bucket_name}${req_path}"可能存在解析偏差，建议改用多行字符串生成：
```
string_to_sign=$(cat <<EOF
GET


${curr_date}
/${bucket_name}${req_path}
EOF
)
```

切换到区域专属端点：usgs-landsat存储桶位于us-west-2区域，通用s3.amazonaws.com端点可能导致签名验证失败，需改用区域端点：

# 修改curl请求的Host头和URL
curl -v -H "Host: ${bucket_name}.s3.us-west-2.amazonaws.com" \
        -H "Date: $curr_date" \
        -H "Authorization: AWS ${s3_key}:${signature}" \
         "https://${bucket_name}.s3.us-west-2.amazonaws.com${req_path}"

升级到AWS V4签名：V2签名已逐步被淘汰，部分存储桶/区域可能限制使用。改用V4签名的完整流程如下：

# 生成V4签名所需的日期参数
date_stamp=$(date -u +"%Y%m%d")
amz_date=$(date -u +"%Y%m%dT%H%M%SZ")
region="us-west-2"
service="s3"

# 生成签名密钥
kSecret="AWS4${secret}"
kDate=$(echo -n "${date_stamp}" | openssl sha256 -hmac "${kSecret}" -binary)
kRegion=$(echo -n "${region}" | openssl sha256 -hmac "${kDate}" -binary)
kService=$(echo -n "${service}" | openssl sha256 -hmac "${kRegion}" -binary)
kSigning=$(echo -n "aws4_request" | openssl sha256 -hmac "${kService}" -binary)

# 生成规范化请求和待签字符串
canonical_request="GET\n${req_path}\n\nhost:${bucket_name}.s3.${region}.amazonaws.com\nx-amz-date:${amz_date}\n\nhost;x-amz-date\nUNSIGNED-PAYLOAD"
canonical_hash=$(echo -en "${canonical_request}" | openssl sha256 -binary | xxd -p -c 256)
string_to_sign="AWS4-HMAC-SHA256\n${amz_date}\n${date_stamp}/${region}/${service}/aws4_request\n${canonical_hash}"

# 计算签名并发起请求
signature=$(echo -en "${string_to_sign}" | openssl sha256 -hmac "${kSigning}" -binary | base64)
curl -v -H "Host: ${bucket_name}.s3.${region}.amazonaws.com" \
        -H "x-amz-date: ${amz_date}" \
        -H "Authorization: AWS4-HMAC-SHA256 Credential=${s3_key}/${date_stamp}/${region}/${service}/aws4_request, SignedHeaders=host;x-amz-date, Signature=${signature}" \
         "https://${bucket_name}.s3.${region}.amazonaws.com${req_path}"

确认IAM权限与存储桶策略：虽然CLI可用，仍需检查IAM用户是否拥有s3:GetObject权限（资源需覆盖usgs-landsat/*），同时确认USGS存储桶的资源策略未限制付费用户的访问。

内容的提问来源于stack exchange，提问作者Bằng Rikimaru