使用AWS boto3创建S3客户端触发boto3/s3/inject.py语法错误求助

问题现象 #

• 使用AWS官方Python SDK boto3对接服务时，仅初始化S3客户端环节触发语法报错，初始化EC2、Machine Learning等其他服务客户端可正常运行
• 已尝试卸载冲突futures包的通用方案，但当前环境未安装该包，方案无效

复现流程 #

步骤1：导入依赖包 #

from pkg_dir.config import *
from pkg_dir.src.utils import *
import boto3

步骤2：读取AWS连接凭证 #

执行代码读取YAML格式凭证配置文件：

creds = read_yaml(creds_file_path)

读取完成后creds变量存储的凭证字典结构如下：

{
    'aws': {
        'aws_access_key_id': 'XXX',
        'aws_secret_access_key': 'YYY'
    }
}

步骤3：创建AWS会话 #

传入凭证参数初始化boto3会话实例：

dev = boto3.Session(
    aws_access_key_id=creds['aws']['aws_access_key_id'],
    aws_secret_access_key=creds['aws']['aws_secret_access_key'],
)

步骤4：初始化S3客户端触发报错 #

执行如下代码创建S3客户端时抛出异常：

s3 = dev.client('s3')

完整错误栈信息：

Traceback (most recent call last):
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3398 in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  Input In [6] in <cell line: 1>
    dev.client('s3')
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/boto3/session.py:299 in client
    return self._session.create_client(
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/botocore/session.py:948 in create_client
    client = client_creator.create_client(
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/botocore/client.py:112 in create_client
    cls = self._create_client_class(service_name, service_model)
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/botocore/client.py:168 in _create_client_class
    self._event_emitter.emit(
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/botocore/hooks.py:412 in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/botocore/hooks.py:256 in emit
    return self._emit(event_name, kwargs)
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/botocore/hooks.py:239 in _emit
    response = handler(**kwargs)
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/boto3/utils.py:65 in _handler
    module = import_module(module)
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/boto3/utils.py:56 in import_module
    __import__(name)
  File ~/Documents/ReposRob_RobPer/personal/kaggle/kgl_space_titanic/.venv/lib/python3.9/site-packages/boto3/s3/inject.py:233
    return self.meta.client.upload_file(,
                                        ^
SyntaxError: invalid syntax

环境信息 #

已安装依赖 #

执行pip3 freeze输出的已安装包列表如下：

anyio==3.6.1
appnope==0.1.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.5
attrs==21.4.0
awscli==1.25.10
Babel==2.10.1
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.0
boto==2.49.0
boto3==1.24.10
botocore==1.27.10
certifi==2022.5.18.1
cffi==1.15.0
charset-normalizer==2.0.12
colorama==0.4.4
cycler==0.11.0
debugpy==1.6.0
decorator==5.1.1
defusedxml==0.7.1
docutils==0.16
entrypoints==0.4
et-xmlfile==1.1.0
executing==0.8.3
fastjsonschema==2.15.3
fonttools==4.33.3
idna==3.3
importlib-metadata==4.11.4
ipykernel==6.13.0
ipython==8.4.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
jmespath==1.0.0
json5==0.9.8
jsonschema==4.6.0
jupyter-client==7.3.1
jupyter-core==4.10.0
jupyter-server==1.17.0
jupyterlab==3.1.0
jupyterlab-pygments==0.2.2
jupyterlab-server==2.14.0
kaggle==1.5.12
kiwisolver==1.4.2
MarkupSafe==2.1.1
matplotlib==3.5.2
matplotlib-inline==0.1.3
mistune==0.8.4
mysql-connector-python==8.0.29
nbclassic==0.3.7
nbclient==0.6.4
nbconvert==6.5.0
nbformat==5.4.0
nest-asyncio==1.5.5
notebook==6.4.11
notebook-shim==0.1.0
numpy==1.22.4
openpyxl==3.0.10
packaging==21.3
pandas==1.4.2
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.1.1
prometheus-client==0.14.1
prompt-toolkit==3.0.29
protobuf==4.21.1
psutil==5.9.1
psycopg2==2.9.3
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.4.8
pycparser==2.21
Pygments==2.12.0
pyparsing==3.0.9
pyrsistent==0.18.1
python-dateutil==2.8.2
python-slugify==6.1.2
pytz==2022.1
PyYAML==5.4.1
pyzmq==23.1.0
requests==2.27.1
rsa==4.7.2
s3transfer==0.6.0
scipy==1.8.1
seaborn==0.11.2
Send2Trash==1.8.0
six==1.16.0
sniffio==1.2.0
soupsieve==2.3.2.post1
stack-data==0.2.0
terminado==0.15.0
text-unidecode==1.3
tinycss2==1.1.1
tornado==6.1
tqdm==4.64.0
traitlets==5.2.2.post1
Unidecode==1.3.4
urllib3==1.26.9
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.3.2
zipp==3.8.0

版本信息 #

• Python版本：Python 3.9.1
• 虚拟环境管理工具：virtualenv

根因分析 #

报错指向boto3内置S3扩展模块boto3/s3/inject.py第233行存在语法错误：函数调用括号内直接写逗号无参数，属于非法Python语法。
该问题和futures包冲突无关，本质是boto3安装包文件损坏：

1. S3客户端初始化时会动态加载inject.py模块注入S3专属方法，其他服务客户端不会加载该文件，因此仅S3初始化触发报错
2. 文件损坏通常由安装过程中断、pip缓存损坏、磁盘写入错误导致，正常发布的boto3版本不可能存在此类低级语法错误

修复步骤 #

1. 完全卸载当前环境中相关的AWS SDK包，避免残留损坏文件：

pip uninstall -y boto3 botocore s3transfer awscli

2. 清理pip本地缓存，防止重新安装时读取到损坏的缓存包：

pip cache purge

3. 重新安装对应版本的依赖包：

pip install boto3==1.24.10 awscli==1.25.10

4. 验证修复：重新执行S3客户端初始化代码即可正常运行。若仍报错，可手动打开虚拟环境路径下的boto3/s3/inject.py文件，确认第233行代码完整无语法错误。

内容的提问来源于stack exchange，提问作者robperch