AWS Elastic Beanstalk部署Python应用时mysqlclient安装失败求助

问题描述 #

在按照AWS官方文档为Elastic Beanstalk Python应用配置数据库时，部署阶段触发以下错误：

2023/05/07 14:06:36.847596 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed with error exit status 1. Stderr:  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      /bin/sh: line 1: mysql_config: command not found
      /bin/sh: line 1: mariadb_config: command not found
      /bin/sh: line 1: mysql_config: command not found
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-7hpz1pvs/mysqlclient_f18de82744fa43e9b3b8706c3e581791/setup.py", line 15, in <module>
          metadata, options = get_config()
                              ^^^^^^^^^^^^
        File "/tmp/pip-install-7hpz1pvs/mysqlclient_f18de82744fa43e9b3b8706c3e581791/setup_posix.py", line 70, in get_config
          libs = mysql_config("libs")
                 ^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-install-7hpz1pvs/mysqlclient_f18de82744fa43e9b3b8706c3e581791/setup_posix.py", line 31, in mysql_config
          raise OSError("{} not found".format(_mysql_config_path))
      OSError: mysql_config not found
      mysql_config --version
      mariadb_config --version
      mysql_config --libs
      [end of output]

环境信息：EC2实例使用Amazon Linux 2023 AMI，requirements.txt包含mysqlclient==2.0.3。

已尝试的操作 #

修改.ebextensions配置文件，安装了以下YUM包，但错误依旧：

packages:
  yum:
    mysql-selinux: []
    mariadb105: []
    mariadb-connector-c: []
container_commands:
  01_migrate:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py migrate"
    leader_only: true
  02_createsuperuser:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py createsuperuser --noinput --username aradipatrik2 --email ap@gmail.com"
    leader_only: true
  03_save_data_to_db:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py save_data_to_db"
    leader_only: true
option_settings:
  aws:elasticbeanstalk:application:environment:
    DJANGO_SETTINGS_MODULE: env_info.settings

通过SSH手动安装依赖时，终端提示修改会在实例自动替换时丢失。另外，尝试查找后缀为-dev或-devel的MariaDB包，但在Amazon Linux 2023可用包列表中未找到。

requirements.txt完整内容：

asgiref==3.6.0
branca==0.6.0
certifi==2022.12.7
charset-normalizer==3.1.0
Django==4.2.1
folium==0.14.0
idna==3.4
Jinja2==3.1.2
MarkupSafe==2.1.2
numpy==1.24.3
python-dotenv==1.0.0
requests==2.29.0
sqlparse==0.4.4
tzdata==2023.3
urllib3==1.26.15
mysqlclient==2.0.3

解决方案 #

关键原因 #

mysqlclient是编译型依赖，需要以下组件才能完成安装：

• C编译器（如gcc）
• Python开发头文件
• MariaDB/MySQL开发库（提供mysql_config工具）

Amazon Linux 2023使用dnf作为包管理器，且开发包名称为mariadb-devel，而非旧版的mariadb*-devel。

修改.ebextensions配置 #

更新.ebextensions中的packages部分，改用dnf安装必要依赖：

packages:
  dnf:
    gcc: []
    python3-devel: []
    mariadb-devel: []
container_commands:
  01_migrate:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py migrate"
    leader_only: true
  02_createsuperuser:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py createsuperuser --noinput --username aradipatrik2 --email ap@gmail.com"
    leader_only: true
  03_save_data_to_db:
    command: "source /var/app/venv/*/bin/activate && python3 manage.py save_data_to_db"
    leader_only: true
option_settings:
  aws:elasticbeanstalk:application:environment:
    DJANGO_SETTINGS_MODULE: env_info.settings

说明 #

1. gcc：提供编译C代码的能力
2. python3-devel：提供Python的头文件和开发库，用于编译Python扩展
3. mariadb-devel：提供mysql_config工具及MariaDB开发头文件，满足mysqlclient的编译需求

如果不需要在EC2实例上运行MariaDB服务端，可以移除原来的mariadb105、mysql-selinux和mariadb-connector-c包，进一步精简环境。

重新部署应用后，Elastic Beanstalk会先安装这些系统依赖，再成功安装mysqlclient。

内容的提问来源于stack exchange，提问作者A. Patrik