使用Heroku部署应用遇psycopg2系列模块缺失问题求助

使用Heroku部署Python应用时遇到依赖问题：

• 初始代码中写import psycopg2_binary as psycopg2，运行heroku local提示ModuleNotFoundError: No module named 'psycopg2_binary'
• 修改导入语句为import psycopg2后，依然提示ModuleNotFoundError: No module named 'psycopg2'
• 当前使用Poetry管理依赖，已添加psycopg2/psycopg2-binary相关依赖，但手动执行pip install psycopg2时出现构建wheel失败的问题
• 尝试过的操作：安装psycopg2-binary、修改导入语句、检查Python环境、新建Anaconda环境，均无效

第一次错误日志（导入psycopg2_binary时） #

0xzohan@zohan-MBP Serverlang % heroku local 
[OKAY] Loaded ENV .env File as KEY=VALUE Format
2:12:07 PM web.1 |  [2023-09-14 14:12:07 +0100] [80592] [INFO] Starting gunicorn 20.1.0
2:12:07 PM web.1 |  [2023-09-14 14:12:07 +0100] [80592] [INFO] Listening at: http://0.0.0.0:5001 (80592)
2:12:07 PM web.1 |  [2023-09-14 14:12:07 +0100] [80592] [INFO] Using worker: sync
2:12:07 PM web.1 |  [2023-09-14 14:12:07 +0100] [80594] [INFO] Booting worker with pid: 80594
2:12:12 PM web.1 |  [2023-09-14 14:12:12 +0100] [80594] [ERROR] Exception in worker process
2:12:12 PM web.1 |  Traceback (most recent call last):
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
2:12:12 PM web.1 |      worker.init_process()
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/workers/base.py", line 134, in init_process
2:12:12 PM web.1 |      self.load_wsgi()
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/workers/base.py", line 146, in load_wsgiapp
2:12:12 PM web.1 |      self.wsgi = self.app.wsgi()
2:12:12 PM web.1 |                  ^^^^^^^^^^^^^^^
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
2:12:12 PM web.1 |      self.callable = self.load()
2:12:12 PM web.1 |                      ^^^^^^^^^^^
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
2:12:12 PM web.1 |      return self.load_wsgiapp()
2:12:12 PM web.1 |             ^^^^^^^^^^^^^^^^^^^
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
2:12:12 PM web.1 |      return util.import_app(self.app_uri)
2:12:12 PM web.1 |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/util.py", line 359, in import_app
2:12:12 PM web.1 |      mod = importlib.import_module(module)
2:12:12 PM web.1 |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:12:12 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
2:12:12 PM web.1 |      return _bootstrap._gcd_import(name[level:], package, level)
2:12:12 PM web.1 |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:12:12 PM web.1 |    File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
2:12:12 PM web.1 |    File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
2:12:12 PM web.1 |    File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
2:12:12 PM web.1 |    File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2:12:12 PM web.1 |    File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2:12:12 PM web.1 |    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2:12:12 PM web.1 |    File "/Users/0xzohan/Serverlang/main.py", line 15, in <module>
2:12:12 PM web.1 |      from database import create_user_table
2:12:12 PM web.1 |    File "/Users/0xzohan/Serverlang/database.py", line 3, in <module>
2:12:12 PM web.1 |      import psycopg2_binary as psycopg2
2:12:12 PM web.1 |  ModuleNotFoundError: No module named 'psycopg2_binary'
2:12:12 PM web.1 |  [2023-09-14 14:12:12 +0100] [80594] [INFO] Worker exiting (pid: 80594)
2:12:12 PM web.1 |  [2023-09-14 14:12:12 +0100] [80592] [INFO] Shutting down: Master
2:12:12 PM web.1 |  [2023-09-14 14:12:12 +0100] [80592] [INFO] Reason: Worker failed to boot.
[DONE] Killing all processes with signal  SIGINT
2:12:12 PM web.1 Exited with exit code null

第二次错误日志（修改为导入psycopg2时） #

0xzohan@Harrys-MBP Serverlang % heroku local
[OKAY] Loaded ENV .env File as KEY=VALUE Format
2:24:12 PM web.1 |  [2023-09-14 14:24:12 +0100] [81581] [INFO] Starting gunicorn 20.1.0
2:24:13 PM web.1 |  [2023-09-14 14:24:12 +0100] [81581] [INFO] Listening at: http://0.0.0.0:5001 (81581)
2:24:13 PM web.1 |  [2023-09-14 14:24:12 +0100] [81581] [INFO] Using worker: sync
2:24:13 PM web.1 |  [2023-09-14 14:24:12 +0100] [81582] [INFO] Booting worker with pid: 81582
2:24:16 PM web.1 |  [2023-09-14 14:24:16 +0100] [81582] [ERROR] Exception in worker process
2:24:16 PM web.1 |  Traceback (most recent call last):
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
2:24:16 PM web.1 |      worker.init_process()
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/workers/base.py", line 134, in init_process
2:24:16 PM web.1 |      self.load_wsgi()
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/workers/base.py", line 146, in load_wsgiapp
2:24:16 PM web.1 |      self.wsgi = self.app.wsgi()
2:24:16 PM web.1 |                  ^^^^^^^^^^^^^^^
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
2:24:16 PM web.1 |      self.callable = self.load()
2:24:16 PM web.1 |                      ^^^^^^^^^^^
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
2:24:16 PM web.1 |      return self.load_wsgiapp()
2:24:16 PM web.1 |             ^^^^^^^^^^^^^^^^^^^
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
2:24:16 PM web.1 |      return util.import_app(self.app_uri)
2:24:16 PM web.1 |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gunicorn/util.py", line 359, in import_app
2:24:16 PM web.1 |      mod = importlib.import_module(module)
2:24:16 PM web.1 |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:24:16 PM web.1 |    File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
2:24:16 PM web.1 |      return _bootstrap._gcd_import(name[level:], package, level)
2:24:16 PM web.1 |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:24:16 PM web.1 |    File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
2:24:16 PM web.1 |    File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
2:24:16 PM web.1 |    File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
2:24:16 PM web.1 |    File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2:24:16 PM web.1 |    File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2:24:16 PM web.1 |    File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2:24:16 PM web.1 |    File "/Users/0xzohan/Serverlang/main.py", line 15, in <module>
2:24:16 PM web.1 |      from database import create_user_table
2:24:16 PM web.1 |    File "/Users/0xzohan/Serverlang/database.py", line 3, in <module>
2:24:16 PM web.1 |      import psycopg2
2:24:16 PM web.1 |  ModuleNotFoundError: No module named 'psycopg2'
2:24:16 PM web.1 |  [2023-09-14 14:24:16 +0100] [81582] [INFO] Worker exiting (pid: 81582)
2:24:17 PM web.1 |  [2023-09-14 14:24:17 +0100] [81581] [INFO] Shutting down: Master
2:24:17 PM web.1 |  [2023-09-14 14:24:17 +0100] [81581] [INFO] Reason: Worker failed to boot.
[DONE] Killing all processes with signal  SIGINT
2:24:17 PM web.1 Exited with exit code null

1. 纠正导入语句的误区 #

psycopg2-binary是预编译的包，但安装后模块名依然是psycopg2，不是psycopg2_binary。所以不管你安装的是psycopg2还是psycopg2-binary，代码里统一写：

import psycopg2

之前的import psycopg2_binary as psycopg2是错误写法，直接导致模块找不到。

2. 确保Poetry依赖配置正确 #

• 用Poetry添加psycopg2-binary（推荐，避免编译问题）：

  poetry add psycopg2-binary
  

• 提交pyproject.toml和poetry.lock到Git仓库，Heroku会自动识别Poetry并安装依赖。如果Heroku没自动识别，手动设置Python buildpack：

  heroku buildpacks:set heroku/python
  

3. 解决本地pip安装psycopg2失败的问题 #

psycopg2需要系统级的PostgreSQL依赖库（libpq），本地安装前先补全系统依赖：

• macOS：

  brew install postgresql
  

• Ubuntu/Debian：

  sudo apt-get install libpq-dev python3-dev
  

补完依赖后，再用Poetry安装所有依赖：

poetry install

4. 本地测试用Poetry虚拟环境 #

之前的错误日志显示用的是系统Python环境，可能没激活Poetry的虚拟环境，导致依赖没装在正确的环境里。运行heroku local时用Poetry的虚拟环境启动：

poetry run heroku local

5. 备选：生成requirements.txt给Heroku #

如果Poetry在Heroku上的构建有问题，可以导出标准的requirements.txt：

poetry export --without-hashes --format=requirements.txt > requirements.txt

提交requirements.txt到Git，Heroku会优先用这个文件安装依赖。

内容的提问来源于stack exchange，提问作者Zohan - Harry