如何解决Flask部署Heroku时requests_futures模块找不到的问题？

问题描述 #

将Flask应用部署到Heroku时，触发requests_futures模块的ModuleNotFoundError错误。尽管已在requirements.txt中添加该模块，但问题仍存在；移除该模块后Heroku可正常部署。

错误日志 #

2022-11-07T22:26:40.279231+00:00 app[web.1]: from app.main import app
2022-11-07T22:26:40.279231+00:00 app[web.1]: File "/app/app/main.py", line 3, in <module>
2022-11-07T22:26:40.279231+00:00 app[web.1]: from .tools.web_scraping import web_scrape
2022-11-07T22:26:40.279231+00:00 app[web.1]: File "/app/app/tools/web_scraping/web_scrape.py", line 4, in <module>
2022-11-07T22:26:40.279232+00:00 app[web.1]: from requests_futures import sessions
2022-11-07T22:26:40.279232+00:00 app[web.1]: ModuleNotFoundError: No module named 'requests_futures'
2022-11-07T22:26:40.279381+00:00 app[web.1]: [2022-11-07 22:26:40 +0000] [9] [INFO] Worker exiting (pid: 9)
2022-11-07T22:27:10.906569+00:00 app[web.1]: [2022-11-07 22:27:10 +0000] [4] [INFO] Shutting down: Master
2022-11-07T22:27:10.906610+00:00 app[web.1]: [2022-11-07 22:27:10 +0000] [4] [INFO] Reason: Worker failed to boot.
2022-11-07T22:27:11.054365+00:00 heroku[web.1]: Process exited with status 3
2022-11-07T22:27:11.153534+00:00 heroku[web.1]: State changed from up to crashed
2022-11-07T22:27:11.963376+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=shows-4-u.herokuapp.com request_id=e0cd543f-98cc-486f-a27e-8f07b97c720f fwd="148.101.185.226" dyno=web.1 connect=0ms service= status=503 bytes= protocol=https
2022-11-07T22:57:34.354131+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=shows-4-u.herokuapp.com request_id=937c2b54-5df2-4e42-b056-1582c6137e0c fwd="148.101.185.226" dyno= connect= service= status=503 bytes= protocol=https

requirements.txt内容 #

appdirs==1.4.4
beautifulsoup4==4.11.1
bs4==0.0.1
certifi==2022.9.24
charset-normalizer==2.1.1
click==8.1.3
cssselect==1.2.0
fake-useragent==0.1.14
Flask==2.2.2
gunicorn==20.1.0
idna==3.4
importlib-metadata==5.0.0
itsdangerous==2.1.2
Jinja2==3.1.2
lxml==4.9.1
MarkupSafe==2.1.1
parse==1.19.0
pyee==8.2.2
pyppeteer==1.0.2
pyquery==1.4.3
requests==2.28.1
requests-futures==1.0.0
requests-html==0.10.0
soupsieve==2.3.2.post1
tqdm==4.64.1
urllib3==1.26.12
w3lib==2.0.1
websockets==10.4
Werkzeug==2.2.2
zipp==3.10.0

构建日志 #

-----> Using buildpack: heroku/python
-----> Python app detected
-----> Using Python version specified in runtime.txt
     
     A Python security update is available! Upgrade as soon as possible to: python-3.10.8
     See: https://devcenter.heroku.com/articles/python-runtimes
     
-----> Requirements file has been changed, clearing cached dependencies
-----> Installing python-3.10.7
-----> Installing pip 22.3.1, setuptools 63.4.3 and wheel 0.37.1
-----> Installing dependencies with Pipenv 2020.11.15
       Installing dependencies from Pipfile.lock (3a221c)...
       Collecting beautifulsoup4==4.9.3
         Downloading beautifulsoup4-4.9.3-py3-none-any.whl (115 kB)
            ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.8/115.8 kB 443.5 kB/s eta 0:00:00
       Installing collected packages: beautifulsoup4
       WARNING: Ignoring invalid distribution -ertifi (/tmp/build_1fedda59/.heroku/python/lib/python3.10/site-packages)
       Successfully installed beautifulsoup4-4.9.3
       Ignoring importlib-metadata: markers 'python_version < "3.10"' don't match your environment
-----> Installing SQLite3
-----> Discovering process types
       Procfile declares types -> web
-----> Compressing...
       Done: 48.2M
-----> Launching...
       Released v21
       https://shows-4-u.herokuapp.com/ deployed to Heroku
Starting November 28th, 2022, free Heroku Dynos, free Heroku Postgres, and free Heroku Data for Redis® will no longer be available.
If you have apps using any of these resources, you must upgrade to paid plans by this date to ensure your apps continue to run and to retain your data. For students, we will announce a new program by the end of September. Learn more at https://blog.heroku.com/next-chapter

解决办法 #

从构建日志能看出，Heroku实际是用Pipenv安装依赖，而非直接读取requirements.txt——日志里明确显示Installing dependencies with Pipenv 2020.11.15和Installing dependencies from Pipfile.lock (3a221c)...，这说明你的项目里存在Pipfile和Pipfile.lock文件，Heroku会优先使用Pipenv管理的依赖配置，所以就算requirements.txt里加了requests-futures，只要Pipfile.lock里没有，就不会安装该模块。

针对这个问题，有两种解决方式：

1. 继续使用Pipenv管理依赖：在本地项目目录执行命令pipenv install requests-futures==1.0.0，更新Pipfile和Pipfile.lock，然后将这两个文件和代码一起提交到Git，重新部署到Heroku即可。
2. 改用requirements.txt管理依赖：删除项目根目录下的Pipfile和Pipfile.lock文件，提交Git后重新部署，Heroku就会自动读取requirements.txt来安装所有依赖。

另外，构建日志里出现了WARNING: Ignoring invalid distribution -ertifi的警告，这可能是certifi模块安装异常导致的，你可以尝试重新在requirements.txt中指定certifi版本，或者在本地清理依赖缓存后重新生成requirements.txt，再部署到Heroku。

内容的提问来源于stack exchange，提问作者Varun Sahni