使用PyInstaller打包newspaper3k新闻爬虫程序遇资源文件缺失问题

问题重现 #

开发新闻爬虫程序时，本地PyCharm运行正常，但用PyInstaller打包为可执行文件后，newspaper3k相关功能失效，报错：

Error occurred while processing author and published date for https://www.npr.org/2023/05/04/1174015808/nfl-investigation-claims-gender-discrimination-harassment#:~:text=The%20attorneys%20general%20of%20California: [Errno 2] No such file or directory: '/Users/.../dist/headline_hunter/newspaper/resources/text'

解决方案 #

方案1：修正spec文件，精准指定资源路径 #

collect_data_files有时无法正确识别newspaper3k的资源目录结构，直接手动指定路径更可靠：

1. 找到Python环境中newspaper3k的安装路径，比如虚拟环境下的venv/lib/python3.x/site-packages/newspaper，定位其中的resources文件夹
2. 修改spec文件，替换原有的datas配置：

# -*- mode: python ; coding: utf-8 -*- 

block_cipher = None

# 替换为你实际的newspaper resources路径
newspaper_resources = [
    ('/path/to/your/site-packages/newspaper/resources', 'newspaper/resources')
]

a = Analysis(
    ['headline_hunter.py'],
    pathex=[],
    binaries=[],
    datas=newspaper_resources,
    # 新增必要的隐式依赖
    hiddenimports=['newspaper', 'feedparser', 'tldextract', 'cssselect'],
    hookspath=['.'],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.zipfiles,
    a.datas,
    [],
    name='headline_hunter',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)

3. 清理旧打包产物后重新打包：

rm -rf build dist
pyinstaller headline_hunter.spec

方案2：修改newspaper3k的资源路径获取逻辑 #

如果方案1仍无效，说明库内部用了硬编码路径，需修改源码适配打包环境：

1. 找到newspaper3k源码中的settings.py文件，定位RESOURCES_PATH的定义
2. 替换为动态资源路径获取：

import pkg_resources
RESOURCES_PATH = pkg_resources.resource_filename('newspaper', 'resources')

3. 重新安装修改后的newspaper3k，再执行打包操作

方案3：手动补全资源文件（临时应急） #

打包完成后，直接把Python环境中newspaper/resources下的所有文件，复制到打包产物目录dist/headline_hunter/newspaper/resources中，即可临时解决问题。

内容的提问来源于stack exchange，提问作者user21822637