AWS Lambda调用fitzcli.py遇ModuleNotFoundError问题求助

基于fitz（PyMuPDF）开发文本提取代码，已通过Lambda Layer正确安装该模块且主进程功能正常，但调用官方fitzcli.py脚本时触发错误：

ModuleNotFoundError: No module named 'fitz'

示例代码：

def extract_text(pdf_stream):
    try:
        pdf_doc = fitz.open(stream=pdf_stream, filetype='pdf')
        
        # Save the PDF document to a file
        pdf_doc.save('/tmp/file.pdf') #/tmp is a file destination required to save file, everything else is read only in lambda
        
        logger.info("PDF file saved. Running fitzcli.py.")
        cmd_args = ["python", "fitzcli.py", "gettext", "-input", "file.pdf", "-output", "tmp/extracted_text.txt", "-mode", "layout"]
        subprocess.run(cmd_args, check=True)

        with open('extracted_text.txt', 'r') as open_file:
            read_file = open_file.read()
        # Assuming extract_top_rows function is defined elsewhere in your code
        headers_text = extract_top_rows(read_file)
        
        return headers_text

    except Exception as e:
        logger.error(f"An error occurred: {e}")
        raise

1. 修正路径错误 #

Lambda仅允许/tmp目录写入，你的命令和文件读取路径存在错误：

• 输入文件需指定完整路径/tmp/file.pdf，而非file.pdf
• 输出文件路径应为/tmp/extracted_text.txt，而非tmp/extracted_text.txt
• 读取输出文件时也要使用完整路径/tmp/extracted_text.txt

2. 让subprocess继承Lambda的Python环境 #

Lambda的subprocess默认不会自动加载Layer中的模块，需显式确保环境正确：

方法一：指定Lambda的Python解释器 #

Lambda的Python解释器路径为/var/lang/bin/python3.x（x对应你的Python版本，如3.9则为python3.9），修改命令参数：

cmd_args = [
    "/var/lang/bin/python3.9",
    "/var/task/fitzcli.py",  # 替换为fitzcli.py在部署包中的实际路径
    "gettext",
    "-input", "/tmp/file.pdf",
    "-output", "/tmp/extracted_text.txt",
    "-mode", "layout"
]

方法二：传递PYTHONPATH环境变量 #

将当前进程的PYTHONPATH（包含Layer路径）传递给subprocess：

import os
# 复制当前环境变量并保留PYTHONPATH
subprocess_env = os.environ.copy()

cmd_args = [
    "/var/lang/bin/python3.9",
    "/var/task/fitzcli.py",
    "gettext",
    "-input", "/tmp/file.pdf",
    "-output", "/tmp/extracted_text.txt",
    "-mode", "layout"
]
subprocess.run(cmd_args, check=True, env=subprocess_env)

3. 确认脚本位置 #

确保fitzcli.py已正确打包到Lambda部署包中（通常存放在/var/task/目录下），或在运行时下载至/tmp目录，避免subprocess找不到脚本。

def extract_text(pdf_stream):
    try:
        pdf_doc = fitz.open(stream=pdf_stream, filetype='pdf')
        
        # 保存PDF到/tmp目录
        pdf_path = '/tmp/file.pdf'
        pdf_doc.save(pdf_path)
        
        logger.info("PDF file saved. Running fitzcli.py.")
        # 使用Lambda指定的Python解释器，传递环境变量
        import os
        subprocess_env = os.environ.copy()
        cmd_args = [
            "/var/lang/bin/python3.9",
            "/var/task/fitzcli.py",
            "gettext",
            "-input", pdf_path,
            "-output", "/tmp/extracted_text.txt",
            "-mode", "layout"
        ]
        subprocess.run(cmd_args, check=True, env=subprocess_env)

        # 读取提取的文本
        output_path = '/tmp/extracted_text.txt'
        with open(output_path, 'r') as open_file:
            read_file = open_file.read()
        
        headers_text = extract_top_rows(read_file)
        return headers_text

    except Exception as e:
        logger.error(f"An error occurred: {e}")
        raise

内容的提问来源于stack exchange，提问作者Vandalism