Ghostscript批量处理PDF仅成功3个，剩余7个失败的解决咨询

问题 #

输入文件夹中有10余个PDF文件，尝试用Ghostscript批量转换后转移至输出文件夹，但仅成功处理3个，剩余7个未被读取也未转移至输出目录。

错误信息 #

**** Unable to open the initial device, quitting.
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using <code>tokenizers</code> before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
GPL Ghostscript 9.56.1 (2022-04-04)
Copyright (C) 2022 Artifex Software, Inc.  All rights reserved.
This software is supplied under the GNU AGPLv3 and comes with NO WARRANTY:
see the file COPYING for details.
GPL Ghostscript 9.56.1: Cannot open X display `(null)'.
**** Unable to open the initial device, quitting.

运行代码 #

import os

pdf_folder_path = './pdf' # Replace with the path to your PDF files directory
output_folder_path = './output' # Replace with the path to your output directory

if not os.path.exists(output_folder_path):
   os.makedirs(output_folder_path)

for file_name in os.listdir(pdf_folder_path):
    if file_name.endswith('.pdf'):
      input_path = os.path.join(pdf_folder_path, file_name)
      output_path = os.path.join(output_folder_path, file_name[:-4] + '_output.pdf')
        !gs -o $output_path -sDEVICE=pdfwrite $input_path

loaders = [UnstructuredPDFLoader(os.path.join(output_folder_path, fn)) for fn in
os.listdir(output_folder_path)]
 loaders

预期结果 #

10个PDF文件均被Ghostscript读取并转换至输出文件夹。

解决方案 #

1. 修复Ghostscript的X显示问题 #

Ghostscript在无图形界面环境下运行时，会尝试调用X11显示导致报错。需在命令中添加-dNO_DISPLAY参数禁用X显示，同时加上-dBATCH -dNOPAUSE确保处理完成后自动退出，避免交互阻塞。

2. 处理路径空格问题 #

若文件名或路径包含空格，未加引号的路径会被gs命令解析为多个参数，导致执行失败。需给输入输出路径加上双引号包裹。

3. 关闭tokenizers并行警告 #

在代码开头设置环境变量，避免fork后的并行死锁警告。

4. 修正代码缩进错误 #

原代码中!gs行缩进不正确，需调整为与同层级代码一致的缩进。

修正后的代码 #

import os

# 关闭tokenizers并行避免警告
os.environ['TOKENIZERS_PARALLELISM'] = 'false'

pdf_folder_path = './pdf' # 替换为你的PDF文件夹路径
output_folder_path = './output' # 替换为你的输出文件夹路径

if not os.path.exists(output_folder_path):
    os.makedirs(output_folder_path)

for file_name in os.listdir(pdf_folder_path):
    if file_name.endswith('.pdf'):
        input_path = os.path.join(pdf_folder_path, file_name)
        output_path = os.path.join(output_folder_path, file_name[:-4] + '_output.pdf')
        # 修正后的Ghostscript命令
        !gs -dBATCH -dNOPAUSE -dNO_DISPLAY -o "$output_path" -sDEVICE=pdfwrite "$input_path"

loaders = [UnstructuredPDFLoader(os.path.join(output_folder_path, fn)) for fn in os.listdir(output_folder_path)]
loaders

内容的提问来源于stack exchange，提问作者phdykd