ARM64环境下安装SpaCy日文模型ja_core_web_sm失败求助

我按照SpaCy文档执行以下命令安装库及多语言模型：

pip install -U pip setuptools wheel
pip install -U 'spacy[apple]'
python -m spacy download zh_core_web_sm
python -m spacy download en_core_web_sm
python -m spacy download fr_core_news_sm
python -m spacy download de_core_news_sm
python -m spacy download ja_core_web_sm
python -m spacy download es_core_news_sm

在基于python:3.11-slim镜像的ARM64架构Docker环境中，安装ja_core_web_sm模型时触发sudachipy构建失败，报错核心为找不到Rust编译器。经查sudachipy仅为x86_64架构提供预编译二进制包，ARM架构需从源码构建，必须依赖Rust工具链。

报错信息 #

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.7/71.7 MB 4.4 MB/s eta 0:00:00
 Building wheels for collected packages: sudachipy
   Building wheel for sudachipy (pyproject.toml): started
   Building wheel for sudachipy (pyproject.toml): finished with status 'error'
   error: subprocess-exited-with-error
   
   × Building wheel for sudachipy (pyproject.toml) did not run successfully.
   │ exit code: 1
   ╰─> [39 lines of output]
       running bdist_wheel
       running build
       running build_py
       creating build
       creating build/lib.linux-aarch64-cpython-311
       creating build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/config.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/command_line.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       copying py_src/sudachipy/errors.py -> build/lib.linux-aarch64-cpython-311/sudachipy
       creating build/lib.linux-aarch64-cpython-311/sudachipy/dictionary
       copying py_src/sudachipy/dictionary/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/dictionary
       creating build/lib.linux-aarch64-cpython-311/sudachipy/tokenizer
       copying py_src/sudachipy/tokenizer/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/tokenizer
       creating build/lib.linux-aarch64-cpython-311/sudachipy/morphemelist
       copying py_src/sudachipy/morphemelist/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/morphemelist
       creating build/lib.linux-aarch64-cpython-311/sudachipy/morpheme
       copying py_src/sudachipy/morpheme/__init__.py -> build/lib.linux-aarch64-cpython-311/sudachipy/morpheme
       copying py_src/sudachipy/sudachipy.pyi -> build/lib.linux-aarch64-cpython-311/sudachipy
       creating build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/sudachi.json -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/rewrite.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/unk.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       copying py_src/sudachipy/resources/char.def -> build/lib.linux-aarch64-cpython-311/sudachipy/resources
       warning: build_py: byte-compiling is disabled, skipping.
       
       running build_ext
       running build_rust
       error: can't find Rust compiler
       
       If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
       
       To update pip, run:
       
           pip install --upgrade pip
       
       and then retry package installation.
       
       If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
       [end of output]
   
   note: This error originates from a subprocess, and is likely not a problem with pip.
   ERROR: Failed building wheel for sudachipy
 Failed to build sudachipy
 ERROR: Could not build wheels for sudachipy, which is required to install pyproject.toml-based projects

当前Dockerfile配置 #

# Use the official Python image, with Python 3.11
FROM python:3.11-slim

# Set environment variables to reduce Python bytecode generation and buffering
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

# Set working directory
WORKDIR /app

# Install essential dependencies including Python development headers and GCC
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    libpq-dev \
    gcc \
    ffmpeg \
    python3-dev \
    libc-dev \
    && apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Update pip and install Python packages
COPY ./docker-requirements.txt /app/
RUN pip install --upgrade pip && \
    pip install --no-cache-dir -r docker-requirements.txt

# Copy application code to container
COPY . /app

# Expose the port the app runs on
EXPOSE 5000

# Make the entrypoint script executable
RUN chmod +x /app/shell_scripts/entrypoint.sh /app/shell_scripts/wait-for-it.sh /app/shell_scripts/docker-ngrok-tunnel.sh

# Define entrypoint
ENTRYPOINT ["/app/shell_scripts/entrypoint.sh"]

解决方案 #

方案1：在Dockerfile中添加Rust工具链（推荐） #

修改Dockerfile，添加Rust安装步骤，确保sudachipy能从源码构建：

# Use the official Python image, with Python 3.11
FROM python:3.11-slim

# Set environment variables to reduce Python bytecode generation and buffering
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    # 配置Rust非交互式安装
    RUSTUP_INIT_SKIP_PATH_CHECK=yes \
    CARGO_HOME=/usr/local/cargo \
    PATH=/usr/local/cargo/bin:$PATH

# Set working directory
WORKDIR /app

# Install essential dependencies including Python development headers and GCC
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    libpq-dev \
    gcc \
    ffmpeg \
    python3-dev \
    libc-dev \
    curl \
    && apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# 安装Rust稳定版工具链
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable

# Update pip and install Python packages
COPY ./docker-requirements.txt /app/
RUN pip install --upgrade pip && \
    pip install --no-cache-dir -r docker-requirements.txt

# Copy application code to container
COPY . /app

# Expose the port the app runs on
EXPOSE 5000

# Make the entrypoint script executable
RUN chmod +x /app/shell_scripts/entrypoint.sh /app/shell_scripts/wait-for-it.sh /app/shell_scripts/docker-ngrok-tunnel.sh

# Define entrypoint
ENTRYPOINT ["/app/shell_scripts/entrypoint.sh"]

说明：添加curl用于下载Rust安装脚本，通过环境变量配置非交互式安装，将Rust的cargo命令加入系统PATH，确保sudachipy构建时能找到Rust编译器。

方案2：替换或跳过日语模型 #

如果业务允许，可以：

• 替换为无需Rust的日语NLP工具（如基于MeCab的模型，需确认ARM64预编译包支持）
• 直接跳过ja_core_web_sm模型安装，仅保留需要的语言模型

内容的提问来源于stack exchange，提问作者Ari