在Bitbucket Pipeline中使用spark-py镜像运行pytest无输出及报告缺失问题

现有Bitbucket Pipelines配置 #

image: apache/spark-py:v3.3.2

pipelines:
  default:
    - step:
        name: PyTest with PySpark
        script:
          - pip install --upgrade pip
          - pip install -r requirements.txt
          - pytest -v tests/* --junitxml=test-results/report.xml

遇到的异常情况 #

1. 测试报告未被识别：

Searching for test report files in directories named [test-results,
failsafe-reports, test-reports, TestResults, surefire-reports] down to
a depth of 4 Finished scanning for test reports. Found 0 test report
files. Merged test suites, total number tests is 0, with 0 failures
and 0 errors.

2. 无任何命令输出：即使添加-v参数，pytest无输出；执行echo命令也看不到输出。

本地正常运行的Dockerfile参考 #

FROM apache/spark-py:v3.3.2

USER root

# Copy the requirements file into the container
COPY requirements.txt /

# Install pytest and any additional dependencies
RUN pip install --upgrade pip
RUN pip install -r /requirements.txt

# Copy the source code into the container
COPY . /app

WORKDIR /app

CMD ["pytest"]

问题分析与解决步骤 #

1. 镜像用户权限问题 #

apache/spark-py镜像默认以spark用户运行，而非root。Bitbucket Pipeline默认继承镜像用户执行命令，会导致：

• pip install无权限写入系统目录，依赖安装失败但无报错输出
• 创建test-results目录权限不足，无法生成测试报告文件

解决方法：在Pipeline步骤中指定root用户执行：

image: apache/spark-py:v3.3.2

pipelines:
  default:
    - step:
        name: PyTest with PySpark
        user: root
        script:
          - pip install --upgrade pip
          - pip install -r requirements.txt
          - pytest -v tests/* --junitxml=test-results/report.xml

2. 工作目录与路径匹配问题 #

Bitbucket Pipeline默认工作目录为/opt/atlassian/pipelines/agent/build，与本地Dockerfile设置的/app不同，可能导致测试文件路径不匹配。可添加调试命令验证：

script:
  - pwd
  - ls -la
  - pip install --upgrade pip
  - ...

3. 强制关闭输出缓冲 #

Bitbucket Pipeline可能会缓冲输出，添加环境变量强制实时输出：

script:
  - ...
  - PYTHONUNBUFFERED=1 pytest -v tests/* --junitxml=test-results/report.xml

4. 验证依赖安装状态 #

添加命令确认pytest是否安装成功：

script:
  - pip install --upgrade pip
  - pip install -r requirements.txt
  - pip list | grep pytest
  - pytest --version
  - ...

调试技巧 #

• 逐步简化命令：先执行echo "test"、pwd、ls等基础命令，确认输出是否正常
• 检查命令返回码：在每个命令后添加echo $?查看执行状态，例如：

script:
  - pip install --upgrade pip
  - echo "pip upgrade exit code: $?"
  - pip install -r requirements.txt
  - echo "pip install exit code: $?"

内容的提问来源于stack exchange，提问作者Paddy