GitLab多任务流水线同时运行失败问题排查求助

问题根源 #

你遇到的问题大概率是两个build任务的产物目录冲突了——build-driver-only和build-driver-vfpf都把编译结果输出到components/nic/linux/driver/，并行运行时会互相覆盖文件，导致后续测试任务拿到的是混合的错误产物，自然跑失败。单独运行时没有冲突，所以能正常通过。

下面是几种有效隔离两组任务的方法：

1. 给不同build任务分配独立产物目录 #

直接修改编译脚本的输出路径，让两组产物完全分开，从根源避免冲突：

stages:
  - build
  - test
  
build-driver-only:
  stage: build
  artifacts:
    paths:
      - components/nic/linux/driver-only/  # 改成专属目录
    expire_in: 1h
  script:
    - echo "Build driver-only started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh DRIVER_ONLY ./components/nic/linux/driver-only/  # 给脚本传目录参数
    - wait

build-driver-vfpf:
  stage: build
  artifacts:
    paths:
      - components/nic/linux/driver-vfpf/  # 另一组专属目录
    expire_in: 1h
  script:
    - echo "Build driver-VFPF started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh VFPF ./components/nic/linux/driver-vfpf/
    - wait

# 测试任务也要对应调整读取路径
test-drv-ping:
  stage: test
  needs:
    - job: build-driver-only
      artifacts: true
  script:
    - echo "Ping test for driver-only started"
    - chmod +x ./CI_testing/test_driver_only.sh
    - bash ./CI_testing/test_driver_only.sh PING ./components/nic/linux/driver-only/
    - wait

test-drv-ping-ns:
  stage: test
  needs:
    - job: build-driver-only
      artifacts: true
  script:
    - echo "Namespace ping test for driver-only started"
    - chmod +x ./CI_testing/test_driver_only.sh
    - bash ./CI_testing/test_driver_only.sh PING_NS ./components/nic/linux/driver-only/
    - wait

test-drv-iperf3:
  stage: test
  needs:
    - job: build-driver-only
      artifacts: true
  script:
    - echo "iperf3 test for driver-only started"
    - chmod +x ./CI_testing/test_driver_only.sh
    - bash ./CI_testing/test_driver_only.sh IPERF3 ./components/nic/linux/driver-only/
    - wait

test-drv-vfpf:
  stage: test
  needs:
    - job: build-driver-vfpf
      artifacts: true
  script:
    - echo "iperf3 test for driver-vfpf started"
    - chmod +x ./CI_testing/test_driver_only.sh
    - bash ./CI_testing/test_driver_only.sh VFPF ./components/nic/linux/driver-vfpf/
    - wait

记得同步修改build_driver.sh和test_driver_only.sh，让它们支持接收目录参数，把产物输出到指定位置，测试时从对应目录读取文件。

2. 打包产物为不同压缩包上传 #

如果不想改目录结构，可以让每个build任务把产物打包成独有的压缩包，测试时再解压到独立目录：

build-driver-only:
  stage: build
  artifacts:
    paths:
      - driver-only.tar.gz
    expire_in: 1h
  script:
    - echo "Build driver-only started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh DRIVER_ONLY
    - tar -czf driver-only.tar.gz components/nic/linux/driver/  # 打包产物
    - rm -rf components/nic/linux/driver/  # 清理原目录防止冲突
    - wait

build-driver-vfpf:
  stage: build
  artifacts:
    paths:
      - driver-vfpf.tar.gz
    expire_in: 1h
  script:
    - echo "Build driver-VFPF started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh VFPF
    - tar -czf driver-vfpf.tar.gz components/nic/linux/driver/
    - rm -rf components/nic/linux/driver/
    - wait

test-drv-ping:
  stage: test
  needs:
    - job: build-driver-only
      artifacts: true
  script:
    - echo "Ping test for driver-only started"
    - tar -xzf driver-only.tar.gz  # 解压专属产物
    - chmod +x ./CI_testing/test_driver_only.sh
    - bash ./CI_testing/test_driver_only.sh PING
    - wait

# 其他测试任务同理，对应解压各自的压缩包

3. 让两个build任务串行执行（临时方案） #

如果暂时没时间改脚本，先让两个build任务按顺序跑，避免并行覆盖：

build-driver-only:
  stage: build
  artifacts:
    paths:
      - components/nic/linux/driver/
    expire_in: 1h
  script:
    - echo "Build driver-only started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh DRIVER_ONLY
    - wait

build-driver-vfpf:
  stage: build
  needs:
    - build-driver-only  # 依赖前一个任务，串行执行
  artifacts:
    paths:
      - components/nic/linux/driver/
    expire_in: 1h
  script:
    - echo "Build driver-VFPF started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh VFPF
    - wait

这种方法会拉长流水线总耗时，适合临时排查问题，不建议长期用。

4. 用Runner标签隔离任务 #

给两组任务分配不同的Runner，让它们在完全独立的环境里运行，彻底避免文件系统冲突：

build-driver-only:
  stage: build
  tags:
    - driver-only-runner  # 指定专属Runner标签
  artifacts:
    paths:
      - components/nic/linux/driver/
    expire_in: 1h
  script:
    - echo "Build driver-only started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh DRIVER_ONLY
    - wait

build-driver-vfpf:
  stage: build
  tags:
    - driver-vfpf-runner  # 另一组专属Runner标签
  artifacts:
    paths:
      - components/nic/linux/driver/
    expire_in: 1h
  script:
    - echo "Build driver-VFPF started"
    - chmod +x CI_testing/build_driver.sh
    - bash ./CI_testing/build_driver.sh VFPF
    - wait

前提是你已经配置好带对应标签的Runner，且这些Runner的工作目录完全独立。

内容的提问来源于stack exchange，提问作者Muhammad Mustafa