GitHub Actions中如何让两个作业并行运行并共用同一个虚拟环境(venv)

嗨，我来帮你解决这个GitHub Actions的问题～首先得明确一个核心点：每个Job都是在独立的Runner环境里运行的，所以直接共享venv是做不到的——venv的路径和当前Runner的文件系统绑定，跨Runner根本没法直接用。不过我们可以通过缓存依赖的方式，达到和「共用环境」一样的效果：不用重复安装依赖，同时让两个Job并行运行。下面给你两种可行的方案：

方案一：在两个并行Job中用缓存复用依赖（最推荐） #

这种方式下，两个Job完全没有依赖关系，会同时启动。它们都会先检查缓存，如果有现成的依赖就直接用，只有缓存命中失败时才会重新安装依赖。

name: Parallel Jobs with Shared Dependencies via Cache

on: push

jobs:
  # 并行Job1：运行pytest
  run-pytest:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11.9'

      - name: Cache Python dependencies
        uses: actions/cache@v3
        with:
          path: |
            ~/.cache/pip
            venv
          key: ${{ runner.os }}-python-${{ hashFiles('requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-python-

      - name: Install Dependencies (if cache miss)
        run: |
          python -m venv venv
          source venv/bin/activate
          pip install -r requirements.txt

      - name: Run Pytest
        run: |
          source venv/bin/activate
          pytest

  # 并行Job2：这里可以换成你需要的其他任务（比如代码检查）
  run-code-check:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11.9'

      - name: Cache Python dependencies
        uses: actions/cache@v3
        with:
          path: |
            ~/.cache/pip
            venv
          key: ${{ runner.os }}-python-${{ hashFiles('requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-python-

      - name: Install Dependencies (if cache miss)
        run: |
          python -m venv venv
          source venv/bin/activate
          pip install -r requirements.txt

      - name: Run Code Lint (示例)
        run: |
          source venv/bin/activate
          flake8 .

方案说明 #

• 两个Job没有设置needs依赖，所以会并行启动
• 用actions/cache缓存venv和pip的缓存目录，缓存的Key基于requirements.txt的哈希值——只有当依赖文件变化时，才会触发全新的依赖安装
• 第一个Job安装完依赖后，第二个Job如果启动稍晚，就能直接命中缓存，跳过耗时的安装步骤

方案二：前置Job构建缓存，两个并行Job复用 #

如果你希望只安装一次依赖，再让两个Job并行使用，可以先做一个前置Job专门安装并缓存依赖，之后两个目标Job在前置Job完成后并行启动，直接复用缓存好的依赖。

name: Parallel Jobs with Pre-built Dependencies

on: push

jobs:
  # 前置Job：仅负责安装依赖并缓存
  prepare-dependencies:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11.9'

      - name: Install Dependencies
        run: |
          python -m venv venv
          source venv/bin/activate
          pip install -r requirements.txt

      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: venv
          key: ${{ runner.os }}-python-${{ hashFiles('requirements.txt') }}

  # 并行Job1：运行pytest
  run-pytest:
    runs-on: ubuntu-latest
    needs: prepare-dependencies
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11.9'

      - name: Restore cached dependencies
        uses: actions/cache@v3
        with:
          path: venv
          key: ${{ runner.os }}-python-${{ hashFiles('requirements.txt') }}

      - name: Run Pytest
        run: |
          source venv/bin/activate
          pytest

  # 并行Job2：其他任务示例
  run-script-task:
    runs-on: ubuntu-latest
    needs: prepare-dependencies
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11.9'

      - name: Restore cached dependencies
        uses: actions/cache@v3
        with:
          path: venv
          key: ${{ runner.os }}-python-${{ hashFiles('requirements.txt') }}

      - name: Run Custom Script (示例)
        run: |
          source venv/bin/activate
          python ./scripts/custom_task.py

方案说明 #

• prepare-dependencies是唯一的前置Job，只运行一次，完成依赖安装后将venv缓存起来
• 两个目标Job都设置了needs: prepare-dependencies，会在前置Job完成后并行启动
• 两个并行Job都会从缓存中恢复venv，完全跳过依赖安装步骤

关键注意事项 #

• 绝对不要尝试直接把venv目录共享给其他Job：venv内部的路径是和当前Runner的文件系统绑定的，跨Runner使用会出现大量路径错误
• 缓存Key一定要包含requirements.txt的哈希值：这样只有当依赖文件更新时，才会重新构建缓存，确保环境的一致性
• 如果你的项目中venv的路径有变化，记得同步修改缓存的path参数

备注：内容来源于stack exchange，提问作者VaporProBuild