Azure DevOps流水线中JestJS测试分片优化方案咨询

问题背景 #

使用JestJS扫描代码库时速度较慢，计划通过Jest的分片（shard）功能提升测试效率，但现有教程多针对GitHub Actions或CircleCI的矩阵策略，无法直接适配Azure DevOps。当前实现是为每个分片创建独立任务，重复执行代码检出、依赖安装等步骤，既冗余又低效，需要更优方案。

当前冗余实现代码 #

主流水线配置 (main_pipeline.yaml)

# run each jest passing a shard value to split them and process in parallel
      - job: test1
        displayName: 'test shard 1'
        steps:
          - template: ./steps/git-checkout.yaml
          - template: ./steps/node_install.yaml
          - template: ./steps/yarn_install.yaml
          - template: ./steps/jest_tests.yaml
            parameters:
              shard: 1/4

      - job: test2
        displayName: 'test shard 2'
        steps:
          - template: ./steps/git-checkout.yaml
          - template: ./steps/node_install.yaml
          - template: ./steps/yarn_install.yaml
          - template: ./steps/jest_tests.yaml
            parameters:
              shard: 2/4

      - job: test3
        displayName: 'test shard 3'
        steps:
          - template: ./steps/git-checkout.yaml
          - template: ./steps/node_install.yaml
          - template: ./steps/yarn_install.yaml
          - template: ./steps/jest_tests.yaml
            parameters:
              shard: 3/4

      - job: test4
        displayName: 'test shard 4'
        steps:
          - template: ./steps/git-checkout.yaml
          - template: ./steps/node_install.yaml
          - template: ./steps/yarn_install.yaml
          - template: ./steps/jest_tests.yaml
            parameters:
              shard: 4/4

Jest测试步骤模板 (steps/jest_tests.yaml)

# Pass the test job shard number as a param from the calling pipeline job
  - script: |
      TARGET=native node_modules/jest-expo/bin/jest.js --maxWorkers=100% --forceExit --shard ${{ parameters.shard }}
    displayName: 'running jest test shard ${{ parameters.shard }}'

  - task: PublishTestResults@2
    displayName: 'publish test results'
    inputs:
      testResultsFiles: '**/junit.xml'

优化方案 #

1. 使用Azure DevOps作业矩阵简化配置 #

Azure DevOps支持作业矩阵，可自动生成多个并行作业，彻底避免重复编写任务配置。修改主流水线配置如下：

- job: test_shards
  displayName: 'Jest 分片测试'
  strategy:
    matrix:
      shard_1:
        shard_value: '1/4'
      shard_2:
        shard_value: '2/4'
      shard_3:
        shard_value: '3/4'
      shard_4:
        shard_value: '4/4'
    maxParallel: 4 # 根据可用资源调整并行数量
  steps:
    - template: ./steps/git-checkout.yaml
    - template: ./steps/node_install.yaml
    - template: ./steps/yarn_install.yaml
    - template: ./steps/jest_tests.yaml
      parameters:
        shard: $(shard_value)

只需定义一次步骤模板，矩阵会自动为每个分片值生成独立作业，大幅减少配置冗余。

2. 避免重复安装依赖的两种方案 #

方案一：用缓存任务复用依赖

修改steps/yarn_install.yaml，加入缓存逻辑：

- task: Cache@2
  displayName: '缓存Yarn依赖'
  inputs:
    key: 'yarn | "$(Agent.OS)" | yarn.lock'
    path: 'node_modules'
    cacheHitVar: 'CACHE_RESTORED'

- script: yarn install --frozen-lockfile
  displayName: '安装依赖'
  condition: ne(variables.CACHE_RESTORED, 'true')

通过yarn.lock和操作系统作为缓存键，只有当依赖变更时才重新安装，否则直接复用缓存的node_modules，大幅缩短每个作业的依赖安装时间。

方案二：前置作业+工件共享依赖

先创建一个前置作业完成代码检出和依赖安装，再将node_modules打包为工件，后续分片作业直接下载使用：

主流水线新增前置作业

- job: prepare_dependencies
  displayName: '准备依赖'
  steps:
    - template: ./steps/git-checkout.yaml
    - template: ./steps/node_install.yaml
    - template: ./steps/yarn_install.yaml
    - task: PublishBuildArtifacts@1
      displayName: '发布依赖工件'
      inputs:
        PathtoPublish: 'node_modules'
        ArtifactName: 'node_modules'
        publishLocation: 'Container'

修改分片作业配置复用工件

- job: test_shards
  displayName: 'Jest 分片测试'
  dependsOn: prepare_dependencies
  strategy:
    matrix:
      shard_1:
        shard_value: '1/4'
      shard_2:
        shard_value: '2/4'
      shard_3:
        shard_value: '3/4'
      shard_4:
        shard_value: '4/4'
    maxParallel: 4
  steps:
    - template: ./steps/git-checkout.yaml
    - template: ./steps/node_install.yaml
    - task: DownloadBuildArtifacts@0
      displayName: '下载依赖工件'
      inputs:
        buildType: 'current'
        downloadType: 'single'
        artifactName: 'node_modules'
        downloadPath: '$(System.DefaultWorkingDirectory)'
    - template: ./steps/jest_tests.yaml
      parameters:
        shard: $(shard_value)

这种方式只需安装一次依赖，所有分片作业共享同一套依赖，适合依赖安装耗时较长的场景。

3. 最优组合：矩阵+缓存/工件 #

将作业矩阵与缓存或工件方案结合，既能简化配置，又能避免重复安装依赖。优先推荐缓存方案，配置更简洁；若缓存无法满足需求（如依赖包含编译后的二进制文件），再考虑工件方案。

内容的提问来源于stack exchange，提问作者Scott