如何加速GitLab Runner？缓存及磁盘空间问题求助

环境与CI配置 #

我在AWS t2.medium实例（gp3磁盘）上通过Docker部署了GitLab Runner，使用的.gitlab-ci.yml配置如下：

# This file is a template, and might need editing before it works on your project.
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml

# Official language image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python/tags/
image: python:3.10

# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

# Pip's cache doesn't store the python packages
# https://pip.pypa.io/en/stable/topics/caching/
#
# If you want to also cache the installed packages, you have to install
# them in a virtualenv and cache it as well.
cache:
    paths:
        - .cache/pip
        - venv/

before_script:
    - python --version ; pip --version # For debugging
    - pip install virtualenv
    - virtualenv venv
    - source venv/bin/activate

stages:
    - build
    - lint

build:
    stage: build
    script:
        - pip install -r requirements-dev.txt

lint:
    stage: lint
    script:
        - flake8 .
        - mypy src

formatting:
    stage: lint
    script:
        - black --check .
        - isort --check .

遇到的问题 #

• 运行速度极慢（例如build阶段耗时8分钟），缓存环节尤为突出；
• 每次任务后都会创建缓存，但linting和formatting任务并未修改venv目录，原因不明；
• 多次运行后16GB磁盘被占满，导致缓存创建失败，如何配置GitLab Runner清理磁盘？

解决方案 #

1. 优化运行速度与缓存效率 #

• 拆分缓存策略：给缓存设置基于依赖文件的哈希key，只有当requirements-dev.txt变更时才更新venv缓存，避免无意义的缓存同步。修改cache配置：

  cache:
    paths:
      - .cache/pip
      - venv/
    key:
      files:
        - requirements-dev.txt
      prefix: $CI_JOB_NAME
  

• 替换virtualenv为内置venv：Python 3.10自带venv模块，无需额外安装virtualenv，修改before_script：

  before_script:
    - python --version ; pip --version
    - python -m venv venv
    - source venv/bin/activate
  

• 预构建自定义镜像：将flake8、mypy、black、isort等工具提前打包到自定义Docker镜像中，减少任务启动时的依赖安装时间。
• 改用分布式缓存：如果当前用的是本地缓存，切换到S3或GitLab自带的分布式缓存服务，本地缓存在Docker环境下同步效率极低。

2. 解决非必要缓存更新问题 #

• 设置缓存拉取策略：lint和formatting任务仅需读取缓存，无需写入，给这两个任务添加cache:policy: pull，避免触发缓存更新：

  lint:
    stage: lint
    cache:
      policy: pull
    script:
      - flake8 .
      - mypy src
  
  formatting:
    stage: lint
    cache:
      policy: pull
    script:
      - black --check .
      - isort --check .
  

• 排除venv中的临时文件：检查venv目录是否有工具生成的临时文件（如日志、缓存），在cache paths中排除这些文件，例如：

  cache:
    paths:
      - .cache/pip
      - venv/lib/python3.10/site-packages/
  

3. 配置磁盘自动清理 #

• 修改Runner配置文件：在config.toml中设置缓存最大容量，超过后自动清理旧缓存：

  [[runners]]
    name = "docker-runner"
    url = "https://your-gitlab-instance.com/"
    token = "your-token"
    executor = "docker"
    [runners.cache]
      Type = "docker"
      Path = "/cache"
      Shared = true
      MaxSize = "10GB" # 限制缓存目录最大占用10GB
  

• 添加定时清理任务：给Runner主机添加cron任务，定期清理旧构建目录：

  # 每天凌晨清理7天前的构建文件
  0 0 * * * find /home/gitlab-runner/builds -type d -mtime +7 -exec rm -rf {} \;
  

• 设置缓存过期时间：在CI配置中给cache添加过期规则，自动清理旧缓存：

  cache:
    paths:
      - .cache/pip
      - venv/
    key:
      files:
        - requirements-dev.txt
      prefix: $CI_JOB_NAME
    expire_in: 1 week
  

内容的提问来源于stack exchange，提问作者Dmitry Kabanov