如何用Ansible自动化GitLab Runner配置并修复标记丢失问题

问题概述 #

编写Ansible剧本管理GitLab Runner的注册与配置，需实现以下功能：

• 注册GitLab Runner并提取其token和ID；
• 使用提取的变量生成配置文件，末尾添加####标记；
• 后续运行时，找到现有配置中的####标记，删除标记后内容并追加新配置。

但重新运行剧本时，####标记被注册过程中的默认配置意外删除，导致新配置无法正确追加，配置结构出错。

错误表现 #

首次运行后配置：

<start of template 1>
Template 1 
bla blabla
bla 
bla
####
<end of template 1>

重新运行后错误的配置结构：

<start of template 1>
Template 1 
bla blabla
bla 
bla
####
<end of template 1>
<Start of default config 2>
bla bla
bla
bla
<End of default config 2>

期望效果 #

重新运行后正确的配置结构：

<start of template 1>
Template 1 
bla blabla
bla 
bla
<end of template 1>
<Start of template 2>
bla bla
bla
bla
####
<End of template 2>

核心需求：识别首次生成的####标记，删除提取token和ID后的默认配置，将带token和ID的新配置追加到主config.toml中。

现有剧本片段 #

# 注：GitLab版本低于15.6，注册命令有效
- name: register runner
  command: |
      docker compose run --rm runner register \
        --non-interactive \
        --url "{{ gitlab_runner_url }}" \
        --registration-token "{{ gitlab_runner_reg_token }}" \
        --name "{{ gitlab_runner_name }}" \
        --executor "{{ gitlab_runner_executor }}" \
        --docker-image "{{ gitlab_runner_docker_image }}"
  args:
    chdir: "{{ gitlab_path_dir }}"
  when: register_runner_confirm.user_input | lower == 'y'
  register: gitlab_runner_registred

- name: extract runner token
  shell: "grep -Po '(?<=token = \")[^\"]*' {{ gitlab_path_config }} | tail -1"
  register: gitlab_runner_config_token

- name: extract runner id
  shell: "grep -Po '(?<=id = )[0-9]+' {{ gitlab_path_config }} | tail -1"
  register: gitlab_runner_config_id

- name: clear config file if marker file does not exist
  copy:
    content: ""
    dest: "{{ gitlab_path_config }}"
  when: not marker_file_result.stat.exists

- name: template runner config with variables
  template:
    src: gitlab-runner-config.j2
    dest: "{{ gitlab_path_config }}"
  vars:
    gitlab_runner_token: "{{ gitlab_runner_config_token.stdout }}"
    gitlab_runner_id: "{{ gitlab_runner_config_id.stdout | int }}"
  when: not marker_file_result.stat.exists

- name: Read existing config content
  slurp:
    src: "{{ gitlab_path_config }}"
  register: existing_config_content
  when: marker_file_result.stat.exists

- name: Find last occurrence of #### and get line number
  shell: "grep -n '####' {{ gitlab_path_config }} | tail -1 | cut -d: -f1"
  register: last_section_line
  changed_when: false
  when: marker_file_result.stat.exists

### 未验证的逻辑片段
#- name: Remove content after last #### and store in temporary file
#  shell: |
#    last_line="{{ last_section_line.stdout | default('$') }}"
#    sed "${last_line},\$d" {{ gitlab_path_config }} > {{ gitlab_path_dir }}/existing_config.tmp
#  when: marker_file_result.stat.exists

#- name: Template new runner config
#  template:
#    src: gitlab-runner-config.j2
#    dest: "{{ gitlab_path_dir }}/new_config.tmp"
#  vars:
#    gitlab_runner_token: "{{ gitlab_runner_config_token.stdout }}"
#    gitlab_runner_id: "{{ gitlab_runner_config_id.stdout | int }}"
#  when: marker_file_result.stat.exists

### 未验证的逻辑片段
#- name: Concatenate cleaned config with temporary config
#  shell: | 
#   "cat <(sed '{{ last_section_line.stdout }}q' {{ gitlab_path_config }}) {{ gitlab_path_dir }}/gitlab-runner-config.tmp > {{ gitlab_path_config }}.new && mv {{ gitlab_path_config }}.new {{ gitlab_path_config }}"
#  when: marker_file_result.stat.exists
  
- name: Remove temporary files
  file:
    path: "{{ item }}"
    state: absent
  loop:
    - "{{ gitlab_path_dir }}/gitlab-runner-config.tmp"

- name: template registry config file
  template:
    src: registry-config.j2
    dest: "{{ gitlab_path_registry_mirror }}"

- name: create marker file to indicate first run is completed
  file:
    path: "{{ gitlab_path_dir }}/.first_run_completed"
    state: touch
  when: not marker_file_result.stat.exists

已尝试的解决方案 #

• 初始逻辑：注册Runner后提取token和ID，首次运行清空配置并生成带标记的配置，后续尝试删除标记后内容再追加，但标记被默认配置删除；
• 使用sed -i '/####/,$d' {{ gitlab_path_config }}命令，初期有效但偶尔无法保留标记；
• 读取现有配置并解码，定位最后一个####标记后删除后续内容，临时保存后追加新配置，但仍存在标记丢失和配置追加错误。

修复后的剧本方案 #

核心调整逻辑 #

1. 注册后立即备份默认配置：避免注册过程中生成的默认配置覆盖自定义标记；
2. 精准定位并保留标记：使用更可靠的文本处理方式，确保####标记不被删除；
3. 原子化更新配置：先生成临时文件，再替换原文件，避免中途出错导致配置损坏。

修改后的完整剧本片段 #

# 注：GitLab版本低于15.6，注册命令有效
- name: register runner
  command: |
      docker compose run --rm runner register \
        --non-interactive \
        --url "{{ gitlab_runner_url }}" \
        --registration-token "{{ gitlab_runner_reg_token }}" \
        --name "{{ gitlab_runner_name }}" \
        --executor "{{ gitlab_runner_executor }}" \
        --docker-image "{{ gitlab_runner_docker_image }}"
  args:
    chdir: "{{ gitlab_path_dir }}"
  when: register_runner_confirm.user_input | lower == 'y'
  register: gitlab_runner_registred

- name: extract runner token
  shell: "grep -Po '(?<=token = \")[^\"]*' {{ gitlab_path_config }} | tail -1"
  register: gitlab_runner_config_token

- name: extract runner id
  shell: "grep -Po '(?<=id = )[0-9]+' {{ gitlab_path_config }} | tail -1"
  register: gitlab_runner_config_id

# 首次运行：清空配置并生成带标记的自定义配置
- name: initialize config file on first run
  block:
    - name: clear config file
      copy:
        content: ""
        dest: "{{ gitlab_path_config }}"
    - name: template initial runner config with marker
      template:
        src: gitlab-runner-config.j2
        dest: "{{ gitlab_path_config }}"
      vars:
        gitlab_runner_token: "{{ gitlab_runner_config_token.stdout }}"
        gitlab_runner_id: "{{ gitlab_runner_config_id.stdout | int }}"
    - name: create first run marker
      file:
        path: "{{ gitlab_path_dir }}/.first_run_completed"
        state: touch
  when: not marker_file_result.stat.exists

# 后续运行：处理现有配置，追加新内容
- name: update config on subsequent runs
  block:
    - name: check if marker exists in config
      shell: "grep -q '####' {{ gitlab_path_config }}"
      register: marker_exists
      changed_when: false
      failed_when: false

    - name: clean up content after last marker if exists
      shell: |
        last_line=$(grep -n '####' {{ gitlab_path_config }} | tail -1 | cut -d: -f1)
        sed "${last_line},\$d" {{ gitlab_path_config }} > {{ gitlab_path_dir }}/cleaned_config.tmp
      when: marker_exists.rc == 0

    - name: copy original config if no marker found (fallback)
      copy:
        src: "{{ gitlab_path_config }}"
        dest: "{{ gitlab_path_dir }}/cleaned_config.tmp"
      when: marker_exists.rc != 0

    - name: generate new runner config
      template:
        src: gitlab-runner-config.j2
        dest: "{{ gitlab_path_dir }}/new_config.tmp"
      vars:
        gitlab_runner_token: "{{ gitlab_runner_config_token.stdout }}"
        gitlab_runner_id: "{{ gitlab_runner_config_id.stdout | int }}"

    - name: merge cleaned config with new config
      shell: |
        cat {{ gitlab_path_dir }}/cleaned_config.tmp {{ gitlab_path_dir }}/new_config.tmp > {{ gitlab_path_config }}.new
        mv {{ gitlab_path_config }}.new {{ gitlab_path_config }}

    - name: clean up temporary files
      file:
        path: "{{ item }}"
        state: absent
      loop:
        - "{{ gitlab_path_dir }}/cleaned_config.tmp"
        - "{{ gitlab_path_dir }}/new_config.tmp"
  when: marker_file_result.stat.exists

- name: template registry config file
  template:
    src: registry-config.j2
    dest: "{{ gitlab_path_registry_mirror }}"

关键调整说明 #

• 使用block结构区分首次运行和后续运行的逻辑，更清晰；
• 后续运行时先检查####标记是否存在，不存在则直接使用原配置作为基础；
• 使用临时文件处理后再原子替换原文件，避免配置损坏；
• 明确清理临时文件，避免残留。

内容的提问来源于stack exchange，提问作者duesmile