基于Jenkins Pull方法实现跨服务器文件定期迁移需求咨询

Got it, let's break down your two file migration requirements step by step, since both center on pull-based workflows with Jenkins on the target server.

You have two solid options here—either a scheduled pull or a change-triggered pull (listening for updates):

Pull Method (Scheduled or On-Demand) #

This is the simplest approach, leveraging Jenkins' built-in scheduling and standard file transfer tools:

• Create a Freestyle Project in Jenkins on Server 2.
• Under "Build Triggers", you can set a schedule (if you want regular pulls) or leave it as manual trigger for on-demand migrations.
• Add a "Execute shell" build step with an rsync command to pull files from Server 1. Rsync is ideal here because it does incremental syncs (only transfers changed files) and supports compression:

  # Basic incremental sync (preserves permissions, timestamps)
  rsync -avz jenkins_user@server1:/path/to/source/directory/ /path/on/server2/target/directory/
  
  # Optional: Add --delete to remove files on Server 2 that no longer exist on Server 1
  rsync -avz --delete jenkins_user@server1:/path/to/source/directory/ /path/on/server2/target/directory/
  

• Critical Setup: Configure passwordless SSH between the Jenkins user on Server 2 and Server 1. Run ssh-keygen as the Jenkins user on Server 2, then copy the public key (~/.ssh/id_rsa.pub) to Server 1's ~/.ssh/authorized_keys file for the target user.

Listening/Change-Triggered Pull Method #

If you want to pull files immediately when they change on Server 1 (instead of waiting for a schedule), combine Jenkins with a polling or trigger mechanism:

• Use Jenkins' Poll SCM trigger (even if it's not a code repo). Under "Build Triggers", check "Poll SCM" and set a frequent schedule (e.g., * * * * * to check every minute). Then, in your build step, add a check to see if files on Server 1 have changed before running rsync:

  # Compare file checksums to avoid unnecessary syncs
  rsync -avz --dry-run jenkins_user@server1:/path/to/source/directory/ /path/on/server2/target/directory/ | grep -q "Number of files transferred: [1-9]"
  if [ $? -eq 0 ]; then
      rsync -avz jenkins_user@server1:/path/to/source/directory/ /path/on/server2/target/directory/
      echo "Sync completed - changes detected"
  else
      echo "No changes to sync"
      exit 0
  fi
  

• Alternatively, if you can run a lightweight script on Server 1, set up an inotifywait watcher there that sends a webhook to Jenkins' Generic Webhook Trigger plugin when files change. This triggers the Jenkins job to pull the updated files immediately—still pull-based, since Jenkins initiates the transfer.

Since this requires a strict 15-minute schedule and pull-only workflow, here's the straightforward solution:

• Create a new Freestyle Project in Jenkins on Server B.
• Under "Build Triggers", set the Cron Schedule to */15 * * * *—this tells Jenkins to run the job every 15 minutes, every hour, every day, etc.
• Add an "Execute shell" build step with the rsync command tailored to your version packages:

  # Sync all version packages from Server-A to Server-B
  rsync -avz --progress jenkins_user@serverA:/path/to/version/packages/ /path/on/serverB/storage/directory/
  
  # If you only want the latest version (e.g., files with "latest" in the name)
  rsync -avz jenkins_user@serverA:/path/to/version/packages/*latest*.tar.gz /path/on/serverB/storage/directory/
  

• Extra Tips:
  • Add logging to track syncs: Redirect rsync output to a log file, e.g., rsync ... >> /var/log/jenkins/serverA_sync.log 2>&1
  • Enable Jenkins build history to keep records of every sync run, including success/failure status.
  • Test the cron schedule first with a shorter interval (e.g., */1 * * * *) to ensure it works as expected before switching to 15 minutes.

General Best Practices #

• Always test sync commands manually first before adding them to Jenkins to catch permission or path issues.
• Use rsync --dry-run to preview what will be transferred without making actual changes.
• Ensure the Jenkins user on the target server has write permissions to the destination directory.

内容的提问来源于stack exchange，提问作者K.D.