Jenkins声明式流水线大RPM文件传递替代方案咨询

Great question—stashing large files like your 215MB RPM is definitely not ideal, since Jenkins' stash/unstash is optimized for small artifacts and can get slow or hit limits with bigger files. Let's go through a few better alternatives that fit your pipeline setup:

1. Use a Shared Filesystem #

The simplest fix is to use a shared storage volume (like NFS, AWS EFS, or a network-mounted directory) that all your Jenkins slaves can access. This way, you don't need to transfer the RPM at all—both stages can read/write directly to the same path.

Here's how you'd adjust your Jenkinsfile:

pipeline {
    agent any
    options {
        timestamps()
    }
    // Define a shared path accessible by all slaves
    environment {
        SHARED_RPM_PATH = '/shared/jenkins-artifacts/myapp.rpm'
    }
    stages {
        stage('Gradle: build') {
            agent {
                docker { image 'some-internal-image' }
            }
            steps {
                sh """
                    chmod +x gradlew
                    ./gradlew buildRpm
                    # Copy RPM to shared directory
                    cp Server/target/myapp.rpm ${SHARED_RPM_PATH}
                """
            }
        }
        stage('Gradle: build docker image') {
            steps {
                sh """
                    chmod +x gradlew
                    # Grab RPM directly from shared storage
                    cp ${SHARED_RPM_PATH} Server/target/myapp.rpm
                    ./gradlew buildDockerImage
                """
            }
            post {
                always {
                    // Optional: Clean up to avoid leftover artifacts
                    sh "rm -f ${SHARED_RPM_PATH}"
                }
            }
        }
    }
}

Pros: No file transfer overhead, minimal pipeline changes.
Cons: Requires maintaining a shared storage system accessible to all Jenkins agents.

2. Publish the RPM to an Internal Package Repository #

For a more scalable, production-ready approach, upload your built RPM to an internal package repository (like Nexus or Artifactory) in the first stage, then pull it down in the second stage. This also gives you versioned artifact storage for future reuse.

Modified Jenkinsfile example:

pipeline {
    agent any
    options {
        timestamps()
    }
    environment {
        RPM_REPO_URL = 'http://your-internal-repo/repository/rpms/'
        RPM_FILENAME = 'myapp.rpm'
        // Fetch version from Gradle (adjust command to match your setup)
        RPM_VERSION = sh(script: './gradlew printVersion --quiet', returnStdout: true).trim()
    }
    stages {
        stage('Gradle: build') {
            agent {
                docker { image 'some-internal-image' }
            }
            steps {
                sh """
                    chmod +x gradlew
                    ./gradlew buildRpm
                """
            }
            post {
                success {
                    // Upload to repository (use Jenkins plugins like Nexus Artifact Uploader for cleaner code)
                    sh """
                        curl -u \${REPO_USER}:\${REPO_PASS} -X PUT \
                        -T Server/target/${RPM_FILENAME} \
                        ${RPM_REPO_URL}/${RPM_VERSION}/${RPM_FILENAME}
                    """
                }
            }
        }
        stage('Gradle: build docker image') {
            steps {
                sh """
                    chmod +x gradlew
                    // Pull RPM from repository
                    curl -u \${REPO_USER}:\${REPO_PASS} -O \
                    ${RPM_REPO_URL}/${RPM_VERSION}/${RPM_FILENAME}
                    mv ${RPM_FILENAME} Server/target/
                    ./gradlew buildDockerImage
                """
            }
        }
    }
}

Pros: Versioned artifact management, no dependency on shared storage, reusable across other pipelines.
Cons: Requires setting up and maintaining an internal package repository.

3. Pin to the Same Agent (If Feasible) #

If your pipeline doesn't strictly need to run on different nodes, you can force both stages to use the same agent. Use reuseNode true in the Docker agent to ensure the container runs on the same node as the pipeline, so the RPM stays local.

Example:

pipeline {
    agent { label 'build-slave' } // Lock to a specific agent label
    options {
        timestamps()
    }
    stages {
        stage('Gradle: build') {
            agent {
                docker { 
                    image 'some-internal-image' 
                    reuseNode true // Run Docker container on the same agent node
                }
            }
            steps {
                sh """
                    chmod +x gradlew
                    ./gradlew buildRpm
                """
            }
        }
        stage('Gradle: build docker image') {
            steps {
                sh """
                    chmod +x gradlew
                    ./gradlew buildDockerImage
                """
            }
        }
    }
}

Pros: Zero file transfer, simplest pipeline changes.
Cons: Less flexible—you can't leverage different node resources for each stage.

Final Recommendation #

If you have a shared storage system already in place, go with option 1 for a quick win. For long-term scalability and best practices, option 2 (internal repository) is the way to go, as it aligns with modern CI/CD artifact management patterns.

内容的提问来源于stack exchange，提问作者Ido Ran