如何在Jenkins上触发Serenity BDD任务失败？含阈值配置需求

Great questions! Let's break down each one with practical steps tailored to Jenkins and Serenity BDD:

1. Fail Jenkins Job on Any Feature/Test Failure #

By default, Serenity BDD integrates with Maven/Gradle to fail the build when tests fail, but you need to make sure your setup is configured correctly. Here's how to enforce this:

• Use the right build command: Instead of mvn test, run mvn clean verify. The verify goal triggers Serenity's test result validation step, which will fail the Maven build immediately if any test scenario fails. If you've overridden this behavior via a property, explicitly set it in your command:

  mvn clean verify -Dserenity.fail.on.test.failure=true
  

• Configure Jenkins to parse test results: Add a Publish JUnit test result report post-build action in your Jenkins job. Point it to the Serenity-generated JUnit XML files at target/site/serenity/junit/*.xml. Jenkins will use these files to track test failures and mark the job as failed automatically.
• Double-check Serenity config: Ensure your serenity.properties file doesn't have serenity.fail.on.test.failure=false set—this would override the default behavior and let the build pass even with failed tests.

2. Fail Job Only When Failure Rate Exceeds a Threshold (e.g., 80%) #

For this, you'll need to calculate the test failure rate and add a conditional check in Jenkins. Here are two common approaches depending on your job type:

Option 1: Freestyle Project (Using Groovy Postbuild Plugin) #

1. Install the Groovy Postbuild Plugin from Jenkins Marketplace.
2. Add an Execute Groovy script post-build action, then use this script to calculate failure rate and enforce the threshold:

   def testResult = manager.build.testResultAction
   if (testResult) {
       def totalTests = testResult.totalCount
       def failedTests = testResult.failCount // Include testResult.skipCount if you want to count skipped tests as failures
       
       if (totalTests > 0) {
           def failureRate = (failedTests / totalTests) * 100
           def threshold = 80
           
           if (failureRate > threshold) {
               manager.build.setResult(hudson.model.Result.FAILURE)
               manager.addWarningBadge("Failure rate: ${failureRate.round(2)}% (exceeds ${threshold}% threshold)")
           } else {
               manager.addInfoBadge("Failure rate: ${failureRate.round(2)}% (within ${threshold}% threshold)")
           }
       } else {
           manager.addInfoBadge("No tests executed—skipping threshold check")
       }
   }
   

   Adjust failedTests to include skipped tests if needed, and tweak the threshold value as required.

Option 2: Pipeline Project (Native Scripting) #

If you're using a Jenkinsfile, integrate this logic into your post-build steps:

pipeline {
    agent any
    stages {
        stage('Run Serenity Tests') {
            steps {
                sh 'mvn clean verify'
            }
        }
    }
    post {
        always {
            // Parse Serenity's JUnit results
            junit 'target/site/serenity/junit/*.xml'
            
            script {
                def testResult = currentBuild.testResultAction
                if (testResult) {
                    def totalTests = testResult.totalCount
                    def failedTests = testResult.failCount
                    def threshold = 80
                    
                    if (totalTests > 0) {
                        def failureRate = (failedTests / totalTests) * 100
                        echo "Calculated failure rate: ${failureRate.round(2)}%"
                        
                        if (failureRate > threshold) {
                            error "Build failed: Failure rate (${failureRate.round(2)}%) exceeds threshold of ${threshold}%"
                        }
                    } else {
                        echo "No tests run—threshold check skipped"
                    }
                }
            }
        }
    }
}

Bonus: Using Quality Gates Plugin #

For a no-code option, install the Quality Gates Plugin, then add a post-build action to set up a gate:

• Set the condition to Test Failure Rate > 80%
• Choose "Mark build as failed" when the condition is met

内容的提问来源于stack exchange，提问作者JavaMan