Spring Batch内存泄漏求助：JobExplorer是否为泄漏原因？

启动应用后仅5分钟内存占用就增加5GB，1小时后触发OOM杀死进程。有人指出JobExplorer可能导致泄漏，请问是否应该停用JobExplorer？恳请帮忙排查。

使用环境 #

java 11
spring boot 2.7.1
spring-boot-starter-batch 2.4.0

相关代码 #

BlockProcessConfiguration #

@Configuration
@RequiredArgsConstructor
@Slf4j
@Profile("block")
public class BlockProcessConfiguration {

@Value("${isStanby:false}")
private Boolean isStanby;

    @Scheduled(fixedDelay = 500)
    public String launch() throws JobInstanceAlreadyCompleteException, JobExecutionAlreadyRunningException, JobParametersInvalidException, JobRestartException {
        if (isStanby != null && isStanby) {
            Boolean isRunningJob = jobValidator.isExistLatestRunningJob(JOB_NAME, 5000);
            if (isRunningJob) {
                return "skip";
            }
        }

        return "completed";
    }
}

JobValidator #

import java.util.*;

@RequiredArgsConstructor
@Slf4j
@Component
public class JobValidator {
    public enum batchMode {
        RECOVER, FORWARD
    }
    private final JobExplorer jobExplorer;

    public Boolean isExistLatestRunningJob(String jobName, long jobTTL) {
        List<JobInstance> jobInstances = jobExplorer.findJobInstancesByJobName(jobName, 0, 10000);
        if (jobInstances.size() > 0) {
            List<JobExecution> jobExecutions = jobExplorer.getJobExecutions(jobInstances.get(0));
            jobInstances.clear();
            if (jobExecutions.size() > 0) {
                JobExecution jobExecution = jobExecutions.get(0);
                jobExecutions.clear();
//                boolean isRunning = jobExecution.isRunning();
                Date createTime = jobExecution.getCreateTime();
                long now = new Date().getTime();
                long timeFrame =  now - createTime.getTime();
                log.info("createTime.getTime() : {}", createTime.getTime());
                log.info("isExistLatestRunningJob found jobExecution :  id, status, timeFrame, jobTTL : {}, {}, {}, {}", jobExecution.getJobId(), jobExecution.getStatus(), timeFrame, jobTTL);
//                if (jobExecution.isRunning() && (now.getTime() - createTime.getTime()) < jobTTL) {
                if ( timeFrame < jobTTL ) {
                    log.info("isExistLatestRunningJob result : {}", true);
                    log.info("Job is already running, skip this job, job name : {}", jobName);
                    return true;
                }
            }
        }
        return false;
    }

    public Boolean isExecutableJob(String jobName, String paramKey, Long paramValue) {
        List<JobInstance> jobInstances = jobExplorer.findJobInstancesByJobName(jobName, 0, 1);
        if (jobInstances.size() > 0) {
            List<JobExecution> jobExecutions = jobExplorer.getJobExecutions(jobInstances.get(0));
            if (jobExecutions.size() > 0) {
                JobExecution jobExecution = jobExecutions.get(0);
                JobParameters jobParameters = jobExecution.getJobParameters();
                Optional<Long> blockNumber = Optional.ofNullable(jobParameters.getLong(paramKey));
                if (blockNumber.isPresent() && blockNumber.get().equals(paramValue)) {
                    if (jobExecution.getStatus().equals(BatchStatus.STARTED)) {
//                    throw new RuntimeException("waiting until previous job done");
                        log.info("waiting until previous job done ... : {}", jobName);
                        return false;
                    }
                }
            }
        }
        return true;
    }

    public Long getStartNumberFromBatch(String jobName, String batchMode, String paramKey1, String paramKey2, long defaultValue) {
        List<JobInstance> jobInstances = jobExplorer.findJobInstancesByJobName(jobName, 0, 20);
        ArrayList<Long> failExecutionNumbers = new ArrayList<>();
        ArrayList<Long> successExecutionNumbers = new ArrayList<>();
        ArrayList<Long> successEndExecutionNumbers = new ArrayList<>();
        ArrayList<JobExecution> executions = new ArrayList<>();
        jobInstances.stream().map(jobInstance -> jobExplorer.getJobExecutions(jobInstance)).forEach(jobExecution -> {
            JobParameters jobParameters = jobExecution.get(0).getJobParameters();
            Optional<Long> param1 = Optional.ofNullable(jobParameters.getLong(paramKey1));
            Optional<Long> param2 = Optional.ofNullable(jobParameters.getLong(paramKey2));
            if (param1.isPresent() && param2.isPresent()) {
                if (jobExecution.get(0).getExitStatus().getExitCode().equals("FAILED")) {
                    failExecutionNumbers.add(param1.get());
                } else {
                    successExecutionNumbers.add(param1.get());
                    successEndExecutionNumbers.add(param2.get());
                }
            }
        });

        if (failExecutionNumbers.size() == 0 && successExecutionNumbers.size() == 0) {
            return defaultValue;
        }

        long successMax = defaultValue;
        long failMin = defaultValue;
        if (successEndExecutionNumbers.size() > 0) {
            successMax = Collections.max(successEndExecutionNumbers);
        }
        if (failExecutionNumbers.size() > 0) {
            failExecutionNumbers.removeIf(successExecutionNumbers::contains);
            if (failExecutionNumbers.size() > 0) {
                failMin = Collections.min(failExecutionNumbers);
            } else {
                return successMax;
            }
        }
        if (Objects.equals(batchMode, JobValidator.batchMode.RECOVER.toString())) {
            return Math.min(failMin, successMax);
        } else {
            return Math.max(failMin, successMax);
        }
    }
}

排查分析与解决方案 #

核心结论：不要停用JobExplorer #

JobExplorer是Spring Batch的核心组件，负责查询作业元数据（实例、执行记录、参数等），停用它会导致作业状态跟踪、重启、幂等性校验等核心功能失效，绝非解决内存泄漏的合理方案。

内存泄漏的关键诱因 #

1. 定时任务频率过高：@Scheduled(fixedDelay = 500)意味着每500毫秒就调用一次作业状态查询，每秒执行2次，高频触发大量元数据查询与对象加载。
2. 查询范围过大：isExistLatestRunningJob中一次查询10000个JobInstance，即使后续只用到第一个实例，剩余9999个实例对象仍会占用内存；且每个实例会加载对应的JobExecution对象，进一步加剧内存消耗。
3. 低效的元数据查询方式：多次调用findJobInstancesByJobName后再取第一个实例，不如直接调用jobExplorer.getLastJobExecution(jobName)获取最新执行记录，减少不必要的对象加载。
4. 手动clear集合无效：虽然代码中调用了jobInstances.clear()和jobExecutions.clear()，但这些集合是JobExplorer返回的，底层可能存在框架级引用，无法彻底释放对象；且高频调用下，对象创建速度远快于GC回收速度，导致内存堆积。

具体修复方案 #

1. 降低定时任务频率：根据业务实际需求调整fixedDelay，比如改为5000（5秒），减少查询频次。

   @Scheduled(fixedDelay = 5000) // 调整为5秒一次
   public String launch() throws ... {
       // 原有逻辑
   }
   

2. 缩小查询范围并优化查询方式：
   • 在isExistLatestRunningJob中，直接使用getLastJobExecution获取最新执行记录，避免加载多余实例：

     public Boolean isExistLatestRunningJob(String jobName, long jobTTL) {
         JobExecution lastExecution = jobExplorer.getLastJobExecution(jobName);
         if (lastExecution != null) {
             Date createTime = lastExecution.getCreateTime();
             long now = new Date().getTime();
             long timeFrame = now - createTime.getTime();
             log.info("isExistLatestRunningJob found jobExecution :  id, status, timeFrame, jobTTL : {}, {}, {}, {}", lastExecution.getJobId(), lastExecution.getStatus(), timeFrame, jobTTL);
             if (timeFrame < jobTTL) {
                 log.info("Job is already running, skip this job, job name : {}", jobName);
                 return true;
             }
         }
         return false;
     }
     

3. 优化getStartNumberFromBatch查询：根据业务需求减少查询的JobInstance数量，比如从20调整为10，避免加载过多历史元数据。
4. 内存验证：使用JVM工具（如jmap、jvisualvm）生成堆转储文件，分析内存中占比最高的对象，确认是否为JobInstance、JobExecution等元数据对象，进一步验证修复效果。

内容的提问来源于stack exchange，提问作者Brian