Java EE：分批次循环调用Servlet解决大文件压缩超时问题

Great question—this is a super common pain point when dealing with large batch operations in Servlet-based apps. Let’s fix both the timeout problem and the progress visibility gap with practical, actionable steps:

1. Rearrange Backend Logic: Async Servlet + Batch Processing (Ditch the "Loop Calling Servlet" Approach) #

First off, looping to call your Servlet for every 50 files isn’t ideal—it creates unnecessary HTTP connections and makes progress tracking messy. Instead, use asynchronous Servlets to handle the batch work in the background, while letting you update progress along the way.

Steps for Backend Setup: #

• Enable Async Support: Mark your Servlet with asyncSupported = true to allow long-running tasks without tying up the request thread.

  @WebServlet(name = "CompressionServlet", urlPatterns = "/compress", asyncSupported = true)
  public class CompressionServlet extends HttpServlet {
      // Use a managed thread pool (don't create threads manually!)
      private final ExecutorService executor = Executors.newFixedThreadPool(5);
  
      @Override
      protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
          AsyncContext asyncContext = request.startAsync();
          asyncContext.setTimeout(300000); // 5 mins timeout, adjust as needed
  
          // Get user's selected files (pass this from frontend, or fetch from session)
          List<String> allFiles = (List<String>) request.getSession().getAttribute("selectedFiles");
          String taskId = UUID.randomUUID().toString();
          request.getSession().setAttribute(taskId + "_progress", 0);
  
          executor.submit(() -> {
              int batchSize = 50;
              int totalBatches = (int) Math.ceil((double) allFiles.size() / batchSize);
              File finalZip = new File("/temp/compressed_" + taskId + ".zip");
  
              try (ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(finalZip))) {
                  for (int i = 0; i < totalBatches; i++) {
                      int start = i * batchSize;
                      int end = Math.min(start + batchSize, allFiles.size());
                      List<String> batchFiles = allFiles.subList(start, end);
  
                      // Compress this batch into the zip
                      for (String filePath : batchFiles) {
                          File file = new File(filePath);
                          zos.putNextEntry(new ZipEntry(file.getName()));
                          Files.copy(file.toPath(), zos);
                          zos.closeEntry();
                      }
  
                      // Update progress
                      int progress = (int) (((i + 1.0) / totalBatches) * 100);
                      request.getSession().setAttribute(taskId + "_progress", progress);
                  }
                  // Mark task as complete
                  request.getSession().setAttribute(taskId + "_complete", true);
                  request.getSession().setAttribute(taskId + "_zipPath", finalZip.getAbsolutePath());
              } catch (Exception e) {
                  request.getSession().setAttribute(taskId + "_error", e.getMessage());
              } finally {
                  asyncContext.complete();
              }
          });
  
          // Return task ID to frontend for progress tracking
          response.setContentType("application/json");
          response.getWriter().write("{\"taskId\": \"" + taskId + "\"}");
      }
  }
  

• Add a Progress Endpoint: Create a simple Servlet to return the current progress for a task ID:

  @WebServlet("/progress")
  public class ProgressServlet extends HttpServlet {
      @Override
      protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
          String taskId = request.getParameter("taskId");
          Integer progress = (Integer) request.getSession().getAttribute(taskId + "_progress");
          Boolean isComplete = (Boolean) request.getSession().getAttribute(taskId + "_complete");
          String error = (String) request.getSession().getAttribute(taskId + "_error");
  
          response.setContentType("application/json");
          JSONObject json = new JSONObject();
          json.put("progress", progress != null ? progress : 0);
          json.put("complete", isComplete != null ? isComplete : false);
          json.put("error", error != null ? error : null);
          response.getWriter().write(json.toString());
      }
  }
  

• Add a Download Endpoint: Once progress hits 100%, let frontend fetch the zip:

  @WebServlet("/download")
  public class DownloadServlet extends HttpServlet {
      @Override
      protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
          String taskId = request.getParameter("taskId");
          String zipPath = (String) request.getSession().getAttribute(taskId + "_zipPath");
          if (zipPath == null) {
              response.sendError(HttpServletResponse.SC_NOT_FOUND, "Zip file not ready");
              return;
          }
  
          File zipFile = new File(zipPath);
          response.setContentType("application/zip");
          response.setHeader("Content-Disposition", "attachment; filename=\"compressed_files.zip\"");
          Files.copy(zipFile.toPath(), response.getOutputStream());
  
          // Optional: Delete temp file after download
          zipFile.delete();
          request.getSession().removeAttribute(taskId + "_zipPath");
      }
  }
  

2. Frontend: Track Progress & Trigger Download Automatically #

The key here is to avoid blocking the user and keep them updated. Here’s how to implement the flow:

Sample Frontend JS/HTML: #

<div>
  <button id="downloadBtn">Start Compression & Download</button>
  <div id="progressContainer" style="display: none;">
    <div id="progressBar" style="width: 0%; height: 20px; background: #4CAF50;"></div>
    <p>Progress: <span id="progressText">0%</span></p>
  </div>
  <div id="errorMsg" style="color: red; display: none;"></div>
</div>

<script>
document.getElementById('downloadBtn').addEventListener('click', async () => {
  const btn = document.getElementById('downloadBtn');
  const progressContainer = document.getElementById('progressContainer');
  const progressBar = document.getElementById('progressBar');
  const progressText = document.getElementById('progressText');
  const errorMsg = document.getElementById('errorMsg');

  btn.disabled = true;
  progressContainer.style.display = 'block';
  errorMsg.style.display = 'none';

  try {
    // Step 1: Start the compression task
    const startResponse = await fetch('/compress', { method: 'POST' });
    const { taskId } = await startResponse.json();

    // Step 2: Poll for progress every 1 second
    const progressInterval = setInterval(async () => {
      const progressResponse = await fetch(`/progress?taskId=${taskId}`);
      const { progress, complete, error } = await progressResponse.json();

      if (error) {
        clearInterval(progressInterval);
        errorMsg.textContent = `Error: ${error}`;
        errorMsg.style.display = 'block';
        btn.disabled = false;
        return;
      }

      // Update UI
      progressBar.style.width = `${progress}%`;
      progressText.textContent = `${progress}%`;

      // Step 3: When complete, trigger download
      if (complete) {
        clearInterval(progressInterval);
        const downloadLink = document.createElement('a');
        downloadLink.href = `/download?taskId=${taskId}`;
        downloadLink.click();
        btn.disabled = false;
        progressContainer.style.display = 'none';
      }
    }, 1000);
  } catch (err) {
    errorMsg.textContent = `Failed to start compression: ${err.message}`;
    errorMsg.style.display = 'block';
    btn.disabled = false;
  }
});
</script>

3. Extra Optimizations #

• Use Redis for Progress Tracking: If your app is scaled across multiple servers, storing progress in Session won’t work—switch to Redis (or a shared database) to track task progress globally.
• Clean Up Temp Files: Add a scheduled task to delete old, unclaimed zip files (e.g., files older than 24 hours).
• Cancel Support: Let users cancel ongoing tasks by adding a "Cancel" button that triggers a request to the backend to stop the thread and clean up resources.

内容的提问来源于stack exchange，提问作者user9446905