基于Spring实现并发调用多站点的RESTful接口作业开发求助

1. 补充必要依赖 #

你需要新增HTML解析工具Jsoup来提取页面标题，Maven依赖配置如下：

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.17.2</version>
</dependency>

如果用Gradle，对应引入即可。

2. 定义接口统一返回实体 #

新建ScrapeResult.java用来封装接口返回的三个字段：

package com.example.test;

import java.util.List;

public class ScrapeResult {
    private int successCount;
    private int failCount;
    private List<String> titles;

    public ScrapeResult(int successCount, int failCount, List<String> titles) {
        this.successCount = successCount;
        this.failCount = failCount;
        this.titles = titles;
    }

    // 自动生成getter和setter
    public int getSuccessCount() { return successCount; }
    public void setSuccessCount(int successCount) { this.successCount = successCount; }
    public int getFailCount() { return failCount; }
    public void setFailCount(int failCount) { this.failCount = failCount; }
    public List<String> getTitles() { return titles; }
    public void setTitles(List<String> titles) { this.titles = titles; }
}

3. 优化RestTemplate配置，添加超时避免请求卡住 #

修改TestApplication.java中的RestTemplate Bean配置：

package com.example.test;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;

@SpringBootApplication
public class TestApplication {

    @Bean
    public RestTemplate getRestTemplate(){
        SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
        // 连接超时5秒，读取超时10秒，可自行调整
        factory.setConnectTimeout(5000);
        factory.setReadTimeout(10000);
        return new RestTemplate(factory);
    }

    public static void main(String[] args) {
        SpringApplication.run(TestApplication.class, args);
    }
}

4. 重写Controller接口逻辑 #

修改APIcontroller.java实现并发请求、标题提取、结果统计逻辑：

package com.example.test;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

@RestController
public class APIcontroller {

    @Autowired
    private RestTemplate restTemplate;

    // 固定目标站点列表
    private static final List<String> TARGET_URLS = List.of(
            "https://pizzerijalimbo.si/meni/",
            "https://pizzerijalimbo.si/kontakt/",
            "https://pizzerijalimbo.si/my-account/",
            "https://pizzerijalimbo.si/o-nas/"
    );

    @GetMapping("/scrape")
    public ScrapeResult scrapeSites(@RequestParam Integer concurrency) {
        // 参数校验
        if (concurrency < 1 || concurrency > 4) {
            throw new IllegalArgumentException("并发数取值范围为1-4");
        }

        // 线程安全的统计变量和结果集合
        AtomicInteger successCount = new AtomicInteger(0);
        AtomicInteger failCount = new AtomicInteger(0);
        List<String> titles = new CopyOnWriteArrayList<>();
        CountDownLatch latch = new CountDownLatch(TARGET_URLS.size());

        // 按入参创建对应大小的线程池控制并发度
        ExecutorService executor = Executors.newFixedThreadPool(concurrency);
        try {
            for (String url : TARGET_URLS) {
                executor.submit(() -> {
                    try {
                        // 请求页面内容
                        String htmlContent = restTemplate.getForObject(url, String.class);
                        // 解析提取标题
                        Document doc = Jsoup.parse(htmlContent);
                        String title = doc.title();
                        titles.add(title);
                        successCount.incrementAndGet();
                    } catch (Exception e) {
                        failCount.incrementAndGet();
                    } finally {
                        latch.countDown();
                    }
                });
            }
            // 等待所有请求执行完成
            latch.await();
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        } finally {
            executor.shutdown();
        }

        return new ScrapeResult(successCount.get(), failCount.get(), new ArrayList<>(titles));
    }
}

5. 核心注意点 #

• 多线程场景下必须用原子类统计成功失败次数，用线程安全集合存储标题，避免数据错乱
• 所有请求逻辑必须包裹异常捕获，单个请求失败不会影响其他任务执行
• 必须配置RestTemplate超时时间，避免单个请求卡住导致整个接口超时
• 并发数为1时线程池只有1个线程，所有请求会串行执行；并发数为4时4个线程同时执行，刚好对应4个目标站点，实现全并行

内容的提问来源于stack exchange，提问作者AquaBalls