Java验证URL有效性问题：非200状态码时程序卡顿

需求 #

验证URL是否有效，即请求后返回状态码200。

调研情况 #

已查阅Stack Overflow相关技术讨论线程。

遇到的问题 #

1. 使用原生HttpURLConnection方案时，对返回非200状态码的URL，调用connection.getResponseCode()方法会出现程序卡顿现象。
2. 使用Apache Commons的UrlValidator类仅能验证URL格式合法性，无法判断实际请求后的状态，即使是返回500的无效URL，仍会被判定为有效。

测试资源 #

• 有效URL：https://archives.nseindia.com/content/historical/EQUITIES/2023/JUN/cm23JUN2023bhav.csv.zip（返回状态码200，浏览器可触发下载）
• 无效URL：https://archives.nseindia.com/content/historical/EQUITIES/2023/JUN/cm24JUN2023bhav.csv.zip（返回状态码500）

现有测试代码 #

方法1：原生HttpURLConnection实现 #

try {
    URL u = new URL(url1); // url1 -> 正常工作，url2 -> 出现问题
    HttpURLConnection connection = (HttpURLConnection) u.openConnection();
    connection.setRequestMethod("GET");

    int responseCode = connection.getResponseCode(); // 针对url2时程序卡顿
    if (responseCode == HttpURLConnection.HTTP_OK) {
        System.out.println("URL is working fine.");
    } else {
        System.out.println("URL returned a response code: " + responseCode);
    }
} catch (Exception e) {
    System.out.println("Error occurred while checking the URL: " + e.getMessage());
}

方法2：Apache Commons UrlValidator实现 #

UrlValidator urlValidator = new UrlValidator();
System.out.println(urlValidator.isValid(url1)); // 有效URL返回true
System.out.println(urlValidator.isValid(url2)); // 无效URL仍返回true

解决方案建议 #

1. 优化原生HttpURLConnection解决卡顿问题 #

核心是设置连接超时和读取超时，避免程序无限等待响应：

try {
    URL u = new URL(targetUrl);
    HttpURLConnection connection = (HttpURLConnection) u.openConnection();
    connection.setRequestMethod("GET");
    // 设置超时时间，单位：毫秒
    connection.setConnectTimeout(5000); // 连接超时5秒
    connection.setReadTimeout(5000);    // 读取超时5秒

    int responseCode = connection.getResponseCode();
    if (responseCode == HttpURLConnection.HTTP_OK) {
        System.out.println("URL有效，返回200状态码");
    } else {
        System.out.println("URL返回状态码：" + responseCode);
    }
    // 关闭连接释放资源
    connection.disconnect();
} catch (Exception e) {
    System.out.println("验证URL时出错：" + e.getMessage());
}

2. 使用HEAD请求替代GET请求 #

如果仅需验证状态码，无需下载资源，用HEAD请求可以减少数据传输量，提升效率：

try {
    URL u = new URL(targetUrl);
    HttpURLConnection connection = (HttpURLConnection) u.openConnection();
    connection.setRequestMethod("HEAD"); // 改为HEAD请求
    connection.setConnectTimeout(5000);
    connection.setReadTimeout(5000);

    int responseCode = connection.getResponseCode();
    if (responseCode == HttpURLConnection.HTTP_OK) {
        System.out.println("URL有效，返回200状态码");
    } else {
        System.out.println("URL返回状态码：" + responseCode);
    }
    connection.disconnect();
} catch (Exception e) {
    System.out.println("验证URL时出错：" + e.getMessage());
}

3. 使用Apache HttpClient替代原生实现 #

HttpClient对HTTP请求的处理更灵活、稳定，推荐用于复杂场景：
首先需要引入依赖（Maven为例）：

<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5</artifactId>
    <version>5.2.1</version>
</dependency>

实现代码：

import org.apache.hc.client5.http.classic.methods.HttpHead;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.core5.http.HttpStatus;

public class UrlValidator {
    public static boolean isUrlValid(String targetUrl) {
        try (CloseableHttpClient client = HttpClients.createDefault()) {
            HttpHead request = new HttpHead(targetUrl);
            // 设置超时
            request.setHeader("Connection", "close");
            try (CloseableHttpResponse response = client.execute(request)) {
                return response.getCode() == HttpStatus.SC_OK;
            }
        } catch (Exception e) {
            return false;
        }
    }

    public static void main(String[] args) {
        String validUrl = "https://archives.nseindia.com/content/historical/EQUITIES/2023/JUN/cm23JUN2023bhav.csv.zip";
        String invalidUrl = "https://archives.nseindia.com/content/historical/EQUITIES/2023/JUN/cm24JUN2023bhav.csv.zip";
        System.out.println("有效URL验证结果：" + isUrlValid(validUrl));
        System.out.println("无效URL验证结果：" + isUrlValid(invalidUrl));
    }
}

内容的提问来源于stack exchange，提问作者iCoder