如何正确关闭RestClient/HttpClient套接字以高效检测大量代理

需要快速检测5万+代理（以azenv.net作为验证地址），但当前代码存在套接字耗尽问题：

• 用using包裹RestClient并自动Dispose后，套接字并未实际关闭，运行期间执行netstat -ano | find /c "TCP"可看到约5000个TCP套接字处于Established等状态，中途因套接字耗尽仅能收到超时错误。
• 即使代理连接超时，也希望立即关闭对应套接字（不会复用该代理）；尝试传入SocketsHttpHandler手动释放无效，同时想了解是否能获取套接字状态，等待完全释放后再执行下一次检测。

当前代码片段：

string proxyFilePath = @"proxyList.txt";
string[] proxyLines = await File.ReadAllLinesAsync(proxyFilePath);
var tasks = new List<Task>();
var good = 0;
var bad = 0;
ParallelOptions parallelOptions = new()
{
    MaxDegreeOfParallelism = 300,
};
await Parallel.ForEachAsync(proxyLines, parallelOptions, async (proxyLine, token) =>
{
    var trim = proxyLine.Split(":");
    WebProxy proxy = new WebProxy(new Uri($"socks5://{trim[0]}:{trim[1]}"));
    var rest = new RestClientOptions()
    {
        Proxy = proxy,
        MaxTimeout = 5000,
    };

    using (var client = new RestClient(rest))
    {
        try
        {
            var request = new RestRequest("http://azenv.net");
            var response = await client.ExecuteAsync(request);
            if (response.IsSuccessStatusCode)
            {
                Console.WriteLine($"Proxy {proxyLine} is working.");
                Interlocked.Increment(ref good);
            }
            else
            {
                Interlocked.Increment(ref bad);
                Console.WriteLine($"Proxy {proxyLine} is not working. Status code: {response.ErrorMessage}");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error checking proxy {proxyLine}: {ex.Message}");
        }
    }
});

1. 配置SocketsHttpHandler强制关闭连接，禁用连接池 #

RestClient底层依赖HttpClient，而HttpClient默认复用连接池，导致套接字不会立即释放。通过自定义SocketsHttpHandler禁用连接池，并添加Connection: close头部强制请求完成后关闭连接：

var handler = new SocketsHttpHandler
{
    // 禁用连接池，每个请求使用独立连接
    PooledConnectionLifetime = TimeSpan.Zero,
    PooledConnectionIdleTimeout = TimeSpan.Zero,
    // 禁用自动重定向和Cookie，减少不必要的资源占用
    AllowAutoRedirect = false,
    UseCookies = false,
};
// 添加关闭连接的HTTP头部
handler.AdditionalRequestHeaders.Add("Connection", "close");

var restOptions = new RestClientOptions()
{
    Proxy = proxy,
    MaxTimeout = 5000,
    // 将自定义Handler注入RestClient
    ConfigureMessageHandler = _ => handler,
};

2. 下调并行度，避免超出系统套接字限制 #

当前设置的MaxDegreeOfParallelism = 300可能过高，系统临时端口范围通常为1024-65535（约6万），还要考虑其他进程占用的套接字。建议将并行度下调至100-200，根据系统实际负载调整，避免短时间内创建过量套接字。

3. 结合CancellationToken及时终止超时请求 #

在请求中添加独立的取消令牌，配合RestClient的超时设置，确保超时请求能及时终止并释放资源：

await Parallel.ForEachAsync(proxyLines, parallelOptions, async (proxyLine, token) =>
{
    // 创建独立的取消令牌，避免受外部令牌影响
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(token);
    cts.CancelAfter(5000); // 与RestClient超时保持一致

    var trim = proxyLine.Split(":");
    WebProxy proxy = new WebProxy(new Uri($"socks5://{trim[0]}:{trim[1]}"));
    
    var handler = new SocketsHttpHandler
    {
        PooledConnectionLifetime = TimeSpan.Zero,
        PooledConnectionIdleTimeout = TimeSpan.Zero,
        AdditionalRequestHeaders = { { "Connection", "close" } },
    };
    
    using var client = new RestClient(new RestClientOptions()
    {
        Proxy = proxy,
        MaxTimeout = 5000,
        ConfigureMessageHandler = _ => handler,
    });

    try
    {
        var request = new RestRequest("http://azenv.net");
        var response = await client.ExecuteAsync(request, cts.Token);
        if (response.IsSuccessStatusCode)
        {
            Console.WriteLine($"Proxy {proxyLine} is working.");
            Interlocked.Increment(ref good);
        }
        else
        {
            Interlocked.Increment(ref bad);
            Console.WriteLine($"Proxy {proxyLine} is not working. Error: {response.ErrorMessage}");
        }
    }
    catch (OperationCanceledException)
    {
        Console.WriteLine($"Proxy {proxyLine} timed out.");
        Interlocked.Increment(ref bad);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Error checking proxy {proxyLine}: {ex.Message}");
        Interlocked.Increment(ref bad);
    }
});

4. 临时调整系统TCP参数（可选） #

若必须保持高并行度，可临时调整系统TCP参数减少TIME_WAIT/FIN_WAIT_2状态的套接字留存时间：

• Windows执行命令：

  netsh int tcp set global timestamps=disabled
  netsh int tcp set global tcpfinwait2timeout=3000
  

  （将FIN_WAIT_2超时改为3秒）
• 注意：此操作会影响整个系统的TCP行为，测试完成后建议恢复默认值。

关于获取套接字状态的说明 #

RestClient/HttpClient并未直接暴露套接字状态的API，底层连接由SocketsHttpHandler管理。若要监控套接字状态，需通过解析netstat输出实现，成本高且不高效，不如通过上述方法强制关闭连接来解决问题。

内容的提问来源于stack exchange，提问作者AnyAid