Unity中HttpClient发送约1500次请求后出现TlsException握手失败问题

问题分析 #

你遇到的TlsException: Handshake failed - error code: UNITYTLS_INTERNAL_ERROR错误，本质是Unity基于Mono的HttpClient实现中，大量请求后出现了TLS资源泄漏，导致系统无法创建新的TLS连接。重启Unity会释放这些泄漏的资源，所以能恢复，但重复请求到阈值后会再次触发。测试httpbin也出现相同问题，排除目标服务器限制的可能，问题出在本地HttpClient的资源管理或配置上。

解决方案 #

1. 修复HttpClient配置与资源释放 #

调整TLS协议与连接池设置

你的代码中SslProtocols = SslProtocols.None会让系统自动选择协议，但Mono的实现可能存在兼容性问题，明确指定TLS版本能减少握手异常。同时设置连接池大小与并发数一致，避免连接过载。

修改HttpClient配置：

private static readonly Lazy<HttpClient> httpClient = new(() => new HttpClient(new HttpClientHandler
{
    ServerCertificateCustomValidationCallback = (_, _, _, _) => true,
    ClientCertificateOptions = ClientCertificateOption.Automatic,
    // 明确指定支持的TLS版本，避免自动选择的兼容性问题
    SslProtocols = SslProtocols.Tls12 | SslProtocols.Tls13,
    // 连接池最大连接数与并发数匹配
    MaxConnectionsPerServer = 5
})
{
    BaseAddress = new Uri("https://yugioh.fandom.com/"),
    Timeout = TimeSpan.FromSeconds(HTTPCLIENT_TIMEOUT),
    // 改为复用连接，减少重复握手的TLS资源消耗
    DefaultRequestHeaders =
    {
        ConnectionClose = false
    }
});

正确释放HttpResponseMessage资源

原代码中HttpResponseMessage没有被Dispose，会持有TLS连接资源导致泄漏。需要修改返回值，在流处理完成后手动释放响应。

修改DownloadAsStreamAsync：

private static async Task<(CustomHttpStatusCode StatusCode, Stream? ContentStream, HttpResponseMessage? Response)> DownloadAsStreamAsync(URL _URL, CancellationToken _CancellationToken)
{
    var _semaphoreAcquired = false;
    var _attempt = 1;
    var _httpStatusCode = CustomHttpStatusCode.Null;
    Stream? _contentStream = null;
    HttpResponseMessage? _response = null;
    
    do
    {
        try
        {
            await Task.Delay(1000, _CancellationToken);
            await semaphoreSlim.Value.WaitAsync(_CancellationToken);
            _semaphoreAcquired = true;
        
            _response = await httpClient.Value.GetAsync(_URL.AbsoluteUri, HttpCompletionOption.ResponseHeadersRead, _CancellationToken);
            
            _httpStatusCode = _response.EnsureSuccessStatusCode().StatusCode.ToCustomHttpStatusCode(); 
            _contentStream = await _response.Content.ReadAsStreamAsync();
        
            return (_httpStatusCode, _contentStream, _response);
        }
        catch (TaskCanceledException) when (_CancellationToken.IsCancellationRequested) { /* Expected behavior. */ }
        catch (Exception _exception)
        {
            Debug.LogException(_exception.CustomMessage($"Download attempt [{_attempt}] aborted: {_URL.Bold()} [{_httpStatusCode}]"));
            _response?.Dispose(); // 异常时及时释放响应资源
            await Task.Delay(5000 * _attempt++, _CancellationToken);
        }
        finally
        {
            if (_semaphoreAcquired)
            {
                semaphoreSlim.Value.Release();
            }
        }
        
    } while (!_CancellationToken.IsCancellationRequested && _attempt <= 3);
    
    _response?.Dispose(); // 循环结束后释放未使用的响应
    return (_httpStatusCode, _contentStream, null);
}

修改DownloadAsync，确保响应在流处理完成后被释放：

internal static async Task<CustomHttpStatusCode> DownloadAsync(URL _URL, Func<string, Task> _ProcessLine, Func<bool> _StopCondition, CancellationToken _CancellationToken)
{
    using var _cancellationTokenSource = new CancellationTokenSource();
    using var _linkedCancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(_CancellationToken, _cancellationTokenSource.Token);

    var (_statusCode, _contentStream, _response) = await DownloadAsStreamAsync(_URL, _linkedCancellationTokenSource.Token);

    if (_contentStream == null)
    {
        _response?.Dispose();
        return _statusCode;
    }

    // 使用using包裹响应，确保流处理完成后自动释放资源
    using (_response)
    using (var _streamReader = new StreamReader(_contentStream))
    {
        while (await _streamReader.ReadLineAsync() is {} _currentLine)
        {
            await _ProcessLine(_currentLine);
    
            if (_StopCondition())
            {
                _cancellationTokenSource.Cancel();
                return CustomHttpStatusCode.Stopped;
            }
        }
    }

    return _statusCode;
}

2. 定期重建HttpClient实例 #

如果资源释放调整后仍有问题，说明Mono的HttpClient单例存在长期资源泄漏，可以定期重建HttpClient，比如每1000次请求替换一次实例：

private static int _requestCount = 0;
private static readonly object _clientLock = new object();
private static HttpClient? _httpClient;

private static HttpClient GetHttpClient()
{
    lock (_clientLock)
    {
        // 每1000次请求重建一次HttpClient，释放累积的资源
        if (_httpClient == null || Interlocked.Increment(ref _requestCount) % 1000 == 0)
        {
            _httpClient?.Dispose();
            _httpClient = new HttpClient(new HttpClientHandler
            {
                ServerCertificateCustomValidationCallback = (_, _, _, _) => true,
                ClientCertificateOptions = ClientCertificateOption.Automatic,
                SslProtocols = SslProtocols.Tls12 | SslProtocols.Tls13,
                MaxConnectionsPerServer = 5
            })
            {
                BaseAddress = new Uri("https://yugioh.fandom.com/"),
                Timeout = TimeSpan.FromSeconds(HTTPCLIENT_TIMEOUT),
                DefaultRequestHeaders =
                {
                    ConnectionClose = false
                }
            };
        }
        return _httpClient;
    }
}

之后在DownloadAsStreamAsync中用GetHttpClient()替换httpClient.Value即可。

3. 额外优化建议 #

• 增加请求延迟到1.5-2秒，进一步降低服务器压力的同时，给本地TLS资源更多回收时间。
• 捕获TlsException并加入重试逻辑，避免单次握手失败中断整个爬虫流程。

内容的提问来源于stack exchange，提问作者Mark Herdt