如何解决.NET中多方法发起WebRequest时的冲突超时问题

问题根源 #

• 首先是WebResponse资源泄漏：你在WriteLogX、StatusCheckX方法中，判断完返回状态直接Return，后续的Res.Close()、Res.Dispose()代码完全不会执行，导致HTTP连接一直被占用没有释放
• 其次是.NET默认的同域名HTTP并发连接数限制：默认情况下，同一个域名的HTTP请求并发上限是2，大量未释放的连接占满配额后，新的请求只能排队等待，直到超时
• 另外每次调用LogHandler都新建独立线程，频繁创建销毁线程也会额外消耗系统资源，加重请求阻塞概率

修复方案 #

1. 替换所有手动资源释放逻辑为Using语句，自动管理资源生命周期，就算方法提前返回也能正确释放连接
2. 调高同域名HTTP请求并发上限，在程序启动时添加对应配置
3. 优化日志线程的实现，改用线程池代替每次新建线程，降低资源消耗

修复后的示例代码：

' 程序启动时添加此行，调高并发限制，可根据实际业务调整数值
ServicePointManager.DefaultConnectionLimit = 10

Public Function Important_Method1() As Boolean
    Dim VariableX As Boolean = True
    Try
        'do something
        'do something
        
        Call LogHandler()
        
        If VariableX = True Then
            Return StatusCheckX()
        End If
        Return True
    Catch ex As Exception
        Return False
    End Try
End Function

Public Function Important_Method2() As Boolean
    Dim VariableX As Boolean = True
    Try
        'do something
        'do something
        
        Call LogHandler()
        
        If VariableX = True Then
            Return StatusCheckX()
        End If
        Return True
    Catch ex As Exception
        Return False
    End Try
End Function

Private Function LogHandler() As Boolean
    ' 改用线程池线程，不需要手动管理线程生命周期
    ThreadPool.QueueUserWorkItem(AddressOf WriteLogX)
    Return True
End Function

Private Function WriteLogX() As Boolean
    Dim url As String = "www.xyz.com"
    Try
        ' Using语句自动释放WebRequest、WebResponse资源
        Using Req As HttpWebRequest = CType(WebRequest.Create(url), HttpWebRequest),
              Res As HttpWebResponse = CType(Req.GetResponse(), HttpWebResponse)
            Return Res.StatusDescription = "ALL GOOD"
        End Using
    Catch ex As Exception
        Return False
    End Try
End Function

Private Function StatusCheckX() As Boolean
    Try
        'do something
        'do something
        'do something

        Using Req As HttpWebRequest = CType(WebRequest.Create("www.ijk.com"), HttpWebRequest),
              Res As HttpWebResponse = CType(Req.GetResponse(), HttpWebResponse)
            If Res.StatusDescription = "OK" Then
                Using ResStream As System.IO.Stream = Res.GetResponseStream(),
                      SReader As System.IO.StreamReader = New System.IO.StreamReader(ResStream)
                    Dim i As String = SReader.ReadToEnd()
                    Dim dict = JsonConvert.DeserializeObject(Of Dictionary(Of String, String))(i)
                    If dict.Values(0) = "OK PROCEED" Then
                        'do something
                        Return True
                    Else
                        'do something
                        Return False
                    End If
                End Using
            End If
            Return False
        End Using
    Catch ex As Exception
        Return False
    End Try
End Function

额外注意 #

如果业务调用频率很高，还可以考虑给日志请求加批量合并逻辑，避免短时间内发起大量HTTP请求，进一步降低连接占满的概率。

内容的提问来源于stack exchange，提问作者Ari Kurita