如何解决Newtonsoft Json序列化时的集合修改枚举异常

使用Newtonsoft.Json的JsonConvert.SerializeObject方法序列化DestinationItemList集合时，反复抛出如下异常：

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.

确认未显式修改待序列化的_chunk集合，但问题仍未解决。相关代码及堆栈跟踪如下：

相关代码 #

GetJson方法 #

Public Overloads Function GetJson(ByVal destination As PathDefinition) As String
     Dim chunk As DestinationItemList = Nothing
     If destination.Name = "PI" Then
          chunk = _chunk
     Else
          'Translate chunk to InfluxCompatible object
          '<Placeholder>
     End If
     Try
          Return JsonConvert.SerializeObject(chunk)
     Catch ex As Exception
          Throw
     End Try
End Function

实体类Destinationtem #

Public Class Destinationtem
     Private _tagAttribute As String
     Private _tagRef As String
     Private _tagName As String
     Private _timeStamp As String
     Private _quality As String
     Private _value As String
     Private _serverName As String
End Class

ProcessData方法（调用GetJson的异步方法） #

Public Overloads Async Sub ProcessData()
   Dim cnt As Long
   Dim jsonString As String
   Dim destinationItem As Destinationtem
   Dim tagRef As String
   Dim token As String

   Try
        _oldDestinationItem = New Destinationtem
        _logger.SendLog("Processing data - " & _lvItems.Count & " records.", LogLevel.Trace)
        'Do until we have no more items to process
        Do While _lastCnt < _lvItems.Count - 1
             Try
                  'Get the next chunk of data
                  GetChunk()
                  If _chunk IsNot Nothing Then
                       If _chunk.Count > 0 Then
                            'If export tags is switched  on then export the tags
                            If _exportTagPath <> "" Then
                                 If _tagsForExport Is Nothing Then
                                      _tagsForExport = New DestinationItemList
                                 End If
                                 'For each entry in the chunk
                                 For Each destinationItem In _chunk
                                      tagRef = destinationItem.TagRef
                                      'Then add to the  list of tags to export
                                      'if the tag is not already in the list
                                      If _tagsForExport.Find(Function(p) p.TagRef = tagRef) Is Nothing Then
                                           'Add the item
                                           _tagsForExport.Add(destinationItem)
                                      End If
                                 Next destinationItem
                            End If
                            'Send the data to the destinations
                            For Each destination In _destinations
                                 'If this is the client for obtaining a token then get the token
                                 If destination.Name.ToUpper.IndexOf("TOKEN") <> -1 Then
                                      _token = Await GetToken(destination.HttpClient, destination.AuthInfo, destination.Path)
                                 Else
                                      jsonString = GetJson(destination)
                                      Send(destination.HttpClient, jsonString, destination.Path)
                                 End If
                            Next
                       End If
                  End If
             Catch ex As MissingItemException
                  _logger.SendLog("The transformer identified by ID = '" & _lvItems(_lastCnt).TransformerId & "' was not found", LogLevel.Trace)
             End Try
        Loop
        _logger.SendLog("Number of records processed -  " & _lvItems.Count & " records.", LogLevel.Info)
   Catch ex As Exception
        _logger.SendLog("LV Data Loader encountered an exception whilst processing the data", LogLevel.Error, ex)
        Throw
   Finally
        'if export tags is true then save and close the file and dispose of the object
        _chunk.CloseFile()
   End Try

End Sub

异常堆栈跟踪 #

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at Newtonsoft.Json.JsonConvert.SerializeObject(Object value)
   at SPEN.LVDataLoader.ImportJson.GetJson(PathDefinition destination) in C:\_VSSWorkArea\LVDataLoader\LVDataLoader\ImportJson.vb:line 1136
   at SPEN.LVDataLoader.ImportJson.VB$StateMachine_199_ProcessData.MoveNext() in C:\_VSSWorkArea\LVDataLoader\LVDataLoader\ImportJson.vb:line 871

GetDestinations方法 #

Public Sub GetDestinations()

      Dim dst As PathDefinition
      Dim dstCnt As Integer


      Try

           'Get the number of source from the app.config file
           dstCnt = ConfigurationManager.AppSettings("Destinations")

           'For each destination  specified in the config file
           For cnt = 0 To dstCnt - 1
                dst = New PathDefinition
                dst.Active = ConfigurationManager.AppSettings("DestinationActive" & cnt)
                If dst.Active Then
                     dst.Name = ConfigurationManager.AppSettings("DestinationName" & cnt)
                     dst.AuthInfo = ConfigurationManager.AppSettings("AuthInfo" & cnt)
                     dst.BaseAddress = ConfigurationManager.AppSettings("DestinationBaseAddress" & cnt)
                     dst.Token = ConfigurationManager.AppSettings("DestinationTokenUrl" & cnt)
                     dst.Path = ConfigurationManager.AppSettings("DestinationUrl" & cnt)
                     dst.HttpClient = GetHttpClient(dst)
                     _destinations.Add(dst)
                End If
           Next
      Catch ex As Exception
           _logger.SendLog("LVDataLoader.GetDestinations", NLog.LogLevel.Error, ex)
           Throw
      Finally
      End Try
 End Sub

发送数据循环片段 #

'Send the data to the destinations

For Each destination In _destinations
     'If this is the client for obtaining a token then get the token
     If destination.Name.ToUpper.IndexOf("TOKEN") <> -1 Then
          _token = Await GetToken(destination.HttpClient, destination.AuthInfo, destination.Path)
     Else
          jsonString = GetJson(destination)
          Send(destination.HttpClient, jsonString, destination.Path)
     End If
Next

1. 异步流程中的并发修改：ProcessData是Async Sub，执行Await GetToken时会释放当前线程，后续循环可能在序列化_chunk的同时，调用GetChunk修改_chunk集合。Newtonsoft.Json序列化集合时会枚举元素，此时集合被修改就会触发异常。
2. 共享集合的线程不安全访问：_chunk是共享成员，若GetChunk或其他代码在未同步的情况下修改_chunk，而序列化操作同时在枚举该集合，就会导致冲突。

1. 序列化前创建集合副本 #

在GetJson方法中，不要直接引用原_chunk集合，而是创建一个副本进行序列化，避免原集合修改影响序列化过程：

' 先定义同步根对象（类级别私有成员）
Private _chunkSyncRoot As New Object()

Public Overloads Function GetJson(ByVal destination As PathDefinition) As String
    Dim chunk As DestinationItemList = Nothing
    If destination.Name = "PI" Then
        ' 加锁确保复制过程中集合不被修改
        SyncLock _chunkSyncRoot
            ' 假设DestinationItemList支持通过IEnumerable构造函数创建副本
            chunk = New DestinationItemList(_chunk)
        End SyncLock
    Else
        'Translate chunk to InfluxCompatible object
        '<Placeholder>
    End If
    Try
        Return JsonConvert.SerializeObject(chunk)
    Catch ex As Exception
        Throw
    End Try
End Function

2. 同步所有访问共享集合的操作 #

对所有读写_chunk的代码块加锁，确保同一时间只有一个线程能访问该集合：

• 修改GetChunk方法：

Public Sub GetChunk()
    SyncLock _chunkSyncRoot
        ' 原GetChunk的实现代码
    End SyncLock
End Sub

• 修改ProcessData中处理_chunk的代码块：

GetChunk()
SyncLock _chunkSyncRoot
    If _chunk IsNot Nothing Then
        If _chunk.Count > 0 Then
            ' 处理tags导出的代码...
            ' 发送数据的代码...
        End If
    End If
End SyncLock

3. 替换Async Sub为Async Function #

Async Sub的异常处理和线程调度不可控，改为Async Function提升异步流程的可控性：

Public Overloads Async Function ProcessData() As Task
    ' 原ProcessData的实现代码，保持逻辑不变
End Function

4. 调整循环逻辑避免并发修改 #

确保在完成当前_chunk的序列化和发送后，再调用GetChunk获取下一批数据，避免在处理当前集合时修改它：

Do While _lastCnt < _lvItems.Count - 1
     Try
          SyncLock _chunkSyncRoot
              'Get the next chunk of data
              GetChunk()
              If _chunk IsNot Nothing AndAlso _chunk.Count > 0 Then
                   ' 处理tags导出、序列化、发送数据的所有代码
              End If
          End SyncLock
     Catch ex As MissingItemException
          ' 异常处理代码
     End Try
Loop

内容的提问来源于stack exchange，提问作者Paul Johnson