使用JsonSerializer与MemoryMappedViewStream时触发JSON无效0x00错误

问题根源 #

MemoryMappedViewStream基于固定大小的内存区域，序列化完成后，流的剩余未写入部分会被填充为0字节。而System.Text.Json的反序列化器会读取整个流直到结束，当遇到这些末尾的0字节时，会判定为无效的JSON字符，从而抛出'0x00' is invalid after a single JSON value异常。

普通MemoryStream会跟踪实际写入的字节长度，反序列化时只会读取有效数据，因此不会触发该问题。

解决方案 #

核心思路是让反序列化器仅处理实际写入的JSON有效数据，以下是两种可行方案：

方案一：先序列化到字节数组，记录长度后写入流 #

先将对象序列化为UTF-8字节数组，记录数组长度并写入流，反序列化时先读取长度，再读取对应长度的字节进行反序列化：

string dataFilePath = Guid.NewGuid().ToString();
using (MemoryMappedFile dataFile = MemoryMappedFile.CreateNew(dataFilePath, 104857600, MemoryMappedFileAccess.ReadWrite))
using (MemoryMappedViewStream dataStream = dataFile.CreateViewStream())
{
    SimpleDataSource original = new SimpleDataSource
    {
        Name = "Example Name",
        Date = new DateTime(1999, 12, 31),
        Number = 191
    };

    // 序列化为字节数组，获取有效数据长度
    byte[] jsonBytes = JsonSerializer.SerializeToUtf8Bytes(original);
    // 先写入长度（用4字节int存储）
    dataStream.Write(BitConverter.GetBytes(jsonBytes.Length), 0, 4);
    // 写入JSON数据
    dataStream.Write(jsonBytes, 0, jsonBytes.Length);
    
    dataStream.Seek(0, SeekOrigin.Begin);
    
    // 读取长度
    byte[] lengthBuffer = new byte[4];
    dataStream.Read(lengthBuffer, 0, 4);
    int jsonLength = BitConverter.ToInt32(lengthBuffer, 0);
    // 读取对应长度的JSON字节
    byte[] jsonReadBytes = new byte[jsonLength];
    dataStream.Read(jsonReadBytes, 0, jsonLength);
    // 反序列化
    SimpleDataSource deserialized = JsonSerializer.Deserialize<SimpleDataSource>(jsonReadBytes);
}

方案二：自定义限定长度的Stream包装器 #

通过自定义Stream类限制读取的字节数，确保反序列化器只处理有效数据：

string dataFilePath = Guid.NewGuid().ToString();
using (MemoryMappedFile dataFile = MemoryMappedFile.CreateNew(dataFilePath, 104857600, MemoryMappedFileAccess.ReadWrite))
using (MemoryMappedViewStream dataStream = dataFile.CreateViewStream())
{
    SimpleDataSource original = new SimpleDataSource
    {
        Name = "Example Name",
        Date = new DateTime(1999, 12, 31),
        Number = 191
    };

    // 记录序列化前的起始位置
    long startPos = dataStream.Position;
    JsonSerializer.Serialize(dataStream, original);
    // 计算实际写入的字节数
    long bytesWritten = dataStream.Position - startPos;
    
    dataStream.Seek(startPos, SeekOrigin.Begin);
    
    // 使用自定义限定流读取有效数据
    using (var limitedStream = new LimitedStream(dataStream, bytesWritten))
    {
        SimpleDataSource deserialized = JsonSerializer.Deserialize<SimpleDataSource>(limitedStream);
    }
}

// 自定义限定读取长度的Stream类
public class LimitedStream : Stream
{
    private readonly Stream _innerStream;
    private long _remainingBytes;

    public LimitedStream(Stream innerStream, long length)
    {
        _innerStream = innerStream;
        _remainingBytes = length;
    }

    public override bool CanRead => _innerStream.CanRead;
    public override bool CanSeek => false;
    public override bool CanWrite => false;
    public override long Length => throw new NotSupportedException();
    public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }

    public override int Read(byte[] buffer, int offset, int count)
    {
        int bytesToRead = (int)Math.Min(count, _remainingBytes);
        if (bytesToRead <= 0) return 0;
        int bytesRead = _innerStream.Read(buffer, offset, bytesToRead);
        _remainingBytes -= bytesRead;
        return bytesRead;
    }

    public override void Flush() => _innerStream.Flush();
    public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
    public override void SetLength(long value) => throw new NotSupportedException();
    public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}

说明 #

两种方案都能有效避免读取内存映射流末尾的空字节，方案一实现更简洁，适合大多数场景；方案二无需提前序列化到字节数组，适合大对象的序列化场景。

内容的提问来源于stack exchange，提问作者Valuator