使用DeflateStream读取时流位置与预期不符的问题排查

背景 #

我需要将离散二进制数据写入流并存储到磁盘，为降低磁盘IO开销采用带缓冲的FileStream，整体处理流程为：BinaryWriter -> DeflateStream -> 带缓冲的FileStream。

定义的文件数据结构包含头部信息与压缩后原始数据：

• 签名：1字节
• 时间戳：8字节
• 未压缩数据大小：8字节
• 经DeflateStream压缩的数据：X字节

问题现象 #

执行反向读取操作时，流位置与预期值不匹配，导致后续数据读取失败。例如处理大小为240000的数据时：

• 写入阶段：完成头部写入与压缩数据写入后，流最终位置为8099
• 读取阶段：读取压缩数据后，流位置变为8306，与预期的8099不符

解决方案 #

在文件头部额外存储压缩数据的长度，读取压缩数据后手动修正流位置，跳过DeflateStream读取时额外偏移的部分。

写入代码 #

var fileStream = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None, 200 * 1_048_576);
var binaryWriter = new BinaryWriter(fileStream);

var item = new Item
{
    Signature = (byte)1,
    TimeStamp = DateTime.Now.Ticks,
    Data = new byte[] { .... }
};

binaryWriter.Write(item.Signature); //1字节
binaryWriter.Write(item.TimeStamp); //8字节
binaryWriter.Write(item.Data.LongLength); //8字节，未压缩数据长度

var start = fileStream.Position;
binaryWriter.Write(0L); //预留8字节存储压缩数据长度

using (var compressStream = new DeflateStream(fileStream, UserSettings.All.CaptureCompression, true))
{
    compressStream.Write(item.Data);
    compressStream.Flush();
}

var end = fileStream.Position;
var compressedLength = end - start - 8; //减去预留的8字节长度

fileStream.Position = start;
binaryWriter.Write(compressedLength); //写入实际压缩数据长度
fileStream.Position = end;

读取代码 #

await using var fileStream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read);
using var binaryReader = new BinaryReader(fileStream);

var items = new List<Item>();

while (fileStream.Position < fileStream.Length)
{
    var item = new Item
    {
        StreamPosition = fileStream.Position
    };

    fileStream.Position += 1; //跳过签名

    item.TimeStampInTicks = binaryReader.ReadInt64();
    item.DataLength = binaryReader.ReadInt64();
    var compressedLength = binaryReader.ReadInt64();
    var currentPosition = fileStream.Position;
    
    await using (var compressStream = new DeflateStream(fileStream, CompressionMode.Decompress, true))
    {
        compressStream.ReadBytes((int)item.DataLength);
    }

    fileStream.Position = currentPosition + compressedLength; //手动修正流位置

    items.Add(item);
}

内容的提问来源于stack exchange，提问作者Nicke Manarin