.NET HttpClient如何分块发送大JSON POST请求？

嘿，我之前刚好处理过几乎一模一样的场景——要流式发送大量JSON数据，避免把整个大字符串塞进内存里。其实你不用自定义Stream，.NET Core里的PushStreamContent就是HttpWebRequest.GetRequestStream的完美替代，而且用起来很顺手！

PushStreamContent的设计初衷就是让你异步写入请求流，完全符合你“逐个用户构建JSON、边生成边发送”的需求，根本不需要预先累积整个JSON字符串。

实现步骤&代码示例 #

// 注意：实际项目里HttpClient建议复用单例，不要每次请求创建新实例
using var httpClient = new HttpClient();

// 创建PushStreamContent，传入写入流的委托
var streamContent = new PushStreamContent(async (requestStream, _, __) =>
{
    // 用StreamWriter来简化字符串写入，指定UTF8编码，leaveOpen设为true避免提前关闭请求流
    using var jsonWriter = new StreamWriter(requestStream, Encoding.UTF8, bufferSize: 1024, leaveOpen: true);
    
    // 先写入JSON数组的开头
    await jsonWriter.WriteAsync("[");
    bool isFirstUser = true;

    // 遍历你的用户数据源（这里假设GetUsers()是逐个返回用户的枚举）
    foreach (var user in GetUsers())
    {
        // 非第一个用户要加逗号分隔
        if (!isFirstUser)
        {
            await jsonWriter.WriteAsync(",");
        }
        isFirstUser = false;

        // 手动构建单个用户的JSON（这里替换成你实际的自定义JSON逻辑）
        // 注意：要手动转义特殊字符，避免生成无效JSON
        var userJson = $"{{\"UserId\":{user.Id},\"UserName\":\"{EscapeJsonSpecialChars(user.Name)}\"}}";
        
        await jsonWriter.WriteAsync(userJson);
        await jsonWriter.FlushAsync(); // 强制把当前数据推送到请求流，避免内存累积
    }

    // 写入JSON数组的结尾
    await jsonWriter.WriteAsync("]");
    await jsonWriter.FlushAsync();
}, "application/json"); // 指定Content-Type为JSON

// 发送POST请求
var response = await httpClient.PostAsync("https://your-target-endpoint.com/batch-users", streamContent);
response.EnsureSuccessStatusCode(); // 确保请求成功

辅助方法：转义JSON特殊字符 #

手动拼接JSON一定要处理特殊字符，不然会导致JSON格式错误：

private string EscapeJsonSpecialChars(string input)
{
    return input.Replace("\\", "\\\\")
                .Replace("\"", "\\\"")
                .Replace("\n", "\\n")
                .Replace("\r", "\\r")
                .Replace("\t", "\\t");
}

如果你的场景对吞吐量要求极高，可以用.NET Core引入的PipeWriter（System.IO.Pipelines）来做，它的内存管理比StreamWriter更高效，不过需要简单包装一个Stream适配StreamContent：

using var httpClient = new HttpClient();

var pipeContent = new StreamContent(new PipeStreamContent(async (writer) =>
{
    // 写入JSON数组开头
    await writer.WriteAsync(Encoding.UTF8.GetBytes("["));
    bool isFirst = true;

    foreach (var user in GetUsers())
    {
        if (!isFirst)
        {
            await writer.WriteAsync(Encoding.UTF8.GetBytes(","));
        }
        isFirst = false;

        var userJson = $"{{\"UserId\":{user.Id},\"UserName\":\"{EscapeJsonSpecialChars(user.Name)}\"}}";
        var jsonBytes = Encoding.UTF8.GetBytes(userJson);
        await writer.WriteAsync(jsonBytes);
        await writer.FlushAsync();
    }

    // 写入数组结尾并完成写入
    await writer.WriteAsync(Encoding.UTF8.GetBytes("]"));
    await writer.CompleteAsync();
}));
pipeContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");

var response = await httpClient.PostAsync("https://your-target-endpoint.com/batch-users", pipeContent);
response.EnsureSuccessStatusCode();

自定义PipeStreamContent类 #

这个类用来把PipeWriter包装成Stream，适配StreamContent：

public class PipeStreamContent : Stream
{
    private readonly Pipe _pipe = new Pipe();
    private readonly Func<PipeWriter, Task> _writeAction;
    private bool _writeInitiated;

    public PipeStreamContent(Func<PipeWriter, Task> writeAction)
    {
        _writeAction = writeAction;
    }

    public override async Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
    {
        // 启动写入逻辑（只执行一次）
        if (!_writeInitiated)
        {
            _writeInitiated = true;
            _ = Task.Run(async () =>
            {
                try
                {
                    await _writeAction(_pipe.Writer);
                }
                catch (Exception ex)
                {
                    _pipe.Writer.Complete(ex);
                }
            }, cancellationToken);
        }

        var readResult = await _pipe.Reader.ReadAsync(cancellationToken);
        var bytesToCopy = Math.Min(count, readResult.Buffer.Length);
        readResult.Buffer.Slice(0, bytesToCopy).CopyTo(buffer.AsSpan(offset));
        
        _pipe.Reader.AdvanceTo(readResult.Buffer.Start, readResult.Buffer.End);

        if (readResult.IsCompleted)
        {
            _pipe.Reader.Complete();
        }

        return bytesToCopy;
    }

    // 以下是Stream抽象方法的默认实现，因为我们只需要ReadAsync用于发送请求
    public override bool CanRead => true;
    public override bool CanSeek => false;
    public override bool CanWrite => false;
    public override long Length => throw new NotSupportedException();
    public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }
    public override void Flush() => throw new NotSupportedException();
    public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException();
    public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
    public override void SetLength(long value) => throw new NotSupportedException();
    public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}

1. 复用HttpClient：不要每次请求创建新的HttpClient，否则会耗尽系统socket连接，建议用单例或者依赖注入的方式复用。
2. 及时Flush：写入后一定要调用FlushAsync()，确保数据及时发送到服务器，避免在内存里累积。
3. JSON格式正确性：手动拼接JSON时一定要处理特殊字符，或者可以考虑用Utf8JsonWriter（System.Text.Json里的类）来生成单个用户的JSON，比手动拼接更可靠。
4. 异常处理：要处理写入过程中可能出现的异常（比如服务器断开连接），及时终止写入并清理资源。

内容的提问来源于stack exchange，提问作者Martin Sykora