如何用boost::beast持续流式传输被追加写入的本地文件

Great question! You’re absolutely on the right track here — using boost::beast::http::response_serializer and chunked encoding is exactly the way to build this continuous file streaming service, just like tail -f does for stdout. Let’s break down how to make this work:

Key Background #

Your initial approach with http::response<http::file_body> and async_write stops at EOF because the serializer assumes the file has a fixed length (from the initial file size) and marks the response as complete once it reaches the end. To stream ongoing updates, we need two critical changes:

1. Use chunked transfer encoding (required by HTTP/1.1 for streaming content of unknown length)
2. Manually control the serializer to keep sending new content as the file grows

Step 1: Configure the Response for Chunked Encoding #

First, you must enable chunked encoding on your response. This tells the client to expect a stream of data chunks instead of a single fixed-length body:

namespace http = boost::beast::http;
namespace beast = boost::beast;

// Create a response with file_body
http::response<http::file_body> res{http::status::ok, 11};
// Enable chunked encoding (critical for streaming)
res.chunked(true);
// Set necessary headers (e.g., content type for text files)
res.set(http::field::content_type, "text/plain");
// Open the file in read mode
beast::error_code ec;
res.body().open("/path/to/your/target/file", beast::file_mode::read, ec);
if (ec) {
    // Handle file open error (e.g., log and send 500 response)
    return;
}

Step 2: Use response_serializer for Manual Control #

Instead of using http::async_write (which sends the entire response in one go), use http::response_serializer and its async_write_some method. This lets you send partial content and resume streaming when new data becomes available:

// Create the serializer with our preconfigured response
http::response_serializer<http::file_body> serializer{std::move(res)};
// Start the first write operation
http::async_write_some(
    your_tcp_stream, // Replace with your active TCP stream
    serializer,
    std::bind(
        &your_class::on_write,
        this,
        std::placeholders::_1,
        std::placeholders::_2,
        std::ref(serializer)
    )
);

Step 3: Handle EOF and Wait for File Updates #

In your write completion handler, check if the serializer has finished. If it hits EOF, don’t close the connection — instead, monitor the file for changes, then resume streaming when new content is added:

The Write Handler #

void on_write(
    beast::error_code ec,
    std::size_t bytes_transferred,
    http::response_serializer<http::file_body>& serializer
) {
    if (ec) {
        // Handle error (e.g., client disconnected or network issue)
        cleanup_resources(); // Close streams, file handles, etc.
        return;
    }

    // Check if there's still content left to send from the current file state
    if (!serializer.is_done()) {
        // More content available, keep sending
        http::async_write_some(
            your_tcp_stream,
            serializer,
            std::bind(
                &your_class::on_write,
                this,
                std::placeholders::_1,
                std::placeholders::_2,
                std::ref(serializer)
            )
        );
        return;
    }

    // We've reached EOF — wait for the file to be updated
    wait_for_file_modification([this, &serializer]() {
        resume_streaming(serializer);
    });
}

Monitoring File Changes #

You’ll need a way to detect when the file is appended to. Platform-specific APIs are most efficient:

• Linux: Use inotify to watch for IN_MODIFY events
• Windows: Use ReadDirectoryChangesW
• macOS: Use FSEvents

For simplicity, here’s a basic Linux inotify example wrapped in Boost.Asio:

void wait_for_file_modification(std::function<void()> on_change) {
    int inotify_fd = inotify_init();
    if (inotify_fd == -1) {
        // Handle inotify init error
        return;
    }

    // Watch the target file for modification events
    inotify_add_watch(inotify_fd, "/path/to/your/target/file", IN_MODIFY);

    // Use Boost.Asio to asynchronously wait for inotify events
    boost::asio::posix::stream_descriptor sd(your_io_context, inotify_fd);
    sd.async_read_some(
        boost::asio::null_buffers(),
        [on_change, sd=std::move(sd)](beast::error_code ec, std::size_t) {
            if (!ec) {
                on_change();
            }
        }
    );
}

Resuming Streaming After File Update #

When the file is modified, update the file body’s read position and tell the serializer there’s more content to send:

void resume_streaming(http::response_serializer<http::file_body>& serializer) {
    beast::error_code ec;
    auto& file = serializer.get().body();
    
    // Get current file size and last read position
    auto current_size = file.size(ec);
    auto last_pos = file.pos(ec);
    
    if (current_size > last_pos && !ec) {
        // Tell the serializer the response isn't complete yet
        serializer.need_more(true);
        // Resume sending the new content
        http::async_write_some(
            your_tcp_stream,
            serializer,
            std::bind(
                &your_class::on_write,
                this,
                std::placeholders::_1,
                std::placeholders::_2,
                std::ref(serializer)
            )
        );
    } else {
        // No new content detected, wait again for modifications
        wait_for_file_modification([this, &serializer]() {
            resume_streaming(serializer);
        });
    }
}

Critical Notes #

• Chunked Encoding is Mandatory: Without it, the client will expect a fixed Content-Length and close the connection once that length is received. Chunked encoding lets you send an indefinite stream of data chunks.
• Serializer Control: Calling serializer.need_more(true) tells Beast the response isn’t complete, even if the file body has reached EOF. This keeps the serializer in a state where it can accept more write operations.
• Resource Cleanup: Don’t forget to close file handles, inotify descriptors, and TCP streams when the client disconnects or an error occurs to avoid resource leaks.
• Cross-Platform Fallback: If platform-specific file monitoring is too complex, you can use a periodic Boost.Asio timer to check the file size (e.g., every 500ms) and resume streaming if the size has increased.

内容的提问来源于stack exchange，提问作者Martchus