使用asio::async_read批量读取大量文件的性能问题排查

我正在测试最新版独立Asio（asio-1.30.2），尝试异步读取某目录及其子目录下的约17000个文件，发现异步方式的性能远低于同步方式。

测试代码 #

// All necessary includes

static constexpr const char* DIR = "<your directory>";
static constexpr size_t DATA_SIZE = 16 * 1024;

asio::io_context io_context;
asio::strand strand = asio::make_strand(io_context);

template<typename Iter>
void hash(Iter begin, Iter end, uint32_t& hash) {
    for (auto it = begin; it != end; it++) {
        hash += *it;
    }
}


std::vector<std::thread> start_threads(int num_threads) {
    std::vector<std::thread> threads;
    for (int i = 0; i < num_threads; ++i) {
        threads.emplace_back([]() { io_context.run(); });
    }
    return threads;
}

asio::awaitable<void> read_file_async(std::string fileName)
{
    asio::stream_file file{ strand, fileName, asio::stream_file::flags::read_only };

    std::array<char, DATA_SIZE> data;
    uint32_t h = 0;
    for (;;) {
        auto [ec, n] = co_await asio::async_read(file, asio::buffer(data), asio::transfer_at_least(data.size()), asio::as_tuple(asio::use_awaitable));
        if (!ec) {
            hash(data.begin(), data.begin() + n, h);
        }
        else if (ec == asio::error::eof) {
            hash(data.begin(), data.begin() + n, h);
            co_return;
        }
        else {
            std::cout << "Error: " << ec << "\n";
            co_return;
        }
    }
}

int main()
{
#if 1
    {
        auto start = std::chrono::steady_clock::now();

        for (const auto& entry : std::filesystem::recursive_directory_iterator(DIR, std::filesystem::directory_options::follow_directory_symlink)) {
            if (!entry.is_regular_file())
                continue;

            auto filePath = std::filesystem::path(entry.path()).lexically_normal().generic_string();
            asio::stream_file file{ io_context, filePath, asio::stream_file::flags::read_only };

            std::array<char, DATA_SIZE> data = { 0 };
            uint32_t h = 0;

            size_t n = 0;
            try {
                n = asio::read(file, asio::buffer(data), asio::transfer_at_least(data.size()));
                hash(data.begin(), data.begin() + n, h);
            }
            catch (const std::system_error& e) {
                if (e.code() == asio::error::eof && n > 0) {
                    hash(data.begin(), data.begin() + n, h);
                };
            }
        }
        auto end = std::chrono::steady_clock::now();

        std::cout << "Execution (sync) time: " << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << " milliseconds." << std::endl;
    }
#endif
#if 1
    {
        auto start = std::chrono::steady_clock::now();
        auto workGuard = asio::make_work_guard(strand);
        auto threads = start_threads(8);

        for (const auto& entry : std::filesystem::recursive_directory_iterator(DIR, std::filesystem::directory_options::follow_directory_symlink)) {
            if (!entry.is_regular_file())
                continue;

            auto filePath = std::filesystem::path(entry.path()).lexically_normal().generic_string();
            asio::co_spawn(
                strand,
                read_file_async(filePath),
                asio::detached
            );
        }

        workGuard.reset();

        for (auto& thread : threads)
            thread.join();

        auto end = std::chrono::steady_clock::now();

        std::cout << "Execution (async) time: " << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << " milliseconds." << std::endl;
    }
#endif
    return 0;
}

测试结果 #

Windows（IOCP）平台，MSVC Release编译 #

Execution (sync) time: 3013 milliseconds.
Execution (async) time: 27203 milliseconds.

Linux（liburing）平台，g++ -std=c++20 -O3编译 #

Execution (sync) time: 143 milliseconds.
Execution (async) time: 21275 milliseconds.

性能分析与尝试 #

异步性能比同步慢10到150倍。通过strace -fc分析Linux版本，发现Asio大部分时间消耗在futex（类似互斥锁）操作上。

• 调整异步会话线程数：设为1时Windows性能变化不大，Linux则异常缓慢（疑似死锁）
• 使用std::counting_semaphore控制并发数：线程数为1、信号量为8时Linux性能有所提升，但线程数增加后性能反而下降，整体仍比同步慢10倍以上

现寻求优化方案以提升Asio异步文件读取的性能。

内容的提问来源于stack exchange，提问作者user28336473