使用-fopenmp编译时文件写入不安全问题求助

问题描述 #

使用Eigen进行并行计算后将结果写入文件，编译时添加-fopenmp选项。代码在-g -o编译时运行正常，但并行化时出现停滞，即使添加了#pragma omp critical，程序仍卡在ofstream outputFile(path, ios::app);处，说明文件写入操作仍存在线程安全问题。相关代码示例如下：

class Loss
{
public:
    double loss_value;
    string choosen_loss;
    string path;

    Loss(string loss_function, string filepath)
    {
        choosen_loss = loss_function;
        path = filepath;
    };

    void calculator(variant<double, VectorXd> NN_outputs, variant<double, VectorXd> targets, int data_size)
    {
        #pragma omp critical //for safe writing; 
        if (choosen_loss == "MSE")
        {
            choice = MSE;
            loss_value += choice(NN_outputs, targets) / (double)data_size;
        }
        else if (choosen_loss == "BCE")
        {
            choice = BCE;
            loss_value += choice(NN_outputs, targets) / (double)data_size;
        }
        else if (choosen_loss == "MEE")
        {
            choice = MEE;
            loss_value += choice(NN_outputs, targets) / (double)data_size;
        }
        else
        {
            cout << "Unvailable choice as loss function. " << endl;
        }
        counter++;
        if (counter == data_size)
        {
            ofstream outputFile(path, ios::app);
            if (outputFile.is_open())
            {
                outputFile << loss_value << endl;
                outputFile.close();
            }
            else
            {
                cerr << "Errore: impossibile aprire il file " << path << endl;
            }
            counter = 0;
            loss_value = 0;
        }
    };
};

解决方案 #

问题根源 #

1. Critical区域范围过大：当前#pragma omp critical覆盖了loss计算、变量修改等所有操作，完全丧失并行效率，且counter、loss_value等共享变量的修改仍存在竞态风险。
2. 文件写入未完全保护：多个线程可能同时触发counter == data_size条件，导致同时打开同一个文件，引发资源竞争和停滞。
3. 共享变量无原子性保障：counter和loss_value的修改未做原子化处理，多线程同时修改会导致数据混乱。

修复步骤及代码示例 #

1. 拆分Critical区域，保留并行效率

将loss计算部分移出critical区域，让每个线程独立计算自身负责的loss值，仅在修改共享变量和文件写入时使用critical保护：

#include <atomic>
#include <fstream>
#include <string>
#include <variant>
#include <Eigen/Dense>

class Loss
{
public:
    double loss_value;
    std::string choosen_loss;
    std::string path;
    std::atomic<int> counter{0}; // 使用原子变量避免counter的竞态修改

    Loss(std::string loss_function, std::string filepath)
        : choosen_loss(std::move(loss_function)), path(std::move(filepath)), loss_value(0.0)
    {};

    void calculator(std::variant<double, Eigen::VectorXd> NN_outputs, std::variant<double, Eigen::VectorXd> targets, int data_size)
    {
        double current_loss = 0.0;

        // 每个线程独立计算loss，无需临界区
        if (choosen_loss == "MSE")
        {
            current_loss = MSE(NN_outputs, targets) / static_cast<double>(data_size);
        }
        else if (choosen_loss == "BCE")
        {
            current_loss = BCE(NN_outputs, targets) / static_cast<double>(data_size);
        }
        else if (choosen_loss == "MEE")
        {
            current_loss = MEE(NN_outputs, targets) / static_cast<double>(data_size);
        }
        else
        {
            // 错误输出需保护，避免多线程打印混乱
            #pragma omp critical
            std::cout << "Unavailable choice as loss function." << std::endl;
            return;
        }

        // 仅在修改共享变量和文件操作时使用临界区
        #pragma omp critical
        {
            loss_value += current_loss;
            counter++;

            // 确保只有一个线程执行文件写入和重置操作
            if (counter == data_size)
            {
                std::ofstream outputFile(path, std::ios::app);
                if (outputFile.is_open())
                {
                    outputFile << loss_value << std::endl;
                    outputFile.close();
                }
                else
                {
                    std::cerr << "Errore: impossibile aprire il file " << path << std::endl;
                }
                counter = 0;
                loss_value = 0.0;
            }
        }
    };
};

2. 更高效的归约替代方案（推荐）

使用OpenMP的reduction指令直接归约loss总和，避免使用critical区域，进一步提升并行效率：

#include <fstream>
#include <string>
#include <variant>
#include <vector>
#include <Eigen/Dense>

class Loss
{
public:
    std::string choosen_loss;
    std::string path;

    Loss(std::string loss_function, std::string filepath)
        : choosen_loss(std::move(loss_function)), path(std::move(filepath))
    {};

    double calculate_single(std::variant<double, Eigen::VectorXd> NN_outputs, std::variant<double, Eigen::VectorXd> targets, int data_size)
    {
        if (choosen_loss == "MSE")
        {
            return MSE(NN_outputs, targets) / static_cast<double>(data_size);
        }
        else if (choosen_loss == "BCE")
        {
            return BCE(NN_outputs, targets) / static_cast<double>(data_size);
        }
        else if (choosen_loss == "MEE")
        {
            return MEE(NN_outputs, targets) / static_cast<double>(data_size);
        }
        else
        {
            std::cout << "Unavailable choice as loss function." << std::endl;
            return 0.0;
        }
    };

    void write_total(double total_loss)
    {
        std::ofstream outputFile(path, std::ios::app);
        if (outputFile.is_open())
        {
            outputFile << total_loss << std::endl;
            outputFile.close();
        }
        else
        {
            std::cerr << "Errore: impossibile aprire il file " << path << std::endl;
        }
    };
};

// 并行调用示例
int main() {
    Loss loss("MSE", "loss_log.txt");
    int data_size = 1000;
    std::vector<std::variant<double, Eigen::VectorXd>> outputs(data_size);
    std::vector<std::variant<double, Eigen::VectorXd>> targets(data_size);

    // 填充outputs和targets数据...

    double total_loss = 0.0;
    #pragma omp parallel for reduction(+:total_loss)
    for (int i = 0; i < data_size; i++) {
        total_loss += loss.calculate_single(outputs[i], targets[i], data_size);
    }

    // 主线程单独写入文件，完全避免线程竞争
    loss.write_total(total_loss);
    return 0;
}

关键说明 #

• 原子变量：std::atomic<int>保证counter的修改是原子操作，避免多线程同时修改导致的计数错误。
• 临界区最小化：仅保护必要的共享操作，最大化并行计算的效率。
• 归约优化：reduction(+:total_loss)由OpenMP内部优化，比critical区域的串行操作效率更高，适合大规模数据的并行计算。

内容的提问来源于stack exchange，提问作者Fra_liturri