MPI并行FFT程序崩溃求助：exit code 0xc0000374

核心问题分析 #

退出码0xc0000374表示堆损坏，结合代码来看，主要问题集中在自定义MPI数据类型错误、进程间通信不匹配和内存越界访问三个方面：

1. 自定义MPI数据类型错误 #

创建num1type时存在两处关键错误：

• blocklen数组表示每个字段的元素个数，而非字节数。num1的real和img都是单个float，所以blocklen应设为{1, 1}，而非{4, 4}。
• 硬编码偏移量disp[1] = 4不可靠，不同编译器的内存对齐规则可能导致偏移量变化，需用MPI_Get_address动态获取字段偏移。

错误的类型定义会导致MPI在发送/接收时读写超出结构体范围的内存，直接引发堆损坏。

2. 通信不匹配导致死锁 #

rank 0进程的接收循环for (i=1; i<num; i++)试图接收num-1个消息，但实际只有size-1个非0进程（每个进程发送1个消息）。这会导致rank 0一直等待不存在的消息，触发死锁，最终引发崩溃。

3. 内存越界与任务分配逻辑缺陷 #

• 当进程数size大于num时，rank >= num的进程访问seq[rank]会直接越界，破坏内存。
• 当前逻辑让每个非0进程只处理与自身rank对应的单个元素，当size < num时，大部分元素没有进程处理，计算结果不完整，同时rank 0的接收逻辑依然错误等待过多消息。

4. 位反转计算的潜在误差 #

log2(num)/log2(2)是冗余计算，且浮点数运算可能产生精度误差，应直接使用key变量（初始值为log2(num)）作为位反转的位数。

修复后的代码 #

#include <iostream>
#include <cmath>
#include <mpi.h>
#include <cstdlib>
#include <ctime>

#define PI 3.14159265

struct num1
{
    float real;
    float img;
};

int reverseBits(int number, int NO_OF_BITS)
{
    int reverse_num = 0, i, temp;
    for (i = 0; i < NO_OF_BITS; i++)
    {
        temp = (number & (1 << i));
        if (temp)
            reverse_num |= (1 << ((NO_OF_BITS - 1) - i));
    }
    return reverse_num;
}

int main(int argc, char** argv)
{
    int rank, size, num, i, key, div;
    double bb1time, ab1time, bb2time, ab2time, starttime, endtime;

    MPI_Status status;

    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    MPI_Datatype num1type;
    MPI_Datatype type[2] = { MPI_FLOAT, MPI_FLOAT };
    int blocklen[2] = { 1, 1 };
    MPI_Aint disp[2];

    num = 256;

    if (size > num) {
        if (rank == 0) {
            std::cerr << "Error: Number of processes cannot exceed " << num << std::endl;
        }
        MPI_Finalize();
        return 1;
    }

    bb1time = MPI_Wtime();
    MPI_Bcast(&num, 1, MPI_INT, 0, MPI_COMM_WORLD);
    ab1time = MPI_Wtime();

    struct num1* input = new num1[num + 1];
    struct num1* seq = new num1[num + 1];
    struct num1* temp = new num1[num + 1];

    input[0].real = 0.0;
    input[0].img = 0.0;
    seq[0].real = 0.0;
    seq[0].img = 0.0;
    temp[0].real = 0.0;
    temp[0].img = 0.0;

    MPI_Get_address(&input[0].real, &disp[0]);
    MPI_Get_address(&input[0].img, &disp[1]);
    disp[1] -= disp[0];
    disp[0] = 0;

    MPI_Type_create_struct(2, blocklen, disp, type, &num1type);
    MPI_Type_commit(&num1type);

    if (rank == 0)
    {
        bb1time = MPI_Wtime();
        srand(time(NULL));
        for (i = 1; i < num + 1; i++)
        {
            input[i].real = static_cast<float>(rand()) / static_cast<float>(RAND_MAX);
            input[i].img = 0.0;
        }

        key = log2(num);
        for (i = 1; i < num + 1; i++)
        {
            seq[i].real = input[reverseBits(i - 1, key) + 1].real;
            seq[i].img = 0.0;
        }
    }

    bb2time = MPI_Wtime();
    MPI_Bcast(seq, num + 1, num1type, 0, MPI_COMM_WORLD);
    ab2time = MPI_Wtime();

    if (rank == 0) {
        key = log2(num);
        div = 1;
    }
    MPI_Bcast(&key, 1, MPI_INT, 0, MPI_COMM_WORLD);
    MPI_Bcast(&div, 1, MPI_INT, 0, MPI_COMM_WORLD);

    starttime = MPI_Wtime();

    while (key--)
    {
        if (rank != 0)
        {
            for (i = rank; i < num + 1; i += size)
            {
                if (((i + div - 1) / div) % 2 == 0)
                {
                    if (i - div < 1) continue;
                    temp[i].real = seq[i - div].real +
                        cos(PI * ((i - 1) % (div * 2)) / div) * seq[i].real +
                        sin(PI * ((i - 1) % (div * 2)) / div) * seq[i].img;

                    temp[i].img = seq[i - div].img +
                        cos(PI * ((i - 1) % (div * 2)) / div) * seq[i].img -
                        sin(PI * ((i - 1) % (div * 2)) / div) * seq[i].real;
                }
                else
                {
                    if (i + div > num) continue;
                    temp[i].real = seq[i].real +
                        cos(PI * ((i - 1) % (div * 2)) / div) * seq[i + div].real +
                        sin(PI * ((i - 1) % (div * 2)) / div) * seq[i + div].img;

                    temp[i].img = seq[i].img +
                        cos(PI * ((i - 1) % (div * 2)) / div) * seq[i + div].img -
                        sin(PI * ((i - 1) % (div * 2)) / div) * seq[i + div].real;
                }
                MPI_Send(&temp[i], 1, num1type, 0, 0, MPI_COMM_WORLD);
            }
        }
        else
        {
            for (i = 1; i < num + 1; i++) {
                temp[i] = seq[i];
            }
            int recv_count = 0;
            while (recv_count < num - 1) {
                num1 val;
                MPI_Recv(&val, 1, num1type, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, &status);
                int src_rank = status.MPI_SOURCE;
                int idx = src_rank + (recv_count / ((num - 1)/size)) * size;
                temp[idx] = val;
                recv_count++;
            }
        }

        MPI_Barrier(MPI_COMM_WORLD);

        if (rank == 0)
        {
            for (i = 1; i < num + 1; i++)
            {
                seq[i] = temp[i];
            }
            div *= 2;
        }

        MPI_Bcast(seq, num + 1, num1type, 0, MPI_COMM_WORLD);
        MPI_Bcast(&div, 1, MPI_INT, 0, MPI_COMM_WORLD);
    }

    endtime = MPI_Wtime();

    if (0 == rank)
    {
        std::cout << std::endl;

        for (i = 1; i < num + 1; i++)
        {
            std::cout << "X[" << i - 1 << "] : " << seq[i].real;

            if (seq[i].img >= 0)
                std::cout << "+j" << seq[i].img << std::endl;
            else
                std::cout << "-j" << -seq[i].img << std::endl;
        }

        std::cout << std::endl;
        std::cout << "Time to broadcast num variable: " << (ab1time - bb1time) * 1000 << " ms" << std::endl;
        std::cout << "Time to broadcast input and seq arrays: " << (ab2time - bb2time) * 1000 << " ms" << std::endl;
        std::cout << "Time to compute FFT parallely: " << (endtime - starttime) * 1000 << " ms" << std::endl;
        std::cout << "Total Time: " << ((endtime - starttime) + (ab2time - bb2time) + (ab1time - bb1time)) * 1000 << " ms" << std::endl;
        std::cout << std::endl;
    }

    MPI_Type_free(&num1type);
    delete[] input;
    delete[] seq;
    delete[] temp;

    MPI_Finalize();

    return 0;
}

额外优化点 #

• 添加了内存释放逻辑（delete[]和MPI_Type_free），避免内存泄漏。
• 简化了复数负号的输出逻辑，修复原代码中冗余计算。
• 添加了进程数检查，避免rank >= num导致的内存越界。
• 调整了任务分配策略，让每个进程处理多个元素（按步长size分配），适配任意小于等于num的进程数。

内容的提问来源于stack exchange，提问作者user23994688