MPI_Get同时对源和目标使用跨步访问是否符合规范且可移植?
MPI_Get同时使用源端与目标端向量类型的合规性说明
结论先行
这种用法完全符合MPI规范,属于具备可移植性的标准行为。
规范依据
MPI标准中MPI_Get的函数签名为:
int MPI_Get(void *origin_addr, int origin_count, MPI_Datatype origin_datatype, int target_rank, MPI_Aint target_disp, int target_count, MPI_Datatype target_datatype, MPI_Win win)
参数里的origin_*系列对应本地接收缓冲区的描述,target_*系列对应远程源端内存的描述。MPI标准对这两端的数据类型没有额外限制——只要是已通过MPI_Type_commit提交的合法派生数据类型(比如MPI_Type_vector创建的向量类型),都可以正常使用。无论源端还是目标端,都支持用自定义的跨步、向量等类型来描述非连续内存布局,本质是让MPI库自动完成非连续内存的读写映射。
你的代码验证
你提供的测试代码中:
- 源端(rank 1)通过
MPI_Type_vector定义了适配自身数组布局的type_origin - 目标端(rank 2)定义了适配自身数组布局的
type_target - 调用
MPI_Get时分别指定两端的数据类型,完全符合标准要求。OpenMPI环境下的正常运行也验证了这一点,其他符合MPI标准的实现(如MPICH)也应当支持该用法。
注意事项
- 必须保证两端数据类型描述的数据总量一致:即
origin_count×origin_datatype的元素总数,要等于target_count×target_datatype的元素总数,否则会触发未定义行为。 - 所有自定义数据类型必须先调用
MPI_Type_commit才能用于RMA操作,你的代码已经满足这一要求。 - 确保
MPI_Win_create创建的窗口内存区域覆盖源端需要访问的所有数据,避免越界访问。
完整测试代码
#define OMPI_SKIP_MPICXX 1 #include <mpi.h> #include <Eigen/Dense> #include <iostream> using namespace Eigen; MPI_Datatype getVectorType( const Ref<const Array2i>& totalBlockSize, const Ref<const Array2i>& subBlockSize, Index nComponents ){ MPI_Datatype vec; MPI_Type_vector( subBlockSize.y(), subBlockSize.x() * nComponents, totalBlockSize.x() * nComponents, MPI_DOUBLE, &vec ); return vec; } int getDisp( const Ref<const Array2i>& start, const Ref<const Array2i>& size, Index nComponents ){ return ( start.y() * size.x() + start.x() ) * nComponents; } int main(int argc, char* argv[]){ MPI_Init(&argc,&argv); MPI_Comm comm {MPI_COMM_WORLD}; int nRanks, rank; MPI_Comm_size(comm, &nRanks); MPI_Comm_rank(comm, &rank); /* let's just say it's ranks 1 and 2 that have to communicate */ int rank_origin {1}, rank_target {2}; /* and what they have to communicate is a block of data, * which is not contiguous on either rank */ Array2i size_origin { 8,12}, size_target { size_origin + 1 }, start_block { 3, 4}, size_block { 4, 6}; ArrayXXd arr_origin, arr_target; /* number of components per cell, equals number of rows in arrays */ /* to make it simple, it's set to 1 here, so it can be ignored below */ Index nComp {1}; auto reshaped = [&](ArrayXXd& arr, const Array2i& size){ return arr.reshaped( nComp * size.x(), size.y() ); }; auto reshapedBlock = [&](auto& resh, const Array2i& start_block, const Array2i& size_block ){ return resh.block( nComp * start_block.x(), start_block.y(), nComp * size_block .x(), size_block .y() ); }; auto print = [&](const auto& resh){ std::cout << "On rank " << rank << ", array content (reshaped):\n" << resh << "\n"; }; if ( rank == rank_origin ){ arr_origin.resize( nComp, size_origin.prod() ); /* set here as a default value so that we know where it's from */ arr_origin = -rank_origin; auto resh { reshaped(arr_origin, size_origin) }; auto reshBlock { reshapedBlock(resh, start_block, size_block) }; reshBlock = rank_origin; print(resh); } MPI_Datatype type_origin, type_target; if ( rank == rank_target ){ arr_target.resize( nComp, size_target.prod() ); arr_target= -rank_target; type_origin = getVectorType(size_origin, size_block, nComp); type_target = getVectorType(size_target, size_block, nComp); MPI_Type_commit(&type_origin); MPI_Type_commit(&type_target); } MPI_Win win; constexpr int disp { sizeof(double) }; MPI_Win_create( arr_origin.data(), arr_origin.size() * disp, disp, MPI_INFO_NULL, comm, &win ); MPI_Win_fence(0, win); if ( rank == rank_target ){ int disp_origin { getDisp(start_block, size_origin, nComp) }, disp_target { getDisp(start_block, size_target, nComp) }; MPI_Get( arr_target.data() + disp_target, 1, type_target, rank_origin, disp_origin, 1, type_origin, win ); MPI_Type_free(&type_origin); MPI_Type_free(&type_target); } MPI_Win_fence(0, win); if ( rank == rank_target ){ print( reshaped(arr_target, size_target) ); } MPI_Win_free(&win); MPI_Finalize(); return 0; }
内容的提问来源于stack exchange,提问作者RL-S
相关产品推荐
相关产品推荐

