如何用C++实现Python HDF5二维数据读写的等价逻辑？

需求背景 #

寻求与给定Python HDF5代码等价的简化C++实现，核心需求：

• 创建包含一个或多个数据集的.h5文件，文件存在时读写，不存在则创建后进行读写操作；
• 数据集存储来自二维集合的数据，该集合的具体大小无法提前确定；
• 希望基于C++内置数据结构实现，避免额外的数据复制操作。

参考Python代码 #

import numpy as np
import h5py

file_name = "f.h5"
dataset_name = "dset_1"


# list-of-lists representing a function call that returns a list-of-lists.
l_2d = [[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4]]


f = h5py.File(f"{file_name}", "a")
# f.flush()


if dataset_name not in f.keys():
  f.create_dataset(dataset_name, (0, 2), maxshape=(None, 2), dtype="i")
  # f.flush()


a_2d = np.array(l_2d)

f[dataset_name].resize(f[dataset_name].shape[0] + a_2d.shape[0], axis=0)
f[dataset_name][-a_2d.shape[0]:] = a_2d

f.flush()
f.close()

当前C++进展代码 #

// Here are comments about some of the tests made while working to get Python equivalence.

#include <array>
#include <iostream>
#include <string>
#include <vector>
#include "H5Cpp.h"

const H5std_string FILE_NAME("f.h5");
const H5std_string DATASET_NAME("dset_1");

int main()
{
  // int                               a_2d[4][3] = {{1, 1, 1}, {2, 2, 2}, {3, 3, 3}, {4, 4, 4}};
  // std::array<std::array<int, 3>, 4> a_2d{{ {1, 1, 1}, {2, 2, 2}, {3, 3, 3}, {4, 4, 4} }};
  std::vector<std::array<int, 3>>      a_2d{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}, {4, 4, 4}};
  // std::array<std::vector<int>, 4>   a_2d{{ {1, 1, 1}, {2, 2, 2}, {3, 3, 3}, {4, 4, 4} }};
  // std::vector<std::vector<int>>     a_2d{{1, 1, 1}, {2, 2, 2}, {3, 3, 3}, {4, 4, 4}};
  
  try
  {
    Exception::dontPrint();
    
    // About `h5py.File(f"{file_name}", "a")`, what would be the "a" mode equivalence in C++?
    H5::H5File f(FILE_NAME, H5F_ACC_TRUNC);
    
    // Extra process for vector-of-vectors and array-of-vectors, more details below ...
    
    hsize_t dims[2];
    dims[0] = 4;  // a_2d.size();
    dims[1] = 3;  // a_2d[0].size();
    H5::DataSpace dspace(2, dims);
    
    H5::DataSet dset = f.createDataSet(DATASET_NAME, H5::PredType::NATIVE_INT32, dspace);  // H5::DataSet dset = f.createDataSet(DATASET_NAME, H5::PredType::STD_I32BE, dspace);
    
    // H5::H5File file(FILE_NAME, H5F_ACC_RDWR);
    // H5::DataSet dset = file.openDataSet(DATASET_NAME);
    
    // dset.write(a_2d, H5::PredType::NATIVE_INT32);     // for C-style arrays.
    dset.write(a_2d.data(), H5::PredType::NATIVE_INT32); // for non C-style arrays.
    
    
    // After checking some HDF5 examples in github, it is not clear exactly what
    // is needed about the following last part in the HDF5 process:
    dspace.close();
    dset.close();
    // f.flush();  // What would be the required `H5F_scope_t scope`?
    f.close();
    
  }

  catch (FileIException error)
  {
      error.printErrorStack();
      return -1;
  }

  catch (DataSetIException error)
  {
      error.printErrorStack();
      return -1;
  }

  catch (DataSpaceIException error)
  {
      error.printErrorStack();
      return -1;
  }

  return 0; // successfully terminated
}

当前问题 #

目前C代码可直接处理C风格二维数组、std::array<std::array>、std::vector<std::array>，但处理std::array<std::vector>、std::vector<std::vector>时，需将数据复制到新结构（如hvl_t），操作繁琐。同时需明确Python中h5py.File的"a"模式在C中的等价写法，以及数据集动态扩容的实现方式，希望基于C++内置结构简化实现。

内容的提问来源于stack exchange，提问作者L1L2L3