处理dlis测井文件时如何将二级嵌套numpy数组转为普通结构化数组

我们使用Python dlisio库提取.dlis测井文件的数据，多数厂商的文件数据结构一致，但某厂商的文件数据以嵌套numpy数组形式存储。

正常dlis读取的数据示例 #

selected_curves_data[0:3]
Output
array([(172600., 1318.3775, 1130.0346, -1130.0301),
       (172590., 1331.5   , 1130.0346, -1130.0301),
       (172580., 1343.5   , 1130.046 , -1130.001 )],
      dtype={'names':['TDEP','A','B','C'], 'formats':['<f4','<f4','<f4','<f4'], 'offsets':[4,8,12,16], 'titles':['T.CHANNEL-I.TDEP','T.CHANNEL-I.A','T.CHANNEL-I.B','T.CHANNEL-C'], 'itemsize':20})

特殊厂商文件的数据结构示例 #

每个值都嵌套在两层子列表中：

selected_curves_data[0:3]
Output
array([([[6860. ]], [[7.887773]], [[65.23707 ]], [[83.41805]], [[98.60489 ]], [[76.93024]], [[305.9046]], [[  1.435147 ]], [[0.]]),
       ([[6859.9]], [[7.594969]], [[65.16657 ]], [[83.31693]], [[98.35259 ]], [[76.18296]], [[305.8163]], [[-10.156202 ]], [[0.]]),
       ([[6859.8]], [[7.539917]], [[65.115074]], [[83.21918]], [[98.084015]], [[75.37859]], [[305.7146]], [[  2.4681084]], [[0.]])],
      dtype={'names':['DEPTH','A','B','C','D','E','F','G','H'], 'formats':[('<f8', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1)),('<f4', (1, 1))], 'offsets':[4,12,16,20,24,62,66,1070,1074], 'titles':['T.CHANNEL-I.DEPTH','T.CHANNEL-I.A','T.CHANNEL-I.B','T.CHANNEL-I.C','T.CHANNEL-I.D','T.CHANNEL-I.E','T.CHANNEL-I.F','T.CHANNEL-I.G','T.CHANNEL-I.H'], 'itemsize':1078}) 

已尝试的转换代码 #

a = np.empty_like (selected_curves_data)
selected_curves_data[0:3]
ithRowList = [] 
i=0
for item in selected_curves_data:
   if i < 3:
       print("Type: ", type(item), " shape: ", item.shape) 
       print("----------------------------- Row ",i, " Original ----------------------------- \n", item)
       for subItem in item:
           ithRowList.append(float(subItem))
       
       print("Type: ", type(ithRowList))
       print("----------------------------- Row ",i, " After ----------------------------- \n", ithRowList)
      
       arr = np.array(ithRowList)
       print("Type: ", type(arr), " shape: ", item.shape) 
       print("----------------------------- Row ",i, " Array ----------------------------- \n", arr)
       print("\n")
      
       np.append(a, ithRowList)
       i+=1
       ithRowList = []
          
       
a[0:3]

运行输出 #

Type:  <class 'numpy.void'>  shape:  ()
----------------------------- Row  0  Original ----------------------------- 
 ([[6860.]], [[7.887773]], [[65.23707]], [[83.41805]], [[98.60489]], [[76.93024]], [[305.9046]], [[1.435147]], [[0.]])

 Type:  <class 'list'>
----------------------------- Row  0  After ----------------------------- 
 [6860.0, 7.887773036956787, 65.23706817626953, 83.41805267333984, 98.60488891601562, 76.93023681640625, 305.90460205078125, 1.4351470470428467, 0.0]

 Type:  <class 'numpy.ndarray'>  shape:  ()
----------------------------- Row  0  Array ----------------------------- 
 [6.86000000e+03 7.88777304e+00 6.52370682e+01 8.34180527e+01
 9.86048889e+01 7.69302368e+01 3.05904602e+02 1.43514705e+00
 0.00000000e+00]

报错信息 #

TypeError: The DTypes <class 'numpy.dtype[float64]'> and <class 'numpy.dtype[void]'> do not have a common DType. For example they cannot be stored in a single array unless the dtype is object.

预期目标 #

得到和正常读取的数组结构、维度一致的最终数组，仅移除值的嵌套结构，同时保留numpy数组的列信息，单条数据格式如下：
(6860., 7.887773, 65.23707, 83.41805, 98.60489,76.93024, 305.9046, 1.435147, 0.)

报错核心原因有两点：

1. 初始化的a是和原数组一致的结构化数组，dtype为带字段信息的void类型，而尝试写入的是普通float64类型数组，两类dtype无法兼容
2. np.append不会修改原数组，仅会返回新数组，原有写法的赋值逻辑不生效

以下是直接可用的转换代码，完全保留原数组的列元信息，向量化操作效率更高：

import numpy as np

# 1. 提取原数组dtype信息，构造无嵌套维度的新dtype
old_dtype = selected_curves_data.dtype
new_fields = []
for name in old_dtype.names:
    # 保留基础数据类型、偏移量、字段标题，去掉每个字段的(1,1)嵌套维度
    base_type = old_dtype.fields[name][0].base
    offset = old_dtype.fields[name][1]
    title = old_dtype.fields[name][2]
    new_fields.append((name, base_type, offset, {"title": title}))

new_dtype = np.dtype({
    "names": [f[0] for f in new_fields],
    "formats": [f[1] for f in new_fields],
    "offsets": [f[2] for f in new_fields],
    "titles": [f[3]["title"] for f in new_fields],
    "itemsize": old_dtype.itemsize
})

# 2. 初始化新结构化数组
new_curve_data = np.empty(len(selected_curves_data), dtype=new_dtype)

# 3. 逐列填充数据，用squeeze去掉嵌套的多余维度
for col_name in old_dtype.names:
    new_curve_data[col_name] = selected_curves_data[col_name].squeeze()

# 验证前3条数据
print(new_curve_data[0:3])

执行后得到的new_curve_data结构和正常dlis读取的数组完全一致，可直接适配后续处理流程。

内容的提问来源于stack exchange，提问作者maithask