解决NumPy hstack维度不匹配ValueError问题（音频特征提取）

基于librosa进行音频特征提取时遭遇ValueError，提示所有输入数组必须具有相同维度，索引0的数组为1维，索引1的数组为2维。原本计划将2D数组转换为1D数组，但当前实现未达到预期效果。

特征提取函数 #

#DataFlair - Extract features (mfcc, chroma, mel) from a sound file
def extract_feature(file_name, mfcc, chroma, mel):
    with soundfile.SoundFile(file_name) as sound_file:
        X = sound_file.read(dtype="float32")
        sample_rate=sound_file.samplerate
        if chroma:
            stft=np.abs(librosa.stft(X))
        result=np.array([])
        if mfcc:
            mfccs=np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
            result=np.hstack((result, mfccs))
        if chroma:
            chroma=np.mean(librosa.feature.chroma_stft(S=stft, sr=sample_rate).T,axis=0)
            result=np.hstack((result, chroma))
        if mel:
            mel=np.mean(librosa.feature.melspectrogram(y=X, sr=sample_rate).T,axis=0)
            result=np.hstack((result, mel))
        return result

数据加载函数 #

#DataFlair - Load the data and extract features for each sound file
def load_data(test_size=0.2):
    x,y=[],[]
    for file in glob.glob("D:\\NFT\Data\Ravdess\\Actor_*\\*.wav"):
        file_name=os.path.basename(file)
        emotion=emotions[file_name.split("-")[2]]
        if emotion not in observed_emotions:
            continue
        feature=extract_feature(file, mfcc=True, chroma=True, mel=True)
        x.append(feature)
        y.append(emotion)
    return train_test_split(np.array(x), y, test_size=test_size, random_state=9)

数据集分割代码 #

#DataFlair - Split the dataset
x_train,x_test,y_train,y_test=load_data(test_size=0.25)

报错信息 #

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[57], line 2
      1 #DataFlair - Split the dataset
----> 2 x_train,x_test,y_train,y_test=load_data(test_size=0.25)

Cell In[56], line 9, in load_data(test_size)
      7 if emotion not in observed_emotions:
      8     continue
----> 9 feature=extract_feature(file, mfcc=True, chroma=True, mel=True)
     10 x.append(feature)
     11 y.append(emotion)

Cell In[54], line 11, in extract_feature(file_name, mfcc, chroma, mel)
      9 if mfcc:
     10     mfccs=np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
---> 11     result=np.hstack((result, mfccs))
     12 if chroma:
     13     chroma=np.mean(librosa.feature.chroma_stft(S=stft, sr=sample_rate).T,axis=0)

File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\numpy\core\shape_base.py:357, in hstack(tup, dtype, casting)
    355 # As a special case, dimension 0 of 1-dimensional arrays is "horizontal"
    356 if arrs and arrs[0].ndim == 1:
---> 357     return _nx.concatenate(arrs, 0, dtype=dtype, casting=casting)
    358 else:
    359     return _nx.concatenate(arrs, 1, dtype=dtype, casting=casting)

ValueError: all the input arrays must have same number of dimensions, but the array at index 0 has 1 dimension(s) and the array at index 1 has 2 dimension(s)

问题根源 #

报错核心原因是部分音频文件为多声道（如立体声），读取后得到的X是2维数组（形状为(采样数, 声道数)）。传入librosa特征提取函数后，返回的特征数组会保留声道维度，经过np.mean(..., axis=0)处理后得到2维数组，而初始的result是1维空数组，np.hstack无法拼接不同维度的数组，因此抛出错误。

修复步骤 #

1. 将多声道音频转换为单声道：读取音频后，对声道维度取均值，将2维数组转为1维。
2. 确保所有特征数组均为1维，避免维度不匹配。

修改后的代码 #

#DataFlair - Extract features (mfcc, chroma, mel) from a sound file
def extract_feature(file_name, mfcc, chroma, mel):
    with soundfile.SoundFile(file_name) as sound_file:
        X = sound_file.read(dtype="float32")
        sample_rate=sound_file.samplerate
        
        # 将多声道音频转为单声道
        if X.ndim > 1:
            X = np.mean(X, axis=1)
        
        stft = None
        if chroma:
            stft=np.abs(librosa.stft(X))
        result=np.array([])
        if mfcc:
            mfccs=np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
            result=np.hstack((result, mfccs))
        if chroma:
            chroma=np.mean(librosa.feature.chroma_stft(S=stft, sr=sample_rate).T,axis=0)
            result=np.hstack((result, chroma))
        if mel:
            mel=np.mean(librosa.feature.melspectrogram(y=X, sr=sample_rate).T,axis=0)
            result=np.hstack((result, mel))
        return result

额外验证 #

如果仍有问题，可在提取特征后打印数组形状，确认是否为1维：

print(mfccs.shape)  # 应为(40,)
print(chroma.shape) # 应为(12,)
print(mel.shape)    # 应为(128,) 默认n_mels=128

内容的提问来源于stack exchange，提问作者SAKET KUMAR