Google Colab构建HuBERT时libsox.so缺失错误求助

在Google Colab中按照官方文档构建HuBERT模型的kmeans程序，已生成符合要求的TSV文件，但执行dump_mfcc_feature.py脚本时触发以下错误：

OSError: libsox.so: cannot open shared object file: No such file or directory

尝试手动编译安装SOX及相关依赖后问题依旧。

相关信息 #

TSV文件内容 #

/content/drive/MyDrive/dataset_myvoice/data/
emotion1.wav    222578
emotion2.wav    188475
emotion3.wav    203631
emotion4.wav    201335

执行命令 #

!python '/content/drive/MyDrive/fairseq-main/examples/hubert/simple_kmeans/dump_mfcc_feature.py' '/content/' paths  8 7 '/content/'

完整报错日志 #

2024-03-15 01:31:39.711269: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-03-15 01:31:39.711324: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-03-15 01:31:39.712768: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-03-15 01:31:39.719939: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-15 01:31:40.845096: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-03-15 01:31:41 | INFO | numexpr.utils | NumExpr defaulting to 8 threads.
2024-03-15 01:31:42 | INFO | fairseq.tasks.text_to_speech | Please install tensorboardX: pip install tensorboardX
2024-03-15 01:31:43 | INFO | dump_mfcc_feature | Namespace(tsv_dir='/content/', split='paths', nshard=8, rank=7, feat_dir='/content/', sample_rate=16000)
2024-03-15 01:31:43 | INFO | feature_utils | rank 7 of 8, process 53 (371-424) out of 424
  0% 0/53 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/content/drive/MyDrive/fairseq-main/examples/hubert/simple_kmeans/dump_mfcc_feature.py", line 74, in <module>
    main(**vars(args))
  File "/content/drive/MyDrive/fairseq-main/examples/hubert/simple_kmeans/dump_mfcc_feature.py", line 58, in main
    dump_feature(reader, generator, num, split, nshard, rank, feat_dir)
  File "/content/drive/MyDrive/fairseq-main/examples/hubert/simple_kmeans/feature_utils.py", line 61, in dump_feature
    feat = reader.get_feats(path, nsample)
  File "/content/drive/MyDrive/fairseq-main/examples/hubert/simple_kmeans/dump_mfcc_feature.py", line 37, in get_feats
    x = self.read_audio(path, ref_len=ref_len)
  File "/content/drive/MyDrive/fairseq-main/examples/hubert/simple_kmeans/dump_mfcc_feature.py", line 31, in read_audio
    wav = get_features_or_waveform(path, need_waveform=True, use_sample_rate=self.sample_rate)
  File "/usr/local/lib/python3.10/dist-packages/fairseq/data/audio/audio_utils.py", line 168, in get_features_or_waveform
    return get_waveform(
  File "/usr/local/lib/python3.10/dist-packages/fairseq/data/audio/audio_utils.py", line 106, in get_waveform
    waveform, sample_rate = convert_waveform(
  File "/usr/local/lib/python3.10/dist-packages/fairseq/data/audio/audio_utils.py", line 58, in convert_waveform
    converted, converted_sample_rate = ta_sox.apply_effects_tensor(
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/sox_effects/sox_effects.py", line 156, in apply_effects_tensor
    return sox_ext.apply_effects_tensor(tensor, sample_rate, effects, channels_first)
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 121, in __getattr__
    self._import_once()
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 135, in _import_once
    self.module = self.import_func()
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 85, in _init_sox
    ext = _import_sox_ext()
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 80, in _import_sox_ext
    _load_lib("libtorchaudio_sox")
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 60, in _load_lib
    torch.ops.load_library(path)
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 933, in load_library
    ctypes.CDLL(path)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libsox.so: cannot open shared object file: No such file or directory

已尝试的解决命令 #

!apt-get update
!apt-get install -y --no-install-recommends \
    software-properties-common \
    build-essential \
    libasound2-dev \
    libmp3lame-dev \
    libogg-dev \
    libvorbis-dev \
    libgsm1-dev \
    libopus-dev \
    libpulse-dev \
    libavcodec-dev \
    libavformat-dev \
    libswresample-dev \
    libavutil-dev \
    libsndfile1-dev

!apt-get install -y wget
!wget https://sourceforge.net/projects/sox/files/sox/14.4.2/sox-14.4.2.tar.gz
!tar -xvzf sox-14.4.2.tar.gz
%cd sox-14.4.2

!./configure 
!make
!make install

!ldconfig -p | grep libsox

import sys
sox_path = "/lib/x86_64-linux-gnu/libsoxr.so.0"  # 替换为合适路径
sys.path.append(sox_path)

!pip install soundfile

解决方案 #

问题出在手动编译安装SOX后，系统无法正确找到libsox.so库文件。直接通过系统包管理器安装预编译的SOX即可解决，无需手动编译。执行以下命令：

!apt-get update
!apt-get install -y sox libsox-dev libsox-fmt-all

安装完成后，重新运行dump_mfcc_feature.py脚本即可。

说明 #

手动编译的SOX可能没有被正确添加到系统库路径中，而通过apt安装的预编译包会自动处理库路径配置，确保torchaudio能找到libsox.so。另外，libsox-fmt-all会安装所有SOX支持的音频格式插件，避免后续出现音频格式不兼容的问题。

内容的提问来源于stack exchange，提问作者anisonbiyori