如何计算STFT移调后的相位?PyTorch实现及自动微分问题
基于STFT的音频移调:相位计算、PyTorch内置函数与Autograd支持问题
我使用torch.stft()生成频谱图,希望直接对音频执行移调操作,最终得到移调后音频的STFT。由于phase_vocoder→istft→resample→stft的流程速度过慢,我编写了一段代码,通过插值原频谱的对应频率仓来生成新频谱:
def interpolate(frequencies: torch.Tensor, sgram: torch.Tensor): start = frequencies.int() frac = (frequencies - start)[:, None] return sgram[start, :] * (1 - frac) + sgram[start + 1, :] * frac def pitch_shift_spectrogram(sgram: torch.Tensor, semitones: torch.Tensor): scaling_factor = 2 ** (-semitones / 12) frequencies = torch.arange(0, sgram.shape[0], 1, device=sgram.device) shifted_frequencies = frequencies * scaling_factor shifted_mags = interpolate(shifted_frequencies, sgram.abs()) phases = sgram.angle() # ??? what do i do? help return torch.polar(shifted_mags, phases)
我在CantinaBand3.wav音频文件上进行了测试,生成的幅度谱效果尚可;若使用完整移调实现的相位,音频效果正常:
import torch import torchaudio import IPython.display as display waveform, sample_rate = torchaudio.load("CantinaBand3.wav") waveform = waveform[0] shifted_waveform = torchaudio.functional.pitch_shift(waveform, sample_rate, 2) working_sgram = torch.stft(shifted_waveform, 1024, return_complex=True) unshifted_sgram = torch.stft(waveform, 1024, return_complex=True) broken_sgram = pitch_shift_spectrogram(unshifted_sgram, torch.tensor(2)) broken_sgram = torch.polar(broken_sgram.abs(), working_sgram.angle()) display.display(display.Audio(torch.istft(broken_sgram, 1024), rate=sample_rate))
现提出以下问题:
- 能否仅通过STFT计算移调信号的相位信息?
- PyTorch是否提供相关内置函数?
- 当
semitones作为模型参数时,插值操作能否正常支持autograd反向传播?
内容的提问来源于stack exchange,提问作者potatoportato
相关产品推荐
相关产品推荐

