设置model.config.decoder_start_token_id后，训练MusicGen仍报错ValueError的问题求助

大家好，我在尝试用LoRA微调MusicGen-small模型时遇到了个头疼的问题——明明已经手动设置了model.config.decoder_start_token_id，但启动训练后还是抛出了ValueError，提示我必须设置这个配置项。下面是我的完整代码流程和报错信息，麻烦各位帮忙排查一下问题！

1. 模型初始化与配置设置 #

我首先加载了MusicGen模型和对应的处理器、分词器，并且明确设置了解码器的起始token ID：

from transformers import AutoProcessor, MusicgenForConditionalGeneration, EncodecModel

model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
processor = AutoProcessor.from_pretrained("facebook/musicgen-small")

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("facebook/musicgen-small")
decoder_start_token = "<s>"  
model.config.decoder_start_token_id = tokenizer.convert_tokens_to_ids(decoder_start_token)

2. LoRA配置与PEFT模型创建 #

接着我定义了LoRA的配置，并生成了PEFT模型：

from peft import LoraConfig, TaskType
import torch.nn as nn

target_modules = [
    name for name, module in model.named_modules()
    if isinstance(module, (nn.Linear))
]

peft_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    target_modules=target_modules,
    inference_mode=False,
    r=8,
    lora_alpha=32,
    lora_dropout=0.1
)

from peft import get_peft_model

peft_model = get_peft_model(model, peft_config)

peft_model.print_trainable_parameters()

3. 训练参数设置 #

然后我配置了训练的相关参数：

from transformers import TrainingArguments, Trainer

training_args = TrainingArguments(
    output_dir="./musicgen_results",
    learning_rate=1e-3,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    num_train_epochs=2,
    weight_decay=0.01,
    eval_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    report_to="none"
)

4. 数据预处理函数 #

我写了预处理函数来处理音频和文本数据：

from transformers import EncodecModel
import torch

def preprocess_function(example):
    audio = example["audio_path"]
    encodec_model = EncodecModel.from_pretrained("facebook/encodec_32khz")

    audio_array = example["audio_path"]["array"]
    sampling_rate = 32000  

    audio_tensor = torch.tensor(audio_array, dtype=torch.float32)  
    audio_tensor = audio_tensor.unsqueeze(0).unsqueeze(0)  

    audio_inputs = encodec_model.encode(audio_tensor)
    audio_tokens = audio_inputs.audio_codes  
    example["labels"] = audio_tokens
    example.update(processor(
        audio=audio["array"],
        text=example["description"],
        sampling_rate=32000
    ))
    return example

train_dataset = train_dataset.map(preprocess_function, remove_columns=["description", "audio_path"])
eval_dataset = eval_dataset.map(preprocess_function, remove_columns=["description", "audio_path"])

5. 启动训练与报错信息 #

最后我初始化了Trainer并启动训练：

from torch.nn.utils.rnn import pad_sequence
from transformers import DataCollatorForSeq2Seq

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
)

trainer.train()

但运行后立刻抛出了如下错误：

/usr/local/lib/python3.10/dist-packages/transformers/models/musicgen/modeling_musicgen.py in shift_tokens_right(input_ids, pad_token_id, decoder_start_token_id)
102     shifted_input_ids[..., 1:] = input_ids[..., :-1].clone()
103     if decoder_start_token_id is None:
---> 104         raise ValueError("Make sure to set the decoder_start_token_id attribute of the model's configuration.")
105     shifted_input_ids[..., 0] = decoder_start_token_id
106

ValueError: Make sure to set the decoder_start_token_id attribute of the model's configuration.

我真的很困惑——明明在最开始就已经设置了model.config.decoder_start_token_id，为什么训练时还是会触发这个错误呢？有没有大佬能帮我找到问题的根源呀？

备注：内容来源于stack exchange，提问作者김동연