问题描述 #

我正在Google Colab中使用trl库的SFTTrainer微调语言模型，但遇到了以下错误：

TypeError Traceback (most recent call last)
in <cell line: 0>()
53
54
---> 55 trainer = SFTTrainer(
56 model=model,
57 train_dataset=data,

/usr/local/lib/python3.11/dist-packages/transformers/utils/deprecation.py in wrapped_func(*args, **kwargs)
170 warnings.warn(message, FutureWarning, stacklevel=2)
171
---> 172 return func(*args, **kwargs)
173
174 return wrapped_func

TypeError: SFTTrainer.init() got an unexpected keyword argument 'dataset_text_field'

我的代码 #

import torch
from datasets import load_dataset, Dataset
from peft import LoraConfig, AutoPeftModelForCausalLM, prepare_model_for_kbit_training, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig, TrainingArguments
from trl import SFTTrainer
import os

# Load dataset
data = load_dataset("tatsu-lab/alpaca", split="train")
data_df = data.to_pandas()
data_df = data_df[:5000]
data_df["text"] = data_df[["input", "instruction", "output"]].apply(lambda x: "###Human: " + x["instruction"] + " " + x["input"] + " ###Assistant: "+ x["output"], axis=1)
data = Dataset.from_pandas(data_df)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("TheBloke/Mistral-7B-Instruct-v0.1-GPTQ")
tokenizer.pad_token = tokenizer.eos_token

# Load model
quantization_config_loading = GPTQConfig(bits=4, disable_exllama=True, tokenizer=tokenizer)
model = AutoModelForCausalLM.from_pretrained(
                            "TheBloke/Mistral-7B-Instruct-v0.1-GPTQ",
                            quantization_config=quantization_config_loading,
                            device_map="auto"
                        )

model.config.use_cache = False
model.config.pretraining_tp = 1
model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model)

# Apply LoRA configuration
peft_config = LoraConfig(
    r=16, lora_alpha=16, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", target_modules=["q_proj", "v_proj"]
)
model = get_peft_model(model, peft_config)

# Training arguments
training_arguments = TrainingArguments(
        output_dir="mistral-finetuned-alpaca",
        per_device_train_batch_size=8,
        gradient_accumulation_steps=1,
        optim="paged_adamw_32bit",
        learning_rate=2e-4,
        lr_scheduler_type="cosine",
        save_strategy="epoch",
        logging_steps=100,
        num_train_epochs=1,
        max_steps=250,
        fp16=True,
        push_to_hub=True
)

# Initialize Trainer
trainer = SFTTrainer(
        model=model,
        train_dataset=data,
        peft_config=peft_config,
        dataset_text_field="text",  # This argument is causing the error
        args=training_arguments,
        tokenizer=tokenizer,
        packing=False,
        max_seq_length=512
)

trainer.train()

我已经尝试的方法 #

• 查阅SFTTrainer官方文档，确认dataset_text_field是否为合法参数
• 执行pip install -U trl命令更新trl库到最新版本
• 反复核对dataset_text_field在SFTTrainer中的使用逻辑是否正确

我的疑问 #

• dataset_text_field这个参数是不是已经被trl库弃用，或者现在SFTTrainer不再需要这个参数了？
• 如果确实不需要这个参数，那我该怎么修改现有代码，才能让SFTTrainer正常完成模型的微调？

备注：内容来源于stack exchange，提问作者User