使用Transformers的Trainer时Accelerate包抛出TypeError参数错误

跟着Hugging Face的NLP教程使用Trainer训练模型时，初始化Trainer步骤报错。

代码示例 #

from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForSequenceClassification, DataCollatorWithPadding, TrainingArguments, Trainer

checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
raw_datasets = load_dataset("glue", "mrpc")

def tokenize_function(example):
    return tokenizer(example["sentence1"], example["sentence2"], truncation=True)

tokenized_datasets = raw_datasets.map(tokenize_function, batched=True)

data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

training_args = TrainingArguments("test-trainer")

model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)

# 以上代码正常运行
# 下面这行报错
trainer = Trainer(
    model,
    training_args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["validation"], 
    tokenizer=tokenizer,
)

报错信息 #

File "tutorial.py", line 21, in <module>
    trainer = Trainer(
              ^^^^^^^^
  File "/opt/miniconda3/envs/py3env/lib/python3.12/site-packages/transformers/trainer.py", line 388, in __init__
    self.create_accelerator_and_postprocess()
  File "/opt/miniconda3/envs/py3env/lib/python3.12/site-packages/transformers/trainer.py", line 4364, in create_accelerator_and_postprocess
    self.accelerator = Accelerator(**args)
                       ^^^^^^^^^^^^^^^^^^^
TypeError: Accelerator.__init__() got an unexpected keyword argument 'use_seedable_sampler'

环境版本 #

Python: 3.12.3
Transformers: 4.40.2
Datasets: 2.19.1
Accelerate: 0.21.0

这个错误是因为Transformers 4.40.2版本需要更高版本的Accelerate库，use_seedable_sampler参数是在Accelerate 0.22.0及以上版本中新增的，当前Accelerate版本（0.21.0）不支持该参数。

执行以下命令升级Accelerate即可解决：

pip install --upgrade accelerate

如果需要指定兼容版本，可安装0.22.0及以上版本：

pip install accelerate>=0.22.0

内容的提问来源于stack exchange，提问作者raka