使用Accelerator与Trainer时遇AcceleratorState无distributed_type属性错误

尝试结合Accelerator与Trainer进行模型训练，代码如下：

tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)

config = AutoConfig.from_pretrained(model_args.model_name_or_path)
model = AutoModelForSeq2SeqLM.from_pretrained(
    model_args.model_name_or_path, config=config)
collator = DataCollatorForSeq2Seq(tokenizer, model=model)

train_set = CorefDataset(tokenizer, data_args, training_args, 'train')
tb_callback = TensorBoardCallback()

accelerator = Accelerator()
trainer = accelerator.prepare(CorefTrainer(
    tokenizer=tokenizer,
    model=model,
    args=training_args,
    train_dataset=train_set,
    #        eval_dataset=dev_set,
    data_collator=collator,
    callbacks=[tb_callback]
))
trainer.train()

在Google Colab中通过以下命令启动：

!accelerate launch --config_file /root/.cache/huggingface/accelerate/default_config.yaml Seq2seqCoref/main.py

运行后报错：

Traceback (most recent call last):
  File "/content/Seq2seqCoref/main.py", line 41, in <module>
    trainer = accelerator.prepare(CorefTrainer(
  File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 1248, in prepare
    if self.distributed_type == DistributedType.DEEPSPEED:
  File "/usr/local/lib/python3.10/dist-packages/accelerator/accelerator.py", line 529, in distributed_type
    return self.state.distributed_type
  File "/usr/local/lib/python3.10/dist-packages/accelerate/state.py", line 1076, in __getattr__
    raise AttributeError(
AttributeError: `AcceleratorState` object has no attribute `distributed_type`. This happens if `AcceleratorState._reset_state()` was called and an `Accelerator` or `PartialState` was not reinitialized.

当前环境：transformers 4.40.2，accelerate 0.30.0；直接在Colab中运行代码而非通过main.py，同样报错。

这个错误的核心是手动初始化Accelerator与Trainer内置的Accelerate支持冲突——Hugging Face Trainer本身已经集成了Accelerate的分布式能力，不需要手动创建Accelerator实例再包装Trainer。

修改步骤 #

1. 删除手动初始化的accelerator = Accelerator()和accelerator.prepare(...)代码
2. 直接实例化CorefTrainer并调用训练方法

修改后的代码：

tokenizer = AutoTokenizer.from_pretrained(model_args.model_name_or_path)

config = AutoConfig.from_pretrained(model_args.model_name_or_path)
model = AutoModelForSeq2SeqLM.from_pretrained(
    model_args.model_name_or_path, config=config)
collator = DataCollatorForSeq2Seq(tokenizer, model=model)

train_set = CorefDataset(tokenizer, data_args, training_args, 'train')
tb_callback = TensorBoardCallback()

# 直接初始化Trainer，无需Accelerator包装
trainer = CorefTrainer(
    tokenizer=tokenizer,
    model=model,
    args=training_args,
    train_dataset=train_set,
    # eval_dataset=dev_set,
    data_collator=collator,
    callbacks=[tb_callback]
)
trainer.train()

启动方式调整 #

• 若使用accelerate launch启动，确保脚本中没有手动初始化Accelerator，直接运行命令即可：

!accelerate launch --config_file /root/.cache/huggingface/accelerate/default_config.yaml Seq2seqCoref/main.py

• 也可直接运行脚本，Trainer会自动适配环境：

!python Seq2seqCoref/main.py

额外排查点 #

• 检查是否有其他代码调用了AcceleratorState._reset_state()，若有，需在调用后重新初始化Accelerator（但本次场景更可能是Trainer与手动Accelerator的冲突）
• 可尝试将accelerate更新到与transformers 4.40.2兼容的版本（0.29.0+均可，0.30.0本身无版本兼容问题）

内容的提问来源于stack exchange，提问作者Evelin Amorim