CTCTrainer对象无use_amp/use_cuda_amp属性问题求助

尝试基于Hugging Face Trainer自定义CTCTrainer训练预训练模型时，先后用self.use_amp和self.use_cuda_amp都触发了属性不存在的错误：

• 初始报错：'CTCTrainer' object has no attribute 'use_amp'
• 修改为use_cuda_amp后报错：'CTCTrainer' object has no attribute 'use_cuda_amp'

问题原因 #

新版Hugging Face Transformers库中，Trainer的混合精度相关配置不再作为实例直接属性存在，而是统一通过self.args（即TrainingArguments实例）管理。use_amp/use_cuda_amp是旧版本Trainer的属性，现已被废弃。

修复方案 #

把代码中判断混合精度的逻辑改成读取self.args.fp16，同时保留对AMP可用性的校验，修改后的完整代码如下：

from typing import Any, Dict, Union

import torch
from packaging import version
from torch import nn

from transformers import (
    Trainer,
    is_apex_available,
)

if is_apex_available():
    from apex import amp

if version.parse(torch.__version__) >= version.parse("1.6"):
    _is_native_amp_available = True
    from torch.cuda.amp import autocast


class CTCTrainer(Trainer):
    def training_step(self, model: nn.Module, inputs: Dict[str, Union[torch.Tensor, Any]]) -> torch.Tensor:
        """
        Perform a training step on a batch of inputs.

        Subclass and override to inject custom behavior.

        Args:
            model (:obj:`nn.Module`):
                The model to train.
            inputs (:obj:`Dict[str, Union[torch.Tensor, Any]]`):
                The inputs and targets of the model.

                The dictionary will be unpacked before being fed to the model. Most models expect the targets under the
                argument :obj:`labels`. Check your model's documentation for all accepted arguments.

        Return:
            :obj:`torch.Tensor`: The tensor with training loss on this batch.
        """

        model.train()
        inputs = self._prepare_inputs(inputs)

        # 改用args.fp16判断是否启用混合精度，同时校验原生AMP可用性
        if self.args.fp16 and _is_native_amp_available:
            with autocast():
                loss = self.compute_loss(model, inputs)
        else:
            loss = self.compute_loss(model, inputs)

        if self.args.gradient_accumulation_steps > 1:
            loss = loss / self.args.gradient_accumulation_steps

        if self.args.fp16 and _is_native_amp_available:
            self.scaler.scale(loss).backward()
        elif self.args.fp16 and is_apex_available() and self.use_apex:
            with amp.scale_loss(loss, self.optimizer) as scaled_loss:
                scaled_loss.backward()
        elif self.deepspeed:
            self.deepspeed.backward(loss)
        else:
            loss.backward()

        return loss.detach()

额外说明 #

• self.args.fp16是TrainingArguments中控制混合精度的参数，初始化Trainer时通过fp16=True开启即可。
• 保留_is_native_amp_available判断，确保只有在PyTorch版本支持且开启fp16时才使用autocast。
• 对APEX的判断增加了双重校验，避免环境不支持时触发错误。

内容的提问来源于stack exchange，提问作者stanley101