使用aitextgen调用ai.train时触发AttributeError问题求助

问题描述 #

在Google Colab中使用aitextgen构建文本生成模型时，调用ai.train()方法触发AttributeError: can't set attribute错误，已尝试更换torch、pytorch-lightning及aitextgen版本，问题仍未解决。

调用代码 #

ai.train(file_name,
     line_by_line = False,
     from_cache = False,
     num_steps = 3000,
     generate_every = 1000,
     save_every = 1000,
     save_gdrive = False,
     learning_rate = 1e-3,
     fp16 = True, 
     batch_size = 1,
     )

错误堆栈信息 #

AttributeError                            Traceback (most recent call last)
<ipython-input-29-e611ad4fbd0c> in <module>
----> 1 ai.train(file_name,
      2          line_by_line = False,
      3          from_cache = False,
      4          num_steps = 3000,
      5          generate_every = 1000,

2 frames
/usr/local/lib/python3.8/dist-packages/aitextgen/aitextgen.py in train(self, train_data, output_dir, fp16, fp16_opt_level, n_gpu, tpu_cores, max_grad_norm, gradient_accumulation_steps, seed, learning_rate, weight_decay, adam_epsilon, warmup_steps, num_steps, save_every, generate_every, n_generate, loggers, batch_size, num_workers, benchmark, avg_loss_smoothing, save_gdrive, run_id, progress_bar_refresh_rate, freeze_layers, num_layers_freeze, use_deepspeed, **kwargs)
    673 
    674         # Wrap the model in a pytorch-lightning module
---> 675         train_model = ATGTransformer(self.model, train_data, hparams, self.tokenizer)
    676 
    677         Begin training

/usr/local/lib/python3.8/dist-packages/aitextgen/train.py in __init__(self, model, dataset, hparams, tokenizer)
     19     def __init__(self, model, dataset, hparams, tokenizer):
     20         super(ATGTransformer, self).__init__()
---> 21         self.model, self.dataset, self.hparams, self.tokenizer = (
     22             model,
     23             dataset,

/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py in __setattr__(self, name, value)
   1311                     buffers[name] = value
   1312                 else:
-> 1313                     super().__setattr__(name, value)
   1314 
   1315     def __delattr__(self, name):

AttributeError: can't set attribute

修复建议 #

• 修改aitextgen源码解决属性赋值冲突
  错误原因是ATGTransformer继承自PyTorch Lightning的LightningModule，hparams是该父类的受保护属性，直接批量赋值会触发错误。需修改train.py的初始化逻辑：
  在Colab中执行以下命令直接修改安装包内的文件：

  !sed -i '21,25d' /usr/local/lib/python3.8/dist-packages/aitextgen/train.py
  !sed -i '21i\        self.model = model' /usr/local/lib/python3.8/dist-packages/aitextgen/train.py
  !sed -i '22i\        self.dataset = dataset' /usr/local/lib/python3.8/dist-packages/aitextgen/train.py
  !sed -i '23i\        self.save_hyperparameters(hparams)' /usr/local/lib/python3.8/dist-packages/aitextgen/train.py
  !sed -i '24i\        self.tokenizer = tokenizer' /usr/local/lib/python3.8/dist-packages/aitextgen/train.py
  

  这段代码会替换原有的批量赋值代码，改用Lightning官方的save_hyperparameters方法处理参数，避免属性冲突。

• 安装兼容版本组合
  若不想修改源码，可强制安装经过验证的兼容版本：

  !pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
  !pip install pytorch-lightning==1.7.7
  !pip install aitextgen==0.6.0
  

• 临时规避方案
  将train()调用中的fp16=True改为fp16=False，禁用混合精度训练，可暂时绕开触发错误的代码路径。

内容的提问来源于stack exchange，提问作者Ahmad Jibril Hemdi