使用LoRA恢复Phi-3微调时遇DeepSpeed优化器索引错误求助

背景 #

我已通过以下代码完成Phi-3模型的1轮epoch LoRA微调：

self.model=AutoModelForCausalLM.from_pretrained(self.model_args.model_name_or_path,**kwarg)
self.model = get_peft_model(self.model, self.lora_config)

现在尝试加载已有的LoRA权重恢复微调，代码如下：

self.model = PeftModel.from_pretrained(self.model , adapter)    
self.model.is_trainable = True    

错误信息 #

运行后触发关键错误：

optimizer = DeepSpeedZeroOptimizer_Stage3(
  File "/PATH/miniconda3/envs/llms/lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 149, in __init__
        self.dtype = self.optimizer.param_groups[0]['params'][0].dtype

完整错误栈：

Traceback (most recent call last):
  File "/gpfs/u/scratch/TMPR/TMPRmhbt/research/FoodKG/models/Phi3/train.py", line 20, in <module>
    train()
  File "/gpfs/u/scratch/TMPR/TMPRmhbt/research/FoodKG/models/Phi3/train.py", line 14, in train
    custom_trainer.train()
  File "/gpfs/u/scratch/TMPR/TMPRmhbt/research/FoodKG/models/Phi3/core/model.py", line 273, in train
    trainer.train(resume_from_checkpoint=True)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/transformers/trainer.py", line 1539, in train
    return inner_training_loop(
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/transformers/trainer.py", line 1687, in _inner_training_loop
    model, self.optimizer = self.accelerator.prepare(self.model, self.optimizer)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/accelerate/accelerator.py", line 1220, in prepare
    result = self._prepare_deepspeed(*args)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/accelerate/accelerator.py", line 1605, in _prepare_deepspeed
    engine, optimizer, _, lr_scheduler = deepspeed.initialize(**kwargs)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/__init__.py", line 181, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 307, in __init__
    self._configure_optimizer(optimizer, model_parameters)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1258, in _configure_optimizer
    self.optimizer = self._configure_zero_optimizer(basic_optimizer)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1582, in _configure_zero_optimizer
    optimizer = DeepSpeedZeroOptimizer_Stage3(
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/zero/stage3.py", line 149, in __init__
    self.dtype = self.optimizer.param_groups[0]['params'][0].dtype
IndexError: list index out of range
Traceback (most recent call last):
  File "/gpfs/u/scratch/TMPR/TMPRmhbt/research/FoodKG/models/Phi3/train.py", line 20, in <module>
    train()
  File "/gpfs/u/scratch/TMPR/TMPRmhbt/research/FoodKG/models/Phi3/train.py", line 14, in train
    custom_trainer.train()
  File "/gpfs/u/scratch/TMPR/TMPRmhbt/research/FoodKG/models/Phi3/core/model.py", line 273, in train
    trainer.train(resume_from_checkpoint=True)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/transformers/trainer.py", line 1539, in train
    return inner_training_loop(
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/transformers/trainer.py", line 1687, in _inner_training_loop
    model, self.optimizer = self.accelerator.prepare(self.model, self.optimizer)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/accelerate/accelerator.py", line 1220, in prepare
    result = self._prepare_deepspeed(*args)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/accelerate/accelerator.py", line 1605, in _prepare_deepspeed
    engine, optimizer, _, lr_scheduler = deepspeed.initialize(**kwargs)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/__init__.py", line 181, in initialize
    engine = DeepSpeedEngine(args=args,
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 307, in __init__
    self._configure_optimizer(optimizer, model_parameters)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1258, in _configure_optimizer
    self.optimizer = self._configure_zero_optimizer(basic_optimizer)
  File "/gpfs/u/home/TMPR/TMPRmhbt/scratch/miniconda3/envs/flax2/lib/python3.10/site-packages/deepspeed/runtime/engine.py", line 1582, in _configure_zero_optimizer 

核心现象 #

从头启动LoRA微调可正常运行，但加载已有LoRA权重恢复微调时，会触发上述优化器索引越界错误。

内容的提问来源于stack exchange，提问作者Mohbat Tharani