Llama 3.2 Vision多模态结合Roboflow YOLO数据集训练时频繁终止/报错的问题求助

大家好，我目前在推进一个多模态项目：用从Roboflow下载的YOLO格式数据集，通过Hugging Face Transformers库给Llama 3.2 Vision做监督式微调，但遇到了棘手的问题：

• 训练启动后经常毫无征兆地自行终止，全程没有任何错误提示
• 换了几种微调方法都没解决，现在用的代码还会在训练过程中直接抛出错误

先说明下我的硬件配置：RTX 4090显卡、64GB内存、i9-14900K处理器，理论上资源应该足够支撑训练。

我的数据集是YOLOv8格式，包含图像文件和对应标注文件，标注内容涵盖目标类别与边界框信息。

下面是我当前使用的微调代码片段，麻烦大家帮忙看看哪里可能出问题了：

def train_llama(images, descriptions):
    print("Initializing model and processor...")
    model_id = "meta-llama/Llama-Guard-3-11B-Vision"
    
    # Initialize processor
    processor = AutoProcessor.from_pretrained(model_id)
    
    # Initialize model
    model = AutoModelForImageTextToText.from_pretrained(
        model_id,
        torch_dtype=torch.float16,
        device_map="cuda"
    )
    model.config.use_cache = False

    print("Creating dataset...")
    dataset = CustomImageTextDataset(images, descriptions, processor)

    # Split dataset
    train_size = int(0.9 * len(dataset))
    val_size = len(dataset) - train_size
    train_dataset, val_dataset = torch.utils.data.random_split(dataset, [train_size, val_size])
    print(f"Train size: {train_size}, Validation size: {val_size}")

    # Training arguments
    training_args = TrainingArguments(
        output_dir="./llama_finetuned",
        learning_rate=1e-5,
        num_train_epochs=3,
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_ratio=0.05,
        logging_steps=10,
        eval_strategy="steps",
        eval_steps=50,
        save_strategy="steps",
        save_steps=100,
        fp16=True,
        gradient_checkpointing=True,
        remove_unused_columns=False,
        report_to="tensorboard",
        load_best_model_at_end=True,
        metric_for_best_model="loss",
        dataloader_num_workers=0
    )

    # Initialize trainer
    trainer = CustomTrainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=val_dataset
    )

    print("Starting Llama training...")
    try:
        trainer.train()
        print("Training completed successfully!")
        
        print("Saving model...")
        trainer.save_model("./final_llama_model")
        print("Model saved successfully!")
        return True
        
    except Exception as e:
        print(f"Error during training: {str(e)}")
        print(f"Traceback: {traceback.format_exc()}")
        return False

备注：内容来源于stack exchange，提问作者Rauhan Siddiqui