HF首次使用model.generate无报错中断脚本执行求助

我是Hugging Face新手，第一次尝试使用模型。调用model.generate后脚本没有报错就直接中断了，只输出了"before"，换其他模型也有同样问题。

我的代码 #

from transformers import RobertaTokenizer, T5ForConditionalGeneration

tokenizer = RobertaTokenizer.from_pretrained('Salesforce/codet5-base')
model = T5ForConditionalGeneration.from_pretrained('Salesforce/codet5-base')

text = "write for cycle"
input_ids = tokenizer(text, return_tensors="pt").input_ids

print("before")
generated_ids = model.generate(input_ids, max_length=8)
print("after")

print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

环境信息 #

• transformers版本: 4.30.0.dev0
• 系统平台: macOS-10.15.7-x86_64-i386-64bit
• Python版本: 3.9.16
• Huggingface_hub版本: 0.14.1
• Safetensors版本: 0.3.1
• PyTorch版本（是否用GPU）: 2.0.1 (False)
• Tensorflow版本（是否用GPU）: 未安装 (NA)
• Flax版本（CPU/GPU/TPU）: 未安装 (NA)
• Jax版本: 未安装
• JaxLib版本: 未安装
• 脚本中是否使用GPU？: 否
• 脚本中是否使用分布式/并行设置？: 否

日志信息 #

loading file vocab.json from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/vocab.json
loading file merges.txt from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/merges.txt
loading file added_tokens.json from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/added_tokens.json
loading file special_tokens_map.json from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/special_tokens_map.json
loading file tokenizer_config.json from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/tokenizer_config.json
loading configuration file config.json from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/config.json
Model config T5Config {
  "_name_or_path": "/content/drive/MyDrive/CodeT5/pretrained_models/codet5_base",
  "architectures": [
    "T5ForConditionalGeneration"
  ],
  "bos_token_id": 1,
  "d_ff": 3072,
  "d_kv": 64,
  "d_model": 768,
  "decoder_start_token_id": 0,
  "dense_act_fn": "relu",
  "dropout_rate": 0.1,
  "eos_token_id": 2,
  "feed_forward_proj": "relu",
  "gradient_checkpointing": false,
  "id2label": {
    "0": "LABEL_0"
  },
  "initializer_factor": 1.0,
  "is_encoder_decoder": true,
  "is_gated_act": false,
  "label2id": {
    "LABEL_0": 0
  },
  "layer_norm_epsilon": 1e-06,
  "model_type": "t5",
  "n_positions": 512,
  "num_decoder_layers": 12,
  "num_heads": 12,
  "num_layers": 12,
  "output_past": true,
  "pad_token_id": 0,
  "relative_attention_max_distance": 128,
  "relative_attention_num_buckets": 32,
  "task_specific_params": {
    "summarization": {
      "early_stopping": true,
      "length_penalty": 2.0,
      "max_length": 200,
      "min_length": 30,
      "no_repeat_ngram_size": 3,
      "num_beams": 4,
      "prefix": "summarize: "
    },
    "translation_en_to_de": {
      "early_stopping": true,
      "max_length": 300,
      "num_beams": 4,
      "prefix": "translate English to German: "
    },
    "translation_en_to_fr": {
      "early_stopping": true,
      "max_length": 300,
      "num_beams": 4,
      "prefix": "translate English to French: "
    },
    "translation_en_to_ro": {
      "early_stopping": true,
      "max_length": 300,
      "num_beams": 4,
      "prefix": "translate English to Romanian: "
    }
  },
  "torch_dtype": "float32",
  "transformers_version": "4.30.0.dev0",
  "use_cache": true,
  "vocab_size": 32100
}

loading weights file pytorch_model.bin from cache at /Users/zonder/.cache/huggingface/hub/models--Salesforce--codet5-base/snapshots/4078456db09ba972a3532827a0b5df4da172323c/pytorch_model.bin
Generate config GenerationConfig {
  "_from_model_config": true,
  "bos_token_id": 1,
  "decoder_start_token_id": 0,
  "eos_token_id": 2,
  "pad_token_id": 0,
  "transformers_version": "4.30.0.dev0"
}

All model checkpoint weights were used when initializing T5ForConditionalGeneration.

All the weights of T5ForConditionalGeneration were initialized from the model checkpoint at Salesforce/codet5-base.
If your task is similar to the task the model of the checkpoint was trained on, you can already use T5ForConditionalGeneration for predictions without further training.
Generation config file not found, using a generation config created from the model config.
before
Generate config GenerationConfig {
  "_from_model_config": true,
  "bos_token_id": 1,
  "decoder_start_token_id": 0,
  "eos_token_id": 2,
  "pad_token_id": 0,
  "transformers_version": "4.30.0.dev0"
}

解决方案 #

1. 排查内存不足问题 #

老版本macOS+纯CPU推理，CodeT5-base这类参数规模的模型容易耗尽系统内存，导致进程被系统强制终止。可以试试：

• 改用更小的模型，比如Salesforce/codet5-small
• 启用梯度检查点降低内存占用：

  model = T5ForConditionalGeneration.from_pretrained('Salesforce/codet5-base', gradient_checkpointing=True)
  

• 关闭后台无关程序，释放更多内存资源

2. 降级transformers到稳定版 #

你当前使用的是开发预览版4.30.0.dev0，可能存在未修复的bug。建议降级到稳定版本，比如4.29.2：

pip install transformers==4.29.2

3. 明确指定生成参数 #

给model.generate补充明确的参数，避免潜在的配置冲突：

generated_ids = model.generate(
    input_ids,
    max_length=8,
    early_stopping=True,
    num_beams=1,  # 贪心解码，减少内存消耗
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id
)

4. 开启内存调试确认问题 #

添加以下代码查看内存使用情况，验证是否是内存导致的进程中断：

import torch
print(f"初始内存占用: {torch.cuda.memory_allocated()/1024**2 if torch.cuda.is_available() else 'N/A'} MB")
# 加载模型后打印
print(f"模型加载后内存占用: {torch.cuda.memory_allocated()/1024**2 if torch.cuda.is_available() else 'N/A'} MB")
# generate前打印
print("before")
print(f"Generate前内存占用: {torch.cuda.memory_allocated()/1024**2 if torch.cuda.is_available() else 'N/A'} MB")
generated_ids = model.generate(input_ids, max_length=8)

内容的提问来源于stack exchange，提问作者MegaBomber