基于微调BERT构建Sentence Transformer时遇错的解决咨询

背景与初始代码 #

已在特定领域数据上完成BERT微调，计划基于该模型及自有标注数据训练Sentence Transformer，使用以下代码构建模型：

model_name = "path/to/model" 
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased") # 微调BERT时用的就是这个tokenizer
model = AutoModel.from_pretrained(model_name)

word_embedding_model = models.Transformer(model, tokenizer)
pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension())
sentence_transformer = SentenceTransformer(modules=[word_embedding_model, pooling_model])

初始错误 #

运行上述代码后出现以下错误：

We couldn't connect to 'https://huggingface.co' to load this model, couldn't find it in the cached files and it looks like BertModel(...)  is not the path to a directory containing a {configuration_file} file.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

保存模型后的新错误 #

使用以下代码将模型保存至谷歌云盘：

model.save_pretrained('/content/drive/MyDrive/testForSentTransformer')

之后又出现两个错误：

错误1 #

HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'BertModel(
  (embeddings): BertEmbeddings(
    (word_embeddings): Embedding(30522, 768, padding_idx=0)
    (position_embeddings): Embedding(512, 768)
    (token_type_embeddings): Embedding(2, 768)
    (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )

错误2 #

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/local/lib/python3.9/dist-packages/transformers/configuration_utils.py in _get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
    647             except Exception:
    648                 # For any other exception, we throw a generic error.
--> 649                 raise EnvironmentError(
    650                     f"Can't load the configuration of '{pretrained_model_name_or_path}'. If you were trying to load it"
    651                     " from 'https://huggingface.co/models', make sure you don't have a local directory with the same"

OSError: Can't load the configuration of 'BertModel(
  (embeddings): BertEmbeddings(
    (word_embeddings): Embedding(30522, 768, padding_idx=0)
    (position_embeddings): Embedding(512, 768)
    (token_type_embeddings): Embedding(2, 768)
    (LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
    (dropout): Dropout(p=0.1, inplace=False)
  )

解决方案 #

错误根源 #

models.Transformer的第一个参数需要传入模型路径/模型名称字符串，而不是直接传入加载好的AutoModel对象。你之前直接传模型实例，导致它把模型的字符串表示（BertModel(...)）当成了Hugging Face的仓库ID，从而触发格式验证错误和加载失败。

修复步骤 #

1. 完整保存微调后的模型与Tokenizer
   除了模型，把Tokenizer也一起保存到同一个目录（确保后续加载时能自动匹配）：

   # 保存微调后的模型和tokenizer到谷歌云盘路径
   model.save_pretrained('/content/drive/MyDrive/testForSentTransformer')
   tokenizer.save_pretrained('/content/drive/MyDrive/testForSentTransformer')
   

2. 正确构建Sentence Transformer
   直接给models.Transformer传入保存好的模型路径，内部会自动加载模型和对应的Tokenizer：

   from sentence_transformers import SentenceTransformer, models
   
   # 模型保存的完整路径
   saved_model_path = '/content/drive/MyDrive/testForSentTransformer'
   
   # 用路径初始化Transformer模块
   word_embedding_model = models.Transformer(saved_model_path)
   # 初始化池化层
   pooling_model = models.Pooling(word_embedding_model.get_word_embedding_dimension())
   # 构建最终的Sentence Transformer
   sentence_transformer = SentenceTransformer(modules=[word_embedding_model, pooling_model])
   

3. 离线环境检查
   如果是离线运行，确认保存的目录下包含这些必要文件：config.json、pytorch_model.bin（或model.safetensors）、tokenizer_config.json、vocab.txt，这些都是save_pretrained自动生成的，只要保存过程没出错就会存在。

内容的提问来源于stack exchange，提问作者Ling