使用Huggingface VisionEncoderDecoderModel时遇json2token属性缺失错误求助

问题场景 #

我在Google Colab中尝试微调HuggingFace的Donut文档理解Transformer模型，但创建DonutDataset对象时受阻，原始代码如下：

!pip install transformers datasets sentencepiece donut-python

from google.colab import drive
from donut.util import DonutDataset
from transformers import DonutProcessor, VisionEncoderDecoderModel, VisionEncoderDecoderConfig

drive.mount('/content/drive/')
projectdir = 'drive/MyDrive/donut'


donut_version = 'naver-clova-ix/donut-base-finetuned-cord-v2'  # 'naver-clova-ix/donut-base'
config = VisionEncoderDecoderConfig.from_pretrained(donut_version)
config.decoder.max_length = 768

processor = DonutProcessor.from_pretrained(donut_version)
model = VisionEncoderDecoderModel.from_pretrained(donut_version, config=config)


train_dataset = DonutDataset(f'{projectdir}/input_doc_images', 
                             model,
                             #'naver-clova-ix/donut-base-finetuned-cord-v2',
                             max_length=config.decoder.max_length,
                             split="train", 
                             task_start_token="", 
                             prompt_end_token="",
                             sort_json_key=True,
                             )

报错信息 #

执行上述代码最后一行时抛出以下错误：

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-9d831be996e6> in <cell line: 4>()
      2 
      3 max_length = 768
----> 4 train_dataset = DonutDataset(f'{projectdir}/input_doc_images', 
      5                              model,
      6                              #'naver-clova-ix/donut-base-finetuned-cord-v2',

2 frames
/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
   1612             if name in modules:
   1613                 return modules[name]
-> 1614         raise AttributeError("'{}' object has no attribute '{}'".format(
   1615             type(self).__name__, name))
   1616 

AttributeError: 'VisionEncoderDecoderModel' object has no attribute 'json2token'

问题原因 #

你混淆了两类模型类：

• Donut官方仓库的model.py中定义的DonutModel类确实包含json2token方法，但你用的是HuggingFace Transformers库提供的通用VisionEncoderDecoderModel类加载模型，这个类并没有实现json2token方法，因此触发AttributeError。

解决方案 #

修改模型加载逻辑，使用Donut库自身的DonutModel类加载预训练模型，或者直接给DonutDataset传入预训练模型的名称（而非VisionEncoderDecoderModel实例）。

修改后的代码示例 #

方式一：使用DonutModel加载模型

!pip install transformers datasets sentencepiece donut-python

from google.colab import drive
from donut.util import DonutDataset
from donut.model import DonutModel  # 替换为Donut库的模型类
from transformers import DonutProcessor, VisionEncoderDecoderConfig

drive.mount('/content/drive/')
projectdir = 'drive/MyDrive/donut'


donut_version = 'naver-clova-ix/donut-base-finetuned-cord-v2'
config = VisionEncoderDecoderConfig.from_pretrained(donut_version)
config.decoder.max_length = 768

processor = DonutProcessor.from_pretrained(donut_version)
# 使用DonutModel加载预训练模型
model = DonutModel.from_pretrained(donut_version, config=config)

# 创建Dataset时传入DonutModel实例
train_dataset = DonutDataset(f'{projectdir}/input_doc_images', 
                             model,
                             max_length=config.decoder.max_length,
                             split="train", 
                             task_start_token="", 
                             prompt_end_token="",
                             sort_json_key=True,
                             )

方式二：直接传入模型名称（无需提前加载模型）

train_dataset = DonutDataset(f'{projectdir}/input_doc_images', 
                             'naver-clova-ix/donut-base-finetuned-cord-v2',
                             max_length=768,
                             split="train", 
                             task_start_token="", 
                             prompt_end_token="",
                             sort_json_key=True,
                             )

内容的提问来源于stack exchange，提问作者Max Power