Colab加载DistilBERT模型报错：KerasTensor不被input_ids接受

报错根源 #

报错核心是输入参数传递方式错误：你将input_ids和attention_mask打包成列表传给了TFDistilBertModel，导致模型误把两个KerasTensor都当成了input_ids的输入，触发类型不匹配校验。TF版DistilBERT需要分别接收input_ids和attention_mask参数，而非将两者合并为一个列表传入。

另外，Colab与本地环境的transformers/tensorflow版本差异也可能是诱因——Colab默认库版本可能和你本地不一致，新版本对KerasTensor的兼容性逻辑有调整。

修复步骤 #

1. 修正模型调用方式 #

将原代码中调用DistilBERT的行：

last_hidden_state = transformer([input_ids_layer, input_attention_layer])[0]

改为关键字参数传递（最稳妥，避免顺序混淆）：

last_hidden_state = transformer(input_ids=input_ids_layer, attention_mask=input_attention_layer)[0]

2. 对齐Colab与本地库版本 #

在Colab开头执行以下命令，安装和本地一致的transformers与tensorflow版本（替换为你本地的版本号）：

!pip install transformers==4.30.0 tensorflow==2.12.0

完整修复后代码 #

MAX_LENGTH = 256
LAYER_DROPOUT = 0.5
LEARNING_RATE = 1e-4
RANDOM_STATE = 42

def build_model(transformer, max_length=MAX_LENGTH):
    # 定义权重初始化器，设置随机种子保证可复现
    weight_initializer = tf.keras.initializers.GlorotNormal(seed=RANDOM_STATE) 

    # 定义输入层
    input_ids_layer = tf.keras.layers.Input(shape=(max_length,), 
                                        name='input_ids', 
                                        dtype='int32')
    input_attention_layer = tf.keras.layers.Input(shape=(max_length,),
                                              name='input_attention', 
                                              dtype='int32')

    # DistilBERT输出元组，索引0的元素是模型最后一层的隐藏状态
    # 形状为 (batch_size, sequence_length, hidden_size=768)
    last_hidden_state = transformer(input_ids=input_ids_layer, attention_mask=input_attention_layer)[0]

    # 只取[CLS] token的输出，位于每个编码序列的索引0位置
    # 切片后得到二维数据
    cls_token = last_hidden_state[:, 0, :]

    dense_layer1 = tf.keras.layers.Dense(256, 
                               activation='relu',
                               kernel_initializer=weight_initializer,  
                               kernel_constraint=None,
                               bias_initializer='zeros'
                               )(cls_token)

    dropout_layer1=tf.keras.layers.Dropout(LAYER_DROPOUT)(dense_layer1)
    dense_layer2 = tf.keras.layers.Dense(256, 
                               activation='relu',
                               kernel_initializer=weight_initializer,  
                               kernel_constraint=None,
                               bias_initializer='zeros'
                               )(dropout_layer1)

    dropout_layer2=tf.keras.layers.Dropout(LAYER_DROPOUT)(dense_layer2)
    dense_layer3 = tf.keras.layers.Dense(32, 
                               activation='relu',
                               kernel_initializer=weight_initializer,  
                               kernel_constraint=None,
                               bias_initializer='zeros'
                               )(dropout_layer2)

    dropout_layer3=tf.keras.layers.Dropout(LAYER_DROPOUT)(dense_layer3)

    # 定义输出层（二分类任务，单节点）
    output = tf.keras.layers.Dense(1, 
                               activation='sigmoid',
                               kernel_initializer=weight_initializer,  
                               kernel_constraint=None,
                               bias_initializer='zeros'
                               )(dropout_layer3)

    # 定义模型
    model = tf.keras.Model([input_ids_layer, input_attention_layer], output)

    # 编译模型
    model.compile(tf.keras.optimizers.Adam(learning_rate=LEARNING_RATE), 
              loss=tf.keras.losses.BinaryFocalCrossentropy(),
              metrics=['accuracy'])
    return model 

model=build_model(distilBERT)

内容的提问来源于stack exchange，提问作者Tony