如何在Keras+DistilBERT的model.fit中正确加入验证数据?
问题
我是NLP新手,正在使用DistilBERT在Kaggle的NLP入门数据集上进行二分类任务。其他流程均正常,但在model.fit中加入验证数据后,epoch输出里始终不显示val_accuracy和val_loss。以下是我的代码及当前输出,恳请各位提供解决思路。
代码
#import libraries import pandas as pd import numpy as np from sklearn.model_selection import train_test_split from transformers import DistilBertTokenizer, TFDistilBertForSequenceClassification import tensorflow as tf #import tokenizer and pre-trained model tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased', do_lower_case=True) model = TFDistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased') #random seed random_seed = 42 #set random seed in tensorflow tf.random.set_seed(random_seed) #set random seed in numpy np.random.seed(random_seed) #load the raw training data df_raw_train = pd.read_csv("data/train.csv") #make a copy of df_raw_train df_train = df_raw_train.copy(deep=True) #load the raw test data df_raw_test = pd.read_csv("data/test.csv") #make a copy of df_raw_test df_test = df_raw_test.copy(deep=True) #get target name target = 'target' #drop columns df_train.drop(['id','keyword','location'],axis=1,inplace=True) df_test.drop(['id','keyword','location'],axis=1,inplace=True) #training (80%) and validation (20%) data split df_train, df_val = train_test_split(df_train, train_size=0.8, random_state=random_seed) #reset index df_train, df_val = df_train.reset_index(drop=True), df_val.reset_index(drop=True) #batch tokenize our tweet field X_train = tokenizer.batch_encode_plus(df_train.text, pad_to_max_length=True, return_tensors="tf") X_val = tokenizer.batch_encode_plus(df_val.text, pad_to_max_length=True, return_tensors="tf") X_test = tokenizer.batch_encode_plus(df_test.text, pad_to_max_length=True, return_tensors="tf") #obtain target y_train = df_train['target'].to_numpy() y_val = df_val['target'].to_numpy() #optimize model optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0) loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True) bce = tf.keras.losses.BinaryCrossentropy() metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy') model.compile(optimizer=optimizer, loss=loss, metrics=[metric]) model.fit(x=X_train['input_ids'], y=y_train, epochs=2, batch_size=15, verbose=2, validation_data=(X_val, y_val))
当前输出
Epoch 1/2 343/508 [===================>..........] - ETA: 6:30 - loss: 0.4728 - accuracy: 0.7811
解决思路
- 输入格式不匹配:训练时你只传入了
X_train['input_ids'],但验证数据传入的是完整的X_val字典(包含input_ids和attention_mask)。模型训练和验证的输入结构必须一致,要么都传完整的tokenizer输出,要么都只传input_ids。修改model.fit的训练输入为完整的X_train即可:model.fit(x=X_train, y=y_train, epochs=2, batch_size=15, verbose=2, validation_data=(X_val, y_val)) - 验证指标显示时机:你设置的
verbose=2会在每个epoch结束后输出验证指标,当前输出是训练中的进度提示(ETA),等第一个epoch跑完后才会显示val_loss和val_accuracy,前提是输入格式已修正。 - 指标与损失匹配:你使用的
SparseCategoricalAccuracy和SparseCategoricalCrossentropy搭配是合理的(标签为0/1整数),这部分无需调整,确保验证标签y_val和训练标签格式一致即可。
内容的提问来源于stack exchange,提问作者Paul
相关产品推荐
相关产品推荐

