R语言基于TensorFlow/Keras的BERT输入适配问题求助

我在R语言中运行BERT模型，之前用Keras完成过word2vec等NLP任务，环境配置没问题。参考教程写代码后，卡在输入tokens的适配环节——试了张量、各类数组等多种转换方式，还是搞不清模型预期的输入数据结构、类型和形状。以下是可复现代码和报错信息，求解决。

可复现代码 #

#rm(list=ls())
packages <- c("reticulate", "keras", "tensorflow", "tfdatasets", "tidyverse", "data.table")
for (p in packages) if (!(p %in% installed.packages()[,1])) install.packages(p, character.only = TRUE) else require(p, character.only = TRUE)
rm(packages, p)

#reticulate::install_miniconda(force = TRUE) # 仅需执行一次
reticulate::use_condaenv("~/.local/share/r-miniconda") # Windows系统可改为 reticulate::use_condaenv("r-miniconda")

Sys.setenv(TF_KERAS=1) 
tensorflow::tf_version() # 若返回NULL则执行 install_tensorflow()
reticulate::py_config()

#reticulate::py_install('transformers', pip = TRUE)
#reticulate::py_install('torch', pip = TRUE)
transformer = reticulate::import('transformers')
tf = reticulate::import('tensorflow')
builtins <- import_builtins() # 导入Python内置方法

set.tf.repos <- "distilbert-base-german-cased"

tokenizer <- transformer$AutoTokenizer$from_pretrained(set.tf.repos)  
tokenizer_vocab_size <- length(tokenizer$vocab)

###### 加载模型
model_tf = transformer$TFDistilBertModel$from_pretrained(set.tf.repos, from_pt = T, trainable = FALSE)
model_tf$config

# 设置配置
model_tf$config$output_hidden_states = TRUE
summary(model_tf)

###### 数据与Token处理 #####
data <- data.table::fread("https://raw.githubusercontent.com/michael-eble/nlp-dataset-health-german-language/master/nlp-health-data-set-german-language.txt", encoding = "Latin-1")
txt <- data$V1
y <- data$V2
table(y, exclude = NULL)

set.max_length = 100
tokens <- tokenizer(
  txt,
  max_length = set.max_length %>% as.integer(),
  padding = 'max_length', # 'longest' 可实现动态padding
  truncation = TRUE,
  return_attention_mask = TRUE,
  return_token_type_ids = FALSE
)
#tokens[["input_ids"]] %>% str()
#tokens[["attention_mask"]] %>% str()

tokens <- list(tokens[["input_ids"]], tokens[["attention_mask"]])
str(tokens)



####### 构建自定义模型 ########
input_word_ids <- layer_input(shape = c(set.max_length), dtype = 'int32', name = "input_word_ids")
input_mask <- layer_input(shape = c(set.max_length), dtype = 'int32', name = "input_attention_mask")
#input_segment_ids <- layer_input(shape = c(max_len), dtype = 'int32', name="input_segment_ids")

last_hidden_state <- model_tf(input_word_ids, attention_mask = input_mask)[[1]]
cls_token <- last_hidden_state[, 1,]

output <- cls_token %>%
  layer_dense(units = 32, input_shape = c(set.max_length, 768), activation = 'relu') %>%
  layer_dense(units = 1, activation = 'sigmoid')

model <- keras_model(inputs = list(input_word_ids, input_mask), outputs = output)

model %>% compile(optimizer = "adam",
                  loss = "binary_crossentropy"
)

history = model %>%
  keras::fit(
    x = list(input_word_ids = tokens$input_ids, input_mask = tokens$attention_mask),
    y = y,
    epochs = 2,
    batch_size = 256,
    #metrics = "accuracy",
    validation_split = .2
  )

报错信息 #

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  ValueError: Failed to find data adapter that can handle input: (<class 'dict'> containing {"<class 'str'>"} keys and {"<class 'NoneType'>"} values), <class 'numpy.ndarray'>

Detailed traceback:
  File "/home/sz/.local/share/r-miniconda/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/sz/.local/share/r-miniconda/lib/python3.9/site-packages/keras/engine/data_adapter.py", line 984, in select_data_adapter
    raise ValueError(

核心问题 #

你在处理tokens和标签y时存在两个关键错误：

1. 将tokenizer输出的字典转成了无命名列表，后续在fit中尝试用命名键访问不存在的字段，导致传入NULL值，触发数据适配器错误
2. 原标签y是字符型，不符合二元交叉熵损失对数值型输入的要求

修复步骤 #

1. 修正Token数据结构：
   保留tokenizer输出的命名结构，或手动构建与模型输入层名称匹配的命名列表，替换原Token处理的最后一行：

   # 替换原错误代码：tokens <- list(tokens[["input_ids"]], tokens[["attention_mask"]])
   tokens <- list(input_word_ids = tokens$input_ids, input_mask = tokens$attention_mask)
   

2. 转换标签类型：
   将字符型标签转为数值型，适配二分类损失函数：

   y <- as.numeric(data$V2)
   

3. 优化fit函数调用：
   直接传入命名列表即可，无需重复指定键名，同时建议添加准确率指标：

   history = model %>%
     keras::fit(
       x = tokens,
       y = y,
       epochs = 2,
       batch_size = 256,
       metrics = "accuracy",
       validation_split = .2
     )
   

验证输入（可选） #

修复后可打印数据结构确认：

str(tokens$input_word_ids) # 应为 (样本数, 100) 的整数数组
str(tokens$input_mask)     # 形状与input_ids一致
str(y)                     # 应为长度等于样本数的数值向量

内容的提问来源于stack exchange，提问作者statistikr