R语言神经网络tfdatasets::feature_spec()处理3D数组输入报错问询

在R语言环境下向神经网络传入多组多变量时间序列是实操中的高频难点，目前三类主流实现方案均存在已知缺陷：

• 嵌套tibble是R语言中最符合使用直觉的存储方案，但实测无法直接作为Keras输入正常工作
• 直接使用3D数组作为输入时，对维度顺序的设置要求极高，顺序错误会直接导致模型训练逻辑失效
• 将3D数组转换为tfdatasets数据集后调用tfdatasets::feature_spec()做特征处理会触发固定报错

最小复现场景 #

初始数据为嵌套tibble格式，复现代码如下：

library(magrittr)
reticulate::use_python("/usr/local/bin/python")

data <- tibble::tibble(
  Timeseires = list(
    A = tibble::tibble(
      Period = rep(0:4, each = 10),
      MeasuremntA = runif(50, 20, 30),
      MeasuremntB = runif(50, 0.001, 0.01)
    ),
    C = tibble::tibble(
      Period = rep(0:4, each = 10),
      MeasuremntA = runif(50, 20, 30),
      MeasuremntB = runif(50, 0.001, 0.01)
    ),
    B = tibble::tibble(
      Period = rep(0:4, each = 10),
      MeasuremntA = runif(50, 20, 30),
      MeasuremntB = runif(50, 0.001, 0.01)
    ),
    D = tibble::tibble(
      Period = rep(0:4, each = 10),
      MeasuremntA = runif(50, 20, 30),
      MeasuremntB = runif(50, 0.001, 0.01)
    )
  )
)

measurement.data <- data %>%
  dplyr::mutate(Timeseires = purrr::map(
    .x = Timeseires,
    .f = simplify2array
  )) %>%
  .$Timeseires %>%
  simplify2array() %>%
  tfdatasets::tensor_slices_dataset()

feature.spec.measurment <- measurement.data %>%
  tfdatasets::feature_spec(.) %>%
  tfdatasets::step_numeric_column(
    tfdatasets::all_numeric(),
    normalizer_fn = tfdatasets::scaler_standard()
  ) %>%
  tfdatasets::fit()

运行上述代码会固定触发以下报错：

Error: Unable to resolve features for dataset that does not have named outputs

手动为数据集添加特征名称无法解决该问题。

完整多输入多输出LSTM场景 #

示例目标是构建元数据+时序测量数据的双输入、双输出LSTM模型，剩余代码如下：

##### Data Prep for meta.data #####
meta.data <- data %>%
  dplyr::select(-Timeseires) %>%
  tfdatasets::tensor_slices_dataset()

feature.spec.meta <- meta.data %>%
  tfdatasets::feature_spec(Result ~ .) %>%
  tfdatasets::step_numeric_column(
    tfdatasets::all_numeric(),
    normalizer_fn = tfdatasets::scaler_standard()
  ) %>%
  tfdatasets::step_categorical_column_with_vocabulary_list(
    tfdatasets::all_nominal()
  ) %>%
  tfdatasets::step_indicator_column(
    tfdatasets::all_nominal()
  ) %>%
  tfdatasets::fit()

input.measurement <- keras::layer_input(shape = c(3, 4))

lstm.out <- input.measurement %>%
  keras::layer_lstm(units = 32)

#####

input.meta <- data %>%
  dplyr::select(-Timeseires, -Result) %>%
  tfdatasets::layer_input_from_dataset()

dense.out <- input.meta %>%
  keras::layer_dense_features(feature_columns = tfdatasets::dense_features(feature.spec.meta))

#####

output <- keras::layer_concatenate(c(dense.out, lstm.out)) %>%
  keras::layer_dense(units = 64, activation = "relu") %>%
  keras::layer_dense(units = 64, activation = "relu") %>%
  keras::layer_dense(units = 64, activation = "relu") %>%
  keras::layer_dense(units = 1, activation = "sigmoid")

output.auxiliary <- input.measurement %>%
  keras::layer_dense(units = 1, activation = "sigmoid", name = "aux_output")

model <- keras::keras_model(
  inputs = c(input.meta, input.measurement),
  outputs = c(output, output.auxiliary)
)

model %>% keras::compile(
  loss = keras::loss_binary_crossentropy,
  optimizer = "adam",
  metrics = "binary_accuracy"
)

summary(model)

history <- model %>%
  keras::fit(
    x = list(
      # tfdatasets::dataset_use_spec(meta.data, spec = feature.spec.meta),
      # tfdatasets::dataset_use_spec(measurement.data ,feature.spec.measurment)
      data %>% dplyr::select(-Timeseires, -Result),
      data %>%
        dplyr::mutate(Timeseires = purrr::map(
          .x = Timeseires,
          .f = simplify2array
        )) %>%  .$Timeseires
        %>% simplify2array()
    ),
    y = list(
      data$Result,
      data$Result
    ),
    epochs = 10,
    validation_split = 0.3
  )

原示例中临时将输入层shape设置为c(3,4)仅能让代码勉强运行，该设置存在维度不匹配的错误：实际转换后的时序数组维度为4（样本数）×50（时间步）×3（特征数），正确的输入层应设置为keras::layer_input(shape = c(50, 3))，才能保证训练、评估阶段的时间序列维度完全对齐。

可行修复方案 #

以下步骤经实测可正常运行，完全适配多输入多输出LSTM的训练逻辑：

• 修正3D数组维度顺序
  R中simplify2array()转换嵌套tibble得到的3D数组默认维度顺序为[时间步, 特征数, 样本数]，不符合Keras要求的[样本数, 时间步, 特征数]输入规范，需要用aperm()置换维度：

# 修正后的时序数组转换逻辑
meas_array <- data %>%
  dplyr::mutate(Timeseires = purrr::map(
    .x = Timeseires,
    .f = simplify2array
  )) %>%
  .$Timeseires %>%
  simplify2array() %>%
  aperm(perm = c(3,1,2)) # 置换后维度顺序为[样本,时间步,特征]

• 规避feature_spec的3D张量兼容问题
  当前版本的tfdatasets::feature_spec()仅支持处理2D结构化表格数据，无法识别3D时序张量的命名特征，不需要强行对时序数据调用该接口。时序数据的标准化可提前手动实现，逻辑和tfdatasets::scaler_standard()完全一致，从根源避免"无命名输出"报错：

# 手动实现时序数据Z-score标准化
meas_mean <- mean(meas_array)
meas_sd <- sd(meas_array)
meas_array_scaled <- (meas_array - meas_mean) / meas_sd

修正后输入层shape可直接设置为keras::layer_input(shape = c(50, 3))，和实际数据维度完全匹配。

• 对齐训练输入逻辑
  元数据部分保留原有的feature_spec处理流程即可，时序部分直接传入预处理完成的3D数组，无需额外转换为tfdatasets对象嵌套feature_spec。模型训练阶段的输入调整如下：

history <- model %>%
  keras::fit(
    x = list(
      as.data.frame(data %>% dplyr::select(-Timeseires, -Result)), # 元数据转data.frame适配feature_spec输入要求
      meas_array_scaled # 传入预处理完成的3D时序数组
    ),
    y = list(
      as.integer(data$Result),
      as.integer(data$Result)
    ),
    epochs = 10,
    validation_split = 0.3
  )

若需要使用tfdatasets实现批量加载、数据打乱、数据增强等操作，构造数据集时需给时序字段命名，例如传入tfdatasets::tensor_slices_dataset(list(meas = meas_array_scaled))，但该方式依然无法直接对3D时序字段调用feature_spec的数值标准化步骤，提前在数组层面完成预处理是目前稳定性最高的方案。

内容的提问来源于stack exchange，提问作者Someone2