R语言Keras模型结合tf.data.Dataset处理重复列及代码报错解决方案问询
使用Keras + tf.data.Dataset时遇到的运行问题
我正尝试在Keras模型中使用tf.data.Dataset来处理包含重复列的数据集(避免重复列导致的问题),但目前卡在了代码运行环节。先从测试代码的问题说起:
测试代码(无法正常运行)
这段代码旨在让Keras模型在包含3个特征列(c1、c2、c3)和目标列target的数据集上拟合/预测:
library(keras) library(tfaddons) library(tfdatasets) alldata <- data.frame(c1=rnorm(100),c2=rnorm(100),c3=rnorm(100), target=rnorm(100)) alldata_xvar <- c("c1","c2","c3") write.csv(alldata,file = "alldata.csv", row.names=F) alldata_spec <- csv_record_spec("alldata.csv") xfeatures <- alldata_xvar model <- keras_model_sequential() model %>% layer_dense(units = length(xfeatures), activation = "relu", input_shape = length(xfeatures)) %>% layer_dense(units = 1) dataset <- text_line_dataset("alldata.csv", record_spec = alldata_spec) dataset %>% dataset_prepare(x = xfeatures,y="target") predicted <- model %>%predict(dataset)
报错信息
Error in py_call_impl(callable, dots$args, dots$keywords) : ValueError: in user code: File "/.local/share/r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/keras/engine/training.py", line 1621, in predict_function * return step_function(self, iterator) File "/.local/share/r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/keras/engine/training.py", line 1611, in step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) File "/.local/share/r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/keras/engine/training.py", line 1604, in run_step ** outputs = model.predict_step(data) File "/.local/share/r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/keras/engine/training.py", line 1572, in predict_step return self(x, training=False) File "/.local/share/r-miniconda/envs/r-reticulate/lib/python3.7/site-packages/keras/utils/traceback_utils.py", line 67, in error_handl
最终需求与完整代码
我的实际需求是处理一个包含3个特征列(c1、c2、c3)以及100个全行列值相同的额外列(RegionVar1至RegionVar100)的数据集。由于把这100列存入CSV会导致文件过大,我计划用dataset_map函数为从CSV读取的数据添加这些共用的区域信息,但这段代码也无法正常工作:
library(keras) library(tfaddons) library(tfdatasets) numregion <- 50 numvarregion <- 100 alldata <- data.frame(c1=rnorm(100),c2=rnorm(100),c3=rnorm(100), target=rnorm(100)) alldata_xvar <- c("c1","c2","c3") region <- matrix( rnorm(50*numvarregion), 50, numvarregion) regionvar_name1 <- rep("RegionVar",numvarregion) regionvar_name2 <- seq(1,numvarregion) regionvar_name <- cbind(regionvar_name1,regionvar_name2) regionvar_name <- apply(regionvar_name,1,paste,collapse="") region <- as.data.frame(region) names(region) <- regionvar_name region <- array_reshape(as.matrix(region), c(nrow(region), ncol(region))) write.csv(alldata,file = "alldata.csv", row.names=F) alldata_spec <- csv_record_spec("alldata.csv") xfeatures <- c(alldata_xvar,names(region)) model <- keras_model_sequential() model %>% layer_dense(units = length(xfeatures), activation = "relu", input_shape = length(xfeatures)) %>% layer_dense(units = 1) regionvar <- 1 dataset <- text_line_dataset("alldata.csv", record_spec = alldata_spec) dataset <- dataset %>% dataset_map(function(record) { record$Region <- region[regionvar,] record }) dataset %>% dataset_prepare(x = xfeatures,y="target") predicted <- model %>%predict(dataset)
希望能得到让代码正常运行的建议,或是解决重复列问题的其他可行方案。
内容的提问来源于stack exchange,提问作者Fan
相关产品推荐
相关产品推荐

