You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

在R的foreach循环中用dbFetch执行多查询时出现‘外部指针无效’错误,求原因

问题描述

我正在使用R的foreach包加速通过SQL和dbFetch语法从服务器下载数据的循环,代码如下:

#library packages
library(RPostgres)
library(dplyr)
library(foreach)
library(doParallel)

#Setup backend to use many processors
totalCores = detectCores()
#Leave one core to avoid overload your computer
cluster <- makeCluster(totalCores[1]-1) 
registerDoParallel(cluster)

# connect the server
wrds <- dbConnect(Postgres(),host='wrds-pgdata.wharton.upenn.edu',
              port=9737,user=userid,password=pw,dbname='wrds',sslmode='require')

# Download Data from server
foreach (j = 1:length(Datetemp), .packages = "RPostgres") %dopar% {
      date <- as.character(Datetemp[j])
      query <- paste0("select * from ", "taqmsec.ctm_", date, 
                     " where sym_root = '", sym, "' and ", 
                     "time_m between '09:30:00.000' and '16:00:00.000'")
      res <- dbSendQuery(wrds, query)
      data <- dbFetch(res, n = -1)
      dbClearResult(res) 
      write.csv(data, row.names = FALSE, file = paste0(sympath, "/", sym, "_", date, ".csv"))
      print(paste0("Finished Stock ", sym, " for Date ", date))
}

运行错误

> task 1 failed - "external pointer is not valid"

错误原因

  • 数据库连接无法跨集群节点共享:主进程中创建的wrds数据库连接是外部指针对象,这类对象无法被序列化传递给doParallel的集群子进程。子进程拿到的是无效的指针引用,调用dbSendQuery时就会触发该错误。
  • 集群子进程的独立性:makeCluster创建的每个子进程都是独立的R会话,不会继承主进程的数据库连接资源,必须在每个子进程内部单独建立连接才能正常操作数据库。

内容的提问来源于stack exchange,提问作者Clay ZHAI

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.07.07 20:38:15