如何从R语言上传Data Frame至BigQuery?解决函数调用报错问题
R向BigQuery上传DataFrame的踩坑记录与解决方法
多年来R向BigQuery上传DataFrame/表格的方法多次变更,跟进起来较为麻烦,以下是实际操作中的问题与解决过程:
1. 旧函数insert_upload_job已废弃
最初使用insert_upload_job上传,代码如下:
job = insert_upload_job( project = "projectid", dataset = "dataset", table = "tablename", values = table_wanting_to_upload, write_disposition='WRITE_TRUNCATE', create_disposition = "CREATE_IF_NEEDED") # pushing table to BQ
执行后收到错误与弃用提示:
Error: billing is not a string (a length one character vector). In addition: Warning message: 'insert_upload_job' is deprecated. Use 'bq_perform_upload' instead. See help("Deprecated") and help("bigrquery-deprecated").
2. 改用bq_perform_upload的问题
按照提示改用bq_perform_upload,代码如下:
job = bq_perform_upload( project = "projectid", dataset = "dataset", table = "table_name", values = table_wanting_to_upload, write_disposition='WRITE_TRUNCATE', create_disposition = "CREATE_IF_NEEDED") # pushing table to BQ
出现参数缺失错误:
Error in as_bq_table(x) : argument "x" is missing, with no default
尝试传入x参数(将DataFrame传给x):
job = bq_perform_upload(x = predictions, project = "projectid", dataset = "dataset", table = "table_name", write_disposition='WRITE_TRUNCATE', create_disposition = "CREATE_IF_NEEDED") # pushing table to BQ
仍收到类型不支持错误:
Error in UseMethod("as_bq_table") : no applicable method for 'as_bq_table' applied to an object of class "c('grouped_df', 'tbl_df', 'tbl', 'data.frame')"
注:此前该函数的调用更为简洁,例如:
job <- bq_perform_upload("project","dataset","table_name", dataframe)
3. 最终可行方案:使用bqr_upload_data
最终通过bqr_upload_data函数成功上传,代码如下:
bqr_upload_data(projectId = 'project', datasetId = 'dataset', tableId = 'table_name', upload_data = predictions , create = c("CREATE_IF_NEEDED"), schema = NULL, sourceFormat = c("CSV", "DATASTORE_BACKUP", "NEWLINE_DELIMITED_JSON", "AVRO"), wait = TRUE, autodetect = TRUE, nullMarker = NULL, maxBadRecords = NULL, allowJaggedRows = FALSE, allowQuotedNewlines = FALSE, fieldDelimiter = ",")
内容的提问来源于stack exchange,提问作者Jaskeil
相关产品推荐
相关产品推荐

