Terraform中BigQuery作业显示成功但云端未完成,报错求助
问题排查与解决方案
核心原因
日志报错Cannot set destination table in jobs with DDL statements明确指出问题:当查询语句是DDL(如CREATE TABLE ... AS SELECT)时,不能同时在Terraform的query块中配置destination_table。因为DDL语句本身已经指定了要创建的目标表,额外设置destination_table会导致BigQuery作业冲突,最终作业标记为完成但实际未执行表创建操作。
额外问题
原SQL语句存在两处语法错误:
- 字段列表末尾多了逗号:
email,FROM应改为email FROM - 项目名拼写错误:
myporject应为myproject
修正后的Terraform配置
移除destination_table块,修正SQL语法:
resource "google_bigquery_job" "user_data_job" { job_id = "job_query_${random_id.random_job_id.hex}" location = "US" labels = { "job_type" = "create_table" } query { query = "CREATE TABLE myproject.shop45_dataset.user_data AS SELECT first_name, last_name, email FROM myproject.shop45_dataset.shop_data" allow_large_results = true flatten_results = true } depends_on = [ google_bigquery_job.import_job ] }
替代方案(不使用DDL)
如果希望通过destination_table指定目标表,可改用普通SELECT语句,去掉DDL部分:
resource "google_bigquery_job" "user_data_job" { job_id = "job_query_${random_id.random_job_id.hex}" location = "US" labels = { "job_type" = "create_table" } query { query = "SELECT first_name, last_name, email FROM myproject.shop45_dataset.shop_data" destination_table { project_id = "myproject" dataset_id = "shop45_dataset" table_id = "user_data" } create_disposition = "CREATE_IF_NEEDED" write_disposition = "WRITE_EMPTY" allow_large_results = true flatten_results = true } depends_on = [ google_bigquery_job.import_job ] }
内容的提问来源于stack exchange,提问作者PRABHAT KUMAR
相关产品推荐
相关产品推荐

