You need to enable JavaScript to run this app.
优惠活动
大模型
产品
解决方案
定价
更多

使用dbt Redshift Unload宏导出数据至S3时执行失败求助

Redshift Unload宏在dbt执行失败但直接运行SQL正常的问题

我尝试使用Redshift Package Macro将数据卸载至S3,编译后的代码直接在Redshift中运行正常,但通过dbt执行或构建模型时持续失败。

dbt代码

{{
  config(
      {
        "post-hook": [
            "{{ redshift.unload_table('analytics_dev_josh',
                            'stg_payment_transaction',
                            s3_path='s3://bucket/folder',
                            iam_role='arn:aws:iam::xxxxxxxxx:role/xxxxxxxxxx',
                            header=True,
                            delimiter=',',
                            overwrite=true) }}"
            ]
        }
    )
}}

编译后的SQL代码

-- compile UNLOAD statement
  UNLOAD ('SELECT * FROM "analytics_dev_josh"."stg_payment_transaction"')
  TO 's3://bucket/folder'  
  IAM_ROLE 'arn:aws:iam::xxxxxxxxx:role/xxxxxxxxxx'  
  HEADER  
  DELIMITER AS ','  
  NULL AS ''
  MAXFILESIZE AS 6 GB  
  ESCAPE  
  ALLOWOVERWRITE  
  PARALLEL OFF

报错信息

14:03:06  Began running node model.analytics.unload_test
14:03:06  1 of 1 START sql table model analytics_dev_josh.unload_test .................... [RUN]
14:03:06  Acquiring new redshift connection 'model.analytics.unload_test'
14:03:06  Began compiling node model.analytics.unload_test
14:03:06  Writing injected SQL for node "model.analytics.unload_test"
14:03:06  Timing info for model.analytics.unload_test (compile): 2023-02-27 14:03:06.181313 => 2023-02-27 14:03:06.184251
14:03:06  Began executing node model.analytics.unload_test
14:03:06  Writing runtime sql for node "model.analytics.unload_test"
14:03:06  Using redshift connection "model.analytics.unload_test"
14:03:06  On model.analytics.unload_test: BEGIN
14:03:06  Opening a new connection, currently in state closed
14:03:06  Redshift adapter: Connecting to Redshift using 'database' credentials
14:03:06  SQL status: BEGIN in 0 seconds
14:03:06  Using redshift connection "model.analytics.unload_test"
14:03:06  On model.analytics.unload_test: /* {"app": "dbt", "dbt_version": "1.4.3", "profile_name": "user", "target_name": "default", "node_id": "model.analytics.unload_test"} */

  
    

  create  table
    "domusodatawarehouse"."analytics_dev_josh"."unload_test__dbt_tmp"
    
    
    
  as (
    
  );
  
14:03:06  Postgres adapter: Postgres error: syntax error at or near ")"
LINE 13:   );
           ^
14:03:06  On model.analytics.unload_test: ROLLBACK
14:03:06  Timing info for model.analytics.unload_test (execute): 2023-02-27 14:03:06.184611 => 2023-02-27 14:03:06.337104
14:03:06  On model.analytics.unload_test: Close
14:03:06  Database Error in model unload_test (models/stage/unload_test.sql)
  syntax error at or near ")"
  LINE 13:   );
             ^
  compiled Code at target/run/analytics/models/stage/unload_test.sql
14:03:06  Sending event: {'category': 'dbt', 'action': 'run_model', 'label': '5db3e73a-88f6-4c8c-bd6f-570e08d52b43', 'context': [<snowplow_tracker.self_describing_json.SelfDescribingJson object at 0x7fce41a11130>]}
14:03:06  1 of 1 ERROR creating sql table model analytics_dev_josh.unload_test ........... [ERROR in 0.16s]
14:03:06  Finished running node model.analytics.unload_test

编译后的代码复制到Redshift中直接运行可正常工作,但通过dbt执行时会报错,看起来dbt尝试将Unload语句写入临时表时触发了语法错误。


内容的提问来源于stack exchange,提问作者Josh Dobson

相关产品推荐
方舟 Agent Plan

超全模态模型 × Harness 升级,最新支持 Deepseek-V4.1-Flash、GLM-5.3 系列、Doubao-Seedream-5.0-pro、Kimi-K3 (部分), 限时 9.9 元起

最近更新时间:2026.07.30 08:35:36