如何创建Snowflake任务执行含动态日期路径的COPY命令?
解决方案
方案一:直接在COPY语句中内嵌动态日期逻辑
无需提前设置变量,直接将日期拼接逻辑写入pattern参数,任务每次执行时会自动计算当前日期生成对应路径:
create or replace task sv_task_copy_command_trial warehouse = compute_wh schedule = '1 MINUTE' AS copy into expose_sf_data_trial.decoded.CALLS_COPY_COMMAND from (select $1:LOAD_ID::varchar::TIMESTAMP_LTZ, $1:TIMESTAMP::timestamp from @expose_sf_data_trial.decoded.trial_calls_copy_command) pattern = CONCAT('CALLS/', TO_CHAR(CURRENT_DATE(), 'YYYY/MM/DD'), '/', '*.parquet');
方案二:同会话执行SET变量+COPY命令
如果偏好使用变量的方式,可在任务的SQL块中先设置变量,再执行COPY命令,两者在同一会话中执行,变量可正常生效:
create or replace task sv_task_copy_command_trial warehouse = compute_wh schedule = '1 MINUTE' AS BEGIN SET date_pattern = CONCAT('CALLS/', TO_CHAR(CURRENT_DATE(), 'YYYY/MM/DD'), '/', '*.parquet'); copy into expose_sf_data_trial.decoded.CALLS_COPY_COMMAND from (select $1:LOAD_ID::varchar::TIMESTAMP_LTZ, $1:TIMESTAMP::timestamp from @expose_sf_data_trial.decoded.trial_calls_copy_command) pattern = $date_pattern; END;
以上两种方案均无需使用存储过程,可直接实现每分钟按当前日期路径执行COPY命令的需求。
内容的提问来源于stack exchange,提问作者svalls
相关产品推荐
相关产品推荐

