如何在Java代码中指定Snowflake的FILE_FORMAT无需提前在Snowflake配置
解决方案
1. 实现可行性说明
Snowflake的COPY INTO语法完全支持内联定义文件格式参数,不需要预先在Snowflake中创建FILE FORMAT对象,直接把配置参数写在FILE_FORMAT的括号内即可。
2. 改造后的COPY INTO语句
把你原来预定义的CSV_TEST的参数全部内联进去,语句如下:
COPY INTO s3://snowflake/[你的userId] from "EPICEROS"."PUBLIC"."USER_TABLE_TEMP" storage_integration = s3_int file_format = ( TYPE = 'CSV' COMPRESSION = 'NONE' FIELD_DELIMITER = ',' RECORD_DELIMITER = '\n' SKIP_HEADER = 0 FIELD_OPTIONALLY_ENCLOSED_BY = 'NONE' TRIM_SPACE = TRUE ERROR_ON_COLUMN_COUNT_MISMATCH = FALSE ESCAPE = 'NONE' ESCAPE_UNENCLOSED_FIELD = '\134' DATE_FORMAT = 'AUTO' TIMESTAMP_FORMAT = 'AUTO' NULL_IF = ('\\N') ) OVERWRITE=TRUE;
3. Java代码适配
注意Java字符串中需要对特殊字符做转义:反斜杠\要写成\\,双引号保留原转义写法即可,改造后的代码片段如下:
String q = "COPY INTO s3://snowflake/" + userId + " from \"EPICEROS\".\"PUBLIC\".\"USER_TABLE_TEMP\" " + "storage_integration = s3_int " + "file_format = (" + "TYPE = 'CSV' " + "COMPRESSION = 'NONE' " + "FIELD_DELIMITER = ',' " + "RECORD_DELIMITER = '\\n' " + "SKIP_HEADER = 0 " + "FIELD_OPTIONALLY_ENCLOSED_BY = 'NONE' " + "TRIM_SPACE = TRUE " + "ERROR_ON_COLUMN_COUNT_MISMATCH = FALSE " + "ESCAPE = 'NONE' " + "ESCAPE_UNENCLOSED_FIELD = '\\134' " + "DATE_FORMAT = 'AUTO' " + "TIMESTAMP_FORMAT = 'AUTO' " + "NULL_IF = ('\\\\N')" + ") OVERWRITE=TRUE;"; jdbcTemplatePerBrand.get(brand).query(q, s -> {});
转义注意事项:
- 换行符
\n在Java字符串里要写成\\n,传给Snowflake后会被解析为正确的换行符 NULL_IF里的\\N在Java里需要转义两次,写成\\\\N,最终传到Snowflake时才会是正确的\N配置
内容的提问来源于stack exchange,提问作者user9347049
相关产品推荐
相关产品推荐

