Snowflake卸载至S3时文件后缀含义及异常问题咨询

Let’s start by unpacking the filename pattern you’re seeing: foobar_[slice_id]_[segment_id]_[retry_count].csv.gz. Each part has a specific purpose, which will help explain your observations:

Filename Component Definitions #

• slice_id (first number): This is the ID of a parallel processing slice Snowflake uses for the unload. Snowflake splits your source table into slices based on warehouse size and table partitioning, processing each slice in parallel to speed up the export.
• segment_id (second number): Within each slice, Snowflake may split data into smaller segments. Each segment gets an ID starting at 0. If a segment has no rows (e.g., that slice subset is empty), no file is created for it.
• retry_count (third number): This counts how many times Snowflake retried writing that file. If the first write succeeds, this stays at 0. You’ll only see higher numbers if there were transient failures (like S3 connectivity issues) that required retries.

Your Specific Questions #

1. Why is the last number never used? #

That last number is the retry count, and it’s only greater than 0 if Snowflake had to retry writing a file. Since all your unloads succeeded on the first try for every segment, every file shows 0 here. You’d see non-zero values only if there were temporary errors during the unload that forced retries.

2. Why does the second number stop at 7? #

The second number is the segment ID, and its upper limit depends on how Snowflake splits the data in each slice. This is influenced by:

• The size of your Snowflake warehouse (larger warehouses can split data into more segments)
• How your source table is partitioned
• The volume of data in each slice

In your case, Snowflake split each slice into up to 8 segments (IDs 0-7). The gaps you see (like 1_1 and 1_6 missing in the first unload) mean those segments had no data, so no file was generated for them.

3. Why wasn’t foobar_0_0_0.csv.gz generated in the second unload? #

Snowflake doesn’t create empty files during unloads. If the segment corresponding to slice_id=0, segment_id=0 had no rows during the second unload, it skips generating that file entirely.

Common reasons for this include:

• Data in that segment was deleted or modified between the two unload runs
• The source table’s data changed (e.g., new rows were added elsewhere, but this specific segment’s data was removed)
• Snowflake’s data splitting logic might have assigned rows to different segments in the second run (unlikely unless you changed warehouse size or table partitioning between runs)

内容的提问来源于stack exchange，提问作者Martin Thoma