使用for循环批量打包每日CSV文件至独立tar.gz时遇异常

Hey there! Let’s work through this issue step by step — I’ve run into similar headaches when batch-processing date-stamped files, so I’ll break down the most likely causes and fixes for your problem.

Common Issues & Fixes #

1. Your Loop Isn’t Capturing All Files for a Single Date #

If you’re ending up with single-file tarballs, the root cause is almost always incorrect date matching logic. Here’s how to fix it:

First, make sure you’re extracting unique date values from your filenames. Let’s assume your files follow a pattern like sales_20240501_0900.csv, sales_20240501_1400.csv (date in YYYYMMDD format). Run this to grab all distinct dates:

# Extract unique YYYYMMDD dates from CSV filenames
unique_dates=$(ls *.csv | sed -E 's/.*([0-9]{8}).*/\1/' | sort -u)

• Adjust the regex ([0-9]{8}) if your date format is different (e.g., ([0-9]{4}-[0-9]{2}-[0-9]{2}) for YYYY-MM-DD).

Then loop through each date and pack all matching files:

for date in $unique_dates; do
  # Use quotes around filenames to handle spaces/special characters
  tar -czf "daily_backup_${date}.tar.gz" *"${date}"*.csv
done

2. The Process Gets Stuck Mid-Pack #

Stalling usually happens because special characters in filenames (spaces, newlines, weird symbols) are breaking the tar command. For bulletproof handling, use find with null-separated filenames (this avoids parsing issues entirely):

unique_dates=$(ls *.csv | sed -E 's/.*([0-9]{8}).*/\1/' | sort -u)

for date in $unique_dates; do
  # Find all CSV files for the date, pass to tar safely
  find . -maxdepth 1 -type f -name "*${date}*.csv" -print0 | tar -czf "daily_backup_${date}.tar.gz" --null -T -
done

• -print0 tells find to separate filenames with null characters (instead of newlines)
• --null -T - tells tar to read filenames from stdin, using null as the delimiter

3. Test First Before Packing! #

Before running the actual tar command, verify that each date is matching the right files. Add an echo step to debug:

for date in $unique_dates; do
  echo "=== Files for ${date} ==="
  ls *"${date}"*.csv
done

This will show you exactly which files are being grouped — if you see single files here, your regex needs tweaking to match your actual filename pattern.

Quick Checks to Rule Out Edge Cases #

• Ensure all your CSV files use the same date format (mixing YYYYMMDD and YYYY-MM-DD will break matching)
• If you’re on WSL/Git Bash, check for Windows-style line endings (CRLF) in filenames — use dos2unix to convert if needed
• Avoid using ls in scripts if possible (it’s prone to parsing issues); for better reliability, use find to list files and extract dates directly:

  unique_dates=$(find . -maxdepth 1 -type f -name "*.csv" | sed -E 's/.*([0-9]{8}).*/\1/' | sort -u)
  

内容的提问来源于stack exchange，提问作者Xizu