使用for循环处理tar文件：按文件名日期归档指定日期及更早文件

Alright, let's work through this problem. You want to archive files by the YYYYMMDD date in their filename—grouping each day's files into a separate tar.gz archive, covering a specified date and all prior days, using a loop. Here's a practical, bash-based solution that fits your needs:

Solution Script #

You can save this as a bash script (e.g., archive_by_date.sh) and run it with your target date, or hardcode the date directly:

#!/bin/bash

# Option 1: Hardcode your target date (YYYYMMDD format)
TARGET_DATE="20180410"

# Option 2: Pass target date as a command-line argument (uncomment below, comment above)
# if [ $# -ne 1 ]; then
#     echo "Usage: $0 <target_date_in_YYYYMMDD>"
#     exit 1
# fi
# TARGET_DATE="$1"

# Extract all unique YYYYMMDD dates from CSV filenames, filter those <= target date
VALID_DATES=$(ls *.csv | grep -oE '[0-9]{8}' | sort -u | awk -v target="$TARGET_DATE" '$1 <= target')

# Loop through each valid date to create archives
for DATE in $VALID_DATES; do
    # Check if any files exist for this date (suppress error if none)
    matching_files=$(ls *"$DATE"*.csv 2>/dev/null)
    
    if [ -n "$matching_files" ]; then
        echo "Creating archive for date $DATE..."
        # Create compressed tar archive with the date as the filename
        tar -czf "${DATE}.tar.gz" $matching_files
        echo "Successfully created ${DATE}.tar.gz"
    else
        echo "No files found for date $DATE — skipping."
    fi
done

How It Works #

Let's break down each part so you understand what's happening:

• Extract Valid Dates:
  • ls *.csv lists all CSV files in the directory.
  • grep -oE '[0-9]{8}' pulls out every 8-digit sequence (which we assume is your YYYYMMDD date from filenames like ABCSMOKLO2Y_pm24hr_20180410.000006_4.csv).
  • sort -u gets rid of duplicate dates, so we only process each date once.
  • awk filters the list to keep only dates that are earlier than or equal to your target date.
• Loop Through Dates:
  • For each date, we check if there are any CSV files containing that date string. 2>/dev/null hides the annoying "No such file or directory" error if no files exist for a date.
  • If matching files exist, tar -czf creates a compressed tar archive named after the date (e.g., 20180410.tar.gz) and adds all matching files to it.

Quick Tips #

• Test First: Before running the actual archive command, replace the tar -czf line with echo "Would archive: $matching_files into ${DATE}.tar.gz" to verify it's picking the right files and dates.
• Date Format: This script relies on your filenames having an 8-digit YYYYMMDD date. If your date format varies, adjust the grep pattern (e.g., if dates are MMDDYYYY, change [0-9]{8} to match that structure).
• Permissions: Make sure the script is executable with chmod +x archive_by_date.sh before running it.

内容的提问来源于stack exchange，提问作者bosug