Bash中按模式批量处理CSV文件并提取目标列值的方法

Got it, let's tackle this problem. Your original awk command only processes the first file when using wildcards because it wasn't built to handle multiple files properly—plus your requirement has shifted to grabbing the second column of the last row from each matching CSV, not the row with the max value in column 1. Here are two robust solutions tailored to your needs:

1. GNU awk (Simpler, Recommended for Linux) #

GNU awk includes a handy ENDFILE block that runs right after processing each file, making this task straightforward. Use this one-liner:

awk '{last_col2 = $2} ENDFILE {
    fname = FILENAME
    sub(/^report-/, "", fname)  # Strip the "report-" prefix from the filename
    sub(/\.csv$/, "", fname)    # Strip the ".csv" suffix
    print fname ": " last_col2
}' report-*.csv >> test.txt

Breakdown: #

• {last_col2 = $2}: Overwrites the last_col2 variable with the second column of every row. By the end of each file, it will hold the value from the very last row.
• ENDFILE: Triggers once per file after all lines are processed. We clean up the filename to match your desired output format, then print the result.

2. POSIX-Compliant awk (Works on macOS/BSD) #

If you're using a system with BSD awk (like macOS) which doesn't support the ENDFILE block, use this compatible version that works across all POSIX systems:

awk '
FNR == 1 {
    # When starting a new file (and it's not the first one), print the previous file's result
    if (NR != 1) {
        sub(/^report-|\.csv$/, "", prev_fname)
        print prev_fname ": " prev_last
    }
    prev_fname = FILENAME
}
# Save the second column of the current row
{ prev_last = $2 }
# Print the final file's result after all processing completes
END {
    sub(/^report-|\.csv$/, "", prev_fname)
    print prev_fname ": " prev_last
}' report-*.csv >> test.txt

Breakdown: #

• FNR == 1: Detects the start of a new file. We print the previous file's result (if it's not the first file) then save the current filename for later.
• { prev_last = $2 }: Continuously updates the saved second column value as we process each row.
• END: Prints the result for the final file, since there's no new file trigger to handle it automatically.

Example Output #

If your files report-x-1.csv and report-y-1.csv have last rows with second columns 156 and 300 respectively, test.txt will contain:

x-1: 156
y-1: 300

内容的提问来源于stack exchange，提问作者blackbaddl