如何用sed/awk减少分隔符、消除千位逗号并合并多余列

Got it, let's solve this properly. Your issue is twofold: stripping thousand-separator commas and merging extra columns (caused by unintended tabs in fields) back into the 30th column. Sed is great for simple text replacements, but awk is the right tool here since it handles field manipulation seamlessly.

Here's a one-liner that does both jobs:

BEGIN {OFS="\t"}
{
    # Remove all commas from every field first
    for (i=1; i<=NF; i++) gsub(/,/, "", $i)
    # Merge extra fields into the 30th column if we have more than 30
    if (NF > 30) {
        for (i=31; i<=NF; i++) $30 = $30 "\t" $i
        # Trim the field count back to 30 to drop the now-redundant extra fields
        NF = 30
    }
    # Print the cleaned row with tab separators preserved
    print $0
}' "$oldfile" > "$newfile"

Breakdown of how this works: #

• BEGIN {OFS="\t"}: Ensures the output uses tabs as field separators (matches your input format, so no unexpected spaces creep in).
• The first loop iterates through every field and removes all commas—this takes care of those thousand-separator commas you mentioned, regardless of which field they're in.
• When a row has more than 30 fields (thanks to unintended tabs), we loop from the 31st field onward and append each one to the 30th field (keeping the original tab that caused the split, since that's part of the field's actual content).
• Setting NF = 30 tells awk to treat the row as having only 30 fields now, so the extra ones are discarded after being merged.
• Finally, we print the cleaned row.

Example: #

If you had a wonky row like this (tab-separated):

Column1 Column2 Column3 has\ta tab Column4 ... (3 total extra columns)

After running the command, it becomes:

Column1 Column2 ... Column30_with_original_content\tColumn31\tColumn32\tColumn33

All commas are stripped, and the extra columns are merged into the 30th column exactly as they appeared (with their tabs intact).

This approach is robust, handles all edge cases where extra tabs pop up, and replaces your original sed command with a single tool that does both tasks.

内容的提问来源于stack exchange，提问作者David 54321