如何用Unix命令合并不同文件夹同名文件？合并结果异常求助

Hey Anna, let's figure out why your merged files have more lines than the sum of the two original ones—this is a common gotcha with batch file operations, so let's break it down step by step.

First: Did You Accidentally Repeat Files in Your Merge Command? #

The most likely culprit is that your merge operation is including the same file content multiple times. Here are a few scenarios that cause this:

• Overlapping source and target directories: If you're saving merged files into either vanila1 or vanila2, your batch command might be re-merging the already created merged file with the original one (e.g., if your loop runs again and picks up the new merged file as part of the source files).
• Buggy batch loop logic: If you wrote a shell loop to handle all files, double-check that you're only grabbing each pair once. For example, a loop that accidentally processes vanila1/* and vanila2/* into a single merged file (instead of pairing same-named files) would duplicate content if there are overlapping filenames.
• Typos in the command: A simple mistake like adding the same file path twice (e.g., cat vanila1/file.txt vanila1/file.txt vanila2/file.txt > merged/file.txt) would instantly double part of the content.

Let's Test with a Single File First #

To rule out file-specific issues, pick one pair of same-named files and run these commands manually:

1. Count lines in each original file:

   wc -l vanila1/your-test-file.txt
   wc -l vanila2/your-test-file.txt
   

2. Add those two numbers together to get the expected line count.
3. Merge them manually:

   cat vanila1/your-test-file.txt vanila2/your-test-file.txt > merged/test-output.txt
   

4. Count lines in the merged file:

   wc -l merged/test-output.txt
   

If this manual test gives you the correct sum, your batch command is the problem. If it still gives an overcount, check if either original file has hidden duplicate lines (use sort your-test-file.txt | uniq -c to spot duplicates) or unusual line endings (though this usually causes undercounts, not overcounts).

A Reliable Batch Merge Command to Try #

Here's a safe bash loop that pairs same-named files, merges them, and even checks for line count mismatches automatically:

# Create a merged directory if it doesn't exist
mkdir -p merged

# Loop through every file in vanila1
for file in vanila1/*; do
    # Get just the filename (without the vanila1/ path)
    filename=$(basename "$file")
    # Only proceed if the same file exists in vanila2
    if [ -f "vanila2/$filename" ]; then
        # Merge the two files into the merged directory
        cat "$file" "vanila2/$filename" > "merged/$filename"
        # Calculate expected vs actual line counts
        original_total=$(( $(wc -l < "$file") + $(wc -l < "vanila2/$filename") ))
        merged_count=$(wc -l < "merged/$filename")
        # Alert if there's a mismatch
        if [ "$merged_count" -ne "$original_total" ]; then
            echo "⚠️ Mismatch for $filename: Expected $original_total lines, got $merged_count"
        fi
    fi
done

Final Checks #

• Make sure the merged directory is separate from vanila1 and vanila2—this prevents the loop from accidentally reprocessing merged files.
• If you're using a different tool (not bash), verify that its merge logic isn't appending extra content or duplicating files.

内容的提问来源于stack exchange，提问作者Anna1364