分块读取文件时ByteArray签名匹配：改用Array.IndexOf实现方案求助

Got it, let's tackle this performance issue head-on. Your original LINQ approach works, but methods like Skip(), Take(), and SequenceEqual() create unnecessary intermediate arrays and add overhead that's avoidable. Array.IndexOf() is optimized for low-level byte lookups, and manual array comparisons will give us a big performance boost while keeping the core logic intact.

Here's a refactored version of your method that replaces LINQ with Array.IndexOf() and direct byte checks, while still handling that tricky partial match scenario at the end of the buffer:

public static bool Contains(byte[] buffer, byte[] signature, FileStream file)
{
    // Handle edge cases first
    if (signature.Length == 0) return true;
    if (buffer.Length == 0) return false;

    int sigLength = signature.Length;
    int bufLength = buffer.Length;

    // Step 1: Look for full signature matches in the current buffer
    // Start by finding the first occurrence of the signature's starting byte
    int startIndex = Array.IndexOf(buffer, signature[0], 0, bufLength - sigLength + 1);
    while (startIndex != -1)
    {
        // Verify the rest of the signature matches
        bool isMatch = true;
        for (int i = 1; i < sigLength; i++)
        {
            if (buffer[startIndex + i] != signature[i])
            {
                isMatch = false;
                break;
            }
        }

        if (isMatch)
        {
            return true;
        }

        // Look for the next possible starting byte
        startIndex = Array.IndexOf(buffer, signature[0], startIndex + 1, bufLength - (startIndex + 1) - sigLength + 1);
    }

    // Step 2: Check for partial matches at the buffer's end
    // Calculate the maximum possible overlap (can't be longer than signature or buffer)
    int maxOverlap = Math.Min(sigLength - 1, bufLength);
    for (int overlapSize = maxOverlap; overlapSize >= 1; overlapSize--)
    {
        // Check if buffer's end matches signature's start
        bool partialMatch = true;
        for (int i = 0; i < overlapSize; i++)
        {
            if (buffer[bufLength - overlapSize + i] != signature[i])
            {
                partialMatch = false;
                break;
            }
        }

        if (partialMatch)
        {
            // Read remaining bytes needed to complete the signature
            int remainingBytes = sigLength - overlapSize;
            byte[] remainingBuffer = new byte[remainingBytes];
            int bytesRead = file.Read(remainingBuffer, 0, remainingBytes);

            // If we can't read all required bytes, no match
            if (bytesRead != remainingBytes)
            {
                return false;
            }

            // Verify the remaining bytes match the rest of the signature
            bool fullMatch = true;
            for (int i = 0; i < remainingBytes; i++)
            {
                if (remainingBuffer[i] != signature[overlapSize + i])
                {
                    fullMatch = false;
                    break;
                }
            }

            return fullMatch;
        }
    }

    // No matches found anywhere
    return false;
}

Key Performance & Functional Improvements: #

• Array.IndexOf() for fast initial lookup: This runtime-optimized method finds the signature's starting byte way faster than manual full scans or LINQ operations.
• Direct byte comparisons: We avoid creating intermediate arrays with LINQ methods, cutting down on memory allocations and unnecessary processing.
• Targeted edge case handling: Added checks for empty signatures/buffers to prevent unexpected behavior.
• Efficient partial match validation: Instead of SequenceEqual(), we compare bytes directly, which is far faster for small overlap lengths.

How It Works: #

1. Full Match Check: We use Array.IndexOf() to locate every possible starting position of the signature in the buffer (only where a full signature could fit). For each hit, we manually verify the rest of the bytes match.
2. Partial Match Check: We check from the largest possible overlap down to 1 byte to see if the buffer's end aligns with the signature's start. If we find a partial match, we read the missing bytes from the file and validate if they complete the signature.

This version keeps the exact same functionality as your original code but runs significantly faster, especially for large buffers or when the method is called frequently.

内容的提问来源于stack exchange，提问作者PyroDeathAdder