使用AWS Lambda从S3传输Zip/Excel至SFTP时文件损坏问题求助

Hey there! I spot the exact issue right away—your code is treating binary files (like Zip and Excel) as plain text, which is why they’re getting mangled during transfer. Let’s break down what’s wrong and fix it.

The Root Cause #

You’re using StreamReader to read the file content into a string with ReadToEnd(), then converting that string back to bytes with Encoding.Default.GetBytes(). This works fine for text files (txt/csv) because their content is human-readable, but binary files contain non-text bytes that get corrupted when forced through a string conversion. That’s exactly why your Zip/Excel files won’t open after being sent to SFTP.

The Fix #

Instead of messing with string conversions, use the raw binary stream directly from the S3 GetObjectResponse. Here’s the corrected version of your code:

var s3Event = evnt.Records?[0].S3;
if (s3Event == null)
{
    return "NoBucketFound";
}
try
{
    // Use 'using' to ensure proper cleanup of S3 response resources
    using GetObjectResponse response = await this.S3Client.GetObjectAsync(s3Event.Bucket.Name, s3Event.Object.Key);
    // Access the raw binary stream directly—no text conversion needed!
    using Stream fileStream = response.ResponseStream;

    using (SftpClient sftp = new SftpClient("sftpserver", 22, "Test", "test"))
    {
        sftp.Connect();
        if (sftp.IsConnected)
        {
            try
            {                                
                sftp.BufferSize = 4 * 1024;
                // Ensure we start reading from the beginning of the stream
                fileStream.Seek(0, SeekOrigin.Begin);
                sftp.UploadFile(fileStream, "/" + s3Event.Object.Key.ToString(), true, null);
                Console.WriteLine($"Successfully uploaded {s3Event.Object.Key} to SFTP");
            }
            catch (Exception exp)
            {
                Console.WriteLine($"SFTP Upload Error: {exp.Message}");
            }
        }
        sftp.Disconnect();
    }

    return $"Successfully processed {s3Event.Object.Key}";
}
catch (Exception e)
{
    context.Logger.LogLine($"Error getting object {s3Event.Object.Key} from bucket {s3Event.Bucket.Name}. Make sure they exist and your bucket is in the same region as this function.");
    context.Logger.LogLine(e.Message);
    context.Logger.LogLine(e.StackTrace);
    throw;
}

Key Changes Explained #

• Removed text-based stream handling: Ditched StreamReader and string conversion entirely. Using response.ResponseStream directly preserves all original binary data, so your Zip/Excel files stay intact.
• Added using statements: Wrapped GetObjectResponse and Stream in using blocks to ensure resources are cleaned up properly—super important in Lambda to avoid memory leaks.
• Reset stream position: Added fileStream.Seek(0, SeekOrigin.Begin) to guarantee we start uploading from the start of the stream (a safe guard even though S3 streams usually start at position 0).
• Improved logging: Swapped Console.Write for Console.WriteLine for better readability, and added clear success/error messages to debug easier.

Why This Worked Locally? #

Chances are you only tested local transfers with text files, or your local environment’s encoding happened to not mangle the binary data by accident. Either way, this raw stream approach works reliably for all file types—text or binary—in both local and Lambda environments.

内容的提问来源于stack exchange，提问作者Pradeep SIngh