C#从Amazon S3下载文件：捕获异常避免崩溃并继续执行

我有一份需要从Amazon S3下载的文件名列表，正常下载流程没问题，但遇到不存在的文件时程序会直接崩溃。尝试列出存储桶内容时收到Access Denied（403 Forbidden）错误，供应商声称该操作权限正常，但我已放弃尝试通过列存储桶内容来提前判断文件是否存在。

想请教：如何在异步下载方法中捕获不存在文件的错误，避免程序崩溃？希望某个文件下载失败时直接跳过，继续处理列表中的下一个文件。

错误信息 #

Amazon.S3.AmazonS3Exception: Access Denied --->   
Amazon.Runtime.Internal.HttpErrorResponseException: The remote server returned an error: (403) 
Forbidden. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden. at 
System.Net.HttpWebRequest.GetResponse() at Amazon.Runtime.Internal.HttpRequest.GetResponse() --- 
End of inner exception stack trace 

原代码 #

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.IO.Compression;
using Amazon;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.IO;
using System.Threading;

namespace AlgoSeekDecompressor
{
    class Program
    {
        static void Main(string[] args)
        {
            var list = new List<string>()
            {
                "Apple.csv",
                "Banana.csv",
                "Canteloupe.csv",
            };

            var downloadPath = @"E:\My_Folder";

            Task.Run(async () => { await DownloadFilesAsync(list, downloadPath); }).Wait();

            Console.WriteLine("Done downloading");
        }

        static async Task DownloadFilesAsync(List<string> list, string downloadPath)
        {
            var accessKey = "#################";
            var secretAccessKey = "#########################";

            var client = new AmazonS3Client(
                accessKey,
                secretAccessKey,
                RegionEndpoint.USEast1
            );

            foreach (var file in list)
            {
                var bucketName = "amazon_s3_bucket";

                // 文件名前缀为文件名首字母大写
                var filename = file.Substring(0, 1).ToUpper() + "/" + file + ".csv.gz";

                var request = new GetObjectRequest()
                {
                    BucketName = bucketName,
                    Key = filename,
                    RequestPayer = RequestPayer.Requester
                };

                if (!Directory.Exists(downloadPath))
                {
                    Directory.CreateDirectory(downloadPath);
                    Console.WriteLine("Directory {0} does not exist - creating directory", downloadPath);
                }

                var filePath = Path.Combine(downloadPath, filename);
                var appendToFile = false;

                using (var response = await client.GetObjectAsync(request))
                {
                    await response.WriteResponseStreamToFileAsync(filePath, appendToFile, CancellationToken.None);
                }
            }
        }
    }
}

解决方案 #

核心是给单个文件的下载逻辑添加异常捕获，将GetObjectAsync和文件写入操作包裹在try-catch块中，捕获AmazonS3Exception并判断错误类型，跳过失败文件继续执行循环。

修改后的代码 #

重点修改DownloadFilesAsync方法中的循环部分：

static async Task DownloadFilesAsync(List<string> list, string downloadPath)
{
    var accessKey = "#################";
    var secretAccessKey = "#########################";

    var client = new AmazonS3Client(
        accessKey,
        secretAccessKey,
        RegionEndpoint.USEast1
    );

    // 目录判断移到循环外，避免重复执行
    if (!Directory.Exists(downloadPath))
    {
        Directory.CreateDirectory(downloadPath);
        Console.WriteLine("Directory {0} does not exist - creating directory", downloadPath);
    }

    foreach (var file in list)
    {
        var bucketName = "amazon_s3_bucket";
        var filename = file.Substring(0, 1).ToUpper() + "/" + file + ".csv.gz";
        var request = new GetObjectRequest()
        {
            BucketName = bucketName,
            Key = filename,
            RequestPayer = RequestPayer.Requester
        };

        var filePath = Path.Combine(downloadPath, filename);
        var appendToFile = false;

        try
        {
            using (var response = await client.GetObjectAsync(request))
            {
                await response.WriteResponseStreamToFileAsync(filePath, appendToFile, CancellationToken.None);
                Console.WriteLine($"成功下载文件: {filename}");
            }
        }
        catch (AmazonS3Exception ex)
        {
            // 根据状态码区分不同错误场景
            if (ex.StatusCode == System.Net.HttpStatusCode.NotFound)
            {
                Console.WriteLine($"文件不存在，跳过: {filename}");
            }
            else if (ex.StatusCode == System.Net.HttpStatusCode.Forbidden)
            {
                Console.WriteLine($"无权限访问文件，跳过: {filename}");
            }
            else
            {
                Console.WriteLine($"下载文件 {filename} 时发生未知S3错误: {ex.Message}");
            }
            // 不抛出异常，继续处理下一个文件
        }
        catch (Exception ex)
        {
            // 捕获非S3相关的其他异常
            Console.WriteLine($"处理文件 {filename} 时发生错误: {ex.Message}");
        }
    }
}

关键说明 #

1. 异常范围控制：单个文件的下载逻辑完全包裹在try-catch中，确保单个文件的错误不会中断整个循环。
2. 精准错误判断：针对AmazonS3Exception通过StatusCode区分文件不存在（404）、权限拒绝（403）等场景，给出对应日志提示。
3. 性能优化：将目录创建逻辑移到循环外，避免重复执行判断和创建操作。
4. 日志辅助排查：添加成功/失败的日志输出，方便后续定位问题。

内容的提问来源于stack exchange，提问作者eek