基于AWS与.NET C#的FTP文件夹定期检测及上传解决方案咨询

Hey Shane, great question! Let’s dive into the top AWS + .NET C# solutions for your FTP file upload and periodic directory monitoring workflow—focused on scalable, production-grade approaches that fit a high-level solution need.

First, let’s cover reliable ways to handle file uploads to FTP from .NET, with AWS integrations for robustness.

1. Modern .NET FTP Libraries (FluentFTP) #

Forget the clunky FtpWebRequest—FluentFTP is the de facto modern library for FTP/FTPS/SFTP operations in .NET. It’s async-first, well-maintained, and supports all the security features you’ll need for a high-level solution.

Here’s a quick example of uploading a file with FTPS (secure FTP):

using FluentFTP;

public async Task UploadToFtpAsync(string localFile, string ftpRemotePath, string ftpHost, string username, string password)
{
    using var ftpClient = new FtpClient(ftpHost, username, password);
    // Enable FTPS for secure transfer
    ftpClient.EnableSsl = true;
    await ftpClient.ConnectAsync();

    // Overwrite existing files if needed (adjust behavior as required)
    await ftpClient.UploadFileAsync(localFile, ftpRemotePath, FtpRemoteExists.Overwrite);

    await ftpClient.DisconnectAsync();
}

2. AWS-Integrated Upload Workflows (Serverless-First) #

For a scalable, managed solution, pair .NET with AWS services:

• Step 1: Let users upload files to an Amazon S3 bucket (use pre-signed URLs for secure browser/app uploads).
• Step 2: Trigger an AWS Lambda (written in .NET) when a new file lands in S3.
• Step 3: The Lambda uses FluentFTP to sync the file to your FTP server.

This approach offloads server management to AWS, scales automatically, and adds built-in fault tolerance.

Next, let’s tackle the "regularly check for files and process them" requirement.

1. .NET Scheduled Task Frameworks (Quartz.NET/Hangfire) #

If you’re running a traditional .NET app (e.g., ASP.NET Core), use a mature scheduling library:

• Quartz.NET: Full-featured, enterprise-grade scheduler with cron support, retry policies, and job persistence.
• Hangfire: Simpler to set up, with a built-in dashboard for monitoring jobs.

Example Quartz.NET job to scan an FTP directory:

using Quartz;

public class FtpDirectoryScanJob : IJob
{
    private readonly IFtpService _ftpService; // Inject your FTP service

    public FtpDirectoryScanJob(IFtpService ftpService)
    {
        _ftpService = ftpService;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        // List files in the target FTP directory
        var files = await _ftpService.ListFilesAsync("/your-monitor-directory");
        
        foreach (var file in files)
        {
            // Run your post-processing logic (e.g., parse, sync to S3, notify)
            await ProcessFtpFileAsync(file);
            
            // Optional: Mark file as processed or delete it
            await _ftpService.MarkFileAsProcessedAsync(file);
        }
    }
}

// Register the job in your .NET app's startup
public static void AddQuartzSetup(IServiceCollection services)
{
    services.AddQuartz(q =>
    {
        var jobKey = new JobKey("FtpScanJob");
        q.AddJob<FtpDirectoryScanJob>(opts => opts.WithIdentity(jobKey));

        // Run every hour (adjust cron schedule as needed)
        q.AddTrigger(opts => opts
            .ForJob(jobKey)
            .WithIdentity("FtpScanTrigger")
            .WithCronSchedule("0 0 * ? * * *"));
    });

    // Host Quartz as a background service
    services.AddQuartzHostedService(q => q.WaitForJobsToComplete = true);
}

2. Serverless Monitoring with AWS EventBridge + Lambda #

For a fully managed, serverless approach:

• Use Amazon EventBridge to schedule a recurring trigger (e.g., every 30 minutes).
• The trigger runs a .NET Lambda function that connects to your FTP server, scans the directory, and processes files.

Key benefits: No servers to maintain, auto-scaling, and built-in logging via CloudWatch.

Add security by storing FTP credentials in AWS Secrets Manager (never hardcode them!). Here’s a snippet from the Lambda:

using Amazon.Lambda.Core;
using Amazon.SecretsManager;
using Amazon.SecretsManager.Model;
using FluentFTP;
using System.Text.Json;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace FtpMonitorLambda;

public class Function
{
    private readonly IAmazonSecretsManager _secretsManager;

    public Function()
    {
        _secretsManager = new AmazonSecretsManagerClient();
    }

    public async Task FunctionHandler(ILambdaContext context)
    {
        // Fetch FTP credentials from Secrets Manager
        var secretResponse = await _secretsManager.GetSecretValueAsync(new GetSecretValueRequest
        {
            SecretId = "prod/ftp-credentials"
        });

        var ftpCreds = JsonSerializer.Deserialize<FtpCredentials>(secretResponse.SecretString);

        using var ftpClient = new FtpClient(ftpCreds.Host, ftpCreds.Username, ftpCreds.Password);
        ftpClient.EnableSsl = true;
        await ftpClient.ConnectAsync();

        var directoryItems = await ftpClient.GetListingAsync("/monitor-dir");
        foreach (var item in directoryItems.Where(i => i.Type == FtpObjectType.File))
        {
            context.Logger.LogInformation($"Processing file: {item.Name}");
            // Add your processing logic here (e.g., upload to S3, send to a queue)
            await ProcessFileAsync(item, context);
        }

        await ftpClient.DisconnectAsync();
    }

    private async Task ProcessFileAsync(FtpListItem file, ILambdaContext context)
    {
        // Implement your processing workflow here
        // Example: Upload to S3 using Amazon.S3 library
    }
}

// Helper class to deserialize secrets
public class FtpCredentials
{
    public string Host { get; set; }
    public string Username { get; set; }
    public string Password { get; set; }
}

• Security First: Always use FTPS/SFTP (not unencrypted FTP) and store credentials in AWS Secrets Manager or .NET User Secrets (for non-production).
• Retry Logic: Use libraries like Polly to add retry policies for FTP connection failures or timeouts.
• Error Handling: For serverless workflows, use Amazon SQS to queue failed file processing jobs for later retries.
• Monitoring: Use Amazon CloudWatch to track Lambda execution, set up alerts for failures, and log detailed workflow steps.
• Idempotency: Ensure your processing logic is idempotent (can run multiple times without side effects) in case of retries.

内容的提问来源于stack exchange，提问作者ShaneKm