Azure Data Factory能否复制非亚马逊托管S3存储桶文件？

Great question! I’ve dealt with this exact scenario when working with non-AWS S3-compatible storage (like MinIO, DigitalOcean Spaces, or on-prem S3 clones) in Azure Data Factory. The good news is ADF does support configuring a custom host/endpoint for S3-compatible storage—it just isn’t always prominently highlighted in the basic documentation.

To set up your custom S3 storage, you need to add the serviceUrl property to the typeProperties section of your Amazon S3 Linked Service configuration. This property lets you specify the base URL of your non-AWS S3 endpoint.

Here’s the updated configuration with the custom endpoint included:

"name": "AmazonS3LinkedService",
"properties": {
    "type": "AmazonS3",
    "typeProperties": {
        "accessKeyId": "<access key id>",
        "secretAccessKey": {
            "type": "SecureString",
            "value": "<secret access key>"
        },
        "serviceUrl": "https://your-custom-s3-endpoint.com" // Add this line for your custom host
    },
    "connectVia": {
        "referenceName": "<name of Integration Runtime>",
        "type": "IntegrationRuntimeReference"
    }
}

A few key things to keep in mind:

• Integration Runtime Access: Make sure your chosen Integration Runtime (either Azure-hosted or self-hosted) can reach the custom S3 endpoint. If your storage is in a private network, you’ll need a self-hosted IR deployed in that network.
• Credential Compatibility: The accessKeyId and secretAccessKey must be valid credentials for your non-AWS S3 provider—these work the same way as they do for AWS S3, so follow your provider’s credential creation guidelines.
• Path Format: When setting up datasets, use the same bucket/path structure you’d use with standard S3; the serviceUrl will handle routing requests to your custom host.

内容的提问来源于stack exchange，提问作者Wilco