无法创建关联AWS的Databricks工作区：存储配置验证失败

创建关联AWS的Databricks工作区时，最后一步失败，报错信息如下：

MALFORMED_REQUEST: Failed storage configuration validation checks: List,Put,PutWithBucketOwnerFullControl,Delete

已在S3策略、Databricks角色权限及信任关系中配置了所需权限，但仍出现该错误。

S3策略 #

{
"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::account-number:role/databricks-access"
        },
        "Action": [
            "s3:ListBucket",
            "s3:GetBucketLocation"
        ],
        "Resource": "arn:aws:s3:::learning-databricks"
    },
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::account-number:role/databricks-access"
        },
        "Action": [
            "s3:PutObject",
            "s3:GetObject",
            "s3:DeleteObject"
        ],
        "Resource": "arn:aws:s3:::learning-databricks/*"
    },
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::account-number:role/databricks-access"
        },
        "Action": "s3:PutObjectAcl",
        "Resource": "arn:aws:s3:::learning-databricks/*",
        "Condition": {
            "StringEquals": {
                "s3:x-amz-acl": "bucket-owner-full-control"
            }
        }
    },
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::414351767826:root"
        },
        "Action": "s3:*",
        "Resource": [
            "arn:aws:s3:::learning-databricks",
            "arn:aws:s3:::learning-databricks/*"
        ],
        "Condition": {
            "StringEquals": {
                "s3:x-amz-acl": "bucket-owner-full-control"
            }
        }
    }
]
}

Databricks角色权限 #

{
"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "ec2:DescribeAvailabilityZones",
            "ec2:DescribeInstances",
            "ec2:DescribeInstanceStatus",
            "ec2:DescribeRouteTables",
            "ec2:DescribeSecurityGroups",
            "ec2:DescribeSubnets",
            "ec2:DescribeVolumes",
            "ec2:DescribeVpcs",
            "ec2:CreateInternetGateway",
            "ec2:CreateVPC",
            "ec2:DeleteVPC",
            "ec2:AllocateAddress",
            "ec2:ReleaseAddress",
            "ec2:DescribeNatGateways",
            "ec2:DeleteNatGateway",
            "ec2:DeleteVpcEndpoints",
            "ec2:CreateRouteTable",
            "ec2:DisassociateRouteTable",
            "ec2:CreateSecurityGroup",
            "ec2:AuthorizeSecurityGroupIngress",
            "ec2:AuthorizeSecurityGroupEgress",
            "ec2:CreateSubnet",
            "ec2:DeleteSubnet",
            "ec2:CreateNetworkInterface",
            "ec2:DeleteNetworkInterface",
            "ec2:AttachNetworkInterface",
            "ec2:DetachNetworkInterface",
            "ec2:ModifyNetworkInterfaceAttribute",
            "iam:PassRole",
            "ec2:RunInstances",
            "ec2:StopInstances",
            "ec2:TerminateInstances",
            "ec2:StartInstances"
        ],
        "Resource": "*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:*"
        ],
        "Resource": [
            "arn:aws:s3:::learning-databricks",
            "arn:aws:s3:::learning-databricks/*"
        ],
        "Condition": {
            "StringEquals": {
                "s3:x-amz-acl": "bucket-owner-full-control"
            }
        }
    }
]
}

Databricks角色信任关系 #

{
"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::414351767826:root"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
            "StringEquals": {
                "sts:ExternalId": "External-uuid"
            }
        }
    },
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::aws-account:root"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
            "StringEquals": {
                "sts:ExternalId": "External-uuid"
            }
        }
    },
    {
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::aws-account:role/databricks-access"
        },
        "Action": "sts:AssumeRole",
        "Condition": {
            "StringEquals": {
                "sts:ExternalId": "External-uuid"
            }
        }
    }
]
}

问题排查与解决方案 #

1. 移除S3权限的条件限制 #

当前Databricks角色的S3权限被强制要求携带s3:x-amz-acl=bucket-owner-full-control条件，但Databricks在验证存储配置时不会自动添加该ACL头，导致权限校验失败。

修改Databricks角色的S3权限语句，删除条件限制：

{
    "Effect": "Allow",
    "Action": [
        "s3:GetBucketLocation",
        "s3:ListBucket"
    ],
    "Resource": "arn:aws:s3:::learning-databricks"
},
{
    "Effect": "Allow",
    "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:PutObjectAcl"
    ],
    "Resource": "arn:aws:s3:::learning-databricks/*"
}

同时，S3桶策略中给databricks-access角色的权限也无需额外条件，保持基础权限即可。

2. 修正信任关系中的账户ID #

信任关系里的aws-account占位符必须替换为你的实际AWS账户ID，确保Databricks服务能正确承担该角色。

3. 检查S3桶的访问设置 #

• 确认S3桶归当前AWS账户所有，无跨账户所有权冲突；
• 检查桶的公共访问设置，确保没有阻止内部角色的访问（无需开启公共访问，只需避免强制的访问阻止规则）。

4. 手动验证权限 #

用AWS CLI模拟Databricks角色执行操作，确认权限生效：

# 承担databricks-access角色
aws sts assume-role --role-arn arn:aws:iam::你的账户ID:role/databricks-access --role-session-name test-databricks

# 使用返回的临时凭证配置CLI后，执行测试
aws s3 ls s3://learning-databricks
aws s3 cp test.txt s3://learning-databricks/test.txt
aws s3 rm s3://learning-databricks/test.txt

如果测试失败，根据报错信息进一步定位权限问题。

内容的提问来源于stack exchange，提问作者Priyanka Gupta