无法创建并推送Iceberg表至S3的问题求助

环境概述 #

基于EKS集群与RDS PostgreSQL搭建环境，EKS内创建new-trino命名空间，部署了服务账号、Iceberg REST实例与Trino集群，目标是利用PostgreSQL数据创建Iceberg表并推送至S3。

相关配置 #

service-account.yaml #

apiVersion: v1
kind: ServiceAccount
metadata:
  name: trino-service-account
  namespace: new-trino
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::385346727387:role/EKS-Trino-Iceberg-Access"

iceberg-rest.yaml #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: iceberg-rest
  namespace: new-trino
spec:
  replicas: 1
  selector:
    matchLabels:
      app: iceberg-rest
  template:
    metadata:
      labels:
        app: iceberg-rest
    spec:
      serviceAccountName: trino-service-account
      containers:
      - name: iceberg-rest
        image: tabulario/iceberg-rest:0.6.0
        ports:
        - containerPort: 8181
        env:
        - name: AWS_REGION
          value: "us-west-2"
        - name: CATALOG_WAREHOUSE
          value: "s3://my-iceberg-bucket/iceberg-warehouse"
        - name: AWS_WEB_IDENTITY_TOKEN_FILE
          value: "/var/run/secrets/eks.amazonaws.com/serviceaccount/token"
        - name: AWS_ROLE_ARN
          value: "arn:aws:iam::3385346727387::role/EKS-Trino-Iceberg-Access"
        volumeMounts:
        - name: aws-iam-token
          mountPath: /var/run/secrets/eks.amazonaws.com/serviceaccount
          readOnly: true
        resources:
          limits:
            memory: "2Gi"
            cpu: "1"
          requests:
            memory: "1Gi"
            cpu: "500m"
      volumes:
      - name: aws-iam-token
        projected:
          sources:
          - serviceAccountToken:
              audience: "sts.amazonaws.com"
              expirationSeconds: 86400
              path: token
---
apiVersion: v1
kind: Service
metadata:
  name: iceberg-rest
  namespace: new-trino
spec:
  selector:
    app: iceberg-rest
  ports:
  - port: 8181
    targetPort: 8181
  type: ClusterIP

trino-values.yaml #

image:
  tag: "463"
  pullPolicy: IfNotPresent
server:
  workers: 2
  config:
    properties:
      "iceberg.s3.endpoint": "s3.us-west-2.amazonaws.com"
      "iceberg.s3.region": "us-west-2"
      "iceberg.aws.credentials-provider": "InstanceProfile"
      "iceberg.fs.s3.path-style-access": "true"
serviceAccount:
  create: false
  name: trino-service-account
coordinator:
  service:
    type: LoadBalancer
    port: 8080
  jvm:
    maxHeapSize: "3G"
  resources:
    limits:
      memory: "4Gi"
      cpu: "2"
    requests:
      memory: "2Gi"
      cpu: "1"
worker:
  jvm:
    maxHeapSize: "3G"
  resources:
    limits:
      memory: "4Gi"
      cpu: "2"
    requests:
      memory: "2Gi"
      cpu: "1"
additionalExchangeManagerProperties:
  - exchange.s3.region=us-west-2
  - exchange.s3.endpoint=s3.us-west-2.amazonaws.com
  
catalogs:
  postgresql: |-
    connector.name=postgresql
    connection-url=jdbc:postgresql://mit-read-replica.c7battky0i8b.us-west-2.rds.amazonaws.com:5432/dev
    connection-user=user
    connection-password=password
  iceberg: |-
    connector.name=iceberg
    iceberg.catalog.type=rest
    iceberg.rest-catalog.uri=http://iceberg-rest.new-trino.svc.cluster.local:8181
    iceberg.file-format=PARQUET
    iceberg.compression-codec=SNAPPY

错误信息 #

容器运行正常，可访问Trino CLI，但执行创建Iceberg表语句时失败：

trino> CREATE TABLE iceberg.test.change_log_simple (
    ->     id integer,
    ->     organization_id integer,
    ->     change_date timestamp,
    ->     resource_type varchar,
    ->     severity varchar
    -> ) WITH (
    ->     format = 'PARQUET',
    ->     partitioning = ARRAY['year(change_date)'], 
    ->     location = 's3://my-iceberg-trino-bucket/iceberg-warehouse/test/change_log_simple'
    -> );
Query 20241106_104052_00012_ue4ps failed: No factory for location: s3://my-iceberg-trino-bucket/iceberg-warehouse/test/change_log_simple

已验证内容 #

• 已创建对应VPC端点及路由表
• EKS-Trino-Iceberg-Access IAM角色拥有my-iceberg-trino-bucket的完全访问权限，信任实体配置如下：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::12345678:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/C7EDBFBFC7F70E4FE"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.us-west-2.amazonaws.com/id/C7EDBFBFC7F70E4FE:sub": "system:serviceaccount:new-trino:trino-service-account"
                }
            }
        }
    ]
}

• 测试Pod可正常访问S3：

kubectl exec -it s3-test-pod -n new-trino -- bash

bash-4.2# yum install -y aws-cli

Complete!
bash-4.2# aws --version
aws-cli/1.18.147 Python/2.7.18 Linux/5.10.225-213.878.amzn2.x86_64 botocore/1.18.6
bash-4.2# aws s3 ls s3://my-iceberg-bucket/ --region us-west-2
                           PRE iceberg-warehouse/

求助需求 #

无法定位问题根源，请求提供排查方向或解决方案。

内容的提问来源于stack exchange，提问作者Raj