使用MinIO+Iceberg REST+Trino创建Iceberg表失败求配置方案

问题背景 #

通过Docker Compose搭建了包含MinIO、Iceberg REST Catalog、Trino、Superset的数据湖环境，Schema创建正常，但执行建表SQL时触发AWS凭证加载失败错误，尝试在Trino的iceberg.properties中添加S3配置后导致Trino容器无法启动。

当前配置 #

docker-compose.yml（关键部分） #

iceberg-rest:
    image: tabulario/iceberg-rest:1.5.0
    container_name: iceberg-rest
    ports:
      - "8181:8181"
    environment:
      - CATALOG_WAREHOUSE=s3://icebergdata/warehouse
      - CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO
      - S3_ENDPOINT=http://minio:9000
      - AWS_ACCESS_KEY_ID=admin
      - AWS_SECRET_ACCESS_KEY=admin123
      - AWS_REGION=us-east-1
      - AWS_S3_PATH_STYLE_ACCESS=true
      #---------------------
      #- S3_PATH_STYLE_ACCESS=true
      - CATALOG_S3_ENDPOINT=http://minio:9000
      - CATALOG_S3_ACCESS_KEY_ID=admin
      - CATALOG_S3_SECRET_ACCESS_KEY=admin123
      - CATALOG_S3_PATH_STYLE_ACCESS=true
    depends_on:
      - minio

trino/catalog/iceberg.properties #

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://iceberg-rest:8181

# IO implementation
iceberg.file-format=parquet

# --- S3 settings for MinIO ---
# iceberg.s3.endpoint=http://minio:9000
# iceberg.s3.region=us-east-1
# iceberg.s3.access-key=admin
# iceberg.s3.secret-key=admin123
# iceberg.s3.path-style-access=true

报错信息 #

Query 20250917_060551_00001_k6u6c failed: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@5e38db07: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@29828a29: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]

问题原因 #

1. Iceberg REST Catalog服务的环境变量存在冗余配置（同时使用AWS_前缀和CATALOG_S3_前缀），导致凭证加载逻辑冲突
2. 当使用REST Catalog模式时，Trino不需要直接配置S3凭证（表的IO操作由Iceberg REST Catalog服务处理），强行添加S3配置会让Trino连接器尝试直接访问S3，引发启动失败

解决方案 #

1. 修正Iceberg REST Catalog的环境变量 #

保留CATALOG_前缀的配置（这是tabulario/iceberg-rest镜像的标准配置方式），移除冗余的AWS_前缀变量：

iceberg-rest:
    image: tabulario/iceberg-rest:1.5.0
    container_name: iceberg-rest
    ports:
      - "8181:8181"
    environment:
      - CATALOG_WAREHOUSE=s3://icebergdata/warehouse
      - CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO
      # 统一使用CATALOG_S3_前缀的配置
      - CATALOG_S3_ENDPOINT=http://minio:9000
      - CATALOG_S3_ACCESS_KEY_ID=admin
      - CATALOG_S3_SECRET_ACCESS_KEY=admin123
      - CATALOG_S3_REGION=us-east-1
      - CATALOG_S3_PATH_STYLE_ACCESS=true
    depends_on:
      - minio

2. 保持Trino的iceberg.properties配置不变 #

不需要启用任何S3相关配置，因为Trino仅通过REST API与Iceberg Catalog通信，IO操作由Catalog服务完成：

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=http://iceberg-rest:8181

# IO implementation
iceberg.file-format=parquet

# 不要启用下面的S3配置
# --- S3 settings for MinIO ---
# iceberg.s3.endpoint=http://minio:9000
# iceberg.s3.region=us-east-1
# iceberg.s3.access-key=admin
# iceberg.s3.secret-key=admin123
# iceberg.s3.path-style-access=true

3. 重启服务 #

执行以下命令重新部署：

docker-compose down
docker-compose up -d

4. 验证建表 #

进入Trino CLI执行建表语句：

CREATE TABLE iceberg.demo.sales (
    id INT,
    product VARCHAR,
    amount DOUBLE
);

建表成功后，可通过MinIO控制台查看icebergdata/warehouse/demo/sales路径下生成的Iceberg表数据文件。

关键说明 #

• 使用Iceberg REST Catalog时，所有S3相关配置都应该放在Catalog服务端，Trino仅负责与Catalog的REST API交互
• tabulario/iceberg-rest镜像的配置参数需要使用CATALOG_前缀，直接使用AWS_前缀的环境变量可能无法被正确识别
• 若Trino配置了iceberg.s3.*参数，连接器会切换为直接访问S3模式，此时需要确保Trino容器内包含AWS SDK依赖，且参数配置正确，否则会启动失败

内容的提问来源于stack exchange，提问作者Jisson