通过AWS Go SDK查询Delta表遇DELTA_LAKE_INVALID_SCHEMA报错求助

我在S3中存储了一张Delta表，元数据存储于Glue中。已为该表添加新字段并更新了Glue中的表结构，完成操作后通过Athena UI执行以下查询完全正常：

select status from deletes where request_id = '1234'

但使用AWS Golang SDK执行相同查询时，却抛出异常：

DELTA_LAKE_INVALID_SCHEMA: Metadata not found in transaction log for deletes

查询函数 #

func queryAthena(s *StorageAthena, query string) (string, error) {
    var input athena.StartQueryExecutionInput
    input.SetQueryString(query)

    var executionContext athena.QueryExecutionContext
    executionContext.SetDatabase(s.dbName)

    var resultConfig athena.ResultConfiguration
    resultConfig.SetOutputLocation(s.bucketName)

    input.SetQueryExecutionContext(&executionContext)
    input.SetResultConfiguration(&resultConfig)

    result, err := s.Db.StartQueryExecution(&input)

    if err != nil {
        log.Error("StartQueryExecution: Error executing the status query", err.Error())
        return "", err
    }

    var queryExecutionInput athena.GetQueryExecutionInput
    queryExecutionInput.SetQueryExecutionId(*result.QueryExecutionId)
    duration := time.Duration(5) * time.Second

    var queryState *athena.GetQueryExecutionOutput

    for {
        queryState, err = s.Db.GetQueryExecution(&queryExecutionInput)
        if err != nil {
            log.Error("GetQueryExecution: Error executing the status query", err.Error())
            return "", err
        }

        s := *queryState.QueryExecution.Status.State
        if s != "RUNNING" && s != "QUEUED" {
            break
        }

        time.Sleep(duration)
    }

    if *queryState.QueryExecution.Status.State == "SUCCEEDED" {
        var resultsInput athena.GetQueryResultsInput
        resultsInput.SetQueryExecutionId(*result.QueryExecutionId)

        rows, err := s.Db.GetQueryResults(&resultsInput)

        if err != nil {
            log.Error("GetQueryResults: Error executing the status query", err.Error())
            return "", err
        }

        resultRows := rows.ResultSet.Rows

        if len(resultRows) > 0 {
            return resultRows[0].String(), nil // Fetch first row as status query will return only 1 row.
        } else {
            return "", errors.New(fmt.Sprintf("Error while fetching rows. Fetched %d rows", len(resultRows)))
        }

    } else {
        return "", errors.New(fmt.Sprintf("ExcutionId : %s, State : %s. Reason : %s", *result.QueryExecutionId, *queryState.QueryExecution.Status.State, queryState.QueryExecution.Status.AthenaError.String()))
    }

}

StorageAthena结构体 #

type StorageAthena struct {
    Db         *athena.Athena //exported for testing
    bucketName string
    dbName     string
}

部署在EKS中的JSON-RPC服务主体已具备以下权限：

• Athena权限：

  "athena:BatchGet*",
  "athena:Get*",
  "athena:List*",
  "athena:CancelQueryExecution",
  "athena:StartQueryExecution",
  

• Glue权限：

  "glue:BatchGet*",
  "glue:Get*",
  "glue:List*",
  

• S3权限：

  "s3:Get*",
  "s3:List*",
  

已经排查了权限配置和Glue表结构更新，仍无法定位问题原因。请问除了Glue之外，还有哪些需要更新表结构/元数据的地方？

内容的提问来源于stack exchange，提问作者kaysush