如何用Cloud Run服务账号与GOOGLE_APPLICATION_CREDENTIALS本地运行Docker容器

问题概述 #

尝试本地运行连接Cloud SQL的Ktor应用Docker容器，计划后续部署到Cloud Run，采用IAM服务账号通过环境变量认证。已排查一周未解决，主要遇到两类错误：

• 直接执行docker run抛出java.lang.NullPointerException
• 使用docker-compose时抛出应用默认凭证（ADC）不可用错误：

  Caused by: java.io.IOException: The Application Default Credentials
  are not available. They are available if running in Google Compute
  Engine. Otherwise, the environment variable GOOGLE_APPLICATION_
  CREDENTIALS must be defined pointing to a file defining the
  credentials.
  

操作步骤 #

1. 创建IAM服务账号foo-cloudrun@foo-foo-121212.iam.gserviceaccount.com
2. 为账号配置角色：

   Cloud Build Service Agent
   Cloud Run Admin
   Cloud Run Service Agent
   Cloud SQL Client
   Cloud SQL Instance User
   Viewer
   

3. 生成服务账号密钥并保存至桌面
4. CMD中激活服务账号：

   gcloud auth activate-service-account --key-file="C:\\Users\\user\\Desktop\\cloudrun-key.json"
   

5. 设置环境变量：

   set GOOGLE_APPLICATION_CREDENTIALS="C:\\Users\\user\\Desktop\\cloudrun-key.json"
   

6. 构建Docker镜像：

   docker build -t my-application .
   

7. 启动容器：

   docker run -p 8080:8080 my-application
   

配置文件 #

Dockerfile #

FROM gradle:7-jdk11 AS build
COPY --chown=gradle:gradle . /home/gradle/src
WORKDIR /home/gradle/src
RUN gradle shadowJar --no-daemon 

FROM openjdk:11
EXPOSE 8080:8080
RUN mkdir /app
COPY --from=build /home/gradle/src/build/libs/*.jar /app/my-application-docker.jar
ENTRYPOINT  [ "java", "-jar", "/app/my-application-docker.jar" ]

docker-compose.yml #

services:
  web:
    build: .
    ports:
      - "8080:8080"
    environment:
      INSTANCE_CONNECTION_NAME: ${INSTANCE_CONNECTION_NAME}
      DB_IAM_USER: ${DB_IAM_USER}
      DB_USER: ${DB_USER}
      DB_NAME: ${DB_NAME}
      PORT: ${PORT}
      JDBC_DRIVER: ${JDBC_DRIVER}
      GOOGLE_APPLICATION_CREDENTIALS: ${GOOGLE_APPLICATION_CREDENTIALS}

.env文件 #

JDBC_DRIVER=org.postgresql.Driver
DB_IAM_USER=foo-cloudrun@foo-foo-121212.iam
DB_NAME=my-database
INSTANCE_CONNECTION_NAME=foo-foo-121212:us-central1:database-sql-instance
PORT=8080
GOOGLE_APPLICATION_CREDENTIALS=C:\\Users\\user\\Desktop\\cloudrun-key.json

DatabaseFactory.kt #

object DatabaseFactory {

    private val INSTANCE_CONNECTION_NAME = System.getenv("INSTANCE_CONNECTION_NAME");
    private val DB_IAM_USER = System.getenv("DB_IAM_USER")
    private val DB_NAME = System.getenv("DB_NAME")


    fun init() {
        Database.connect(hikari())

        transaction {
            SchemaUtils.create(FirstTable)
            SchemaUtils.create(SecondTable)
        }
    }


    private fun hikari(): HikariDataSource {

        val config = HikariConfig()

        // IAM AUTHENTICATION
        config.driverClassName = System.getenv("JDBC_DRIVER")
        config.jdbcUrl = String.format("jdbc:postgresql:///%s", DB_NAME)
        config.addDataSourceProperty("socketFactory", "com.google.cloud.sql.postgres.SocketFactory")
        config.addDataSourceProperty("cloudSqlInstance", INSTANCE_CONNECTION_NAME)
        config.addDataSourceProperty("enableIamAuth", "true")
        config.addDataSourceProperty("user", DB_IAM_USER)
        config.addDataSourceProperty("password", "password")
        config.addDataSourceProperty("sslmode", "disable")

        config.connectionTimeout = 10000 // 10s
        config.maximumPoolSize = 5
        config.minimumIdle = 3
        config.maxLifetime = 1800000; // 30 minutes
        config.isAutoCommit = false
        config.transactionIsolation = "TRANSACTION_REPEATABLE_READ"
        config.validate()

        return HikariDataSource(config);
    }

    suspend fun <T> dbQuery(block: () -> T): T =
        withContext(Dispatchers.IO) {
            transaction {
                addLogger(StdOutSqlLogger)
                block()
            }
        }
}

已尝试但无效的方法 #

1. 挂载凭证文件到容器：

   docker run -p 8080:8080 -v ${GOOGLE_APPLICATION_CREDENTIALS}:/tmp/keys/cloudrun-key.json:ro my-application-api
   docker run -p 8080:8080 -e GOOGLE_APPLICATION_CREDENTIALS=/tmp/keys/cloudrun-key.json -v ${GOOGLE_APPLICATION_CREDENTIALS}:/tmp/keys/cloudrun-key.json:ro my-application-api
   

2. 使用个人Gmail账号认证：
   • 清空GOOGLE_APPLICATION_CREDENTIALS环境变量
   • 执行gcloud auth login和gcloud auth application-default login
   • 启动容器仍报错文件不存在
3. 在Windows系统环境变量中设置GOOGLE_APPLICATION_CREDENTIALS，问题依旧

解决方案 #

1. 修复容器内凭证文件映射问题 #

Windows本地路径无法直接在Docker容器内识别，需将本地密钥文件挂载到容器内部路径，并设置容器内的环境变量指向该路径：

修改docker-compose.yml：

services:
  web:
    build: .
    ports:
      - "8080:8080"
    environment:
      INSTANCE_CONNECTION_NAME: ${INSTANCE_CONNECTION_NAME}
      DB_IAM_USER: ${DB_IAM_USER}
      DB_USER: ${DB_USER}
      DB_NAME: ${DB_NAME}
      PORT: ${PORT}
      JDBC_DRIVER: ${JDBC_DRIVER}
      GOOGLE_APPLICATION_CREDENTIALS: "/app/keys/cloudrun-key.json"  # 容器内的路径
    volumes:
      - "C:\\Users\\user\\Desktop\\cloudrun-key.json:/app/keys/cloudrun-key.json:ro"  # 本地路径映射到容器内

同时删除.env中的GOOGLE_APPLICATION_CREDENTIALS配置项，避免冲突。

2. 修复空指针异常（NPE） #

原代码未对环境变量做空值校验，添加校验确保变量存在：

object DatabaseFactory {

    private val INSTANCE_CONNECTION_NAME = System.getenv("INSTANCE_CONNECTION_NAME") 
        ?: error("INSTANCE_CONNECTION_NAME environment variable not set")
    private val DB_IAM_USER = System.getenv("DB_IAM_USER")
        ?: error("DB_IAM_USER environment variable not set")
    private val DB_NAME = System.getenv("DB_NAME")
        ?: error("DB_NAME environment variable not set")
    private val JDBC_DRIVER = System.getenv("JDBC_DRIVER")
        ?: error("JDBC_DRIVER environment variable not set")


    fun init() {
        Database.connect(hikari())

        transaction {
            SchemaUtils.create(FirstTable)
            SchemaUtils.create(SecondTable)
        }
    }


    private fun hikari(): HikariDataSource {

        val config = HikariConfig()

        // IAM AUTHENTICATION
        config.driverClassName = JDBC_DRIVER
        config.jdbcUrl = String.format("jdbc:postgresql:///%s", DB_NAME)
        config.addDataSourceProperty("socketFactory", "com.google.cloud.sql.postgres.SocketFactory")
        config.addDataSourceProperty("cloudSqlInstance", INSTANCE_CONNECTION_NAME)
        config.addDataSourceProperty("enableIamAuth", "true")
        config.addDataSourceProperty("user", DB_IAM_USER)
        config.addDataSourceProperty("password", "")  // IAM认证时密码留空，SocketFactory会自动生成临时密码
        config.addDataSourceProperty("sslmode", "disable")

        config.connectionTimeout = 10000 // 10s
        config.maximumPoolSize = 5
        config.minimumIdle = 3
        config.maxLifetime = 1800000; // 30 minutes
        config.isAutoCommit = false
        config.transactionIsolation = "TRANSACTION_REPEATABLE_READ"
        config.validate()

        return HikariDataSource(config);
    }

    // 其他代码不变
}

注意：启用IAM认证时，password字段可设为空字符串，Cloud SQL Socket Factory会自动使用IAM凭证生成临时密码，无需手动填写固定密码。

3. 修正服务账号邮箱格式 #

原.env中DB_IAM_USER仅填写foo-cloudrun@foo-foo-121212.iam，缺少.gserviceaccount.com后缀，需修改为完整的服务账号邮箱：

DB_IAM_USER=foo-cloudrun@foo-foo-121212.iam.gserviceaccount.com

4. 本地Docker运行的正确命令 #

若不用docker-compose，直接执行docker run需明确挂载密钥并传递所有必要环境变量：

docker run -p 8080:8080 ^
  -e INSTANCE_CONNECTION_NAME="foo-foo-121212:us-central1:database-sql-instance" ^
  -e DB_IAM_USER="foo-cloudrun@foo-foo-121212.iam.gserviceaccount.com" ^
  -e DB_NAME="my-database" ^
  -e JDBC_DRIVER="org.postgresql.Driver" ^
  -e GOOGLE_APPLICATION_CREDENTIALS="/app/keys/cloudrun-key.json" ^
  -v "C:\\Users\\user\\Desktop\\cloudrun-key.json:/app/keys/cloudrun-key.json:ro" ^
  my-application

5. 验证Cloud SQL IAM权限 #

确保服务账号已被添加为Cloud SQL实例的IAM用户：

• 进入Cloud SQL实例详情页 → 权限 → 添加用户账号
• 选择"IAM用户"，填写完整的服务账号邮箱，设置角色为Cloud SQL Client

内容的提问来源于stack exchange，提问作者Theseus