GKE部署应用上传Google Cloud Storage遇Stream is closed错误求助

问题详情 #

使用Google Cloud Storage服务数月，2023年1月30日突发异常：本地环境通过IAM服务账号JSON密钥配置凭证后运行正常，但部署到GKE生产环境后，上传图片至GCS时出现如下错误：

Caused by: java.io.IOException: Error getting access token for service account: Stream is closed, iss: refund@pg-user.iam.gserviceaccount.com
    at com.google.auth.oauth2.ServiceAccountCredentials.refreshAccessToken(ServiceAccountCredentials.java:605) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.auth.oauth2.OAuth2Credentials$1.call(OAuth2Credentials.java:257) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.auth.oauth2.OAuth2Credentials$1.call(OAuth2Credentials.java:254) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[na:na]
    at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) ~[guava-31.0.1-jre.jar!/:na]
    at com.google.auth.oauth2.OAuth2Credentials$AsyncRefreshResult.executeIfNew(OAuth2Credentials.java:580) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.auth.oauth2.OAuth2Credentials.asyncFetch(OAuth2Credentials.java:220) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.auth.oauth2.OAuth2Credentials.getRequestMetadata(OAuth2Credentials.java:170) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.auth.oauth2.ServiceAccountCredentials.getRequestMetadata(ServiceAccountCredentials.java:1018) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.auth.http.HttpCredentialsAdapter.initialize(HttpCredentialsAdapter.java:96) ~[google-auth-library-oauth2-http-1.4.0.jar!/:na]
    at com.google.cloud.http.HttpTransportOptions$1.initialize(HttpTransportOptions.java:159) ~[google-cloud-core-http-2.4.0.jar!/:2.4.0]
    at com.google.cloud.http.CensusHttpModule$CensusHttpRequestInitializer.initialize(CensusHttpModule.java:109) ~[google-cloud-core-http-2.4.0.jar!/:2.4.0]
    at com.google.api.client.http.HttpRequestFactory.buildRequest(HttpRequestFactory.java:91) ~[google-http-client-1.41.2.jar!/:1.41.2]
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:521) ~[google-api-client-1.33.1.jar!/:1.33.1]
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455) ~[google-api-client-1.33.1.jar!/:1.33.1]
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565) ~[google-api-client-1.33.1.jar!/:1.33.1]
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:311) ~[google-cloud-storage-2.4.4.jar!/:2.4.4]
    at com.google.cloud.storage.StorageImpl.lambda$internalCreate$2(StorageImpl.java:198) ~[google-cloud-storage-2.4.4.jar!/:2.4.4]

已尝试更换服务账号JSON密钥、更换集群，问题仍未解决。

相关代码 #

凭证初始化代码 #

Credentials credentials = GoogleCredentials.fromStream(new ByteArrayInputStream(Base64.getDecoder().decode(base64Credentials)))
                .createScoped(Collections.singletonList(StorageScopes.DEVSTORAGE_FULL_CONTROL));

return StorageOptions.newBuilder().setCredentials(credentials)
        .setTransportOptions(HttpTransportOptions.newBuilder().
                setConnectTimeout(300000)
                .build())
        .setProjectId(projectId).build().getService();

GCS上传代码 #

storage.create(BlobInfo
                .newBuilder(cloudStorageProperties.getBucketName(), filename)
                .setAcl(listAcl)
                .setContentType(FileUtils.getMimeType(extension))
                .build(), content);

Gradle依赖 #

implementation platform('com.google.cloud:libraries-bom:24.4.0')
implementation 'com.google.cloud:google-cloud-storage'

解决方案建议 #

1. 修复凭证流复用问题 #

GoogleCredentials.fromStream会关闭传入的InputStream，如果后续有重试、多线程场景尝试重新读取凭证，就会触发「Stream is closed」错误。修改方式：

• 缓存Base64解码后的字节数组，每次创建新的ByteArrayInputStream传入：

// 提前解码并缓存，避免重复Base64解码
private final byte[] decodedCredentials = Base64.getDecoder().decode(base64Credentials);

public Storage getStorage() {
    Credentials credentials = GoogleCredentials.fromStream(new ByteArrayInputStream(decodedCredentials))
            .createScoped(Collections.singletonList(StorageScopes.DEVSTORAGE_FULL_CONTROL));

    return StorageOptions.newBuilder().setCredentials(credentials)
            .setTransportOptions(HttpTransportOptions.newBuilder()
                    .setConnectTimeout(300000)
                    .build())
            .setProjectId(projectId).build().getService();
}

• 更优方案：将Storage实例设为单例，避免重复初始化凭证和流操作。

2. 改用GKE工作负载身份（Workload Identity） #

JSON密钥存在安全风险，GKE环境下优先使用Workload Identity：

• 创建Kubernetes服务账号（KSA）
• 将GCP服务账号（GSA）绑定到该KSA，授予GSA对目标GCS桶的读写权限
• 在Deployment/YAML中指定serviceAccountName为该KSA
• 代码无需手动加载JSON密钥，SDK会自动从环境获取凭证，彻底规避流相关问题。

3. 升级依赖版本 #

当前使用的libraries-bom 24.4.0对应的auth库版本可能存在已知bug，尝试升级到最新稳定版：

implementation platform('com.google.cloud:libraries-bom:26.26.0') // 示例最新稳定版
implementation 'com.google.cloud:google-cloud-storage'

4. 排查GKE网络限制 #

• 确认GKE集群节点能访问metadata.google.internal（元数据服务器）和storage.googleapis.com
• 检查集群防火墙规则是否允许 outbound 流量到上述地址
• 若使用私有集群，确认已启用私有访问配置。

内容的提问来源于stack exchange，提问作者Ario Bintang