Bitbucket流水线部署GCP App Engine遇gcloud Unicode解码错误求助

问题背景 #

此前正常运行的Bitbucket流水线，在将微服务部署至GCP App Engine时突然失败，执行命令为mvn gcloud deploy。代码无变更，路径未修改，甚至部署过往成功的构建包也出现相同错误，DEV、SIT、UAT环境均受影响。已知Bamboo服务器新增安装了Python 3.9版本，保留原有Python 2.7，流水线正常运行一周后出现该问题。

核心报错日志 #

build   29-Jun-2023 13:12:41    org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.appengine:gcloud-maven-plugin:2.0.9.133.v201611104:deploy (default-cli) on project egsp-sga-admin: Error: gcloud app command exit code is: 1
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:51)
build   29-Jun-2023 13:12:41        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
build   29-Jun-2023 13:12:41        at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:309)

同时伴随编码错误：
ERROR: gcloud crashed (UnicodeDecodeError): 'ascii' codec can't decode byte 0xe2 in position 1438: ordinal not in range 128

POM.xml相关配置 #

gcloud-maven-plugin #

<plugin>
    <groupId>com.google.appengine</groupId>
    <artifactId>gcloud-maven-plugin</artifactId>
    <version>2.0.9.133.v201611104</version>
    <configuration>
        <gcloud_directory>/usr/lib64/google-cloud-sdk/</gcloud_directory>
        <verbosity>info</verbosity>
        <promote>true</promote>
    </configuration>
</plugin>

appengine-maven-plugin #

<plugin>
    <groupId>com.google.appengine</groupId>
    <artifactId>appengine-maven-plugin</artifactId>
    <version>${gae.version}</version>
    <configuration>
        <enableJarClasses>false</enableJarClasses>
        <version>${app.version}</version>
        <address>0.0.0.0</address>
        <jvmFlags>
            <jvmFlag>-Xdebug</jvmFlag>
            <jvmFlag>-agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n</jvmFlag>
        </jvmFlags>
    </configuration>
</plugin>

服务器环境信息 #

GCP SDK版本 #

[root@bambooagent-4 ~]# gcloud version
Google Cloud SDK 287.0.0
alpha 2020.03.30
app-engine-java 1.9.79
app-engine-python 1.9.89
beta 2020.03.30
bq 2.0.56
cloud-datastore-emulator 2.1.0
core 2020.03.30
gsutil 4.49
kubectl 2020.03.30

网络诊断结果 #

[root@bambooagent-4 ~]# gcloud info --run-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.                                                                                                   
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).

Property diagnostic detects issues that may be caused by properties.
Checking hidden properties...done.                                                                                                    
Hidden Property Check passed.
Property diagnostic passed (1/1 checks passed).

详细环境信息 #

[root@bambooagent-4 ~]# gcloud info
Google Cloud SDK [287.0.0]

Platform: [Linux, x86_64] ('Linux', 'bambooagent-4', '3.10.0-1062.9.1.el7.x86_64', '#1 SMP Mon Dec 2 08:31:54 EST 2019', 'x86_64', 'x86_64')
Locale: ('en_GB', 'UTF-8')
Python Version: [2.7.5 (default, Jun 11 2019, 14:33:56)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]]
Python Location: [/bin/python2]
Site Packages: [Disabled]

Installation Root: [/usr/lib64/google-cloud-sdk]
Installed Components:
  core: [2020.03.30]
  app-engine-python: [1.9.89]
  kubectl: [2020.03.30]
  app-engine-java: [1.9.79]
  beta: [2020.03.30]
  gsutil: [4.49]
  bq: [2.0.56]
  alpha: [2020.03.30]
  cloud-datastore-emulator: [2.1.0]
System PATH: [/usr/local/rbenv/shims:/usr/local/rbenv/bin:/opt/gradle/gradle-7.2/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/bin]
Python PATH: [/usr/lib64/google-cloud-sdk/lib/third_party:/usr/lib64/google-cloud-sdk/lib:/usr/lib64/python27.zip:/usr/lib64/python2.7/:/usr/lib64/python2.7/plat-linux2:/usr/lib64/python2.7/lib-tk:/usr/lib64/python2.7/lib-old:/usr/lib64/python2.7/lib-dynload]
Cloud SDK on PATH: [False]
Kubectl on PATH: [/bin/kubectl]

Installation Properties: [/usr/lib64/google-cloud-sdk/properties]
User Config Directory: [/root/.config/gcloud]
Active Configuration Name: [default]
Active Configuration Path: [/root/.config/gcloud/configurations/config_default]

解决方案建议 #

• 强制指定Python 2.7路径：新增的Python 3.9可能干扰了gcloud的默认Python调用，在执行mvn gcloud deploy前设置环境变量：

  export CLOUDSDK_PYTHON=/bin/python2
  

  可以直接在Bamboo任务的环境变量配置中添加，或者写在流水线的前置脚本里。

• 升级gcloud SDK：当前使用的2020年版本SDK存在Unicode解码bug的概率极高，升级到最新稳定版本修复问题：

  gcloud components update
  

  升级前记得备份现有SDK配置，避免意外。

• 修复编码环境：Python 2.7默认使用ASCII编码，遇到UTF-8字符会触发解码错误，在流水线脚本中添加编码配置：

  export LC_ALL=en_GB.UTF-8
  export LANG=en_GB.UTF-8
  

• 检查gcloud配置文件：打开/root/.config/gcloud/configurations/config_default，确认文件中没有非ASCII字符，或者确保文件编码为UTF-8。

• 替换老旧插件：当前使用的gcloud-maven-plugin是2016年的废弃版本，换成官方维护的最新appengine-maven-plugin，改用mvn appengine:deploy命令部署，兼容性更强。

内容的提问来源于stack exchange，提问作者user22154619