Kerberos环境下Spark连接GreenPlum误用GP用户执行kinit问题

问题现象 #

普通Java JDBC应用可正常连接GreenPlum，但Spark应用连接失败，核心问题是Spark尝试使用<GP_USER>而非正确的<HDP_PRINCIPAL>进行Kerberos认证。

普通JDBC成功案例 #

代码实现 #

try (Connection conn = DriverManager.getConnection(url, user, pwd)) {
    ResultSet resultSet = conn.prepareStatement("select 1").executeQuery();
    resultSet.next();
    resultSet.getInt(1);
    System.out.println("Connection successful.");
} catch (SQLException e) {
    e.printStackTrace();
}

启动命令 #

java -jar -Djava.security.auth.login.config=pgjdbc.conf -Dsun.security.jgss.debug=true ./test-jdbc-connection-1.0-SNAPSHOT.jar param:url=<URL> param:user=<GP_USER>

成功日志 #

Debug is  true storeKey false useTicketCache true useKeyTab false doNotPrompt true ticketCache is null isInitiator true KeyTab is null refreshKrb5Config is false principal is null tryFirstPass is false useFirstPass is false storePass is false clearPass is false
Acquire TGT from Cache
Principal is <HDP_PRINCIPAL>
Commit Succeeded

Search Subject for Kerberos V5 INIT cred (<HDP_PRINCIPAL>, sun.security.jgss.krb5.Krb5InitCredential)
Connection successful.

Spark连接失败案例 #

代码实现 #

ss = SparkSession.builder().getOrCreate();
ss.read().format("jdbc")
        .option("url", url)
        .option("user", user)
        .option("dbschema", schema)
        .option("dbtable", table)
        .load()
        .show();

启动脚本 #

export SPARK_MAJOR_VERSION=3
spark-submit \
        --name "test_GP_connect" \
        --master yarn \
        --deploy-mode "client" \
        --queue <MY_YARN_QUEUE> \
        --conf ..... \
        ........
        --conf spark.driver.extraJavaOptions="-Dsun.security.jgss.native=true -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.jgss.debug=true -Dsun.security.krb5.debug=true -Djava.security.auth.login.config=pgjdbc.conf " \
        --conf spark.executor.extraJavaOptions="-Dsun.security.jgss.native=true -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.jgss.debug=true -Dsun.security.krb5.debug=true -Djava.security.auth.login.config=pgjdbc.conf " \
        --num-executors 1 \
        --executor-memory 20g \
        --driver-memory 5g \
        --files pgjdbc.conf \
        --jars ./gsp.jar \
        --class ru.pal.testsparkjdbcconn.Main \
        test-spark-jdbc-conn-3.0.jar \
        param:url=<URL> \
        param:user=<GP_USER> \
        param:schema=<GP_SCHEMA> \
        param:table=<GP_TABLE>

失败日志 #

Debug is  true storeKey false useTicketCache true useKeyTab false doNotPrompt true ticketCache is null isInitiator false KeyTab is null refreshKrb5Config is false principal is null tryFirstPass is false useFirstPass is false storePass is false clearPass is false
Acquire TGT from Cache
......................
get normal credential
Principal is <HDP_PRINCIPAL>
Commit Succeeded 

Search Subject for Kerberos V5 INIT cred (<GP_USER>, sun.security.jgss.wrapper.GSSCredElement)

异常信息 #

Exception in thread "main" com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: GSS Authentication failed
        ......
        at ru.pal.testsparkjdbcconn.Main.main(Main.java:85)
        ......
Caused by: org.postgresql.util.PSQLException: GSS Authentication failed
        ......
        ... 25 more
Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Can't find client principal <GP_USER>@<HDP_REALM> in cache collection)
        ......
        ... 40 more

相关配置 #

pgjdbc.conf内容 #

pgjdbc {
com.sun.security.auth.module.Krb5LoginModule required
isInitiator=true
useTicketCache=true
useKeyTab=false
renewTGT=false
doNotPrompt=true
debug=true
;
};

补充说明 #

看起来是PostgreSQL JDBC驱动执行了kinit操作，导致异常行为。

内容的提问来源于stack exchange，提问作者pal548