已创建好集群并添加Kyuubi 服务。
您可以使用 beeline 工具连接到 Kyuubi 服务。
您需要指定某个具体的用户以连接到 Kyuubi 服务,用户名为已在系统内创建的用户,可进入“用户管理”页面查看:
$ source /etc/emr/kyuubi/kyuubi-env.sh $ /usr/lib/emr/current/kyuubi/bin/beeline -n ${user} -u "jdbc:hive2://${zookeeper_address}:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi"
zookeeper_address
为集群 zk 地址,可通过 “运维管理” 页面进行查看,例如 master-1:2181,master-2:2181,master-3:2181
。您也可以通过 Thrift 端口,直连 Kyuubi 服务。
$ /usr/lib/emr/current/kyuubi/bin/beeline -n user1 -u "jdbc:hive2://${kyuubi_server_address}:10009"
${kyuubi_server_address}
为 Kyuubi Server 地址,例如:$ /usr/lib/emr/current/kyuubi/bin/beeline -n user1 -u "jdbc:hive2://master-1:10009"
您也可以使用 JDBC 的方式,进行 Java 编程通过 Kyuubi JDBC Driver 连接到 Kyuubi。
<dependency> <groupId>org.apache.kyuubi</groupId> <artifactId>kyuubi-hive-jdbc-shaded</artifactId> <version>1.7.1</version> </dependency>
import java.sql.*; public class KyuubiJDBC { private static String driverName = "org.apache.kyuubi.jdbc.KyuubiHiveDriver"; private static String kyuubiJdbcUrl = "jdbc:kyuubi://${kyuubi_zookeeper_adress}/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi"; public static void main(String[] args) throws SQLException { try (Connection conn = DriverManager.getConnection(kyuubiJdbcUrl)) { try (Statement stmt = conn.createStatement()) { try (ResultSet rs = stmt.executeQuery("show databases")) { while (rs.next()) { System.out.println(rs.getString(1)); } } } } } }
如果您需要在开启了 Kerberos 的集群中访问 Kyuubi,请参考以下两种方式进行连接。
$ beeline -u 'jdbc:kyuubi://${zookeeper_address}/default;kyuubiServerPrincipal=${kyuubi_server_principal};kyuubiClientPrincipal=${user_principal};kyuubiClientKeytab=${user_keytab_location};serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;#spark.yarn.queue=root.default' -n ${user}
kyuubiServerPrincipal
需要填写 Kyuubi Server 使用的 principal,可在运维管理 > 服务列表 > Kyuubi > 服务参数中搜索参数 kyuubi.kinit.principal
获得。kyuubiClientPrincipal
和 kyuubiClientKeytab
需填入需要进行访问的用户的 principal 和 keytab 文件路径。import java.sql.*; public class KyuubiJDBCDemo { private static String driverName = "org.apache.kyuubi.jdbc.KyuubiHiveDriver"; private static String kyuubiJdbcUrlTemplate = "jdbc:kyuubi://${zookeeper_address}/default;kyuubiClientPrincipal=%s;kyuubiClientKeytab=%s;kyuubiServerPrincipal=%s;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;#spark.yarn.queue=root.default"; public static void main(String[] args) throws SQLException { String clientPrincipal = args[0]; // Kerberos principal String clientKeytab = args[1]; // Keytab file location String serverPrincipal = args[2]; // Kerberos principal used by Kyuubi Server String kyuubiJdbcUrl = String.format(kyuubiJdbcUrlTemplate, clientPrincipal, clientKeytab, serverPrincipal); try (Connection conn = DriverManager.getConnection(kyuubiJdbcUrl)) { try (Statement stmt = conn.createStatement()) { try (ResultSet rs = stmt.executeQuery("show databases")) { while (rs.next()) { System.out.println(rs.getString(1)); } } } } } }
说明
参数kyuubiServerPrincipal
、kyuubiClientPrincipal
和 kyuubiClientKeytab
填写参考上一章节。