Logstash从Kafka写入Kerberos集群WebHDFS时GSSAPI类初始化错误

问题场景 #

我们搭建了Logstash 6.2.4管道，从Kafka主题读取数据并输出到WebHDFS。测试环境使用解压版logstash-6.2.4.tar.gz，目标Hadoop集群已启用Kerberos认证。执行管道配置文件时出现插件注册错误，错误信息为uninitialized constant GSSAPI::GssApiError::LibGSSAPI，完整执行日志如下：

[user@hostname]$ bin/logstash -f /datan1/logstash-6.2.4/bin/pipeline.conf
Sending Logstash's logs to /datan1/logstash-6.2.4/logs which is now configured via log4j2.properties
[2018-05-04T15:04:06,566][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/datan1/logstash-6.2.4/modules/netflow/configuration"}
[2018-05-04T15:04:06,607][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/datan1/logstash-6.2.4/modules/fb_apache/configuration"}
[2018-05-04T15:04:07,880][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-05-04T15:04:09,193][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.2.4"}
[2018-05-04T15:04:10,034][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-05-04T15:04:15,906][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>32, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-05-04T15:04:19,373][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::OutputDelegator:0x100380a9 @namespaced_metric=#<LogStash::Instrument::NamespacedMetric:0x5ba440f8 @metric=#<LogStash::Instrument::Metric:0x2affe8ea @collector=#<LogStash::Instrument::Collector:0x4954cc3d @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x45de0b76 @store=#<Concurrent::Map:0x00000000000fac entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x41868944>, @fast_lookup=#<Concurrent::Map:0x00000000000fb0 entries=63 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :outputs, :d4fc8a80f489c5060bccfb1317f8c420c21a88b6fd6135075b8f7131c356cf29]>, @metric=#<LogStash::Instrument::NamespacedMetric:0x5c5513c0 @metric=#<LogStash::Instrument::Metric:0x2affe8ea @collector=#<LogStash::Instrument::Collector:0x4954cc3d @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x45de0b76 @store=#<Concurrent::Map:0x00000000000fac entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x41868944>, @fast_lookup=#<Concurrent::Map:0x00000000000fb0 entries=63 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :outputs]>, @out_counter=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @strategy=#<LogStash::OutputDelegatorStrategies::Legacy:0x636843ca @worker_count=1, @workers=[<LogStash::Outputs::WebHdfs host=>"xxx.xx.xx.xx", port=>50070, path=>"/user/logstash/ocs_cdr_data/dt=%{+YYYY-MM-dd}/ocs_cdr_data-%{+HH}.log", user=>"user", use_kerberos_auth=>true, kerberos_keytab=>"/home/user/user.keytab", id=>"d4fc8a80f489c5060bccfb1317f8c420c21a88b6fd6135075b8f7131c356cf29", enable_metric=>true, codec=><LogStash::Codecs::Line id=>"line_80e1b3a5-9a38-4674-85cc-c7e519b32c2e", enable_metric=>true, charset=>"UTF-8", delimiter=>"\n">, workers=>1, standby_host=>false, standby_port=>50070, idle_flush_time=>1, flush_size=>500, open_timeout=>30, read_timeout=>30, use_httpfs=>false, single_file_per_thread=>false, retry_known_errors=>true, retry_interval=>0.5, retry_times=>5, compression=>"none", snappy_bufsize=>32768, snappy_format=>"stream", use_ssl_auth=>false>], @worker_queue=#<SizedQueue:0x3a5f35e0>>, @in_counter=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @id="d4fc8a80f489c5060bccfb1317f8c420c21a88b6fd6135075b8f7131c356cf29", @time_metric=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x25b6080a @metric=#<LogStash::Instrument::Metric:0x2affe8ea @collector=#<LogStash::Instrument::Collector:0x4954cc3d @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x45de0b76 @store=#<Concurrent::Map:0x00000000000fac entries=4 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x41868944>, @fast_lookup=#<Concurrent::Map:0x00000000000fb0 entries=63 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :outputs, :d4fc8a80f489c5060bccfb1317f8c420c21a88b6fd6135075b8f7131c356cf29, :events]>, @output_class=LogStash::Outputs::WebHdfs>", :error=>"uninitialized constant GSSAPI::GssApiError::LibGSSAPI\nDid you mean? GSSAPI", :thread=>"#<Thread:0x2d7a4e66 run>"}

问题原因 #

这个错误本质是Logstash的WebHDFS插件依赖的JRuby GSSAPI绑定无法找到系统级的GSSAPI库，通常是服务器缺少Kerberos相关依赖包，或者插件安装时未正确拉取GSSAPI相关gem导致的。

解决方案 #

1. 安装系统级Kerberos依赖 #

首先确保服务器上安装了Kerberos客户端和开发包，这是GSSAPI正常工作的基础：

• RHEL/CentOS系统：

  sudo yum install krb5-workstation krb5-devel -y
  

• Debian/Ubuntu系统：

  sudo apt-get install krb5-user libkrb5-dev -y
  

2. 重新安装WebHDFS插件 #

默认安装的插件可能缺失GSSAPI相关依赖，重新安装可以确保所有依赖被正确拉取：

cd /datan1/logstash-6.2.4
bin/logstash-plugin remove logstash-output-webhdfs
bin/logstash-plugin install logstash-output-webhdfs

安装过程中会自动下载并安装所需的JRuby GSSAPI gem，确保服务器网络可以访问RubyGems源。

3. 验证Kerberos配置和权限 #

• 检查keytab文件的权限，确保Logstash运行用户（这里是user）可以读取：

  sudo chown user:user /home/user/user.keytab
  sudo chmod 600 /home/user/user.keytab
  

• 测试keytab是否能正常获取Kerberos票据：

  kinit -kt /home/user/user.keytab user@YOUR_KERBEROS_REALM
  

  用klist命令确认票据已成功获取且未过期。

4. 配置Logstash JVM的Kerberos路径（可选） #

如果系统的krb5.conf不在默认路径，或者需要明确指定，可以修改Logstash的JVM参数：
编辑/datan1/logstash-6.2.4/config/jvm.options，添加以下行：

-Djava.security.krb5.conf=/etc/krb5.conf

替换/etc/krb5.conf为你的实际Kerberos配置文件路径。

5. 测试管道配置 #

修复后，先验证配置文件是否有效：

bin/logstash -f /datan1/logstash-6.2.4/bin/pipeline.conf --config.test_and_exit

如果没有错误提示，再启动Logstash运行管道。

内容的提问来源于stack exchange，提问作者Anushke Hewawaitharana