Logstash中对比两个JSON字段获取差异的问题与疑问

问题场景 #

我在Logstash中尝试对比JDBC获取的两个JSON对象差异，示例数据如下：

before = '{"heroes":[{"id":1,"name":"pudge"},{"id":2,"name":"slark"},{"id":3,"name":"techies"}]}'

after = '{"heroes":[{"id":1,"name":"pudge"},{"id":2,"name":"slark"},{"id":3,"name":"invoker"}]}'

报错信息 #

运行时触发如下错误：

[ERROR][logstash.filters.ruby][main][88122xx6957217be6831d4e0f72ec6a5e9ec4dccc693227d1256f06b0d14bad03] Ruby exception occurred: undefined method `deep_diff' for #<Hash:0x5480e379> {:class=>"NoMethodError", :backtrace=>["(ruby filter code):30:in `block in filter_method'", "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:178:in `block in multi_filter'", "org/jruby/RubyArray.java:1865:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:175:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:300:in `block in start_workers']}

问题细节 #

我最初在Logstash的filter块中通过扩展Hash类定义deep_diff方法来实现差异对比，这种写法在普通Ruby环境中可以正常运行，但在Logstash里就会报错。而改用独立函数的写法后，功能完全正常：

def deep_diff(hash1, hash2)
        (hash1.keys | hash2.keys).inject({}) do |diff, key|
          next diff if hash1[key] == hash2[key]
          if hash1[key].is_a?(Hash) && hash2[key].is_a?(Hash)
            diff[key] = deep_diff(hash1[key], hash2[key])
          else
            diff[key] = [hash1[key], hash2[key]]
          end
          diff
        end
      end

diff = deep_diff(hash1, hash2)

疑问 #

这种情况是否正常？我是否应该优先使用独立函数而非扩展Hash类的方式？

解答 #

这种情况是正常的，核心原因在于Logstash的Ruby filter运行环境存在特殊的隔离机制：

• Logstash基于JRuby运行Ruby脚本，对全局核心类的修改做了上下文隔离限制。你在filter块中扩展Hash类的代码，无法被加载到处理实际Hash对象的运行上下文，导致调用时触发undefined method错误。
• 独立函数的写法属于局部作用域内的定义，不依赖全局类的修改，能在Logstash的隔离环境中正常执行。

结论：必须优先使用独立函数的方式。Logstash的插件运行环境不推荐修改Ruby核心类（如Hash、Array），除了会出现这类上下文加载问题，还可能与Logstash内部代码或其他插件产生冲突，引发不可预料的异常。

内容的提问来源于stack exchange，提问作者onur