Solr 6.6升级至7.3后高亮DatePointField字段触发日期格式异常

原本使用Solr 6.6，升级到7.3版本后，执行基于ID的查询并将CreationDate日期字段加入高亮列表时，抛出日期格式无效的异常。数据通过dataimporter从SQL导入，日期存储格式为"2021-06-08T17:12:26Z"，该字段配置为DatePointField类型。

异常信息 #

o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Invalid Date String:'1623172346000'
    at org.apache.solr.util.DateMathParser.parseMath(DateMathParser.java:247)
    at org.apache.solr.util.DateMathParser.parseMath(DateMathParser.java:226)
    at org.apache.solr.schema.DatePointField.toNativeType(DatePointField.java:113)
    at org.apache.solr.schema.DatePointField.readableToIndexed(DatePointField.java:184)
    at org.apache.solr.schema.PointField.toInternalByteRef(PointField.java:193)
    at org.apache.solr.schema.FieldType$DefaultAnalyzer$1.incrementToken(FieldType.java:492)
    at org.apache.solr.highlight.TokenOrderingFilter.incrementToken(DefaultSolrHighlighter.java:809)
    at org.apache.lucene.search.highlight.OffsetLimitTokenFilter.incrementToken(OffsetLimitTokenFilter.java:42)
    at org.apache.lucene.analysis.CachingTokenFilter.fillCache(CachingTokenFilter.java:91)
    at org.apache.lucene.analysis.CachingTokenFilter.incrementToken(CachingTokenFilter.java:70)
    at org.apache.lucene.index.memory.MemoryIndex.storeTerms(MemoryIndex.java:583)
    at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:474)
    at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:452)
    at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:431)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getLeafContext(WeightedSpanTermExtractor.java:405)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extractWeightedTerms(WeightedSpanTermExtractor.java:364)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:141)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:112)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:513)
    at org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:218)
    at org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:186)
    at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:201)
    at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:631)
    at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingOfField(DefaultSolrHighlighter.java:480)
    at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:442)
    at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:183)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
    at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.Server.handle(Server.java:530)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
    at java.lang.Thread.run(Thread.java:750)

字段配置 #

<fieldType name="date" class="solr.DatePointField" omitNorms="true" positionIncrementGap="0" />

<field name="CreationDate" type="date" indexed="true" stored="true"/>

问题原因与解决方法 #

原因 #

Solr 7.x中DatePointField的高亮处理逻辑发生变化：高亮器处理该字段时，错误将存储的毫秒时间戳字符串1623172346000当作可读日期格式解析，而DateMathParser无法识别纯数字时间戳，导致报错。Solr 6.6使用的是TrieDateField，其高亮逻辑不会触发此问题。

解决方法 #

• 方法一：移除日期字段的高亮配置
  如果业务不需要对日期字段高亮，直接从高亮字段列表中删除CreationDate即可，这是最简便的方式。

• 方法二：新增字符串类型的日期字段用于高亮
  创建一个string类型字段存储格式化后的日期字符串，对该字段启用高亮：

  <field name="CreationDate_str" type="string" indexed="false" stored="true"/>
  

  在dataimporter中，将CreationDate的格式化值导入到CreationDate_str，查询时对CreationDate_str配置高亮。

• 方法三：自定义分析器（不推荐）
  为DatePointField配置可处理时间戳的分析器，或修改DateMathParser解析逻辑，但需修改Solr源码，成本较高。

内容的提问来源于stack exchange，提问作者Rahul Sonar