Lucene 9.10.0 MemoryIndex整数范围查询异常问题排查

问题场景 #

需要在MemoryIndex中存储两个字段：字符串类型的status和整数类型的portno，目标是匹配portno处于50至90范围内的记录，但遇到两个问题：

• 将portno以字符串形式索引时，数字按字典序处理，出现非预期匹配结果
• 改用IntField存储portno时，符合范围的记录完全无法匹配

测试输出 #

字符串索引方式输出 #

查询条件：status: Open AND portno: [50 TO 90]
portno: 4 匹配结果：false
portno: 5 匹配结果：false
portno: 6 匹配结果：true
portno: 7 匹配结果：true
portno: 8 匹配结果：true
portno: 9 匹配结果：true
portno: 49 匹配结果：false
portno: 50 匹配结果：true
portno: 89 匹配结果：true
portno: 90 匹配结果：true
portno: 91 匹配结果：false
portno: 100 匹配结果：false

IntField索引方式输出 #

查询条件：status: Open AND portno: [50 TO 90]
portno: 4 匹配结果：false
portno: 5 匹配结果：false
portno: 6 匹配结果：false
portno: 7 匹配结果：false
portno: 8 匹配结果：false
portno: 9 匹配结果：false
portno: 49 匹配结果：false
portno: 50 匹配结果：false
portno: 89 匹配结果：false
portno: 90 匹配结果：false
portno: 91 匹配结果：false
portno: 100 匹配结果：false

问题代码 #

import org.apache.lucene.analysis.core.WhitespaceAnalyzer;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.IntField;
import org.apache.lucene.index.memory.MemoryIndex;
import org.apache.lucene.queryparser.classic.MultiFieldQueryParser;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.TopDocs;

import java.util.HashMap;
import java.util.List;
import java.util.Map;


public class CriteriaEvaluatorTest {

    private static void indexNumberAsString() {
        List<Integer> ilist = List.of(4, 5, 6, 7, 8, 9, 49, 50, 89, 90, 91, 100);
        String criteria = "status: Open AND portno: [50 TO 90]";
        System.out.println("Criteria: " + criteria);
        ilist.forEach(number -> {
                    MemoryIndex memoryIndex = new MemoryIndex();
                    WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
                    Map<String, Object> fmap = new HashMap<>();
                    fmap.put("status", "Open");
                    fmap.put("portno", String.valueOf(number));
                    for (Map.Entry<String, Object> entry : fmap.entrySet()) {
                        String fldName = entry.getKey();
                        String fldValue = entry.getValue().toString();
                        memoryIndex.addField(fldName, fldValue, analyzer);
                    }
                    IndexSearcher indexSearcher = memoryIndex.createSearcher();
                    String[] fldArray = new String[3];
                    int j = 0;
                    for (String fld : fmap.keySet()) {
                        fldArray[j++] = fld;
                    }
                    MultiFieldQueryParser multiFieldQueryParser = new MultiFieldQueryParser(fldArray, analyzer);
                    try {
                        Query query = multiFieldQueryParser.parse(criteria);
                        TopDocs topDocs = indexSearcher.search(query, 1);
                        boolean result = topDocs.totalHits.value > 0;
                        System.out.println("portno: " + number + " Criteria matched: " + result);
                    } catch (Exception e) {
                        String message = e.getMessage();
                    }
                }
        );
    }

    private static void indexNumberAsIntPoint() {
        List<Integer> ilist = List.of(4, 5, 6, 7, 8, 9, 49, 50, 89, 90, 91, 100);
        String criteria = "status: Open AND portno: [50 TO 90]";
        System.out.println("Criteria: " + criteria);
        ilist.forEach(number -> {
                    MemoryIndex memoryIndex = new MemoryIndex(true, true);
                    WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
                    Map<String, Object> fmap = new HashMap<>();
                    fmap.put("status", "Open");
                    fmap.put("portno", number);
                    for (Map.Entry<String, Object> entry : fmap.entrySet()) {
                        String fldName = entry.getKey();
                        String fldValue = entry.getValue().toString();
                        if (fldName.equals("portno")) {
                            IntField intField = new IntField("portno", number, Field.Store.YES);
                            memoryIndex.addField(intField, analyzer);
                        } else {
                            memoryIndex.addField(fldName, fldValue, analyzer);
                        }
                    }
                    IndexSearcher indexSearcher = memoryIndex.createSearcher();
                    String[] fldArray = new String[3];
                    int j = 0;
                    for (String fld : fmap.keySet()) {
                        fldArray[j++] = fld;
                    }
                    MultiFieldQueryParser multiFieldQueryParser = new MultiFieldQueryParser(fldArray, analyzer);
                    try {
                        Query query = multiFieldQueryParser.parse(criteria);
                        TopDocs topDocs = indexSearcher.search(query, 1);
                        boolean result = topDocs.totalHits.value > 0;
                        System.out.println("portno: " + number + " Criteria matched: " + result);
                    } catch (Exception e) {
                        String message = e.getMessage();
                    }
                }
        );
    }

    public static void main(String args[]) {
        indexNumberAsString();
        System.out.println("==============================================================");
        indexNumberAsIntPoint();
    }
}

问题分析与解决方案 #

1. 字符串索引的问题根源 #

字符串类型的portno按字典序排序，而非数值序。例如字符串"6"的字典序大于"50"（因为首字符'6' > '5'），所以会被[50 TO 90]的范围查询匹配，这就是为什么6-9这些数字会被错误命中。

如果必须用字符串存储数值，需要对数字做补零处理（比如统一转成3位字符串："004"、"050"），确保字典序和数值序一致，但这种方式不够优雅，优先推荐使用数值类型字段。

2. IntField索引的问题根源 #

MultiFieldQueryParser默认会把范围查询解析为TermRangeQuery，但IntField是数值型字段，需要使用NumericRangeQuery才能正确匹配。另外，MemoryIndex添加IntField时，需要确保字段被正确识别为数值类型。

修改后的IntField索引查询代码

import org.apache.lucene.search.BooleanClause;
import org.apache.lucene.search.BooleanQuery;
import org.apache.lucene.search.NumericRangeQuery;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.index.Term;

// 新增修正后的方法
private static void indexNumberAsIntFieldFixed() {
    List<Integer> ilist = List.of(4, 5, 6, 7, 8, 9, 49, 50, 89, 90, 91, 100);
    System.out.println("Criteria: status: Open AND portno: [50 TO 90]");
    ilist.forEach(number -> {
        MemoryIndex memoryIndex = new MemoryIndex(true, true);
        WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
        // 添加字符串字段status
        memoryIndex.addField("status", "Open", analyzer);
        // 添加数值字段portno
        IntField intField = new IntField("portno", number, Field.Store.YES);
        memoryIndex.addField(intField, analyzer);
        
        IndexSearcher indexSearcher = memoryIndex.createSearcher();
        try {
            // 手动构建布尔查询：同时满足status=Open和portno在50-90之间
            BooleanQuery.Builder booleanQueryBuilder = new BooleanQuery.Builder();
            // status的Term查询
            booleanQueryBuilder.add(new TermQuery(new Term("status", "Open")), BooleanClause.Occur.MUST);
            // portno的数值范围查询，最后两个参数表示是否包含边界值
            Query portRangeQuery = NumericRangeQuery.newIntRange("portno", 50, 90, true, true);
            booleanQueryBuilder.add(portRangeQuery, BooleanClause.Occur.MUST);
            
            TopDocs topDocs = indexSearcher.search(booleanQueryBuilder.build(), 1);
            boolean result = topDocs.totalHits.value > 0;
            System.out.println("portno: " + number + " Criteria matched: " + result);
        } catch (Exception e) {
            e.printStackTrace();
        }
    });
}

修改说明

• 放弃使用MultiFieldQueryParser，手动构建BooleanQuery，分别为status创建TermQuery，为portno创建NumericRangeQuery
• NumericRangeQuery.newIntRange的最后两个参数设为true，表示包含50和90这两个边界值
• 确保IntField被正确添加到MemoryIndex，MemoryIndex会自动处理数值型字段的索引逻辑

最终效果 #

修改后，只有portno在50-90之间的记录（50、89、90）会返回true，其余记录返回false，完全符合预期查询需求。

内容的提问来源于stack exchange，提问作者ParagJ