基于Deep Java Library，如何利用Solr索引实现问答功能？

我们正在探索基于Deep Java Library（DJL）的问答应用，参考了DJL的BERT问答实现示例。当前测试代码使用静态段落作为问答上下文，具体代码如下：

public static String predict() throws IOException, TranslateException, ModelException {
    // String question = "How is the weather";
    // String paragraph = "The weather is nice, it is beautiful day";
    String question = "When did BBC Japan start broadcasting?";
    String paragraph =
            "BBC Japan was a general entertainment Channel. "
                    + "Which operated between December 2004 and April 2006. "
                    + "It ceased operations after its Japanese distributor folded.";

    QAInput input = new QAInput(question, paragraph);
    logger.info("Paragraph: {}", input.getParagraph());
    logger.info("Question: {}", input.getQuestion());

    Criteria<QAInput, String> criteria =
            Criteria.builder()
                    .optApplication(Application.NLP.QUESTION_ANSWER)
                    .setTypes(QAInput.class, String.class)
                    .optFilter("backbone", "bert")
                    .optEngine(Engine.getDefaultEngineName())
                    .optProgress(new ProgressBar())
                    .build();

    try (ZooModel<QAInput, String> model = criteria.loadModel()) {
        try (Predictor<QAInput, String> predictor = model.newPredictor()) {
            return predictor.predict(input);
        }
    }
}

当前代码依赖静态硬编码的paragraph实现问答，替换为Lucene/Solr索引数据的具体实现方案如下：

实现步骤 #

1. 准备Lucene/Solr索引 #

首先将目标文档数据构建为Lucene/Solr索引。以Lucene为例，确保每个文档包含用于检索的文本字段（如命名为content），并配置该字段为可索引、可存储状态，方便后续检索和提取内容。

2. 实现基于问题的检索逻辑 #

根据用户输入的问题，在Lucene/Solr中执行检索，获取与问题最相关的段落或文档。以下是Lucene检索的核心代码片段：

public String retrieveRelevantParagraph(String question) throws IOException {
    // 初始化Lucene检索组件（实际项目建议复用IndexReader，避免重复IO）
    Directory directory = FSDirectory.open(Paths.get("/path/to/your/index"));
    IndexReader reader = DirectoryReader.open(directory);
    IndexSearcher searcher = new IndexSearcher(reader);
    
    // 用分词器处理问题，构建检索查询
    Analyzer analyzer = new StandardAnalyzer();
    QueryParser parser = new QueryParser("content", analyzer);
    Query query = parser.parse(question);
    
    // 检索Top 1最相关的文档
    TopDocs topDocs = searcher.search(query, 1);
    if (topDocs.totalHits.value == 0) {
        return "未找到相关内容";
    }
    
    // 提取文档的content字段内容作为问答上下文
    Document doc = searcher.doc(topDocs.scoreDocs[0].doc);
    String relevantParagraph = doc.get("content");
    
    // 关闭资源（实际建议用try-with-resources自动管理）
    reader.close();
    directory.close();
    return relevantParagraph;
}

3. 集成检索与DJL问答逻辑 #

修改原有predict方法，将静态paragraph替换为检索到的相关段落：

public static String predict(String question) throws IOException, TranslateException, ModelException {
    // 1. 从Lucene检索相关段落作为上下文
    String paragraph = retrieveRelevantParagraph(question);
    
    QAInput input = new QAInput(question, paragraph);
    logger.info("Paragraph: {}", input.getParagraph());
    logger.info("Question: {}", input.getQuestion());

    Criteria<QAInput, String> criteria =
            Criteria.builder()
                    .optApplication(Application.NLP.QUESTION_ANSWER)
                    .setTypes(QAInput.class, String.class)
                    .optFilter("backbone", "bert")
                    .optEngine(Engine.getDefaultEngineName())
                    .optProgress(new ProgressBar())
                    .build();

    try (ZooModel<QAInput, String> model = criteria.loadModel()) {
        try (Predictor<QAInput, String> predictor = model.newPredictor()) {
            return predictor.predict(input);
        }
    }
}

4. 优化建议 #

• 资源复用：Lucene的IndexReader、DJL的ZooModel和Predictor建议全局复用，避免每次请求重新初始化，大幅提升性能。
• 多段落处理：若检索到多个相关段落，可分别传入DJL预测后，根据模型返回的置信度（部分DJL模型支持）选择最优答案，或合并段落为上下文再进行预测。
• Solr适配：如果使用Solr，只需将Lucene检索逻辑替换为SolrJ API或HTTP客户端调用，获取相关文档的content字段即可，核心集成逻辑一致。

内容的提问来源于stack exchange，提问作者user989010