如何在Backstage新后端系统添加自定义Collator并使用OpenSearch索引

一、在Backstage新后端插件中使用自定义Collator #

步骤1：修改插件注册逻辑，注册Collator到搜索系统 #

在你的plugin.ts中引入搜索服务，将自定义CollatorFactory注册到搜索系统：

import {
  coreServices,
  createBackendPlugin,
} from '@backstage/backend-plugin-api';
import { createRouter } from './service/router';
import { searchCollatorRegistry } from '@backstage/plugin-search-backend/alpha';
import { CollatorFactory } from './Collator';

/**
 * myTestPlugin backend plugin
 *
 * @public
 */
export const myTestPlugin = createBackendPlugin({
  pluginId: 'my-test',
  register(env) {
    env.registerInit({
      deps: {
        httpRouter: coreServices.httpRouter,
        logger: coreServices.logger,
        collatorRegistry: searchCollatorRegistry,
      },
      async init({
        httpRouter,
        logger,
        collatorRegistry,
      }) {
        // 注册自定义Collator
        collatorRegistry.addCollator({
          factory: CollatorFactory.fromConfig({ logger }),
        });

        httpRouter.use(
          await createRouter({
            logger,
          }),
        );
        httpRouter.addAuthPolicy({
          path: '/health',
          allow: 'unauthenticated',
        });
      },
    });
  },
});

步骤2：完善Collator的数据生成逻辑 #

你的Collator.ts中execute函数目前无数据返回，需补充文档生成逻辑（示例为模拟数据，后续可替换为OpenSearch数据源）：

async *execute(): AsyncGenerator<IndianCitiesDocument>{
  this.logger.info('Starting to fetch documents for search indexing');
  
  // 模拟数据源，后续替换为OpenSearch查询逻辑
  const sampleData = [
    { id: '1', name: 'Mumbai', state: 'Maharashtra', description: '印度金融中心' },
    { id: '2', name: 'Delhi', state: 'Delhi', description: '印度首都' },
  ];

  for (const item of sampleData) {
    yield {
      id: item.id,
      title: item.name,
      description: item.description,
      location: item.state,
      type: this.type,
    } as IndianCitiesDocument;
  }
}

步骤3：配置搜索索引（可选） #

在app-config.yaml中为自定义文档类型配置索引：

search:
  indexers:
    my-awesome-plugin:
      type: elasticsearch
      documentType: my-awesome-plugin

二、直接使用OpenSearch中的索引数据实现搜索 #

可以通过自定义Collator读取OpenSearch现有索引数据，转换为Backstage搜索文档格式后同步到Backstage搜索索引，实现统一搜索体验。

实现步骤 #

1. 安装OpenSearch客户端依赖

npm install @opensearch-project/opensearch

2. 修改Collator.ts，接入OpenSearch数据源

import { Logger } from 'winston';
import { Readable } from 'stream';
import { DocumentCollatorFactory } from '@backstage/plugin-search-common'
import { IndianCitiesDocument } from './document';
import { Permission } from '@backstage/plugin-permission-common';
import { Client } from '@opensearch-project/opensearch';

export type CollatorFactoryOptions = {
    logger: Logger
}

export class CollatorFactory implements DocumentCollatorFactory{
    private readonly logger: Logger;
    public readonly type: string = 'my-awesome-plugin';
    private constructor(options: CollatorFactoryOptions){
        this.logger = options.logger;
    }
    visibilityPermission?: Permission | undefined;

    static fromConfig(options:CollatorFactoryOptions){
        return new CollatorFactory({ ...options})
    }

    async getCollator(){
        return Readable.from(this.execute())
    }

    async *execute(): AsyncGenerator<IndianCitiesDocument>{
        this.logger.info('Fetching data from OpenSearch');
        
        // 初始化OpenSearch客户端
        const client = new Client({
            node: 'https://你的OpenSearch实例地址:9200',
            auth: {
                username: '你的OpenSearch用户名',
                password: '你的OpenSearch密码',
            },
            // 若使用自签名证书，添加以下配置
            ssl: {
                rejectUnauthorized: false,
            },
        });

        try {
            // 查询OpenSearch目标索引
            const response = await client.search({
                index: '你的印度城市索引名', // 替换为实际索引名
                body: {
                    query: { match_all: {} },
                    size: 1000, // 单次查询数量，可按需调整或实现分页
                },
            });

            // 转换为Backstage搜索文档格式
            for (const hit of response.body.hits.hits) {
                const source = hit._source;
                yield {
                    id: source.id || hit._id,
                    title: source.city_name,
                    description: source.city_description,
                    location: source.state,
                    type: this.type,
                } as IndianCitiesDocument;
            }
        } catch (error) {
            this.logger.error('Failed to fetch data from OpenSearch', error);
            throw error;
        } finally {
            await client.close();
        }
    }
}

3. 配置Backstage搜索后端连接到OpenSearch

在app-config.yaml中配置搜索引擎为OpenSearch（兼容Elasticsearch API）：

search:
  backend:
    search:
      engine:
        type: elasticsearch
        options:
          node: 'https://你的OpenSearch实例地址:9200'
          auth:
            username: '你的OpenSearch用户名'
            password: '你的OpenSearch密码'
          ssl:
            rejectUnauthorized: false

4. 验证索引同步

启动Backstage后端后，搜索服务会定期触发Collator执行数据同步，可通过后端日志确认数据获取状态，或直接查询OpenSearch中前缀为backstage_search_的索引验证文档是否存在。

内容的提问来源于stack exchange，提问作者Daksh Sharma