如何在Supabase Vector Store上传时为project_id列赋值？

我需要在向Supabase Vector Store上传数据时，把project_id存入数据库的独立列中，而不是只放在metadata字段里——因为多用户上传的场景下，得靠project_id做数据筛选。但目前代码里的project_id只进到了metadata，数据库的project_id列还是空的，当前代码如下：

import { supabaseClient } from '../../../utils/browser-supabase';
import { SupabaseVectorStore } from 'langchain/vectorstores/supabase';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
import { createClient } from '@supabase/supabase-js';

//need to figure out hwo to add project_ID here
export async function handleVectorStoreUpload({ token, text, project_id }) {
  console.log(token);
  const textSplitter = new RecursiveCharacterTextSplitter({
    chunkSize: 200,
    chunkOverlap: 50,
  });
  const metadata = { project_id: project_id };
  const docs = await textSplitter.createDocuments([text]);

  try {
    const supabase = await supabaseClient(token);
    const vectorStore = await SupabaseVectorStore.fromDocuments(
      docs,
      new OpenAIEmbeddings({
        openAIApiKey: process.env.NEXT_PUBLIC_OPENAI_API_KEY,
      }),
      {
        client: supabase,
        tableName: 'documents',
        queryName: 'match_documents',
      }
    );
  } catch (error) {
    console.error('Error uploading to vector store:', error);
  }
}

解决方案 #

要让project_id写入数据库的独立列，需要做两处关键修改：

• 拆分文本生成文档后，给每个文档都绑定包含project_id的metadata（原代码里创建的docs没有携带metadata）
• 在SupabaseVectorStore的配置中添加columnMapping，明确指定metadata里的project_id对应数据库表的project_id列

修改后的完整代码：

import { supabaseClient } from '../../../utils/browser-supabase';
import { SupabaseVectorStore } from 'langchain/vectorstores/supabase';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
import { createClient } from '@supabase/supabase-js';

export async function handleVectorStoreUpload({ token, text, project_id }) {
  console.log(token);
  const textSplitter = new RecursiveCharacterTextSplitter({
    chunkSize: 200,
    chunkOverlap: 50,
  });
  
  // 拆分文本时直接给每个文档添加metadata
  const docs = await textSplitter.createDocuments([text], [{ project_id }]);

  try {
    const supabase = await supabaseClient(token);
    const vectorStore = await SupabaseVectorStore.fromDocuments(
      docs,
      new OpenAIEmbeddings({
        openAIApiKey: process.env.NEXT_PUBLIC_OPENAI_API_KEY,
      }),
      {
        client: supabase,
        tableName: 'documents',
        queryName: 'match_documents',
        // 配置metadata字段到数据库列的映射
        columnMapping: {
          metadata: 'metadata',
          embedding: 'embedding',
          project_id: 'project_id' // 指定metadata的project_id对应数据库的project_id列
        }
      }
    );
  } catch (error) {
    console.error('Error uploading to vector store:', error);
  }
}

关键说明 #

1. createDocuments方法的第二个参数是metadata数组，每个元素对应拆分后的一个文档的metadata，传入[{ project_id }]就能让所有拆分出的文档都带上该project_id
2. columnMapping配置会告诉SupabaseVectorStore，把文档metadata里的project_id字段值写入数据库表的project_id列，而不是只存在metadata的JSON结构里

内容的提问来源于stack exchange，提问作者clout_chaser