如何通过AWS Bedrock JS SDK V3为Claude 3 Haiku传递系统提示词

问题说明 #

使用AWS Bedrock的InvokeModelWithResponseStreamCommand调用Claude 3 Haiku时，尝试将{ role: "system", content: systemPrompt }加入messages数组会触发验证错误；尝试设置独立system字段初期未成功，需实现带系统提示约束的流式响应。

错误原因 #

AWS Bedrock上的Claude 3系列模型，其API规范不允许在messages数组中使用system角色，系统提示需通过payload中的独立system字段传递。之前的错误是因为错误地将系统提示放入messages数组，导致角色枚举验证失败：

ValidationException: Malformed input request: #: subject must not be valid against schema {"required":["messages"]}#/messages/0/role: system is not a valid enum value, please reformat your input and try again.

解决方案 #

• 在请求payload中添加独立的system字段，赋值为系统提示文本
• messages数组仅保留user和assistant角色的对话历史，不要包含system角色的消息

修改后的完整代码 #

import { NextResponse } from "next/server";
import { BedrockRuntimeClient, InvokeModelWithResponseStreamCommand } from "@aws-sdk/client-bedrock-runtime";

const systemPrompt = "你的系统提示内容，比如：你是一个专业的技术助手，用简洁易懂的语言回答问题。"

export async function POST(req) {
  const client = new BedrockRuntimeClient({ region: "us-east-1" });
  const data = await req.json();
  const modelId = "anthropic.claude-3-haiku-20240307-v1:0";

  const payload = {
    anthropic_version: "bedrock-2023-05-31",
    max_tokens: 10000,
    // 独立的system字段传递系统提示
    system: systemPrompt,
    messages: [
      {"role": "user", "content": "Hello there."},
      {"role": "assistant", "content": "Hi, I'm Claude. How can I help you?"},
      {"role": "user", "content": "Can you explain LLMs in plain English?"},
    ],
  }

  const command = new InvokeModelWithResponseStreamCommand({
    contentType: "application/json",
    body: JSON.stringify(payload),
    modelId,
  });
  
  const stream = new ReadableStream({
    async start(controller) {
      try {
        const response = await client.send(command);
        let completeMessage = "";
        for await (const item of response.body) {
          const chunk = JSON.parse(new TextDecoder().decode(item.chunk.bytes));
          const chunk_type = chunk.type;
      
          if (chunk_type === "content_block_delta") {
            const text = chunk.delta.text;
            completeMessage += text;
            controller.enqueue(text);
            process.stdout.write(text);
          }
        }
      } catch (err) {
        controller.error(err)
      } finally {
        controller.close()
      }
    }
  })

  return new NextResponse(stream);
}

内容的提问来源于stack exchange，提问作者Colexeco