LangChain v0.3.24：ChatPromptTemplate嵌入Base64图片时variable_name参数报错问题求助

Let's break down what's going wrong and how to fix it properly:

Root Cause #

You're hitting two issues here:

1. Invalid MessagesPlaceholder usage: The variable_name parameter expects a single string (to name a dynamic message variable like chat history), but you passed a list ["im64", str(image_data)]—this is why you get the "Input should be a valid string" error.
2. Incorrect multimodal prompt structure: Using {{im64}} in the system prompt won't inject image data correctly. LangChain requires explicit formatting for multimodal inputs (text + images) that models like GPT-4 Vision can understand.

Solution #

Here's the corrected approach to include your base64 image in the prompt for LangChain v0.3.24:

Basic Fixed Code #

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.messages import HumanMessage, SystemMessage

# Your base64 image data (ensure it's stripped of any extra headers if needed)
image_data = "your_base64_image_string_here"

# Build the prompt with proper multimodal message structure
generation_prompt = ChatPromptTemplate.from_messages(
    [
        SystemMessage(
            content="You are an expert assistant. Your task is to analyze the food items displayed in the image and provide detailed information about them."
        ),
        HumanMessage(
            content=[
                {"type": "text", "text": "Please analyze this image of food:"},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}}
            ]
        )
    ]
)

# Generate the prompt value to pass to your model
prompt_value = generation_prompt.invoke({})

Key Details: #

• Image URI Format: Most multimodal models require the base64 string to be prefixed with data:image/[format];base64, (e.g., data:image/png;base64, for PNGs). Adjust the format to match your image type.
• Multimodal Message Structure: The HumanMessage content uses a list of dictionaries to separate text and image content—this is the standard format LangChain uses for multimodal prompts.

Alternative: Reusable Prompt with Image Variable #

If you want to reuse the prompt with different images, make the image data an input variable:

generation_prompt = ChatPromptTemplate.from_messages(
    [
        SystemMessage(
            content="You are an expert assistant. Your task is to analyze the food items displayed in the image and provide detailed information about them."
        ),
        HumanMessage(
            content=[
                {"type": "text", "text": "Please analyze this image of food:"},
                {"type": "image_url", "image_url": {"url": "{image_data}"}}
            ]
        )
    ]
)

# Invoke with your base64 image (including the data URI prefix)
prompt_value = generation_prompt.invoke({"image_data": f"data:image/jpeg;base64,{image_data}"})

This way, you can swap out image_data for different images without rewriting the prompt.

内容的提问来源于stack exchange，提问作者Mohammed Baashar