Azure部署GPT模型输出格式不一致及识别精度问题求助

一、修复输出格式不一致问题 #

模型输出额外注释的核心原因是提示词约束强度不足，可通过以下方案解决：

1. 强化提示词的格式强制规则 #

优化提示词，明确要求模型必须严格遵循输出格式，不得添加任何额外内容，并补充示例强化认知：

question = """
Task: Identify overly generic line item descriptions from the table below.
Definition of generic: Broad, non-specific terms that don't refer to a unique product/service (e.g., Miscellaneous, Parts, Product, Item, Transportation, Business Service).
Output Rules:
- If no generic descriptions are found, output ONLY the number `0`.
- If generic descriptions exist, output ONLY the terms separated by `|` (no extra text, labels, or explanations).
- **Do NOT add any additional comments, explanations, or labels to your response.**
---Table---
"""

2. 使用结构化输出约束（推荐） #

若使用支持结构化输出的模型版本（如gpt-3.5-turbo-1106、gpt-4-1106-preview），可通过response_format参数强制模型输出JSON格式，彻底避免格式混乱：

model = AzureChatOpenAI(
    openai_api_base=openai_base,
    openai_api_version=openai_version,
    deployment_name=dep_name,
    openai_api_key=api_key,
    openai_api_type=api_type,
    temperature=0.0  # 降低随机性，提升输出稳定性
)

# 调用时指定JSON格式输出
gpt_model = model(
    [
        HumanMessage(content=prompt)
    ],
    response_format={"type": "json_object"}
)

# 解析JSON响应
import json
try:
    response_data = json.loads(gpt_model.content)
    # 输出结构为{"generic_terms": ["term1", "term2"]}或{"generic_terms": []}
    if not response_data["generic_terms"]:
        response = "0"
    else:
        response = "|".join(response_data["generic_terms"])
except json.JSONDecodeError:
    # 处理解析失败的情况，可重试或标记异常
    response = "0"
print("This is response: ", response)

对应的提示词需调整为要求输出JSON：

question = """
Task: Identify overly generic line item descriptions from the table below.
Definition of generic: Broad, non-specific terms that don't refer to a unique product/service (e.g., Miscellaneous, Parts, Product, Item, Transportation, Business Service).
Output Requirements:
- Return a JSON object with a single key "generic_terms".
- If no generic descriptions are found, set "generic_terms" to an empty array.
- If generic descriptions exist, set "generic_terms" to an array of the identified terms.
- **Do NOT include any other content outside the JSON object.**
---Table---
"""

3. 代码层添加输出清洗逻辑 #

若无法使用结构化输出，可在获取响应后做简单清洗，提取符合要求的内容：

# 清洗响应，只保留数字0或用|分隔的词汇
import re
cleaned_response = re.sub(r'[^0-9|a-zA-Z\s]', '', response.strip())
# 进一步过滤多余文字，只保留符合格式的部分
if "0" in cleaned_response and len(cleaned_response) == 1:
    final_response = "0"
else:
    terms = [t.strip() for t in cleaned_response.split("|") if t.strip()]
    final_response = "|".join(terms) if terms else "0"
print("Cleaned response: ", final_response)

二、提升识别精度问题 #

模型误判的核心原因是对“通用描述”的定义不够清晰，可通过以下方案优化：

1. 细化通用描述的判定标准 #

在提示词中明确区分通用与非通用描述，补充反例：

question = """
Task: Identify overly generic line item descriptions from the table below.
Definition of generic: Broad, non-specific terms that cannot be linked to a unique product/service. Examples include: Miscellaneous, Parts, Product, Item, Transportation, Business Service.
Non-generic examples: "Pure Protein Bar" (specific product), "Office Chair - Ergonomic" (specific item), "FedEx Overnight Shipping" (specific service). These should NOT be marked as generic.
Output Rules:
- If no generic descriptions are found, output ONLY `0`.
- If generic descriptions exist, output ONLY the terms separated by `|`.
- **No extra comments or explanations allowed.**
---Table---
"""

2. 添加示例输入输出 #

给模型提供具体的示例，帮助它理解判断逻辑：

question = """
Task: Identify overly generic line item descriptions from the table below.
Definition of generic: Broad, non-specific terms that cannot be linked to a unique product/service. Examples include: Miscellaneous, Parts, Product, Item, Transportation, Business Service.
Non-generic examples: "Pure Protein Bar", "Office Chair - Ergonomic", "FedEx Overnight Shipping" (these should NOT be marked).
Example 1:
Input Table:
item_descriptions
Miscellaneous
Pure Protein Bar
Parts
Output: Miscellaneous|Parts

Example 2:
Input Table:
item_descriptions
Laptop Charger
Ergonomic Mouse
Output: 0

Output Rules for Your Task:
- Follow the same format as the examples.
- If no generic descriptions are found, output ONLY `0`.
- If generic descriptions exist, output ONLY the terms separated by `|`.
- **Do NOT add any extra text, labels, or explanations.**
---Table---
"""

3. 调整模型参数 #

降低temperature参数（设置为0或0.1），减少模型的随机性，让输出更贴合规则：

model = AzureChatOpenAI(
    # 其他参数不变
    temperature=0.0
)

完整优化后的代码示例 #

import pandas as pd
from langchain.chat_models import AzureChatOpenAI
from langchain.schema import HumanMessage
import re

# 假设df是你的数据框
df = pd.DataFrame({'desc': ['Miscellaneous', 'Pure Protein Bar', 'Parts', 'Office Chair']})

# 优化后的提示词
question = """
Task: Identify overly generic line item descriptions from the table below.
Definition of generic: Broad, non-specific terms that cannot be linked to a unique product/service. Examples include: Miscellaneous, Parts, Product, Item, Transportation, Business Service.
Non-generic examples: "Pure Protein Bar" (specific product), "Office Chair" (specific item) — these should NOT be marked as generic.
Example 1:
Input Table:
item_descriptions
Miscellaneous
Pure Protein Bar
Parts
Output: Miscellaneous|Parts

Example 2:
Input Table:
item_descriptions
Laptop Charger
Ergonomic Mouse
Output: 0

Output Rules:
- Follow the example format strictly.
- If no generic descriptions exist, output ONLY `0`.
- If generic descriptions exist, output ONLY the terms separated by `|`.
- **No extra comments, labels, or explanations allowed.**
---Table---
"""

descriptions = "item_descriptions\n" + '\n'.join(df['desc'])
prompt = question + "\n" + descriptions

# 初始化模型，设置低temperature
model = AzureChatOpenAI(
    openai_api_base=openai_base,
    openai_api_version=openai_version,
    deployment_name=dep_name,
    openai_api_key=api_key,
    openai_api_type=api_type,
    temperature=0.0
)

# 调用模型
gpt_model = model(
    [
        HumanMessage(content=prompt)
    ])

# 清洗响应
response = gpt_model.content.strip()
# 处理可能的格式问题
if not response:
    final_response = "0"
elif response == "0":
    final_response = "0"
else:
    terms = [t.strip() for t in response.split("|") if t.strip() and not t.isdigit()]
    final_response = "|".join(terms) if terms else "0"

print("This is response: ", final_response)

内容的提问来源于stack exchange，提问作者user3437212