Azure OpenAI GPT-4o：Python与Java API调用结果差异求助

问题概述 #

使用Python和Java的Azure OpenAI SDK调用GPT-4o时，在相同输入条件下返回结果差异极大：Python版本的输出符合用户提示中的规则要求，而Java版本的输出完全忽略了用户提示里的约束，生成了超出要求的内容。

重现条件 #

• 相同的系统提示
• 相同的用户提示
• 使用Python和Java的Azure AI最新版本包

完整示例 #

用户提示 #

The following text is a customer complaint about Cheque In. Your task is to summarise your understanding of what the customer complaint was about in a few sentences.

<<<COMPLAINT>>>
I received a check from NS&I for £425 and tried to upload it using the check system on my online account but your system does not accept checks from the government there, I now have to drive 16 miles pay for parking ti the nearest Big Bank bank !! All because your system does not accept government checks !!!!!

Always follow these guidelines in your response:

<<<GUIDELINES>>>
* Start with "I understand that…".
* DO NOT include anything beyond your summary.
* Always use active voice.
* Always address the customer directly using "you".
* Refer to the bank as "we" and refer to any human advisor or agent as a colleague.
* Be clear, polite, and use language that is easy to understand.
* NEVER include any details about internal bank processes or systems, instead say "our records".
* Only use facts from the provided context. DO NOT make up information.
* This task is VERY sensitive; you MUST NOT make mistakes.
* DO NOT mention account markers.
* Always format monetary values as £x,xxx.xx.
* Use “day month year” format for dates e.g 22 May 2023
* Only use the last 4 digits of account numbers.
* Always use British spelling, NOT American.

系统提示 #

You are a complaint handler for a customer service department at Big Bank

Python代码 #

from dotenv import load_dotenv
import os, sys
import jwt
import openai
from msal import ConfidentialClientApplication
from openai import AzureOpenAI

def get_jwt_token():
    # org_constants
    auth = os.getenv("MS_AUTH_URL")

    # Provide Azure Client/App ID and Secret to Variables
    client_secret = os.getenv("AZURE_CLIENT_SECRET")
    client_id = os.getenv("AZURE_CLIENT_ID")

    scope_list = [client_id + "/.default"]

    # Getting JWT Token Using MSAL Methods for OpenAI Authentication
    app = ConfidentialClientApplication(client_id=client_id, client_credential=client_secret, authority=auth)
    result = app.acquire_token_for_client(scopes=scope_list)
    return result.get("access_token")

api_base_full_chat = os.getenv("API_BASE_FULL_CHAT")
deployment_model_chat = os.getenv("DEPLOYMENT_MODEL_CHAT")
api_version_chat = os.getenv("AZURE_OPENAI_API_VERSION")

def create_chat_client():
    client_chat = AzureOpenAI(
        api_key=get_jwt_token(),
        azure_deployment=deployment_model_chat,
        azure_endpoint=api_base_full_chat,
        api_version=api_version_chat,
    )
    return client_chat

chat = create_chat_client()

response = chat.chat.completions.create(
                model=deployment_model_chat,
                messages=[
                    {"role": "system", "content": "You are a complaint handler for a customer service department at Big Bank"},
                    {"role": "user", "content": user_prompt},
                ]
                )

print(response.choices[0].message.content)

Java代码 #

this.setEnvironmnet();
//Get JWT
String auth = String.format("%s/%s", "https://login.microsoftonline.com", System.getProperty("AZURE_TENANT_ID"));
IClientSecret mySecret  = ClientCredentialFactory.createFromSecret(System.getProperty("AZURE_CLIENT_SECRET"));
ConfidentialClientApplication app = ConfidentialClientApplication.builder(System.getProperty("AZURE_CLIENT_ID"), mySecret)
        .authority(auth)
        .proxy(getProxy())
        .build();
Set<String> scopes = new HashSet<>();
scopes.add(String.format("%s/.default", System.getProperty("AZURE_CLIENT_ID")));
ClientCredentialParameters params = ClientCredentialParameters.builder(scopes).build();
IAuthenticationResult result = app.acquireToken(params).get(5000, TimeUnit.MILLISECONDS);

//Add JWT to header - get a 401 if try to use TokenCredential!
ClientOptions options = new ClientOptions();
List<Header> headers = new ArrayList<>();
headers.add(new Header("Authorization", String.format("Bearer %s", result.accessToken())));
options.setHeaders(headers);

//get client
OpenAIClient client = new OpenAIClientBuilder()
        .clientOptions(options)
        //  .credential(this.getToken())
        .endpoint(System.getProperty("AZURE_GPT4O_URL"))
        .buildClient();

//Build and submit Prompt
List<ChatRequestMessage> chatMessages = new ArrayList<>();
chatMessages.add(new ChatRequestSystemMessage(this.getUnderstandingSystemPrompt()));
chatMessages.add(new ChatRequestUserMessage(this.getUnderstandingUserPrompt()));
ChatCompletionsOptions chatOptions = new ChatCompletionsOptions(chatMessages);
ChatCompletions response = this.getClient().getChatCompletions("gpt4o_2024-05-13", chatOptions);
response.getChoices().forEach(message -> {
    System.out.println(message.getMessage().getContent());
});

Python结果 #

I understand that you received a cheque for £425.00 from NS&I and tried to upload it using our cheque system on your online account. Your complaint is that our system does not accept government cheques, which now requires you to drive 16 miles and pay for parking to visit the nearest Big Bank branch.

Java结果 #

Dear Customer,

Thank you for reaching out to us. I understand your frustration regarding the check deposit issue involving the sum of $425. Please accept my sincere apologies for the inconvenience this has caused you.

From your message, it appears that you were trying to upload a check using the online system but encountered a problem since our system does not support checks from the government. This has now left you in a situation where you need to drive 16 miles to our nearest branch for parking, which adds to your frustration.

Please allow me to address your concerns systematically:

1. Understanding the Issue:

   • We understand that you received a check from the government and attempted to upload it using our online check system.
   • The system currently does not accept government-issued checks, causing you to resort to visiting a branch in person.

2. Customer Experience:

   • Your inconvenience in driving a significant distance to visit a branch is duly noted. We empathize with the unexpected trouble this has caused you.

Moving forward, here's what we can do to assist you:

1. Alternative Submission:

   • You can consider mailing the check to your branch. If you prefer this option, please let us know so we can provide you with the necessary details and ensure a prompt processing and response time.

2. Feedback to Improve:

   • Your feedback will be forwarded to our IT and service teams to evaluate the possibility of including government-issued checks in the online check-processing system.

3. Compensation for Inconvenience:

   • We would like to offer you a small compensation for your inconvenience. Please provide us with your mailing address, and we will send you a reimbursement for your travel expenses.

4. Personal Assistance:

   • If there are any other specific needs or assistance you require, please do not hesitate to let us know. We are here to help you.

Once again, I apologize for the inconvenience and appreciate your understanding and patience as we work to resolve this issue. Thank you for banking with us, and we look forward to providing you with better services in the future.

Sincerely,

[Your Name]

Customer Service Department

Big Bank

排查方向 #

• 验证提示内容一致性：检查Java代码中getUnderstandingSystemPrompt()和getUnderstandingUserPrompt()返回的内容，是否与Python中使用的提示完全一致（包括换行、特殊字符、空格）。可在Java中添加日志输出实际发送的消息内容，与Python端做逐字符对比。
• 核对API参数配置：确认Python和Java调用时的参数（如temperature、max_tokens、top_p）是否一致；检查Java中指定的部署模型"gpt4o_2024-05-13"是否与Python的deployment_model_chat指向同一部署。
• 检查请求头与API版本：确认Java端的请求头无额外干扰内容，Authorization格式正确；核对Python和Java使用的Azure OpenAI API版本是否一致，Java端需显式指定与Python相同的版本。
• 排查SDK行为差异：验证Java SDK是否对提示内容做了自动转义或格式化，尝试在Java中硬编码提示内容替换方法调用，确认是否为方法返回内容错误导致问题。

内容的提问来源于stack exchange，提问作者Rich