基于LangChain/FastAPI的自动作文反馈API JSON解析错误求助

问题概述 #

我基于LangChain、FastAPI（Python 3.8.18）开发了一款自动作文反馈API，可为输入作文提供纠错、详细反馈及质量评估服务。当前存在以下问题：

• 当作文与给定主题对齐时，API输出不稳定，有时纠错部分为空，甚至句子被拆分为单个字符
• 频繁触发JSON解析错误，典型错误提示：Error parsing JSON response: Expecting ',' delimiter: line 8 column 5 (char 339)

我当前使用的是ChatGPT组织版，想确认免费版是否会引发这类问题。

使用的Prompt模板 #

template = """
        Suppose, you are an English teacher. A student has submitted their essay. You need to provide a response in the following format.
     
        First, check if the {essay} is aligned with the provided {question}. If not, show status code 422 and the following JSON-
        - status: error,
        - message: Your essay is not aligned with the topic provided.
        - topic: {question}
        If yes, provide - 
        1. Sentence-by-sentence correction of the {essay}. List all sentences with what the student wrote and what your corrected version is. If there is no
        error in the sentence, write the sentence as it is
        2. Your feedback of the {essay} in details, your feedback should be like a human teacher. Write about the strong and weak points of the essay,
        3. Evaluate quality for the following essay: {essay}. Choose one of the following for quality: below satisfactory, 
        satisfactory, good, better, excellent.
    
        You are required to extract the feedback and the quality from the output. Construct JSON in the following format and add 
        the extracted data in the respective places
        Output JSON should be:
        - status: success,
        - message: your response is processed successfully,
        - correction: sentence-by-sentence correction of each essay, format - the original sentence is shown first and in the following line the 
        corrected sentence is shown.,
        - feedback: feedback of the essay,
        - quality: quality of the essay,
        - topic: {question}
    
        Keep the response within the token limit, do not give cut-off response. 
    """

API核心代码 #

prompt_template = PromptTemplate(
    input_variables=['question', 'essay'],
    template=template
)

# Pydantic model for input validation
class EssayInput(BaseModel):
    question: str
    essay: str

# Use dependency injection to create llm_chain within the route function
def get_llm_chain():
    return LLMChain(llm=llm, prompt=prompt_template)

MAX_CORRECTION_DISPLAY_LENGTH = 500  # Adjust this value as needed

@app.post('/process-essay')
async def process_essay(essay_input: EssayInput):
    try:
        # Check if the essay is aligned with the question
        if not is_essay_aligned(essay_input.question, essay_input.essay):
            raise HTTPException(status_code=422, detail="Your essay is not aligned with the topic provided.")

        # Convert the provided string to a JSON object
        essay_input_str = '{{"question": "{}", "essay": "{}"}}'.format(essay_input.question, essay_input.essay)
        essay_input_json = json.loads(essay_input_str)

        # Replace line breaks in the essay text
        cleaned_essay = essay_input_json['essay'].replace('\n', ' ')

                # Invoke the language model with only 'question' and cleaned 'essay'
        output = get_llm_chain().invoke({'question': essay_input_json['question'], 'essay': cleaned_essay})
        print("output 1", output)
        # Inside the try block where you process the output
        # Inside the try block where you process the output
        try:
            # Parse the 'text' as JSON
            output_text = json.loads(output['text'])

            # Add a print statement to inspect the content of output['text']
            print("Parsed JSON:", output_text)

            # Correct formatting of the 'correction' list in the JSON response
            # Correct formatting of the 'correction' list in the JSON response
            # Correct formatting of the 'correction' list in the JSON response
            correction_text = output_text.get('correction', [])

            # If correction_text is a string, convert it to a list with a single element
            if isinstance(correction_text, str):
                correction_text = [correction_text]

            # Each sentence in the correction list as a separate dictionary
            formatted_correction = [{'original': correction_text[i], 'corrected': correction_text[i + 1]} for i in range(0, len(correction_text) - 1, 2)]

            # Include generated 'feedback' and 'quality' in the response
            feedback = output_text.get('feedback', '')
            quality = output_text.get('quality', '')

            # Correct the quality field (remove single quote)
            quality = quality.replace("'", "")

            # Construct the response dictionary
            response_dict = {
                'status': 'success',
                'message': 'Your request has been processed successfully',
                'correction': formatted_correction,  # Use the corrected formatted_correction here
                'feedback': feedback,
                'quality': quality,
                'topic': essay_input_json['question']
            }



            # Convert the response dictionary to a nicely formatted JSON string
            response_json = json.dumps(response_dict, ensure_ascii=False, indent=4)

            # Return the JSON response
            return JSONResponse(
                content=response_dict,
                status_code=200,
                media_type='application/json'
            )

        except json.decoder.JSONDecodeError as json_error:
            # Return error response if there's an issue with JSON parsing
            return JSONResponse(
                content={'status': 'error', 'message': f"Error parsing JSON response: {str(json_error)}", 'data': None},
                status_code=500,
                media_type='application/json'
            )




    except HTTPException as http_error:
        # Return error response with status 500 for other errors
        return JSONResponse(
            content={'status': 'error', 'message': str(http_error.detail) if http_error.detail else 'Unknown error', 'data': None},
            status_code=500,
            media_type='application/json'
        )

def is_essay_aligned(question, essay):
    # Extract the expected essay topic from the question
    topic_marker = "on"
    topic_index = question.lower().find(f"{topic_marker} ")

    if topic_index != -1:
        expected_topic = question[topic_index + len(topic_marker) + 1 :]

        # Check if the expected topic is present in the essay
        aligned = expected_topic.lower() in essay.lower()

        return aligned

    # If "on" is not present, consider it as not aligned
    return False

常见错误输出示例 #

{
    "status": "error",
    "message": "Error parsing JSON response: Expecting ',' delimiter: line 8 column 5 (char 339)",
    "data": null
}

内容的提问来源于stack exchange，提问作者Sanji Vinsmoke