微调GPT-3.5 Turbo分类模型：如何解决超定义类别的异常响应？

背景与目标 #

目前正在微调「gpt-3.5-turbo-1106」模型，用于将事件严重度分为'a'、'b'、'c'、'd'四个类别（'a'为最轻微级别，'d'为最严重级别）。训练数据包含事件名称、描述、地点、关联风险及严重度标签。

训练数据实现代码 #

system_message = "This model is trained to classify the potential severity of events into four categories: { a , b , c , d }. These classes represent increasing levels of severity, with 'a' being the least severe and 'd' being the most severe. Please provide a response that accurately reflects the severity of the event."

completions = []
for _, row in df_hazards.iterrows():
    event_name = row['summary']
    event_description = row['detailed_description']
    impact_type = row['potential_severity_type']
    potential_severity = row['categorical_severity']
    exact_place = row['exact_location']
    risk = row['risk_event_title']
    prompt = textwrap.dedent(f"""\
    Event Name: {event_name}
    Event Description: {event_description}
    Exact Place: {exact_place}
    Impact Type: {impact_type}
    Associated Risk: {risk}""")
    # Remove all indentations at the beginning of each line
    prompt = re.sub(r'^\s+', '', prompt, flags=re.MULTILINE)
    completions.append({'messages': [{'role': 'system', 'content': system_message}, {'role': 'user', 'content': prompt}, {'role': 'assistant', 'content': potential_severity}]})

现存问题 #

• 模型验证阶段输出超出'a'、'b'、'c'、'd'限定类别，top logprobs出现「Event」「Action」「Safety」等非定义类token，尽管系统提示已明确限定类别范围。
• 训练数据存在严重类别不平衡：'a'类663条、'b'类146条、'c'类58条、'd'类仅10条。

提问 #

1. 是否可以通过调整现有微调流程，让模型更好地贴合限定的四个类别输出？
2. 若现有微调方案不可行，针对事件严重度精准分类的模型构建，有哪些具体建议？

内容的提问来源于stack exchange，提问作者LLM_Enthusiast