使用apply_chat_template实现函数调用时出错，寻求解决方法

测试场景 #

下载了tokenizer.json和tokenizer_config.json，用于测试函数调用并查看完整提示词，代码如下：

from transformers import AutoTokenizer

model_path = "/data1/ztshao/pretrained_models/deepseek-ai/DeepSeek-V3"
def get_current_temperature(location: str) -> float:
    """
    Get the current temperature at a location.
    
    Args:
        location: Get the location of the temperature in the format of "city, country"
    Returns:
        Displays the current temperature at the specified location as a floating point number (in the specified unit).
    """
    return 22.

tool_call = {"name": "get_current_temperature", "arguments": {"location": "Paris, France"}}
messages = [
  {"role": "system", "content": "You are a robot that responds to weather queries."},
  {"role": "user", "content": "What is the temperature in Paris now?"},
  {"role": "assistant", "tool_calls": [{"type": "function", "function": tool_call}]},
]

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
inputs = tokenizer.apply_chat_template(messages, 
                                       tools=[get_current_temperature],
                                       add_generation_prompt=False, 
                                       tokenize=False,
                                       tools_in_user_message=False)

错误1 #

测试时问题出在最后一条消息，报错信息：

Traceback (most recent call last):
  File "/data1/ztshao/projects/flooding/test/test.py", line 34, in <module>
    inputs = tokenizer.apply_chat_template(messages, 
  File "/data1/ztshao/.conda/envs/flooding/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1695, in apply_chat_template
    rendered_chat = compiled_template.render(
  File "/data1/ztshao/.conda/envs/flooding/lib/python3.10/site-packages/jinja2/environment.py", line 1295, in render
    self.environment.handle_exception()
  File "/data1/ztshao/.conda/envs/flooding/lib/python3.10/site-packages/jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 10, in top-level template code
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'content'

尝试在最后一条消息中添加"content":None键值对：

{"role": "assistant", "tool_calls": [{"type": "function", "function": tool_call}], "content":None},

但出现新错误。

错误2 #

Traceback (most recent call last):
  File "/data1/ztshao/projects/flooding/test/test.py", line 34, in <module>
    inputs = tokenizer.apply_chat_template(messages, 
  File "/data1/ztshao/.conda/envs/flooding/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1695, in apply_chat_template
    rendered_chat = compiled_template.render(
  File "/data1/ztshao/.conda/envs/flooding/lib/python3.10/site-packages/jinja2/environment.py", line 1295, in render
    self.environment.handle_exception()
  File "/data1/ztshao/.conda/envs/flooding/lib/python3.10/site-packages/jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 6, in top-level template code
TypeError: can only concatenate str (not "dict") to str

问题在于tool['function']['arguments']是字典类型，无法直接与字符串拼接。

模板问题分析 #

查看jinja2模板后发现两个问题：

1. 模板定义了message['content']为None的处理逻辑，但原始消息中未包含content字段时会报错；
2. 模板直接将字典类型的tool['function']['arguments']与字符串拼接，导致类型错误。

模板代码片段：

{%- if message['role'] == 'assistant' and message['content'] is none %}
    {%- set ns.is_tool = false -%}
    {%- for tool in message['tool_calls'] %}
        {%- if not ns.is_first %}
            {{ '<｜Assistant｜><｜tool calls begin｜><｜tool call begin｜>' + tool['type'] + '<｜tool sep｜>' + tool['function']['name'] + '\n```json\n' + tool['function']['arguments'] + '\n```<｜tool call end｜>' }}
            {%- set ns.is_first = true -%}
        {%- else %}
            {{ '\n<｜tool call begin｜>' + tool['type'] + '<｜tool sep｜>' + tool['function']['name'] + '\n```json\n' + tool['function']['arguments'] + '\n```<｜tool call end｜>' }}
            {{ '<｜tool calls end｜><｜end of sentence｜>' }}
        {%- endif %}
    {%- endfor %}
{%- endif %}

完整jinja2模板代码：

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='', is_first_sp=true) %}{%- for message in messages %}{%- if message['role'] == 'system' %}{%- if ns.is_first_sp %}{% set ns.system_prompt = ns.system_prompt + message['content'] %}{% set ns.is_first_sp = false %}{%- else %}{% set ns.system_prompt = ns.system_prompt + '\n\n' + message['content'] %}{%- endif %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\n' + '```json' + '\n' + tool['function']['arguments'] + '\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{{'<｜Assistant｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜>'}}{% endif %}

问题咨询 #

请问是tokenizer_config.json存在问题，还是我遗漏了什么配置？

内容的提问来源于stack exchange，提问作者happycat