Skip to content

vllm.entrypoints.openai.chat_completion.message_assembly

Functions:

assemble_chat_message_fields(request, reasoning, content, tool_calls, *, enable_auto_tools, parser_cls)

Pick the assistant message fields for one parsed chat choice.

Shared by OpenAIServingChat and batch chat derender so both surface the same reasoning, content and tool_calls for the same parser output.

Parameters:

  • request

    (ChatCompletionRequest) –

    Supplies tools, tool_choice, include_reasoning and parallel_tool_calls.

  • reasoning

    (str | None) –

    Reasoning from Parser.parse.

  • content

    (str | None) –

    Content from Parser.parse or the raw text without a parser.

  • tool_calls

    (list[FunctionCall] | None) –

    Tool calls from Parser.parse.

  • enable_auto_tools

    (bool) –

    The server's --enable-auto-tool-choice.

  • parser_cls

    (type[Parser] | None) –

    The server's parser class, if any.

Returns:

  • ChatMessageFields –

    The fields for the response ChatMessage.

Source code in vllm/entrypoints/openai/chat_completion/message_assembly.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
def assemble_chat_message_fields(
    request: ChatCompletionRequest,
    reasoning: str | None,
    content: str | None,
    tool_calls: list[FunctionCall] | None,
    *,
    enable_auto_tools: bool,
    parser_cls: type[Parser] | None,
) -> ChatMessageFields:
    """Pick the assistant message fields for one parsed chat choice.

    Shared by `OpenAIServingChat` and batch chat derender so both surface the
    same `reasoning`, `content` and `tool_calls` for the same parser output.

    Args:
        request: Supplies `tools`, `tool_choice`, `include_reasoning` and
            `parallel_tool_calls`.
        reasoning: Reasoning from `Parser.parse`.
        content: Content from `Parser.parse` or the raw text without a parser.
        tool_calls: Tool calls from `Parser.parse`.
        enable_auto_tools: The server's `--enable-auto-tool-choice`.
        parser_cls: The server's parser class, if any.

    Returns:
        The fields for the response `ChatMessage`.

    """
    if not request.include_reasoning:
        reasoning = None

    if not keeps_tool_calls(
        request, enable_auto_tools=enable_auto_tools, parser_cls=parser_cls
    ):
        return ChatMessageFields(reasoning, content, [])
    if _forces_tool_calls(request):
        content = content or ""
    return ChatMessageFields(reasoning, content, _to_tool_calls(tool_calls, request))

keeps_tool_calls(request, *, enable_auto_tools, parser_cls)

Whether a chat choice returns the tool calls its parser extracts.

Shared by the non-streaming and streaming chat paths so both apply tool_choice the same way. Parsers don't all gate on it themselves: HarmonyParser ignores "none" and null is parsed like "auto".

Parameters:

  • request

    (ChatCompletionRequest) –

    Supplies tools and tool_choice.

  • enable_auto_tools

    (bool) –

    The server's --enable-auto-tool-choice.

  • parser_cls

    (type[Parser] | None) –

    The server's parser class, if any.

Returns:

  • bool –

    True for "required", a named tool, or "auto" with tools and auto

  • bool –

    tool choice enabled.

Source code in vllm/entrypoints/openai/chat_completion/message_assembly.py 
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
def keeps_tool_calls(
    request: ChatCompletionRequest,
    *,
    enable_auto_tools: bool,
    parser_cls: type[Parser] | None,
) -> bool:
    """Whether a chat choice returns the tool calls its parser extracts.

    Shared by the non-streaming and streaming chat paths so both apply
    `tool_choice` the same way. Parsers don't all gate on it themselves:
    `HarmonyParser` ignores `"none"` and `null` is parsed like `"auto"`.

    Args:
        request: Supplies `tools` and `tool_choice`.
        enable_auto_tools: The server's `--enable-auto-tool-choice`.
        parser_cls: The server's parser class, if any.

    Returns:
        True for `"required"`, a named tool, or `"auto"` with tools and auto
        tool choice enabled.

    """
    if _forces_tool_calls(request):
        return True
    auto_tools = (
        enable_auto_tools
        and parser_cls is not None
        and parser_cls.tool_parser_cls is not None
    )
    if not auto_tools or request.tool_choice != "auto":
        return False
    if not request.tools:
        logger.error(
            "Cannot determine if tools should be extracted. "
            "Returning a standard chat completion."
        )
        return False
    return True