Skip to content

Updating chat template of LFM2 to render tools in correct format | enabling minja for that model - #4411

Open
przepeck wants to merge 4 commits into
mainfrom
przepeck/lfm2_chat_template_improvement
Open

Updating chat template of LFM2 to render tools in correct format | enabling minja for that model#4411
przepeck wants to merge 4 commits into
mainfrom
przepeck/lfm2_chat_template_improvement

Conversation

@przepeck

Copy link
Copy Markdown
Collaborator

🛠 Summary

CVS-186942

Changing the chat template of LFM2 to render executed tools. This chat templates also works with Minja.

BFCL results:
Multi_turn_base:
Multiturn score with old chat template: 0.266
Multiturn score with new chat template: 0.32

Simple_python:
Llama.cpp: 37.75
OVMS with new chat template: 39.25

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the LFM2 chat template used by OVMS LLM tests/examples so that prior assistant tool calls are serialized in the LFM2 tool-call token format (and can be processed by Minja), aligning prompt rendering with LFM2 tool-call expectations.

Changes:

  • Added Jinja macros to render tool_calls as <|tool_call_start|>[func(arg=...)]<|tool_call_end|> in the LFM2 chat template.
  • Updated end-to-end Minja/Jinja tests to expect LFM2 tool-call rendering and tool-call capability detection.
  • Added an extras/ example LFM2 chat template mirroring the test template.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/test/llm/chat_templates/chat_template_lfm2.jinja Adds tool-call rendering macros and emits tool calls in LFM2 format during prompt construction.
src/test/llm/chat_template_end_to_end_minja_test.cpp Updates Minja end-to-end expectations for LFM2 tool-call rendering/capabilities.
src/test/llm/chat_template_end_to_end_jinja_test.cpp Updates Jinja end-to-end expectations for LFM2 tool-call rendering/capabilities.
extras/chat_template_examples/chat_template_lfm2.jinja Introduces an example LFM2 template matching the updated tool-call rendering behavior.
Comments suppressed due to low confidence (2)

src/test/llm/chat_templates/chat_template_lfm2.jinja:86

  • The template renders tool-call markers whenever tool_calls is defined, even if it is an empty list. That produces <|tool_call_start|>[]<|tool_call_end|> in the prompt, which is likely unintended and can confuse downstream parsing.
    {%- if message["tool_calls"] is defined -%}
        {{- render_tool_calls(message["tool_calls"]) -}}
    {%- endif -%}

extras/chat_template_examples/chat_template_lfm2.jinja:86

  • The template renders tool-call markers whenever tool_calls is defined, even if it is an empty list. That produces <|tool_call_start|>[]<|tool_call_end|> in the prompt, which is likely unintended and can confuse downstream parsing.
    {%- if message["tool_calls"] is defined -%}
        {{- render_tool_calls(message["tool_calls"]) -}}
    {%- endif -%}

Comment on lines +4 to +10
{%- if arg_value is string -%}
{{- "'" + arg_value + "'" -}}
{%- elif arg_value is mapping -%}
{{- arg_value | tojson -}}
{%- else -%}
{{- arg_value | string -}}
{%- endif -%}
Comment on lines +4 to +10
{%- if arg_value is string -%}
{{- "'" + arg_value + "'" -}}
{%- elif arg_value is mapping -%}
{{- arg_value | tojson -}}
{%- else -%}
{{- arg_value | string -}}
{%- endif -%}
Comment thread extras/chat_template_examples/chat_template_lfm2.jinja
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants