Skip to content

Add optional C++ rendering for HF chat templates - #222

Open
eoinjordan wants to merge 1 commit into
meta-pytorch:mainfrom
eoinjordan:codex/hf-chat-templates
Open

eoinjordan wants to merge 1 commit into
meta-pytorch:mainfrom
eoinjordan:codex/hf-chat-templates

Conversation

@eoinjordan

@eoinjordan eoinjordan commented Oct 2, 2026 •

Copy link
Copy Markdown

I've been using Kotlin for the application logic in some Android examples, and had a go at #131 while working on a small VLM example.

This is a first pass at loading the model's HF chat template in C++, so runners can build the prompt from the model files. It handles add_generation_prompt and kwargs like enable_thinking.

I used a pinned Minja dependency behind TOKENIZERS_BUILD_CHAT_TEMPLATE, which is off by default. It loads standalone Jinja, legacy tokenizer/processor config and explicitly selected named templates. There's a small command-line example and an exported tokenizers::chat_template target. The CI test builds enable the option so the new tests run.

I checked 30 cases against Transformers using pinned Qwen3 and SmolVLM templates, including Unicode, conversation history, image blocks, generation prompts and thinking options. All 16 Ubuntu test targets passed, including 14 renderer tests. A separate installed-library consumer passed the same 30 cases. The Android arm64 build passed with NDK r26c, and I ran the validation script on my Razr 50 Ultra: all 30 exact prompt comparisons passed there too.

Runner and Java/JNI integration would be the next step. Minja supports a subset of Jinja, and the added capitalize filter handles ASCII roles; unsupported Unicode case conversion returns an error. The example documents those limits.

Also checked the default-off build configuration, repository formatting/CMake/newline lint and git diff --check.

Related to #131.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 2, 2026
@eoinjordan
eoinjordan marked this pull request as ready for review October 2, 2026 21:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant