Skip to content

mtmd: add n_embd_head - #26342

Merged
ngxson merged 1 commit into
masterfrom
xsn/clip_n_embd_head
Jul 31, 2026
Merged

mtmd: add n_embd_head#26342
ngxson merged 1 commit into
masterfrom
xsn/clip_n_embd_head

Conversation

@ngxson

@ngxson ngxson commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Overview

extracted change from unslothai#48 (comment)

it will be used by more than one model, so this is a dedicated PR to allow git bisect in if there are any problems

Additional information

Test results: some models fail, but it's the same as what we get on master (unrelated bugs)

[vision] OK:   ggml-org/SmolVLM-500M-Instruct-GGUF:Q8_0
[vision] OK:   ggml-org/SmolVLM2-2.2B-Instruct-GGUF:Q4_K_M
[vision] OK:   ggml-org/SmolVLM2-500M-Video-Instruct-GGUF:Q8_0
[vision] OK:   ggml-org/gemma-3-4b-it-GGUF:Q4_K_M
[vision] OK:   THUDM/glm-edge-v-5b-gguf:Q4_K_M
[vision] OK:   second-state/Llava-v1.5-7B-GGUF:Q2_K
[vision] OK:   cjpais/llava-1.6-mistral-7b-gguf:Q3_K_M
[vision] OK:   ibm-research/granite-vision-3.2-2b-GGUF:Q4_K_M
[vision] OK:   second-state/MiniCPM-Llama3-V-2_5-GGUF:Q2_K
[vision] OK:   openbmb/MiniCPM-V-2_6-gguf:Q2_K
[vision] OK:   openbmb/MiniCPM-o-2_6-gguf:Q4_0
[vision] OK:   bartowski/Qwen2-VL-2B-Instruct-GGUF:Q4_K_M
[vision] FAIL: ggml-org/Qwen2.5-VL-3B-Instruct-GGUF:Q4_K_M
[vision] OK:   ggml-org/InternVL2_5-1B-GGUF:Q8_0
[vision] OK:   ggml-org/InternVL3-1B-Instruct-GGUF:Q8_0
[vision] OK:   ggml-org/Qwen2.5-Omni-3B-GGUF:Q4_K_M
[vision] OK:   ggml-org/LFM2-VL-450M-GGUF:Q8_0
[vision] OK:   ggml-org/granite-docling-258M-GGUF:Q8_0
[vision] FAIL: ggml-org/LightOnOCR-1B-1025-GGUF:Q8_0
[vision] OK:   ggml-org/DeepSeek-OCR-GGUF:Q8_0
[vision] FAIL: ggml-org/dots.ocr-GGUF:Q8_0
[vision] OK:   ggml-org/HunyuanOCR-GGUF:Q8_0
[vision] OK:   ggml-org/gemma-4-E2B-it-GGUF:Q8_0
[audio]  OK:   ggml-org/ultravox-v0_5-llama-3_2-1b-GGUF:Q8_0
[audio]  OK:   ggml-org/Qwen2.5-Omni-3B-GGUF:Q4_K_M
[audio]  OK:   ggml-org/Voxtral-Mini-3B-2507-GGUF:Q4_K_M
[audio]  OK:   ggml-org/LFM2-Audio-1.5B-GGUF:Q8_0
[audio]  OK:   ggml-org/gemma-4-E2B-it-GGUF:Q8_0
[audio]  OK:   ggml-org/Qwen3-ASR-0.6B-GGUF:Q8_0

Requirements

Co-authored-by: Daniel Han <unslothai@gmail.com>
@ngxson
ngxson requested review from a team and CISC as code owners July 30, 2026 20:30
@github-actions github-actions Bot added mtmd Related to multimodal functionality (video/image/audio) conversion labels Jul 30, 2026
@ngxson

ngxson commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator Author

bisect track back the problem to #20962 , so confirmed that it's unrelated to this change, and will be fixed in a dedicated PR

@ngxson
ngxson merged commit db7d8b2 into master Jul 31, 2026
24 of 29 checks passed
kashif pushed a commit to kashif/llama.cpp that referenced this pull request Aug 2, 2026
Co-authored-by: Daniel Han <unslothai@gmail.com>
huaxel pushed a commit to huaxel/CachyLLama that referenced this pull request Aug 2, 2026
Co-authored-by: Daniel Han <unslothai@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conversion mtmd Related to multimodal functionality (video/image/audio)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant