Performance of llama.cpp on Intel GPU with SYCL backend #23313
Replies: 36 comments 96 replies
|
compiled with cmake -B build-sycl -DGGML_SYCL=ON -DGGML_SYCL_F16=ON -DGGML_SYCL_TARGET=INTEL -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DCMAKE_C_FLAGS="-march=znver4" -DCMAKE_CXX_FLAGS="-march=znver4" -DCMAKE_BUILD_TYPE=Release && cmake --build build-sycl --config Release -j 16 single b70 dual b70: |
|
If instead compiling and using with f16=off: cmake -B build-sycl -DGGML_SYCL=ON -DGGML_SYCL_F16=OFF -DGGML_SYCL_TARGET=INTEL -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DCMAKE_C_FLAGS="-march=znver4" -DCMAKE_CXX_FLAGS="-march=znver4" -DCMAKE_BUILD_TYPE=Release && cmake --build build-sycl --config Release -j 16 Single B70: Dual B70: |
|
And with a much more interesting model, namely Qwen 3.6 27B: q4 and q8: |
|
@FCLC Thank you! |
|
Ooft. A770 16GB, i5 14600k, current cachyOS. fp16: fp32: Very weird, compared with the one in the table - much better prefill, half the decode performance. |
|
From the change of performance, I guess the Flash-attention is enabled in second case. 34 token/s is very similar with my old test result, which is impacted by driver. Could you run: |
|
Interesting. Just tried it with the FA switch: So...I do get a bit of a boost from FA on! I don't know what's happened with the performance from recompiling it, it's just fallen off a cliff. The Could it be the i915 driver that's the problem? |
|
Yes, this is the root cause. FA's impact is still less than I expected. Please remove one of them. It's risk to remove the GPU driver. |
|
Aha...winning! Mildly frustrating that I can't reproduce that nice PP result now, though. |
|
It's known issue. We are checking it. It's great to see better performance in your test. Thank you! |
|
B580, AMD Ryzen 7 5700X3D, Ubuntu 25.10 built with fp16 in |
|
Update as your feedback! |
|
Intel Arc Pro B50, Intel i7-8700 32GB RAM build: c0c7e14 (9298)
build: 2f6c815 (9397)
Command line arguments: Build options: cmake .. -B build -DGGML_VULKAN=1 -DGGML_RPC=ON nothing else changed between these runs, I tested my old version, ran "git pull", built it and retested |
|
@thordarsen Because your test result differs vastly from the common results as we known, I suggest not adding them in the table. how do you think? Thank you! |
|
It's an Arc Pro B50 and actually looking at the relative specs on Intel Ark I'm kinda inline with the B570 and B580 from the above data for FP16
( I purchased the B50 for the 16GB VRAM and 70W draw ) |
|
I haven't B50. |
B50 owner here as well. The B50 doesnt get the recognition it deserves, 16GB VRAM at max 70W makes it an out of this world addition to something like my unraid server. Ive got it doing Plex transcodes, Ollama, Llama.cpp, A-eye (image cataloguing), Memories (nextcloud). For a 24x7 machine its tough to beat. It isnt a Ferrari, and it isnt meant to be. Its a Camry. It gets you to where youre going, comfortably, and cheaply. |
|
@shansen008 Thank you for your sharing the usage of B50! |
|
I hope it helps. 255H, ARC 140T, 32GB RAM
build: d4c8e2c (9442) |
|
@jlionhan Thank you! |
|
~/llama.cpp$ cmake -B build/ReleaseOV -G Ninja -DCMAKE_BUILD_TYPE=Release -DGGML_OPENVINO=ON :~/llama.cpp$ GGML_OPENVINO_STATEFUL_EXECUTION=1
specs https://www.asrockind.com/en-gb/NUC%20BOX-358H |
|
📊 Intel Panther Lake Xe3 iGPU (12 EU) Benchmark Matrix: OpenVINO vs. Vulkan vs. SYCL Benchmarking sweep across all three major acceleration backends available in Environment
Models Tested
📈 Performance Summary Matrix
🛠️ Deep-Dive Analysis
📋 Raw Build & Execution Logs 1. Qwen3 80B MoE — OpenVINO Crash Log |
|
@bobguns Thank for your sharing! |
Okay i understand, made some new benchmarks with 512 input tokens Hardware & Environment
📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: d5ab083 (9476) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: d5ab083 (9476) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: d5ab083 (9476) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: d5ab083 (9476) |
|
16 bit Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: a468b89 (9477) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: a468b89 (9477) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: a468b89 (9477) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: a468b89 (9477) 📊 Running SYCL Benchmark on Panther Lake Xe3 iGPU
build: a468b89 (9477) |
|
@bobguns Thank you for your sharing! |
|
HW:ryzen5 5600X, DDR4-3600 128GB, ARC B570 FP32 FP16 |
|
@yqYo1 |
|
HW:ryzen5 5700X, DDR4-3600 64GB, ARC B580 + ARC PRO B60 (24gb) llama build b60+b580
build: 65ef50a (9501) ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -fa 1
build: 65ef50a (9501) on arc b60 ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -fa 0 -dev sycl0
build: 65ef50a (9501) ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev sycl0 -fa 1
build: 65ef50a (9501) on arc b580 ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev sycl1 -fa 0
build: 65ef50a (9501) ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev sycl1 -fa 1
build: 65ef50a (9501) VULKAN After promt processing, the GPU frequency is reset to minimum and TG is low ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -fa 0
build: 65ef50a (9501) I set the minimum frequency on the GPU to b60 2300 and b580 2683 ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -fa 0
build: 65ef50a (9501) ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -fa 1
build: 65ef50a (9501) on B60 ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev vulkan1 -fa 0
build: 65ef50a (9501) ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev vulkan1 -fa 1
build: 65ef50a (9501) on b580 ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev vulkan0 -fa 0
build: 65ef50a (9501) ./llama-bench -m ~/llama-2-7b.Q4_0.gguf -dev vulkan0 -fa 1
build: 65ef50a (9501) |
|
@Andr010-lang Thank you for your sharing! |
Arc A380I know, I know... I got this it for its AV1 encoding/decoding, not LLMs, but here we are...
F16
F32
Got a couple of warnings during execution:
Other info: Build config (F16/F32 variations):
Found 1 SYCL devices:
SYCL Optimization Feature:
build: 7c158fb (b9518) |
|
@GerardoNevarez If you don't care the performance and running time, you could run a 2-7B LLM by llama-server on existed Intel GPU (like A380) to help to create a report or abstract based on your raw texts. Improve your work and life without cost. Thank you for your sharing! |
|
Updated Arc A380 bench runs with default build on e82beaa (9534) - you could use these for the table above. Forgot to mention, this rig uses PCIe 3.0 only, but has ReBar enabled (SYCL refuses to use the xe driver otherwise) F16
F32
"Static" build e82beaa (9534) with F16
F32
|
|
@GerardoNevarez But your case shows it is increased. Thank you! |
|
@arthw I ran it twice before posting, I was surprised than the "official" build is slower, I'm running natively on Ubuntu, not in Docker. I just pulled updates from GH and ran them again, results below (without FA) - they are a bit worse than the last time, I can't explain that either. "Official" build "Static" build My CPU is an old Xeon, but it has 45MB L3 cache, on a single die. Maybe communication between CPU/GPU gets improved by that - less long call jumps to the libraries? Does not explain your results on the A770 though. |
|
@GerardoNevarez We will check it later. |
|
Hi,
./llama-bench -fa 0,1 -m ../../models/llama-2-7b.Q4_0.gguf
build: 6471e3c (9607) sycl-ls Thanks for your great work! |
A380 - DockerApologies for the testing in docker, my local env is messed up in all sort of ways, im unable to test on bare metal however Ill share the docker compose and commands to reproduce if anyone is interested. Ill also attach other benchmarks to compare the current state as of this commit e95dae1 All results are the third results printed SYCL F16
SYCL F32
I ran a similar docker image a few days ago and I remember my results being far far better, not sure what has happened with the pp. Vulkanggml_vulkan: Found 1 Vulkan devices:
OpenvinoOpenVINO: using device GPU
Docker composeservices:
bench-openvino:
build:
context: .
dockerfile: .devops/openvino.Dockerfile
target: full
image: llama.cpp:full-openvino-local
devices:
- /dev/dri:/dev/dri
volumes:
- llama-cache:/models
environment:
- LD_LIBRARY_PATH=/app
- GGML_OPENVINO_DEVICE=${GGML_OPENVINO_DEVICE:-GPU}
- GGML_OPENVINO_STATEFUL_EXECUTION=1
- LLAMA_CACHE=/models
entrypoint: /app/llama-bench
command:
- -hf
- ${HF_REPO:-TheBloke/Llama-2-7B-GGUF:Q4_0}
- -fa
- "1"
- -ngl
- "99"
bench-sycl-f16:
build:
context: .
dockerfile: .devops/intel.Dockerfile
target: full
args:
GGML_SYCL_F16: "ON"
image: llama.cpp:full-sycl-f16-local
devices:
- /dev/dri:/dev/dri
volumes:
- llama-cache:/models
environment:
- LLAMA_CACHE=/models
- ONEAPI_DEVICE_SELECTOR=level_zero:0
- ZES_ENABLE_SYSMAN=1
entrypoint: /app/llama-bench
command:
- -hf
- ${HF_REPO:-TheBloke/Llama-2-7B-GGUF:Q4_0}
- -fa
- "1,0"
- -ngl
- "99"
bench-sycl-f32:
build:
context: .
dockerfile: .devops/intel.Dockerfile
target: full
args:
GGML_SYCL_F16: "OFF"
image: llama.cpp:full-sycl-f32-local
devices:
- /dev/dri:/dev/dri
volumes:
- llama-cache:/models
environment:
- LLAMA_CACHE=/models
- ONEAPI_DEVICE_SELECTOR=level_zero:0
- ZES_ENABLE_SYSMAN=1
entrypoint: /app/llama-bench
command:
- -hf
- ${HF_REPO:-TheBloke/Llama-2-7B-GGUF:Q4_0}
- -fa
- "1,0"
- -ngl
- "99"
bench-vulkan:
build:
context: .
dockerfile: .devops/vulkan.Dockerfile
target: full
image: llama.cpp:full-vulkan-local
devices:
- /dev/dri:/dev/dri
volumes:
- llama-cache:/models
environment:
- LLAMA_CACHE=/models
entrypoint: /app/llama-bench
command:
- -hf
- ${HF_REPO:-TheBloke/Llama-2-7B-GGUF:Q4_0}
- -fa
- "1,0"
- -ngl
- "99"
volumes:
llama-cache:
name: llama-cacheCommandsdocker compose run --build --rm bench-openvino # OpenVINO
docker compose run --build --rm bench-sycl-f16 # SYCL F16
docker compose run --build --rm bench-sycl-f32 # SYCL F32
docker compose run --build --rm bench-vulkan # VulkanFor repeated runs remove the |
|
@arthw dpkg -l | grep libze-intel-gpu1 SYCL F16
SYCL F32
Openvino
Vulkan
Performance for SYCL F16 improved after driver update, vulkan still low somehow. Openvino PP is insane however model selection is still catching up. |
|
@toomanybyt3s Could you check the driver by following cmds? |
Ah missing piece drivers, let me update them and ill rerun tests |
|
Uname shows this as my kernal, i might also update my system to 25.10 |
|
@toomanybyt3s Could you disable i915 to enable Xe driver on A380? Thank you! |
|
@toomanybyt3s try adjusting which driver is used, modifying the
This forces enabling the Xe and disabling i915 drivers for the card, value must match the PCIe Device ID for the card ( A380 = |
|
@GerardoNevarez , I did try to play around with XE drivers but since my server is mostly for jellyfin I couldnt get av1 decoding to work with the XE driver, im planning to get a intel arc b60 and continue with sycl backend and forcing the XE driver, since I read somewhere the newer XE driver is able to decode av1 with the b60 but not the a380 |
|
Oh, that's interesting; I'll give it a try. cmake --build build-sycl --config Release -j |
GGML_SYCL_ENABLE_MKL_FA=1It's opened as default. In code: commit 11924d4 (tag: b10223, origin/master, master) It will get performance increase in more LLMs event with fp32 building on B60: Test Script
Environment Configurations
Per Metric
|
GGML_SYCL_FA_ONEDNN=1 on PTL (Intel Arc B390)I was curious about the performance impact of the oneDNN SDPA path introduced by PR #25222, so I benchmarked three paths: oneDNN / MKL / FA-off. I know oneDNN is currently only enabled on BMG (Battlemage) hardware, but I had an LLM run a test against the reproduction scheme of uxlfoundation/oneDNN#5510, and at least on PTL + Level Zero 1.15.39122 there were no computation errors. In code: commit 0713275 (build: b10270) To enable the oneDNN path on PTL, the BMG-only hardware check in Test Script
Per Metricllama-2-7b.Q4_0.gguf (llama 7B Q4_0)
gemma-4-26B-A4B-it-qat-heretic-UD-Q4_K_XL.gguf (gemma4 26B.A4B Q4_0)
Hardware
Software
Notes
|
|
I test it on Arc770 for PR: #25222 Arc770
As the comments of the code: We need to check if the above issue (wrong result) is fixed on target GPU. |
|
Wow, what a difference. I have a benchmark from a week ago to compare with. Spectacular: Qwen3.6-27B-Q5_K_M Performance ComparisonAll benchmarks were run with identical llama.cpp runtime parameters. The only compilation changes were: -DGGML_SYCL_DEVICE_ARCH=xe2 -DGGML_SYCL_DNN=ONThe second run also used a larger context size, but this does not affect these benchmark results. Runtime Parameters (identical in both runs)
Speculative Decoding--spec-type draft-mtp --spec-draft-n-max 3Baseline BuildBuild ConfigurationBackend: SYCL
Precision: FP16Optimized BuildBuild ConfigurationBackend: SYCL
Precision: FP16
-DGGML_SYCL_DEVICE_ARCH=xe2 -DGGML_SYCL_DNN=ONPrompt Processing Performance
|
|
In code: commit f8e3026 (tag: b10322, origin/master, master) Test Script
This is a community fine-tuned version of standard gemma4 12b model - yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF. Other models shared the same command options. Build Configurations
My setup
Metric
Interestingly, error "failed to create context" was raised when fa=0. Besides, I didn't see any notable speed differences between MKL_FA and DNN_FA, only on this gemma4 and granite and not on llama2 though... |
2. Thank you for your sharing! |
|
I have swept a few parameters trying to improve the decode speed of some modern useful models (Muse Glimmer and Qwen 3.8 27B) on my double-GPU setup:
I built llama.cpp from source, commit 4df29be using the following command: Over the sweep, F16 made the biggest impact on the performance, with AOT helping squeeze a few more percent. On Muse Glimmer (bartowski/Muse-Glimmer-30B-Q4_K_S.gguf), this translated into the following results on the i915 driver:
GGML_SYCL_DEV2DEV_MEMCPY had no impact. Xe DriverI wanted to improve FA performance further, as well as get better tensor split decode results. The first step was to move to the Xe driver, which yielded:
Note both PP and TG improvements, especially with Flash Attention. These figures are on par with Vulkan+i915 on decode (~17tps) while being much better on prefill. GGML_SYCL_DEV2DEV_MEMCPY still had no effect. IOMMU PTAs I wanted to improve the tensor split, I set IOMMU to the passthrough mode. The same benchmark command gave the following results:
Note that the layer split is not affected, while in the tensor split mode the prefill is 466->580 - 24% faster, outperforming the layer split mode! The model is coherent. No luck on decode yet :( Counterintuitively (?), GGML_SYCL_DEV2DEV_MEMCPY=1 actually decreases the performance:
Setting attention cache to q8_0 (GGML_SYCL_DEV2DEV_MEMCPY=0) makes both prefill and decode a bit slower:
Bonus: Qwen3.8-27BWith lmstudio-community/Qwen3.8-27B-Q4_K_M.gguf, the best result is:
Takeaways
Hope this is useful (and also hope decode can be made faster). |
|
It's good result. Thank your sharing! |
|
Are these suggestions applicable to Windows? |
|
Same as Linux. |
Environment
Build optionscmake -B build/sycl -G Ninja \
-DGGML_SYCL=ON \
-DGGML_SYCL_TARGET=INTEL \
-DGGML_SYCL_DNN=ON \
-DGGML_SYCL_F16=ON \
-DGGML_SYCL_DEVICE_ARCH=bmg-g31 \
-DGGML_NATIVE=ON \
-DMKL_DIR=/opt/intel/oneapi/mkl/latest/lib/cmake/mkl \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_RPATH="\$ORIGIN/../lib" \
-DCMAKE_CXX_FLAGS="-ffast-math -O3" \
-DCMAKE_C_FLAGS="-ffast-math -O3"Benchmark options./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL \
-ctk q8_0 -ctv q8_0 -fa on -ngl 99 \
-p 512,1024 -n 512
./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL \
-ctk q8_0 -ctv q8_0 -fa on -ngl 99 \
-p 512,1024 -n 512
./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Qwen3.8-27B-GGUF:UD-Q6_K_M \
-ctk q8_0 -ctv q8_0 -fa on -ngl 99 \
-p 512,1024 -n 512Qwen 3.8 27B (Q4, Q5, Q6) - Q5 has the best performance.
build: cd26896 (10553)
build: cd26896 (10553)
build: cd26896 (10553) |
|
@virus-warnning But it won't impact the data in this post be searched by user as reference. Thank you for your sharing! |
|
Should I bench again for pp512 + tg128? |
|
I noticed that the llama-2-7B Q4_0 benchmarks for Intel B70 are very outdated and deserves a refresh. So here we go, I've done a refresher today and also threw in some Qwen3.8-27b UD-Q8_K_L results too! It's amazing to see how far the Intel B70 has come in the recent updates to the llama.cpp project by all you amazing people as well as Intel's own driver team's efforts to improve things. Seriously big shoutout to all the awesome people I've chatted with on here. Environment
Build optionsFor the FP16 build: Benchmark optionsSYCL FP16 - llama-2-7B TheBloke Q4_0
build: d222767 (10630) SYCL FP16 - qwen3.8-27B Unsloth UD Q8_K_L
build: d222767 (10630) SYCL FP32 - llama-2-7B TheBloke Q4_0
build: d222767 (10630) SYCL FP32 - qwen3.8-27B Unsloth UD Q8_K_L
build: d222767 (10630) Vulkan Build from Intel Dev's PRsThese final tables are not SYCL but rather my custom Vulkan build from the Intel dev's work (#24408 (comment)), just posting it here too to have the data in the one place. This is the amazing work they've managed to do with the FA=1 branch and the PP. This shows that the Intel B70 is very competitive against competing products with similar VRAM but are more expensive than the Intel B70. Vulkan Intel Dev Build #24408 (comment) - llama-2-7B TheBloke Q4_0
build: 9da5214fd (9595) Vulkan Intel Dev Build #24408 (comment) - qwen3.8-27B Unsloth UD Q8_K_L
build: 9da5214fd (9595) |
|
If there are any other models you want me to test to update the table above with the current SYCL 32/16 on Windows, please let me know! |
|
@SleepinDevil Yes, SYCL backend has very good improvement of functionility and performance by the support of comminuty developers and user. It's great if you can provide the data of more LLMs. Thank you! |
|
Here is my test data on B70, Ubuntu 26.04, host 48GB, DDR4, Intel i5-12490F,
build: 4d19b28 (10636) Build with fp16:
build: 4d19b28 (10636) |
Environment
Results of Devstral-Small-2-24B-Instruct-2512-GGUF./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_0 \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q5_K_M \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q6_K \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q8_0 \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q4_K_XL \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q5_K_XL \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:UD-Q6_K_XL \
-fa on -ctk q8_0 -ctv q8_0 -ngl 99
build: bb4caa7 (10566) ./llama.cpp/sycl/bin/llama-bench \
-hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_0 \
-fa 0,1 -ngl 99
build: bb4caa7 (10566) |
|
FP32 or FP16 built? |
|
@virus-warnning Thank you! |
I'll follow the comparison table later. Thanks. |
FP16 |
|
Got it! Update is done! |
Environment
Build optionscmake -B build/sycl -G Ninja \
-DGGML_SYCL=ON \
-DGGML_SYCL_TARGET=INTEL \
-DGGML_SYCL_DNN=ON \
-DGGML_SYCL_F16=ON \
-DGGML_SYCL_DEVICE_ARCH=bmg-g31 \
-DGGML_NATIVE=ON \
-DGGML_LTO=ON \
-DGGML_BLAS=ON \
-DGGML_BLAS_VENDOR=Intel10_64lp \
-DMKL_DIR=/opt/intel/oneapi/mkl/latest/lib/cmake/mkl \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_RPATH="\$ORIGIN/../lib" \
-DCMAKE_CXX_FLAGS="-O3 -fp-model=fast=2 -qopenmp" \
-DCMAKE_C_FLAGS="-O3 -fp-model=fast=2 -qopenmp"llama-bench./llama.cpp/sycl/bin/llama-bench -hf bartowski/Qwen3.8-27B-GGUF:IQ3_XXS -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf bartowski/Qwen3.8-27B-GGUF:Q4_0 -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf bartowski/Qwen3.8-27B-GGUF:Q4_K_L -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf bartowski/Qwen3.8-27B-GGUF:Q5_K_L -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf bartowski/Qwen3.8-27B-GGUF:Q6_K_L -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:Q4_0 -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL -fa 0,1
./llama.cpp/sycl/bin/llama-bench -hf unsloth/Qwen3.8-27B-GGUF:UD-Q6_K -fa 0,1Performancebartowski/Qwen3.8-27B-GGUF:IQ3_XXS on SYCL 16
build: d7a2074 (10644) bartowski/Qwen3.8-27B-GGUF:Q4_0 on SYCL 16
build: d7a2074 (10644) bartowski/Qwen3.8-27B-GGUF:Q4_K_L on SYCL 16
build: d7a2074 (10644) bartowski/Qwen3.8-27B-GGUF:Q5_K_L on SYCL 16
build: d7a2074 (10644) bartowski/Qwen3.8-27B-GGUF:Q6_K_L on SYCL 16
build: d7a2074 (10644) unsloth/Qwen3.8-27B-GGUF:Q4_0 on SYCL 16
build: d7a2074 (10644) unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL on SYCL 16
build: d7a2074 (10644) unsloth/Qwen3.8-27B-GGUF:UD-Q6_K on SYCL 16
build: d7a2074 (10644) |
|
Interesting 🤔, in my mind I thought the Q4-Q6 speeds would be a lot higher, I wonder if something is up there. I am downloading the models right now and will have to give it a shot too to see |
|
@virus-warnning |
|
Maybe I have to test again without BLAS options. |
|
@virus-warnning so I tested this some more, my results are close to yours and I tried out the Q4-Q6 K_XL models instead. So there we go, I never really tested it before, but turns out Q4-Q6 of the Qwen3.8 ggufs are actually slower PP to run than the Q8 of the model, but seems there is an improvement of TG as you reduce the Quant 🤔. I guess this is why a lot of people say Q6 is the sweet point for local agentic coding use. Good mix of intelligence and also good TG speed, and not much PP speed lost from Q8 Environment
SYCL FP16 - qwen3.8-27B Unsloth UD Q4_K_XL
SYCL FP16 - qwen3.8-27B Unsloth UD Q5_K_XL
SYCL FP16 - qwen3.8-27B Unsloth UD Q6_K_XL
|
Intel B60 TestsNotesThis is my first time contributing to anything like this I've been mostly messing around with different models and testing different setups and harnesses so I haven't done anything real with local LLMs; I then found this and figured I could contribute and be helpful in some way as I don't see any B60s here. I'm willing to test more models that are of interest, I have a few in mind but I figured but starting with Llama 2 and figured some more experienced people taking a glance before I commit too hard to testing couldn't hurt. Let me know what additional Info Could be Helpful for future runs. Host and Toolboxes should be on the same versions of intel related stuff. Environment: TheBloke llama 2 7B (Llama.cpp F16)
build: fe235f4 (10650) TheBloke llama 2 7B (Llama.cpp F32)
build: fe235f4 (10650) |
|
Looking at some of these other results I'm thinking I should eye up a CPU upgrade 👀 |
|
@caqdev Thank you for your sharing! |
|
Thanks, you're right, but I do like using larger context lengths so I do have to rely on the CPU RAM side of my machine during actual usage. This was actually a fun exercise because I never see such speeds except right at the first few hundred tokens of Prompt Processing on a new request. |
|
@caqdev what is the drop off in PP/TG you have experienced when you are overflowing onto CPU/RAM? I've got the 5950X and I have to admit I haven't tested your scenario a lot, but in the very early day of me dabbling with this, I tried it (about 5 months ago now), I reckon my PP dropped from that 900-500 range at 80k context to something around 50-20 PP ingest speed, and TG dropped from around the 15-10 mark at 80k to less than 1 🫠 - I distinctly remember this drop off and feeling very disheartened about local LLMs being too expensive to run at that point. So then I ended up taking a gamble with the Intel B70 and that turned out to be way better than doing my 3080Ti+5950X combination on larger models. At the moment I am using my Intel B70 + 3080Ti in Vulkan mode with layer split, actually works really well to be honest with you! Maybe getting a second B60 and sticking to the 3700X is an option for you? I think SYCL has implemented tensor split too, would be interesting to see how you go with that 🤔. |
Intel B60 TestsPotentially better formatted for the main table
These are the models that I already had on hand, but if I only had an IQ quant or a sub Q4_K_M quant I downloaded the (UD-)Q4_K_M to help people compare a common recommendation As you can see these use a variety of architecture types so I help it can help people in that regard in case you find finetunes based on some of these models Let me know what additional Info Could be Helpful for future runs. Host and Toolboxes should be on the same versions of intel related stuff. I think this is the bulk of what I wanted to contribute. Though if I remember this thread when new models (especially with different architectures) are released and supported in llama.cpp I may return. I hope this information helps people Environment: Ornith AI (Deepreinforce) 1.5 35B-3B Active Q4_K_M (Mixture of Experts) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--ornith-ai--Ornith-1.5-35B-A3B-GGUF/snapshots/fbbaed45c2f0e200276ffa51701a24d45dc7f57e/Ornith-1.5-35B-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Ornith AI (Deepreinforce) 1.5 35B-3B Active Q4_K_M (Mixture of Experts) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--ornith-ai--Ornith-1.5-35B-A3B-GGUF/snapshots/fbbaed45c2f0e200276ffa51701a24d45dc7f57e/Ornith-1.5-35B-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Ornith AI (Deepreinforce) 1.5 8B Q6_K (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--ornith-ai--Ornith-1.5-9B-GGUF/snapshots/85bf2b98cdcbad4291cb4f46943526cc089f75a0/Ornith-1.5-9B-Q6_K.gguf -fa 0,1
build: ca3d5a3 (10665) Ornith AI (Deepreinforce) 1.5 8B Q6_K (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--ornith-ai--Ornith-1.5-9B-GGUF/snapshots/85bf2b98cdcbad4291cb4f46943526cc089f75a0/Ornith-1.5-9B-Q6_K.gguf -fa 0,1
build: ca3d5a3 (10665) LG AI EXAONE 4.5 33B IQ4_XS (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--LGAI-EXAONE--EXAONE-4.5-33B-GGUF/snapshots/0e969634ef24db05151b435970297a6dee634b7e/EXAONE-4.5-33B-IQ4_XS.gguf -fa 0,1
build: ca3d5a3 (10665) LG AI EXAONE 4.5 33B IQ4_XS (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--LGAI-EXAONE--EXAONE-4.5-33B-GGUF/snapshots/0e969634ef24db05151b435970297a6dee634b7e/EXAONE-4.5-33B-IQ4_XS.gguf -fa 0,1
build: ca3d5a3 (10665) LG AI EXAONE 4.5 33B Q4_K_M (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--LGAI-EXAONE--EXAONE-4.5-33B-GGUF/snapshots/0e969634ef24db05151b435970297a6dee634b7e/EXAONE-4.5-33B-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) LG AI EXAONE 4.5 33B Q4_K_M (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--LGAI-EXAONE--EXAONE-4.5-33B-GGUF/snapshots/0e969634ef24db05151b435970297a6dee634b7e/EXAONE-4.5-33B-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Google Gemma4 QAT UD-Q4_K_XL (Quantization-Aware Training) 12B (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--gemma-4-12b-it-qat-GGUF/snapshots/980b060c40a8539ac159e0501a3e0f66a6365af3/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Google Gemma4 QAT 12B UD-Q4_K_XL (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--gemma-4-12b-it-qat-GGUF/snapshots/980b060c40a8539ac159e0501a3e0f66a6365af3/gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Google Gemma4 QAT 26B-4B Active UD-Q4_K_XL (Mixture of Experts) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--gemma-4-26B-A4B-it-qat-GGUF/snapshots/7b92b5b28818151e8669af2e45e88d6086f490dd/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Google Gemma4 QAT 26B-4B Active UD-Q4_K_XL (Mixture of Experts) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--gemma-4-26B-A4B-it-qat-GGUF/snapshots/7b92b5b28818151e8669af2e45e88d6086f490dd/gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Google Gemma4 QAT 31B UD-Q4_K_XL (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--gemma-4-31b-it-qat-GGUF/snapshots/43cc1aeb31adf47ec06a854507ce552cd9862e6f/gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Google Gemma4 QAT 31B UD-Q4_K_XL (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--gemma-4-31b-it-qat-GGUF/snapshots/43cc1aeb31adf47ec06a854507ce552cd9862e6f/gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Bartowski IBM Granite 4.2 30B Q4_K_M (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--bartowski--granite-4.2-30b-GGUF/snapshots/1847d3b70241af9d656f382a4cf29d5c6573e584/granite-4.2-30b-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Bartowski IBM Granite 4.2 30B Q4_K_M (Dense) F32
build: ca3d5a3 (10665) IBM Granite 4.2 8B Q6_K (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--ibm-granite--granite-4.2-8b-GGUF/snapshots/31239b3e4a93d1c2bc0e0d1160f711bf300cef05/granite-4.2-8b-Q6_K.gguf -fa 0,1
build: ca3d5a3 (10665) IBM Granite 4.2 8B Q6_K (Dense) (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--ibm-granite--granite-4.2-8b-GGUF/snapshots/31239b3e4a93d1c2bc0e0d1160f711bf300cef05/granite-4.2-8b-Q6_K.gguf -fa 0,1
build: ca3d5a3 (10665) Poolside Laguna XS 2.1 30B-3B Active Q4_K_M (Mixture of Experts) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--poolside--Laguna-XS-2.1-GGUF/snapshots/1a37c0a5fb8c7a18e6106decb6be6327d1b63fa6/Laguna-XS-2.1-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Poolside Laguna XS 2.1 30B-3B Active Q4_K_M (Mixture of Experts) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--poolside--Laguna-XS-2.1-GGUF/snapshots/1a37c0a5fb8c7a18e6106decb6be6327d1b63fa6/Laguna-XS-2.1-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Meta Muse Glimmer 30B UD-Q4_K_XL (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Muse-Glimmer-30B-GGUF/snapshots/faa5b025c584459c13febfa5c59883516710ae39/Muse-Glimmer-30B-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Meta Muse Glimmer 30B UD-Q4_K_XL (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Muse-Glimmer-30B-GGUF/snapshots/faa5b025c584459c13febfa5c59883516710ae39/Muse-Glimmer-30B-UD-Q4_K_XL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Cohere Labs North Mini Code 30B-3B Active UD-IQ4_NL (Mixture of Experts) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--North-Mini-Code-1.0-GGUF/snapshots/e306bb4bf0df610f5471d97a01de2b6e0b24d356/North-Mini-Code-1.0-UD-IQ4_NL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Cohere Labs North Mini Code 30B-3B Active UD-IQ4_NL (Mixture of Experts) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--North-Mini-Code-1.0-GGUF/snapshots/e306bb4bf0df610f5471d97a01de2b6e0b24d356/North-Mini-Code-1.0-UD-IQ4_NL.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Cohere Labs North Mini Code 30B-3B Active UD-Q4_K_M (Mixture of Experts) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--North-Mini-Code-1.0-GGUF/snapshots/e306bb4bf0df610f5471d97a01de2b6e0b24d356/North-Mini-Code-1.0-UD-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Cohere Labs North Mini Code 30B-3B Active UD-Q4_K_M (Mixture of Experts) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--North-Mini-Code-1.0-GGUF/snapshots/e306bb4bf0df610f5471d97a01de2b6e0b24d356/North-Mini-Code-1.0-UD-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Alibaba Qwen3.8 27B UD-Q4_K_M (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Alibaba Qwen3.8 27B UD-Q4_K_M (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q4_K_M.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Alibaba Qwen3.8 27B UD-Q4_K_S (Dense) F16 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q4_K_S.gguf -fa 0,1
build: ca3d5a3 (10665) Unsloth Alibaba Qwen3.8 27B UD-Q4_K_S (Dense) F32 llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q4_K_S.gguf -fa 0,1
build: ca3d5a3 (10665) |
|
@SleepinDevil I was working on this getting ready to post when I saw your message I feel like from memory it probably droped to like 10-30% of the max speed shown in the main comment based on how much context I had used (I normally configure 131072 to be my max tokens when actually running). I admit I've been treating this as a fun toy that I thankfully have access too, The slowdowns would be slightly annoying if I was trying to make money with it but I'm generally using this as a background thing that I can kick off and just come back to when it's done. Logically I shouldn't buy more hardware for this right now but I admit getting another Intel card is super tempting for me on a "support the (relative to this particular hardware market) little guy" angle; Alternatively just ripping out a B50 or B580 from my other machines could be a fun test for tensor splits, but I'm not really need to open up my other PC for this just yet. Since my percentage estimates are based on such hazy memories of runs I don't quite fully remember while doing agentic coding and didn't pay a lot of attention to at the time (I gave an big ballpark range to guestimate). These runs were also with a bunch of flags set to change this to get more KV cache so it's different to whats reported here so I don't know how those config's max speed compare to the baseline bench runs we're collecting here. It's totally possible that while it did happen, I just didn't see the performance level you mentioned as I probably just walked away to do something else in the meantime. What I'll do for you is run a couple of these llama bench (F16) with a much large -p and -n flags as I think thand if I get good runs that really work the CPU I'll try and remember to come back and post them. I'm about to sleep so this is likely a tomorrow thing. I'll add another comment here or edit this one. |
|
I'm not sure how to get llama bench to offload large contexts without running out of memory like when running with llama serve but I think the -nkvo 1 --no-kv-offload 1 flags should give a worst case where all the KV cache is on the CPU and System RAM llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q4_K_M.gguf -fa 0,1 -nkvo 1
build: ca3d5a3 (10665) Working on a longer test with big -p and -n values but that's slow; GPU usage was still over 50% on this run but consistently under 10% on this longer test. I hope it doesn't run out of memory while it runs overnight lol Larger results: this is where we see how it really falls apart. Thankfully when actually serving through llama.cpp some of this KV cache can stay on VRAM so it doesn't get this bad but it's still slow. llama-bench -m /home/fred/.cache/huggingface/hub/models--unsloth--Qwen3.8-27B-GGUF/snapshots/4ca720788d1e01f1bff70c033e0d0028fd02e502/Qwen3.8-27B-UD-Q4_K_M.gguf -fa 0,1 -p 81920 -n 20480 -nkvo 1 -r 1
build: ca3d5a3 (10665) |
|
That PP and TG is actually very impressive to see, thanks for sharing that! |
|
@SleepinDevil updated my last comment with the long run, I only ran it once and overnight so IDK how long it actually took but you can probably guestimate from the values |
build: 2cdae80 (10714) |
|
I run with same gguf, version and parameters on Ubuntu. Thank you! GPU: Intel(R) Arc(TM) A770 Graphics
build: 2cdae80 (10714) |
Environment
Build optionsF16 with oneDNNF16 without oneDNNF32Test InstructionsTest Results[F16] unsloth/Qwen3.8-27B-GGUF:UD-Q6_K
[F16] unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
[F16] unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_M
[F16] unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_S
[F16] unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL
[F16] unsloth/Qwen3.8-27B-GGUF:Q4_0
[F16] unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS
[F16 without DNN] unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
[F16 without DNN] unsloth/Qwen3.8-27B-GGUF:Q4_0
[F16 without DNN] unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS
[F32] unsloth/Qwen3.8-27B-GGUF:UD-Q5_K_XL
[F32] unsloth/Qwen3.8-27B-GGUF:Q4_0
[F32] unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS
|
|
@virus-warnning Thank you! |
|
First time using any local model on an Intel card, also the first time using llama.cpp period. Also first time ever trying any of this on Linux. Apologize in advance, I have no clue how to make my posts look as nice as everyone elses. If I can run any other tests I'd be happy to contribute. EnvironmentHost Hardware: VM Hardware Buildcmake -B build -G Ninja cmake --build build -j 16 Test Instructions
Test Results[F16]** TheBloke--Llama-2-7B-GGUF./build/bin/llama-bench -m models/llama-2-7b.Q4_0.gguf -fa 1,0
|
|
@cbelcher Thank you! |
and others
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Purpose
It's used to share the performance data on Intel GPU with SYCL backend.
The performance data is only used as reference, since we don't double check the data.
It can not be used as any commercial purpose.
Rule
Encourage to test with default setting (environment variables).
If you want to update the data with special building or running setting, please create a new table.
Create/update the tables directly following the format.
Insert new record, instead of update it for same keys; Sort the records by col1, col2, col3.
Add your comments in the latest for more discussion.
Don't add table to compare with other hardware, framework or backend.
Please run 1+ times and update with the stable data.
Tips and tricks
Xe driver gives a huge boost over i915.
Combined with IOMMU and Xe,
--split-mode tensorgives decent number on prefill2 x Arc A770 can give usable speeds, at least with benchmark 512+128 regimes.
Refer to: Performance of llama.cpp on Intel GPU with SYCL backend #23313 (comment)
User can choose them according to the test result.
Quick build scripts:
Performance data on Intel GPU
Default setting
Build:
Run:
Data:
FP16
t/s
t/s
DDR5-6400 16GB
DDR5-6400 16GB
- Medium
32GB
- Medium
32GB
DDR5-6400 16GB
DDR5-6400 16GB
64GB
24.04.4
64GB
24.04.4
5600X
DDR4-3600
128GB
24.04.4
6.17.0-29-generic
5600X
DDR4-3600
128GB
24.04.4
6.17.0-29-generic
5600X
DDR4-3600
128GB
24.04.4
6.17.0-29-generic
5600X
DDR4-3600
128GB
24.04.4
6.17.0-29-generic
DDR5-6400 16GB
DDR5-6400 16GB
5700X3D
25.10
5700X3D
25.10
DDR4-3600 16GB
DDR4-3600 16GB
DDR4-3600 16GB
DDR4-3600 16GB
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
64GB DDR5
26.04
64GB DDR5
26.04
More PP/TG Types:
FP16
t/s
t/s
t/s
t/s
(12 EUs)
Panther Lake
26.04
(12 EUs)
Panther Lake
26.04
All reactions