llama.app : website + unified llama binary

[ggerganov]

Overview

We are launching an official website for llama.cpp: https://llama.app/

The main goal of the website is to provide a simple way for new users to install and run llama.cpp on their machines. The page has an installation command (one-liner) and links/instructions to popular GGUF models on the hub. The current version is a first iteration of many.

During install the cross-platform installer ships a single binary called llama. The binary packs all the user-facing tooling of llama.cpp (i.e. llama-server, llama-cli, etc.) with a single CLI entry point. This is mostly following the git example. Currently we ship binaries for the major operating systems and we plan to iterate and improve the packaging pipeline.

The webpage will also provide helpful instructions for running llama in common use cases: chat, agentic coding, etc. These will be combined with current FOTM models from various quantization providers (e.g. Unsloth, Bartowski, etc.). There will be guidelines for integration with 3rd-party agents, creating or finding the best configuration for your device and tips for utilizing advanced llama.cpp features. We will be iterating on this and would love to hear any feedback from the community on how to improve.

• The contents of the webpage are hosted at https://github.com/ggml-org/llama.pages
• The unified llama app is here: https://github.com/ggml-org/llama.cpp/tree/master/app
• The binaries are built here: https://github.com/ggml-org/llama-install.sh
• The binaries are hosted here: https://huggingface.co/buckets/ggml-org/install.sh

HF Team: @ggml-org/hf

[donohara]

This is terrific! Thanks to everyone who contributes to this great tool !

[MerijnHendriks]

Congrats on the achievement and the launch of the website!

If llama-server can remain to exist as a single binary and function independently of the llama binary, then I have no complaints.

[ggerganov]

Yes, llama-server remains as is. Simply put, llama serve and llama-server are the same thing. Also llama cli and llama-cli, etc. Same idea as git commit and git-commit.

[Jipok]

When choosing a model, the size is missing. In my opinion, it would be better to either remove selector and leave the links to the HG, or add a size indication.

[cphlipot]

Aside from the size, i think we could also try to select more sensible defaults. It seems right now we just select whatever the first one in the list is, which seems to result in either selecting the largest or smallest quant depending on the provider, neither of which is probably what the user is most likely to want:

[cphlipot]

This looks like a great step forward in terms of usability.

I was wondering with the CLI, are there any plans to adjust arugments for better UX, or is the aim to just simply expose existing tools as subcommand with all existing arguments left exactly as-is?

[aviallon]

Missed opportunity to register llama.cpp

[Kangaroux]

Site looks good, two critiques:

1. Add a brief callout to what llama.cpp is in the hero. If I stumble on the site the first thing I see is a "C++" logo
2. Nit: would be nice if someone with image editing tools could clean up the llama image to fix the perspective on the computer case https://llama.app/local-ai.png

[allozaur]

hey, thank you for the feebdack :) improvements are coming

[h8f1z]

Congrats 👏 🎉

[luck-tar-gz]

Congrats! I saw a little inconsistency though:

While Qwen's, Gemma's and Step's model tags show "XB MoE · YB active", GPT-OSS's model tag does not show that, making it sound more like the Qwen3.6-27B dense model. Converting it to show that it is MoE would be more accurate.

Great work as always!

[julien-c]

do you want to open a quick PR on https://github.com/ggml-org/llama.pages maybe?

[luck-tar-gz]

Yeah, should be there now.

[overas]

Thank you for the website. One suggestion for improving the website would be to have an option for whether one prefers the Windows one-liner "irm https://llama.app/install.ps1 | iex" or the Linux one-liner "curl -LsSf https://llama.app/install.sh | sh"

[allozaur]

cc @angt

[Kishlay-notabot]

I'm currently trying to get CUDA prebuilt binaries for x64 Linux platforms and I notice some discrepancies which are kinda confusing: The GH actions do not ship CUDA compiled binaries, yet there are linux x64 CUDA enabled docker images. So I am confused with the intended usage or fetching of CUDA binaries for the linux platform. CUDA compilation from scratch on a colab instance takes an awful while, even with parallel jobs. Would be grateful for some guidance over this! Thanks.

[Kishlay-notabot]

ill try this, thanks a lot! I didn't know about the unified binaries.

[Kishlay-notabot]

Tried this on google colab:

Version: b9704
Probing CUDA...
Downloading cuda-probe...
Probing ROCm...
Downloading rocm-probe...
Probing Vulkan...
Downloading vulkan-probe...
Downloading featcode...
Probing CPU...
Found: bmi2
Found: avx512vl
Found: avx512bw
Found: avx512cd
Found: avx512dq
Downloading llama...
Installation completed successfully

To make llama available in future sessions, add ~/.local/bin to your PATH by running:

  echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bash_profile

Then open a new terminal and run:

  llama serve

To start it now without modifying your PATH, run:

  ~/.llama-app/llama serve

Did the GPU probe fail? The inference runs on CPU.

[Kishlay-notabot]

Starting llama-server...
  Trying port 19876: [/root/.llama-app/llama](https://file+.vscode-resource.vscode-cdn.net/root/.llama-app/llama) serve --port 19876 --model [/root/.cache/huggingface/hub/models--bartowski--SmolLM2-135M-Instruct-GGUF/snapshots/09816acd5d99df7be770d85ea30822623dab342c/SmolLM2-135M-Instruct-Q4_K_M.gguf](https://file+.vscode-resource.vscode-cdn.net/root/.cache/huggingface/hub/models--bartowski--SmolLM2-135M-Instruct-GGUF/snapshots/09816acd5d99df7be770d85ea30822623dab342c/SmolLM2-135M-Instruct-Q4_K_M.gguf) --no-webui
    0.00.003.407 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
    0.00.003.433 I device_info:
    0.00.003.439 I   - CPU     : Intel(R) Xeon(R) CPU @ 2.00GHz (12975 MiB, 12975 MiB free)
    0.00.003.455 I system_info: n_threads = 1 (n_threads_batch = 1) [/](https://file+.vscode-resource.vscode-cdn.net/) 2 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | LLAMAFILE = 1 | REPACK = 1 |
    0.00.003.456 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
    0.00.003.464 I srv          init: running without SSL
    0.00.003.490 I srv          init: using 8 threads for HTTP server
    0.00.003.493 I srv          init: The UI is disabled
    0.00.003.494 I srv          init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
    0.00.003.579 I srv         start: binding port with default address family
    0.00.005.532 I srv  llama_server: loading model
    0.00.005.540 I srv    load_model: loading model '/root/.cache/huggingface/hub/models--bartowski--SmolLM2-135M-Instruct-GGUF/snapshots/09816acd5d99df7be770d85ea30822623dab342c/SmolLM2-135M-Instruct-Q4_K_M.gguf'
    0.00.005.626 I common_init_result: fitting params to device memory ...
    0.00.005.632 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
    0.00.117.782 I common_params_fit_impl: projected to use 292 MiB of host memory vs. 12975 MiB of total host memory
    0.00.394.269 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
    0.00.515.113 I srv    load_model: initializing slots, n_slots = 4
    0.00.679.491 W common_speculative_init: no implementations specified for speculative decoding
    0.00.679.521 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 8192
    0.00.679.524 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 8192
    0.00.679.525 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 8192
    0.00.679.526 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 8192
    0.00.679.671 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB

[angt]

Does nvidia-smi works on your machine ?
If yes do you have libnvidia-compute installed ?

[Kishlay-notabot]

hi! sorry for late reply but I went around the issue by getting prebuilt binaries for the T4 GPU.

[ronoaldo]

I installed the app with the install.sh script. I figured it was amazing to work with. It has not being updated recently, however. When I did llama update it appeared to download new stuff however it didn't. Is there a way to update manually? Should I use another install method to get newer versions?

Thanks!

[julien-c]

worst case you can run the install.sh script again, imo – but cc @angt

[angt]

Yes, there are instances where there are minor updates (in the build system) even though the release number remains unchanged. So install.sh always download and update the binary when we execute the llama update command.

New releases are scheduled every week. We only force an update when there’s a significant bugfix commit.

[ronoaldo]

Oh, I see. So the hugging face bucket is updated once a week, that is why I was downloading an older build if compared with the Github releases, correct?

I ended up downloading from Github releases the most fresh build as some of the items in changelog were relevant to me (like a fix for deduplicating models in cache/models.ini and some improvements to Vulkan).

[ronoaldo]

worst case you can run the install.sh script again, imo – but cc @angt

yea, I did that and it was yet in the same version I had already which was not the latest if compared with Github.

Thanks again for the reply!