From 244f77acebb168a5f1a966c7acadd57c1d0a017c Mon Sep 17 00:00:00 2001 From: tintisimone Date: Wed, 8 Jul 2026 09:12:58 +0000 Subject: [PATCH] Add Intel Arc Battlemage (BMG) GPU XPU support Add vLLM-based inference support for Intel Arc Battlemage (BMG) discrete GPUs to the single-node deployment: Helm values for the BMG vLLM serving container, XPU-aware model selection, Intel GPU device-plugin wiring, and setup documentation. Incorporates review feedback: run the vLLM container as non-root, use xpu*-style naming instead of bmg-specific names, replace hardcoded inventory/config values with placeholders, and de-duplicate the kubernetes SDK install tasks. The CodeTranslation changes were split into a separate PR (#117) to keep this one focused on XPU/BMG. Signed-off-by: tintisimone --- README.md | 11 +- core/helm-charts/vllm/bmg-values.yaml | 223 ++++++++ .../vllm/templates/deployment.yaml | 13 +- core/helm-charts/vllm/values.yaml | 2 + core/inference-stack-deploy.sh | 10 +- core/inventory/hosts.yaml | 23 +- core/inventory/inference-config.cfg | 18 +- core/inventory/metadata/all.yml | 8 +- .../inventory/metadata/inference-metadata.cfg | 1 + .../metadata/vars/inference_llm_models.yml | 2 + core/lib/add-node.sh | 2 +- core/lib/cluster/deployment/fresh-install.sh | 16 +- core/lib/cluster/nodes/add-node.sh | 2 +- core/lib/components/intel-gpu-plugin.sh | 15 + core/lib/models/install-model-hf.sh | 2 +- core/lib/models/install-model.sh | 33 +- core/lib/models/model-selection.sh | 165 ++++-- core/lib/models/uninstall-model-hf.sh | 2 +- core/lib/system/config-vars.sh | 4 +- core/lib/system/precheck/read-config-file.sh | 86 ++- core/lib/system/setup-env.sh | 18 +- core/lib/user-menu/parse-user-prompts.sh | 30 +- core/lib/xeon/ballon-policy.sh | 6 +- core/playbooks/deploy-inference-models.yml | 528 ++++++++++++++++++ core/playbooks/deploy-intel-gpu-plugin.yml | 106 ++++ core/roles/inference-tools/tasks/main.yml | 52 +- core/scripts/generate-token.sh | 8 +- core/scripts/generate-vault-secrets.sh | 0 core/scripts/keycloak-fetch-client-secret.sh | 4 +- core/scripts/vllm-quickstart/README.md | 42 +- core/scripts/vllm-quickstart/models.json | 87 ++- .../vllm-quickstart/vllm-model-runner.sh | 193 ++++++- docs/configuring-inference-config-cfg-file.md | 16 +- docs/examples/single-node/README.md | 6 +- docs/intel-arc-bmg-setup.md | 170 ++++++ docs/single-node-deployment.md | 30 +- docs/supported-models.md | 41 +- 37 files changed, 1701 insertions(+), 274 deletions(-) create mode 100644 core/helm-charts/vllm/bmg-values.yaml mode change 100644 => 100755 core/inference-stack-deploy.sh create mode 100644 core/lib/components/intel-gpu-plugin.sh create mode 100644 core/playbooks/deploy-intel-gpu-plugin.yml mode change 100644 => 100755 core/scripts/generate-token.sh mode change 100644 => 100755 core/scripts/generate-vault-secrets.sh create mode 100644 docs/intel-arc-bmg-setup.md diff --git a/README.md b/README.md index e39cb944..adc06793 100644 --- a/README.md +++ b/README.md @@ -4,22 +4,23 @@ Unleash the power of AI Inference on Intel Silicon The Intel® AI for Enterprise Inference is aimed to streamline and enhance the deployment and management of AI inference services on Intel hardware. Utilizing the power of Kubernetes Orchestration, this solution automates the deployment of LLM models to run faster inference, provision compute resources, and configure the optimal settings to minimize the complexities and reduce manual efforts. -It supports a broad range of Intel hardware platforms, including Intel® Xeon® Scalable processors and Intel® Gaudi® AI Accelerators, ensuring flexibility and scalability to meet diverse enterprise needs. +It supports a broad range of Intel hardware platforms, including Intel® Xeon® Scalable processors, Intel® Gaudi® AI Accelerators, and **Intel® Arc™ Battlemage (BMG) GPUs**, ensuring flexibility and scalability to meet diverse enterprise needs. -Intel® AI for Enterprise Inference, powered by OPEA, is compatible with OpenAI standard APIs, enabling seamless integration to enterprise applications both on-premises and in cloud-native environments. This compatibility allows businesses to leverage the full capabilities of Intel hardware while deploying AI models with ease. With this suite, enterprises can efficiently configure and evolve their AI infrastructure, adapting to new models and growing demands effortlessly. +Intel® AI for Enterprise Inference, powered by OPEA, is compatible with OpenAI standard APIs, enabling seamless integration to enterprise applications both on-premises and in cloud-native environments. This compatibility allows businesses to leverage the full capabilities of Intel hardware while deploying AI models with ease. With this suite, enterprises can efficiently configure and evolve their AI infrastructure, adapting to new models and growing demands effortlessly. ![Intel AI for Enterprise Inference](docs/pictures/Enterprise-Inference-Architecture.png) #### Key Components: - **Kubernetes**: A powerful container orchestration platform that automates the deployment, scaling, and management of containerized applications, ensuring high availability and efficient resource utilization. - **Intel Gaudi Base Operator**: A specialized operator that manages the lifecycle of Habana AI resources within the Kubernetes cluster, enabling efficient utilization of Intel® Gaudi® hardware for AI workloads. (Applicable only to Gaudi based deployments) + - **Intel GPU Plugin**: A Kubernetes device plugin that manages Intel® Arc™ GPU resources within the cluster, enabling efficient utilization of Intel® Arc™ Battlemage (BMG) hardware for AI workloads. (Applicable only to BMG based deployments) - **Ingress NGINX Controller**: A high-performance reverse proxy and load balancer for traffic, responsible for routing incoming requests to the appropriate services within the Kubernetes cluster, ensuring seamless access to deployed AI models. - **Keycloak**: An open-source identity and access management solution that provides robust authentication and authorization capabilities, ensuring secure access to AI services and resources within the cluster. - **APISIX**: A cloud-native API gateway, handling API traffic and providing advanced features caching, and authentication, enabling efficient and secure access to deployed AI models. - **Observability**: An open-source monitoring solution designed to operate natively within Kubernetes clusters, providing comprehensive visibility into the performance, health, and resource utilization of deployed applications and cluster components through metrics, visualization, and alerting capabilities. - **Model Deployments**: Automated deployment and management of AI LLM models within the Kubernetes inference cluster, enabling scalable and reliable AI inference capabilities. - **GenAI Gateway**: An integrated gateway leveraging LiteLLM and Langfuse to provide flexible interfaces for routing and managing generative AI models. It enables user and key management, user token telemetry, and analytics for LLM inference workflows. - + ## Table of Contents - [Usage](#usage) - [Support](#support) @@ -32,6 +33,8 @@ Intel® AI for Enterprise Inference, powered by OPEA, is compatible with OpenAI The Usage instructions for the AI Inference as a Service Deployment Automation can be found in the [docs/README.md](docs/README.md) file. To setup, follow the step-by-step instructions provided in the `docs/README.md` file. +For Intel® Arc™ Battlemage (BMG) GPU setup, refer to [docs/intel-arc-bmg-setup.md](docs/intel-arc-bmg-setup.md). + ## Support For feature requests, bugs or questions about the project, [open an issue](https://github.com/opea-project/Enterprise-Inference/issues) on the GitHub Issues page. Provide as much details as possible, including steps to reproduce the issue, expected behavior, and actual behavior. @@ -42,7 +45,7 @@ Intel® AI for Enterprise Inference is licensed under the [Apache License Versio The [Security Policy](SECURITY.md) outlines our guidelines and procedures for ensuring the highest level of security and trust for our users who consume Intel® AI for Enterprise Inference. ## Trademark Information -Intel, the Intel logo, Xeon, and Gaudi are trademarks of Intel Corporation or its subsidiaries. +Intel, the Intel logo, Xeon, Gaudi, and Arc are trademarks of Intel Corporation or its subsidiaries. * Other names and brands may be claimed as the property of others. © Intel Corporation diff --git a/core/helm-charts/vllm/bmg-values.yaml b/core/helm-charts/vllm/bmg-values.yaml new file mode 100644 index 00000000..056a6f0a --- /dev/null +++ b/core/helm-charts/vllm/bmg-values.yaml @@ -0,0 +1,223 @@ +# Copyright (C) 2025-2026 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +# Intel® Arc™ Battlemage (BMG) GPU optimized override values for vLLM deployments. +# This file contains BMG-specific overrides for Intel Arc B-series GPU (e.g., B580, B770). +# Requires the Intel GPU Plugin (intel-device-plugins-gpu) to be installed on the cluster. + +# Intel XPU accelerator device (Arc GPU) +accelDevice: "xpu" +# Kubernetes resource name exposed by the Intel GPU device plugin. +# Use "gpu.intel.com/xe" with the newer Xe kernel driver (Ubuntu 25.10 / Arc B-series), +# or "gpu.intel.com/i915" on hosts still using the i915 driver. +xpuDeviceResource: "gpu.intel.com/xe" + +block_size: 64 # XPU-optimised KV cache block size (must be >= 64 for 0.14.1-xpu IPEX chunked prefill) +max_num_seqs: 128 # Max concurrent sequences (tuned for Arc B-series VRAM) +max_seq_len_to_capture: 2048 +d_type: "float16" +max_model_len: 8192 +tensor_parallel_size: "1" # Arc Pro B50 has 1 GPU; adjust for multi-GPU setups + +image: + repository: intel/vllm + tag: "0.17.0-xpu" + pullPolicy: IfNotPresent + command: ["vllm", "serve"] + +# --- Security context (non-root, scanner-compliant) --- +# The intel/vllm:0.17.0-xpu image defaults to root, but the workload runs fine as +# a non-root user (UID 1001) provided it can reach the Arc render node +# (/dev/dri/renderD*). That device is owned by root:render on the host, so the pod +# must join the host's "render" (and typically "video") group via supplementalGroups. +# HF_HOME is set to /data (the model-volume mount); fsGroup: 1001 makes that volume +# and the /tmp + /dev/shm emptyDirs group-writable, so caches work without root. +# +# IMPORTANT — validate the host GIDs on the BMG node before deploying (they vary by +# distro/kernel). On the target node run: +# getent group render video # e.g. render:x:993 video:x:44 +# stat -c '%g %G' /dev/dri/renderD128 +# then set supplementalGroups below to the matching numeric GIDs. A GPU permission +# error at startup almost always means the render GID here does not match the host. +podSecurityContext: + runAsNonRoot: true + runAsUser: 1001 + runAsGroup: 1001 + fsGroup: 1001 + # [,