[Helm] Allow setting extra env vars on the LMCache cache server container - #1048
Open
pujitha24 wants to merge 1 commit into
Open
[Helm] Allow setting extra env vars on the LMCache cache server container#1048pujitha24 wants to merge 1 commit into
pujitha24 wants to merge 1 commit into
Conversation
…iner
Motivation:
The LMCache cache server Deployment template had no `env:` block at
all, so there was no way to pass any environment variable to the
cache-server container via Helm values. This blocks the workaround
several users on the OOMKilled report converged on: setting
LMCACHE_CACHE_POLICY and LMCACHE_MAX_LOCAL_CPU_SIZE so the cache
server evicts old entries instead of growing unbounded until the
kernel OOM-kills it. This change does not itself pick an eviction
policy or fix the OOM behavior for anyone by default -- it only
removes the blocker that made self-configuring LMCache's own
eviction knobs impossible through the chart.
Approach:
Add a `cacheserverSpec.env` values field (default `[]`, fully
backward compatible) and render it into the container spec via
`{{- with .Values.cacheserverSpec.env }} env: {{- toYaml . | nindent 12 }} {{- end }}`,
mirroring the exact `env` pattern already used for
`routerSpec.env` / `$container.env` elsewhere in this chart
(deployment-router.yaml, deployment-vllm-multi.yaml). Also updated
helm/values.schema.json and helm/README.md to document the new
field, and extended the existing cacheserver_test.yaml Suite to
cover it.
Validation:
- `helm unittest helm -f 'tests/cacheserver_test.yaml'`: 2/2 pass.
- `helm unittest helm` (full suite): 142 passed, 2 failed. Both
failures (tests/deployment-vllm-multi_test.yaml asserts[3];
tests/ray-cluster_test.yaml asserts[1..3]) are pre-existing and
unrelated -- reproduced identically with this diff stashed out
against unmodified main.
- `helm lint helm`: passes (only pre-existing unrelated warnings).
- `helm template` with no `cacheserverSpec.env` set renders the
container with no `env:` key at all, i.e. default behavior is
byte-for-byte unchanged.
- `helm template --set-json 'cacheserverSpec.env=[{"name":"LMCACHE_CACHE_POLICY","value":"LRU"}]'`
renders the env block correctly.
- Installed the `helm-values-schema-json` plugin and ran the exact
`helm-schema` pre-commit hook this repo's .pre-commit-config.yaml
wires up; it passed with zero additional diff, confirming
values.schema.json is byte-identical to the tool's own output.
- Ran the repo's non-manual pre-commit hooks (check-json, check-yaml,
end-of-file-fixer, trailing-whitespace, markdownlint, codespell) on
all changed files: all passed.
- Could NOT run this repo's live-cluster "Functionality test for
helm chart" GitHub Actions jobs (self-hosted minikube runners) --
not available in this sandbox. Mitigated by the default-rendering
check above showing this change is purely additive behind an
empty-by-default list, so it cannot affect any existing values
file used by that workflow.
- Base branch CI (`gh run list --branch main --workflow "Functionality
test for helm chart"`) is currently green.
Report: vllm-project#923
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)
pujitha24
requested review from
ApostaC,
Shaoting-Feng,
YuhanLiu11 and
ruizhang0101
as code owners
August 24, 2026 04:23
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces support for configuring extra environment variables for the cache server container via the cacheserverSpec.env parameter. It updates the Helm chart templates, values schema, default values, and documentation, and adds corresponding unit tests to verify the environment variables are correctly injected. There are no review comments, and I have no feedback to provide.
Author
|
Since it has been open a week without a look, @YuhanLiu11 as noted in the PR template — this is rebased and green, happy to adjust anything that would help review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Helm] Allow setting extra env vars on the LMCache cache server container
FIX #923 (link existing issues this PR will resolve)
The LMCache cache server
Deploymenttemplate had noenv:block at all, sothere was no way to pass any environment variable to the cache-server
container through Helm values. Issue #923 reports the cache server getting
OOMKilled, and several commenters converged on the workaround of setting
LMCACHE_CACHE_POLICY/LMCACHE_MAX_LOCAL_CPU_SIZE(LMCache's own evictionknobs) — but nobody could actually do that through this chart.
This PR does not pick an eviction policy or fix OOM behavior by default.
It only adds a generic, empty-by-default
cacheserverSpec.envvalues field(mirroring the existing
envpattern used forrouterSpec.env/$container.envelsewhere in the chart) so users can self-configure LMCache'seviction env vars, or any other env var, themselves. Default rendering is
byte-for-byte unchanged when
cacheserverSpec.envis left unset.Also updated
helm/values.schema.jsonandhelm/README.mdto document thenew field, and extended the existing
helm/tests/cacheserver_test.yamlsuiteto cover it.
Validation
helm unittest helm -f 'tests/cacheserver_test.yaml': 2/2 pass.helm unittest helm(full suite): 142 passed, 2 failed. Both failures(
tests/deployment-vllm-multi_test.yamlasserts[3];tests/ray-cluster_test.yamlasserts[1..3]) are pre-existing andunrelated to this change — reproduced identically with this diff stashed
out against unmodified
main.helm lint helm: passes (only pre-existing unrelated warnings).helm templatewith nocacheserverSpec.envset renders the containerwith no
env:key at all (unchanged default behavior); with--set-json 'cacheserverSpec.env=[{"name":"LMCACHE_CACHE_POLICY","value":"LRU"}]'it renders the env block correctly.
helm-values-schema-jsonplugin and ran the exacthelm-schemapre-commit hook this repo wires up in.pre-commit-config.yaml— it passed with zero additional diff, confirmingvalues.schema.jsonis byte-identical to the tool's own generated output.end-of-file-fixer, trailing-whitespace, markdownlint, codespell) on all
changed files: all passed.
chart" GitHub Actions jobs (self-hosted minikube runners) — not available
in this sandbox. Mitigated by the default-rendering check above: since this
change is purely additive behind an empty-by-default list, it cannot affect
any existing
tests/assets/values-*.yamlfile used by that workflow.Functionality test for helm chartonmain) is currentlygreen.
BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE
-swhen doinggit commit[Bugfix],[Feat], and[CI].Detailed Checklist (Click to Expand)
Thank you for your contribution to production-stack! Before submitting the pull request, please ensure the PR meets the following criteria. This helps us maintain the code quality and improve the efficiency of the review process.
PR Title and Classification
Please try to classify PRs for easy understanding of the type of changes. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:
[Bugfix]for bug fixes.[CI/Build]for build or continuous integration improvements.[Doc]for documentation fixes and improvements.[Feat]for new features in the cluster (e.g., autoscaling, disaggregated prefill, etc.).[Router]for changes to thevllm_router(e.g., routing algorithm, router observability, etc.).[Misc]for PRs that do not fit the above categories. Please use this sparingly.Note: If the PR spans more than one category, please include all relevant prefixes.
Code Quality
The PR need to meet the following code quality standards:
pre-committo format your code. SeeREADME.mdfor installation.DCO and Signed-off-by
When contributing changes to this project, you must agree to the DCO. Commits must include a
Signed-off-by:header which certifies agreement with the terms of the DCO.Using
-swithgit commitwill automatically add this header.What to Expect for the Reviews
We aim to address all PRs in a timely manner. If no one reviews your PR within 5 days, please @-mention one of YuhanLiu11
, Shaoting-Feng or ApostaC.