Skip to content

[Helm] Allow setting extra env vars on the LMCache cache server container - #1048

Open
pujitha24 wants to merge 1 commit into
vllm-project:mainfrom
pujitha24:auto/issue-923
Open

[Helm] Allow setting extra env vars on the LMCache cache server container#1048
pujitha24 wants to merge 1 commit into
vllm-project:mainfrom
pujitha24:auto/issue-923

Conversation

@pujitha24

Copy link
Copy Markdown

[Helm] Allow setting extra env vars on the LMCache cache server container

FIX #923 (link existing issues this PR will resolve)

The LMCache cache server Deployment template had no env: block at all, so
there was no way to pass any environment variable to the cache-server
container through Helm values. Issue #923 reports the cache server getting
OOMKilled, and several commenters converged on the workaround of setting
LMCACHE_CACHE_POLICY / LMCACHE_MAX_LOCAL_CPU_SIZE (LMCache's own eviction
knobs) — but nobody could actually do that through this chart.

This PR does not pick an eviction policy or fix OOM behavior by default.
It only adds a generic, empty-by-default cacheserverSpec.env values field
(mirroring the existing env pattern used for routerSpec.env /
$container.env elsewhere in the chart) so users can self-configure LMCache's
eviction env vars, or any other env var, themselves. Default rendering is
byte-for-byte unchanged when cacheserverSpec.env is left unset.

Also updated helm/values.schema.json and helm/README.md to document the
new field, and extended the existing helm/tests/cacheserver_test.yaml suite
to cover it.

Validation

  • helm unittest helm -f 'tests/cacheserver_test.yaml': 2/2 pass.
  • helm unittest helm (full suite): 142 passed, 2 failed. Both failures
    (tests/deployment-vllm-multi_test.yaml asserts[3];
    tests/ray-cluster_test.yaml asserts[1..3]) are pre-existing and
    unrelated to this change — reproduced identically with this diff stashed
    out against unmodified main.
  • helm lint helm: passes (only pre-existing unrelated warnings).
  • helm template with no cacheserverSpec.env set renders the container
    with no env: key at all (unchanged default behavior); with
    --set-json 'cacheserverSpec.env=[{"name":"LMCACHE_CACHE_POLICY","value":"LRU"}]'
    it renders the env block correctly.
  • Installed the helm-values-schema-json plugin and ran the exact
    helm-schema pre-commit hook this repo wires up in
    .pre-commit-config.yaml — it passed with zero additional diff, confirming
    values.schema.json is byte-identical to the tool's own generated output.
  • Ran the repo's non-manual pre-commit hooks (check-json, check-yaml,
    end-of-file-fixer, trailing-whitespace, markdownlint, codespell) on all
    changed files: all passed.
  • Could not run this repo's live-cluster "Functionality test for helm
    chart" GitHub Actions jobs (self-hosted minikube runners) — not available
    in this sandbox. Mitigated by the default-rendering check above: since this
    change is purely additive behind an empty-by-default list, it cannot affect
    any existing tests/assets/values-*.yaml file used by that workflow.
  • Base branch CI (Functionality test for helm chart on main) is currently
    green.

BEFORE SUBMITTING, PLEASE READ THE CHECKLIST BELOW AND FILL IN THE DESCRIPTION ABOVE


  • Make sure the code changes pass the pre-commit checks.
  • Sign-off your commit by using -s when doing git commit
  • Try to classify PRs for easy understanding of the type of changes, such as [Bugfix], [Feat], and [CI].
Detailed Checklist (Click to Expand)

Thank you for your contribution to production-stack! Before submitting the pull request, please ensure the PR meets the following criteria. This helps us maintain the code quality and improve the efficiency of the review process.

PR Title and Classification

Please try to classify PRs for easy understanding of the type of changes. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

  • [Bugfix] for bug fixes.
  • [CI/Build] for build or continuous integration improvements.
  • [Doc] for documentation fixes and improvements.
  • [Feat] for new features in the cluster (e.g., autoscaling, disaggregated prefill, etc.).
  • [Router] for changes to the vllm_router (e.g., routing algorithm, router observability, etc.).
  • [Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.

Code Quality

The PR need to meet the following code quality standards:

  • Pass all linter checks. Please use pre-commit to format your code. See README.md for installation.
  • The code need to be well-documented to ensure future contributors can easily understand the code.
  • Please include sufficient tests to ensure the change is stay correct and robust. This includes both unit tests and integration tests.

DCO and Signed-off-by

When contributing changes to this project, you must agree to the DCO. Commits must include a Signed-off-by: header which certifies agreement with the terms of the DCO.

Using -s with git commit will automatically add this header.

What to Expect for the Reviews

We aim to address all PRs in a timely manner. If no one reviews your PR within 5 days, please @-mention one of YuhanLiu11
, Shaoting-Feng or ApostaC.

…iner

Motivation:
The LMCache cache server Deployment template had no `env:` block at
all, so there was no way to pass any environment variable to the
cache-server container via Helm values. This blocks the workaround
several users on the OOMKilled report converged on: setting
LMCACHE_CACHE_POLICY and LMCACHE_MAX_LOCAL_CPU_SIZE so the cache
server evicts old entries instead of growing unbounded until the
kernel OOM-kills it. This change does not itself pick an eviction
policy or fix the OOM behavior for anyone by default -- it only
removes the blocker that made self-configuring LMCache's own
eviction knobs impossible through the chart.

Approach:
Add a `cacheserverSpec.env` values field (default `[]`, fully
backward compatible) and render it into the container spec via
`{{- with .Values.cacheserverSpec.env }} env: {{- toYaml . | nindent 12 }} {{- end }}`,
mirroring the exact `env` pattern already used for
`routerSpec.env` / `$container.env` elsewhere in this chart
(deployment-router.yaml, deployment-vllm-multi.yaml). Also updated
helm/values.schema.json and helm/README.md to document the new
field, and extended the existing cacheserver_test.yaml Suite to
cover it.

Validation:
- `helm unittest helm -f 'tests/cacheserver_test.yaml'`: 2/2 pass.
- `helm unittest helm` (full suite): 142 passed, 2 failed. Both
  failures (tests/deployment-vllm-multi_test.yaml asserts[3];
  tests/ray-cluster_test.yaml asserts[1..3]) are pre-existing and
  unrelated -- reproduced identically with this diff stashed out
  against unmodified main.
- `helm lint helm`: passes (only pre-existing unrelated warnings).
- `helm template` with no `cacheserverSpec.env` set renders the
  container with no `env:` key at all, i.e. default behavior is
  byte-for-byte unchanged.
- `helm template --set-json 'cacheserverSpec.env=[{"name":"LMCACHE_CACHE_POLICY","value":"LRU"}]'`
  renders the env block correctly.
- Installed the `helm-values-schema-json` plugin and ran the exact
  `helm-schema` pre-commit hook this repo's .pre-commit-config.yaml
  wires up; it passed with zero additional diff, confirming
  values.schema.json is byte-identical to the tool's own output.
- Ran the repo's non-manual pre-commit hooks (check-json, check-yaml,
  end-of-file-fixer, trailing-whitespace, markdownlint, codespell) on
  all changed files: all passed.
- Could NOT run this repo's live-cluster "Functionality test for
  helm chart" GitHub Actions jobs (self-hosted minikube runners) --
  not available in this sandbox. Mitigated by the default-rendering
  check above showing this change is purely additive behind an
  empty-by-default list, so it cannot affect any existing values
  file used by that workflow.
- Base branch CI (`gh run list --branch main --workflow "Functionality
  test for helm chart"`) is currently green.

Report: vllm-project#923
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5 (via Claude Code)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for configuring extra environment variables for the cache server container via the cacheserverSpec.env parameter. It updates the Helm chart templates, values schema, default values, and documentation, and adds corresponding unit tests to verify the environment variables are correctly injected. There are no review comments, and I have no feedback to provide.

@pujitha24

Copy link
Copy Markdown
Author

Since it has been open a week without a look, @YuhanLiu11 as noted in the PR template — this is rebased and green, happy to adjust anything that would help review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: LMCache cacheserver OOMKilled

1 participant