Skip to content

feat(helm): add optional PostgreSQL backing store#1579

Open
sauagarwa wants to merge 5 commits into
NVIDIA:mainfrom
sauagarwa:feat/helm-postgres-secret
Open

feat(helm): add optional PostgreSQL backing store#1579
sauagarwa wants to merge 5 commits into
NVIDIA:mainfrom
sauagarwa:feat/helm-postgres-secret

Conversation

@sauagarwa
Copy link
Copy Markdown
Contributor

@sauagarwa sauagarwa commented May 26, 2026

Summary

  • Add optional PostgreSQL support to the Helm chart with postgres.enabled (use PostgreSQL) and postgres.deploy (deploy bundled Bitnami subchart) flags
  • Support both internal (bundled subchart) and external PostgreSQL with credentials managed via Kubernetes Secrets
  • Add postgres.external.existingSecret to let users bring their own pre-existing Secret (e.g. from external-secrets-operator or GitOps)
  • When postgres.deploy=true, reference the Bitnami service-binding secret directly instead of duplicating credentials
  • Externalize JWT signing key file mode via sandboxJwt.secretDefaultMode with 0400 default
  • Add database backend e2e scenarios to Kubernetes e2e (e2e:kubernetes:db) and OpenShift e2e (e2e:openshift)

Closes #1599

Changes

  • values.yaml: Add postgres.* values block including external.existingSecret and sandboxJwt.secretDefaultMode
  • Chart.yaml: Add Bitnami PostgreSQL optional subchart dependency (condition: postgres.deploy)
  • templates/db-secret.yaml: Chart-managed Opaque Secret with individual credential fields + uri key (only created when deploy=false and no existingSecret)
  • templates/_helpers.tpl: openshell.dbSecretName and openshell.postgresFullname helpers for secret resolution across all modes
  • templates/statefulset.yaml: Conditional OPENSHELL_DB_URL env var from Secret uri key, scoped checksum annotation
  • templates/gateway-config.yaml: Omit db_url from TOML when PostgreSQL is enabled
  • tests/gateway_config_test.yaml: Comprehensive test cases covering bundled, external, existingSecret, fullnameOverride, and nameOverride paths
  • README.md.gotmpl: Restructured database backend section with existingSecret-first flow, separate Kubernetes/OpenShift examples
  • e2e/with-kube-gateway.sh: Add helm dependency build before install (fixes CI); add multi-scenario DB backend loop gated by OPENSHELL_E2E_KUBE_DB_SCENARIOS=1 (SQLite, bundled PG, external PG with existingSecret)
  • e2e/rust/e2e-openshift.sh: OpenShift-specific database-backend integration scenarios
  • tasks/test.toml: Add e2e:kubernetes:db and e2e:openshift mise tasks
  • tasks/helm.toml: Add helm dependency build before lint and unittest
  • .gitignore: Ignore subchart tarballs

Test plan

  • mise run helm:test — all tests pass
  • mise run pre-commit — all lint checks pass
  • mise run ci — full local CI passes
  • mise run e2e:kubernetes — existing behavior unchanged (no env var set)
  • mise run e2e:kubernetes:db — all 3 DB scenarios pass
  • mise run e2e:openshift — OpenShift integration scenarios pass
  • Deploy with postgres.external.existingSecret pointing to a pre-existing Secret
  • Verify default SQLite path unchanged when postgres.enabled=false

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 26, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@sauagarwa sauagarwa force-pushed the feat/helm-postgres-secret branch 3 times, most recently from 1fc9bef to 253051f Compare May 26, 2026 22:07
@sauagarwa sauagarwa marked this pull request as draft May 26, 2026 22:07
@sauagarwa sauagarwa force-pushed the feat/helm-postgres-secret branch from 253051f to d8d0be7 Compare May 26, 2026 22:15
@TaylorMutch TaylorMutch self-assigned this May 27, 2026
@sauagarwa sauagarwa force-pushed the feat/helm-postgres-secret branch from 4ba4f67 to b71c3a7 Compare May 28, 2026 00:04
@sauagarwa sauagarwa changed the title feat(helm): add optional PostgreSQL backing store with Secret-based credentials feat(helm): add optional PostgreSQL backing store May 28, 2026
@sauagarwa sauagarwa marked this pull request as ready for review May 28, 2026 00:09
@TaylorMutch
Copy link
Copy Markdown
Collaborator

/ok to test b71c3a7

@TaylorMutch TaylorMutch added the test:e2e Requires end-to-end coverage label May 28, 2026
@github-actions
Copy link
Copy Markdown

Label test:e2e applied for b71c3a7. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@TaylorMutch
Copy link
Copy Markdown
Collaborator

/ok to test 6a2b2ee

Comment thread deploy/helm/openshell/README.md.gotmpl Outdated
Comment thread deploy/helm/openshell/README.md.gotmpl Outdated
@TaylorMutch
Copy link
Copy Markdown
Collaborator

/ok to test db1dbe8

sauagarwa added 5 commits May 28, 2026 14:19
…redentials

- Add postgres.enabled and postgres.deploy values to control database
  backend (SQLite vs PostgreSQL) and subchart deployment independently.
- Introduce db-secret.yaml template for Opaque Secret with assembled
  postgresql:// connection string injected via OPENSHELL_DB_URL env var.
- Add Bitnami PostgreSQL as optional subchart dependency keyed on
  postgres.deploy to prevent subchart deployment in external mode.
- Externalize JWT signing key file mode via sandboxJwt.secretDefaultMode
  with 0400 default matching upstream.
- Add validation guard for postgres.deploy=true without postgres.enabled.
- Add helm unit tests covering internal, external, URL-override, special
  character encoding, and misconfiguration error paths.
- Update README with Kubernetes and OpenShift install examples for
  bundled and external PostgreSQL configurations.
- Add helm dependency build to lint and unittest tasks.
The helm-docs CI check failed because the Database backend section was
added directly to README.md instead of README.md.gotmpl. Move the
content to the template and regenerate so the check passes.
Replace the inline db-url stringData pattern with a proper Secret
containing individual fields plus a uri key.  When postgres.deploy=true
the Bitnami service-binding secret is referenced directly; when
deploy=false users can supply postgres.external.existingSecret to
bring their own Secret, or let the chart generate one from the external
field values.

Also restructures the README database section for clarity, adds
helm-unittest coverage for the new secret resolution paths, and
fixes a markdown lint issue in the root README.
Move test-openshift-scenarios.sh from deploy/helm/openshell/ci/ to
e2e/rust/e2e-openshift.sh, matching the existing e2e script naming
convention. Register it as `e2e:openshift` in tasks/test.toml — not
wired into the `test` or `e2e` aggregates so it only runs on explicit
invocation against a live OpenShift cluster.
Extend with-kube-gateway.sh with an optional multi-scenario loop gated
by OPENSHELL_E2E_KUBE_DB_SCENARIOS=1. When enabled, the script installs
the Helm chart three times — SQLite (default), bundled PostgreSQL, and
external PostgreSQL with existingSecret — running the full test suite
against each backend. When unset, existing single-install behavior is
unchanged.

Also adds helm dependency build before helm install, fixing CI failures
caused by the missing PostgreSQL subchart dependency.
@sauagarwa sauagarwa force-pushed the feat/helm-postgres-secret branch from 31d6a8e to 25ed9f6 Compare May 28, 2026 18:26
@TaylorMutch
Copy link
Copy Markdown
Collaborator

/ok to test 25ed9f6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(helm): add optional PostgreSQL backing store for gateway persistence

2 participants