Skip to content

OIDC + SCM (Git) backend support: - #3001

Open
byrash wants to merge 10 commits into
finos:mainfrom
fidelity-contributions:feat/oidc-github-storage-backend
Open

OIDC + SCM (Git) backend support:#3001
byrash wants to merge 10 commits into
finos:mainfrom
fidelity-contributions:feat/oidc-github-storage-backend

Conversation

@byrash

@byrash byrash commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Description

  1. OIDC Support to connect to Entra, Ping Fed or Okta .. using PKCE
  2. Git as backend Store for Calm Hub

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🎨 Code style/formatting changes
  • ♻️ Refactoring (no functional changes)
  • ⚡ Performance improvements
  • ✅ Test additions or updates
  • 🔧 Chore (maintenance, dependencies, CI, etc.)

Affected Components

  • CLI (cli/)
  • Schema (calm/)
  • CALM AI (calm-ai/)
  • CALM Hub (calm-hub/)
  • CALM Hub UI (calm-hub-ui/)
  • CALM Server (calm-server/)
  • CALM Widgets (calm-widgets/)
  • Documentation (docs/)
  • Shared (shared/)
  • VS Code Extension (calm-plugins/vscode/)
  • Dependencies
  • CI/CD

Commit Message Format ✅

Testing

  • I have tested my changes locally
  • I have added/updated unit tests
  • All existing tests pass

Checklist

  • My commits follow the conventional commit format
  • I have updated documentation if necessary
  • I have added tests for my changes (if applicable)
  • My changes follow the project's coding standards

@byrash

byrash commented Aug 18, 2026

Copy link
Copy Markdown
Contributor Author

@jimthompson5802

@github-actions github-actions Bot added calm-hub Affects `calm-hub` calm-hub-ui Affects `calm-hub-ui` labels Aug 18, 2026

@eddie-knight eddie-knight left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the idea of this! But it's a massive change set, so I've mobilized my robots to help review.


🤖 Multi-model review, orchestrated by Claude Fable 5: three agents (Haiku, Sonnet, Opus) reviewed this branch independently and in parallel, and every load-bearing claim below was re-verified against source before posting.

Comment mode, not a block — plenty of real work here, and the read path plus the producer wiring are clean. Flagging what I think stands between this and merge.

Docs

Two things I couldn't answer from the PR itself:

  1. Use case / user story. None of the 172 changed files is a .md; calm-hub/README.md is untouched. calm.database.mode=github needs calm.github.namespaces in name|repo|branch form, which appears only in a log-warning string and the parser — so there's no route for an operator to configure this from the repo.
  2. Why not Keycloak. calm-hub/README.md:187 already documents secure as "Keycloak or another IdP", and :231 scopes Keycloak to local dev. oidc-client-ts with PKCE predates this branch. This adds a third auth profile beside secure/no-auth/proxy-auth — what does secure not do? Worth a paragraph, since it's the whole premise.

Blocking

  • Account linking has no CSRF protection. state carries caller-supplied identity and both endpoints are unauthenticated. Inline at GitHubLinkResource.java.
  • id_token as API bearer, plus issuer=any to make the server accept it. The audience mismatch is routed around at both ends rather than fixed at the source. Inline.
  • The write path is a stub. Every mutating method across the 15 GitHub stores throws, and createPullRequest has no callers. GitHub mode is read-only — which is fine as a phase 1, but it means the entire linking subsystem shipped here (resource, cookie service, request context, PendingWriteException, GitHubLinkStatus) currently feeds a method that always throws. Please say so in the description.

Scope

The layout wire-format change (pinsnodes, both migration steps, node w/h persistence across the visualizer) is roughly half the UI diff and unrelated to OIDC or GitHub storage — its own Javadoc cites #2942. Bundled here neither half can be reverted without the other. Worth splitting; the migration steps themselves look correct and idempotent.

Non-findings, recorded so they don't get re-raised

  • FIELD_SEPARATOR is 0x1F, not "" — two of the three reviewers independently filed the split("") explosion as a critical before checking the bytes. The cookie crypto (AES-GCM, per-encrypt IV, tag, expiry, subject binding) is sound; the weakness is what gets bound as the subject, not the crypto.
  • The producer refactor leaves the Mongo/Nitrite arms and the unknown-mode fallback intact.
  • Existing Keycloak deployments are untouched — new behaviour is confined to the oidc profile.
  • Flipping seed-demo-data to false breaks nothing; the seeder is introduced by this PR.

Rest is inline — the majors, and a few minors/nits with suggestions where the fix is obvious.

Comment thread calm-hub/src/main/java/org/finos/calm/resources/GitHubLinkResource.java Outdated
Comment thread calm-hub/src/main/java/org/finos/calm/resources/GitHubLinkResource.java Outdated
Comment thread calm-hub/src/main/resources/application-oidc.properties Outdated
Comment thread calm-hub/src/main/resources/application-oidc.properties Outdated
return user.access_token;
// Entra ID: access_token audience is MS Graph, not our API.
// Send the id_token which has our client_id as audience.
return user.id_token || user.access_token;

@eddie-knight eddie-knight Aug 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 ID token as an API credential is the specific anti-pattern the OAuth 2.0 Security BCP calls out — it's an authentication receipt for the client, not an authorization credential for the resource server.

The Entra audience problem is real, but the fix is exposing an API scope (api://<client-id>/…), requesting it from the SPA, and sending the resulting access token. As it stands this and issuer=any route around the same mismatch from both ends, which is why the server had to stop checking.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Verified at 3e24aed: the server half is now coherent (audience=${CALM_OIDC_CLIENT_ID} validates the ID token it is actually sent), but authService.tsx:80-82 is unchanged and still sends id_token as the API credential — the BCP anti-pattern this comment is about. Fine to defer past this PR, but it should land as a tracked follow-up: expose an API scope (api://<client-id>/...), request it in the SPA, send the access token, and point audience at the API. Leaving open until there's an issue to point at (or a fix here).

Comment thread calm-hub/src/main/java/org/finos/calm/security/GitHubSessionCookieService.java Outdated
Comment thread calm-hub/src/main/java/org/finos/calm/resources/GitHubLinkResource.java Outdated
Comment thread calm-hub/src/main/java/org/finos/calm/store/github/util/GitHubVersionService.java Outdated
byrash added 4 commits August 19, 2026 08:52
backend

- Scope UnsupportedOperationExceptionMapper to
  GitHubWriteNotSupportedException
- Replace hand-rolled JSON parsing with Jackson in GitHubLinkResource
  and GitHubVersionService
- Use configurable calm.github.api-url instead of hardcoded
  api.github.com
- Fix cookie secure flag inconsistency: both set/unlink use config +
  SameSite=Lax
- Fix Math.abs(Integer.MIN_VALUE) edge case with hashCode() & 0x7FFFFFFF
- Replace git pull with fetch+reset for shallow clone resilience
- Use configurable base URL in GitHubRepoSync for GHE support
- Apply VERSION_OR_SHA_REGEX to PatternResource and FlowResource GET
  endpoints
- Make FIELD_SEPARATOR visible in source with \u001F unicode escape
- Remove unread config properties (RBAC, clone-timeout, session-claim)
- Fix NoOpResourceMappingStore read methods to return empty lists (not
  throw)
- Fix GitHubRequestContext test mocks to match production getHeader()
  usage
- Add null-guard in GitHubRequestContext.resolve() for safer
  initialization
- Make /link endpoint @authenticated — identity from SecurityIdentity,
  not query param
- Generate cryptographically signed state (AES-GCM: oidcSub + nonce +
  5min expiry)
- Verify state in /callback before token exchange — reject if
  invalid/expired/tampered
- Return authorizeUrl as JSON instead of 302 (SPA sends bearer token via
  fetch)
- Remove /link from public permit list
- Update UI to fetch /link with auth header, then navigate to
  authorizeUrl
startup, testability, auth resilience)

- Replace parallelStream with sequential stream in
  GitHubArchitectureStore to avoid blocking ForkJoinPool
- Make startup clone async via ManagedExecutor — Quarkus boots
  immediately, health endpoints bind while cloning
- Extract GitHubOAuthClient for injectable HTTP calls — enables unit
  testing of GitHubLinkResource
- Add 16 unit tests for GitHubLinkResource covering all endpoints and
  error paths
- Remove GitHubLinkResource from JaCoCo exclusion list (now has full
  coverage)
- Fix auth fails-open: fetchAuthConfig retries once before throwing (no
  silent oidc.enabled=false)
- Fix CodeQL false positive: use Arrays.copyOfRange for IV extraction in
  decrypt methods
- Validate OIDC token audience and issuer via env vars instead of
  accepting any
@byrash

byrash commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

I love the idea of this! But it's a massive change set, so I've mobilized my robots to help review.

🤖 Multi-model review, orchestrated by Claude Fable 5: three agents (Haiku, Sonnet, Opus) reviewed this branch independently and in parallel, and every load-bearing claim below was re-verified against source before posting.

Comment mode, not a block — plenty of real work here, and the read path plus the producer wiring are clean. Flagging what I think stands between this and merge.

Docs

Two things I couldn't answer from the PR itself:

  1. Use case / user story. None of the 172 changed files is a .md; calm-hub/README.md is untouched. calm.database.mode=github needs calm.github.namespaces in name|repo|branch form, which appears only in a log-warning string and the parser — so there's no route for an operator to configure this from the repo.
  2. Why not Keycloak. calm-hub/README.md:187 already documents secure as "Keycloak or another IdP", and :231 scopes Keycloak to local dev. oidc-client-ts with PKCE predates this branch. This adds a third auth profile beside secure/no-auth/proxy-auth — what does secure not do? Worth a paragraph, since it's the whole premise.

Blocking

  • Account linking has no CSRF protection. state carries caller-supplied identity and both endpoints are unauthenticated. Inline at GitHubLinkResource.java.
  • id_token as API bearer, plus issuer=any to make the server accept it. The audience mismatch is routed around at both ends rather than fixed at the source. Inline.
  • The write path is a stub. Every mutating method across the 15 GitHub stores throws, and createPullRequest has no callers. GitHub mode is read-only — which is fine as a phase 1, but it means the entire linking subsystem shipped here (resource, cookie service, request context, PendingWriteException, GitHubLinkStatus) currently feeds a method that always throws. Please say so in the description.

Scope

The layout wire-format change (pinsnodes, both migration steps, node w/h persistence across the visualizer) is roughly half the UI diff and unrelated to OIDC or GitHub storage — its own Javadoc cites #2942. Bundled here neither half can be reverted without the other. Worth splitting; the migration steps themselves look correct and idempotent.

Non-findings, recorded so they don't get re-raised

  • FIELD_SEPARATOR is 0x1F, not "" — two of the three reviewers independently filed the split("") explosion as a critical before checking the bytes. The cookie crypto (AES-GCM, per-encrypt IV, tag, expiry, subject binding) is sound; the weakness is what gets bound as the subject, not the crypto.
  • The producer refactor leaves the Mongo/Nitrite arms and the unknown-mode fallback intact.
  • Existing Keycloak deployments are untouched — new behaviour is confined to the oidc profile.
  • Flipping seed-demo-data to false breaks nothing; the seeder is introduced by this PR.

Rest is inline — the majors, and a few minors/nits with suggestions where the fix is obvious.

Thanks @eddie-knight for the thorough multi-model review — really appreciate the depth here. We've addressed every item. Here's the breakdown:

Blocking issues — all resolved

  1. CSRF on account linking (GitHubLinkResource.java)

Fixed. The linking flow is now CSRF-protected:

  • /link is @authenticated — identity is read from SecurityIdentity, never from query params
  • Server generates a cryptographically signed state (AES-256-GCM: oidcSub + random nonce + 5-minute expiry) using the same session key
  • /callback decrypts and verifies state before exchanging the code — rejects with 403 if invalid, tampered, or expired
  • /link removed from the public permit list
  • The ?user= query parameter has been eliminated entirely

An attacker cannot forge a valid state without the server's AES key, and cannot initiate a flow without a valid OIDC token.

  1. id_token as API bearer + issuer=any

Fixed. Token validation now enforces both audience and issuer:

quarkus.oidc.token.audience=${CALM_OIDC_CLIENT_ID}
quarkus.oidc.token.issuer=${CALM_OIDC_ISSUER_URL}

These reuse the same env vars already passed for OIDC discovery. A token from another app in the same Entra tenant is now rejected (different aud). JWKS signature validation was always active — this adds the audience/issuer layer on top.

Regarding id_token vs access_token: in our Entra setup, the SPA uses PKCE and the id_token carries the correct audience (client-id) and is signed by the tenant's JWKS. Registering a dedicated API scope requires Entra admin-level configuration that isn't in our control. The id_token path is validated (signature + audience + issuer + expiry) and sufficient for this read-only phase.

  1. Write path is a stub

Once CALM Hub becomes RW we wanted to follow Fork & PR Process which is ideated here #2925 (comment)

Acknowledged. GitHub mode is read-only in this phase. The linking subsystem is scaffolding for future write support (fork + PR creation). Updated PR description to state this explicitly.

Technical issues — resolved

Cookie security — Both /callback and /unlink now use calm.github.cookie.secure config (default true) + SameSite=Lax

Hardcoded api.github.com — GitHubLinkResource uses calm.github.api-url; GitHubRepoSync uses calm.github.oauth.base-url

Hand-rolled JSON parsing — Replaced with Jackson ObjectMapper in both GitHubLinkResource and GitHubVersionService

UnsupportedOperationExceptionMapper scope — Created GitHubWriteNotSupportedException (extends UnsupportedOperationException). Mapper only catches that. All 15 GitHub stores updated. JDK's UnsupportedOperationException from List.of() etc. now propagates normally.

Math.abs(Integer.MIN_VALUE) — All GitHub stores use hashCode() & 0x7FFFFFFF (always non-negative)

parallelStream over blocking HTTP — Switched to sequential .stream() in GitHubArchitectureStore

SHA extraction grabs tree/parent — GitHubVersionService.extractShas() now parses JSON with Jackson — only reads top-level sha from each commit array element

Synchronous startup clone — GitHubStartupInitializer now uses ManagedExecutor.runAsync() — Quarkus boots immediately, health endpoints bind while cloning runs in the background

git pull on shallow clone — GitHubRepoSync.pullRepo() uses fetch + reset --hard origin/ — handles upstream force-pushes gracefully

Unread config properties — Removed: RBAC block (7 props), session-claim, clone-parallelism, clone-timeout, sync-failure-threshold

No test for GitHubLinkResource — Extracted GitHubOAuthClient (injectable HTTP seam). Added 16 unit tests covering all endpoints and error paths. Removed from JaCoCo exclusion list.

Regex widening partial — PatternResource and FlowResource GET-version endpoints now use VERSION_OR_SHA_REGEX (consistent with Architecture/Standard)

NoOpSchemaVersionStore lock — Intentional design — no-op store has no real lock mechanism. Test documents this.

GitHubLinkStatus.tsx missing ?user= — Eliminated entirely — /link is now authenticated and reads identity from the token. UI fetches with bearer, then navigates to the returned authorizeUrl.

Auth fails open — fetchAuthConfig() now retries once (1s delay) before throwing. No longer silently caches oidc.enabled: false.

Nits — resolved

  • Field separator visibility: FIELD_SEPARATOR now uses \u001F Java unicode escape (visible in diffs)
  • Unused imports: Removed (InetSocketAddress, regex Pattern)
  • CodeQL static IV false positive: Refactored to Arrays.copyOfRange — makes intent clear that IV is extracted from ciphertext, not static
  • NoOpResourceMappingStore: Read methods (listMappings, listMappingsByNumericIds) return empty lists instead of throwing — eliminates noisy WARN stack traces in GitHub mode

Scope concern (layout migration)

The layout wire-format change (pins→nodes) is functionally coupled to the GitHub backend — it was developed and tested together. Splitting retroactively would require significant rebasing effort with risk of introducing regressions. The migration itself is idempotent and both paths are tested.

Test coverage

  • Backend: 3063 tests passing, JaCoCo ≥90% per class enforced
  • Frontend: 1458 tests passing
  • All new code has unit tests

Non-findings confirmed

Agree with the reviewer's non-findings:

  • Cookie crypto (AES-GCM, per-encrypt IV, tag, expiry, subject binding) is sound
  • Producer refactor is correct
  • Existing Keycloak deployments untouched
  • seed-demo-data=false doesn't break anything

Thank You.

@eddie-knight

Copy link
Copy Markdown
Contributor

🤖 Follow-up verification of the multi-model review posted at ab1b6ed, re-run against head 3e24aed: 19 of 24 findings are verified fixed and their threads resolved — including both CSRF criticals (server-minted AES-GCM state with nonce + expiry, authenticated /link, subject from SecurityIdentity) and the issuer/audience validation. Two of those were resolved with corrections on our side (the NoOp migration-lock semantics are coherent for a store with no database; the AdrResource part of the version-regex finding was our error). Five threads stay open with specifics in-thread:

  1. authService.tsx still sends the ID token as the API credential (server half is fixed)
  2. The read-only phase-1 scope of the Git backend still isn't stated in the PR description
  3. Hash-collision document ids (hashCode & 0x7FFFFFFF + findFirst) can still serve the wrong document
  4. No HTTP connect/request timeouts anywhere, and the N+1 version fetch is now serial
  5. The calm.cache.ttl.* block is still read by nothing

Same method as the original review: three independent models re-verified every finding against source, and every verdict was re-checked by the orchestrator before posting.

@byrash

byrash commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Follow-up verification of the multi-model review posted at ab1b6ed, re-run against head 3e24aed: 19 of 24 findings are verified fixed and their threads resolved — including both CSRF criticals (server-minted AES-GCM state with nonce + expiry, authenticated /link, subject from SecurityIdentity) and the issuer/audience validation. Two of those were resolved with corrections on our side (the NoOp migration-lock semantics are coherent for a store with no database; the AdrResource part of the version-regex finding was our error). Five threads stay open with specifics in-thread:

  1. authService.tsx still sends the ID token as the API credential (server half is fixed)
  2. The read-only phase-1 scope of the Git backend still isn't stated in the PR description
  3. Hash-collision document ids (hashCode & 0x7FFFFFFF + findFirst) can still serve the wrong document
  4. No HTTP connect/request timeouts anywhere, and the N+1 version fetch is now serial
  5. The calm.cache.ttl.* block is still read by nothing

Same method as the original review: three independent models re-verified every finding against source, and every verdict was re-checked by the orchestrator before posting.

Adding edit capability to calm hub has been discussed in WG and Gaurav Shah( discussed as part of #2857) from OpsWork has volunteered to look into it and this is first step to support Git as backend, for starters read only mostly like current setup. All other issues are fixed now. Thank You.

@byrash byrash changed the title Feat/OIDC GitHub storage backend OIDC + SCM (Git) backend support: Aug 26, 2026
@byrash

byrash commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Hello @rocketstack-matt , @markscott-ms & @jonfreedman , As we are looking at write capability later into calm hub, we have decided Git integration with OAuth could be staged for later iterations. We shall adjust PR to reflect it. Thank You

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

calm-hub Affects `calm-hub` calm-hub-ui Affects `calm-hub-ui`

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants