Skip to content

Enhance directUrlDocumentLoader support for End User Specific IDP - #3013

Open
jimthompson5802 wants to merge 17 commits into
finos:mainfrom
jimthompson5802:iss2975-idp-directurl
Open

Enhance directUrlDocumentLoader support for End User Specific IDP#3013
jimthompson5802 wants to merge 17 commits into
finos:mainfrom
jimthompson5802:iss2975-idp-directurl

Conversation

@jimthompson5802

@jimthompson5802 jimthompson5802 commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Description

Closes #2975

This PR adds support for authenticated direct http(s) document loading in the CLI and shared document loader stack. It introduces a separate directUrlAuth configuration path for protected remote schema and document fetches, wires that module into the CLI document loader configuration, and improves error handling and debug logging so authentication failures are clearer without exposing sensitive header values.

The intent of this PR is to provide an interface specification that lets an end-user organization supply a custom authentication module for its own Identity Provider. That module remains outside the CALM project and is fully owned by the end-user organization.

For TLS and certificate validation, this PR relies on Node.js runtime behavior rather than adding CALM-specific certificate handling. In environments that use private or enterprise certificate chains, trust can be extended with NODE_EXTRA_CA_CERTS. In environments where certificate verification must be bypassed for local or controlled use cases, Node also supports NODE_TLS_REJECT_UNAUTHORIZED=0. This keeps certificate handling aligned with Node.js configuration and leaves trust and validation policy under deployment control.

IMPORTANT NOTE: To view how an end user organization integrates with the changes in this PR, see this repo: https://github.com/jimthompson5802/calm-web-repo. The README describes the three test scenarios used for testing the PR. There are sample code and configurations that show how the end user provided authentication module is used and integrates with calm cli DirectUrlDocumentLoader.

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🎨 Code style/formatting changes
  • ♻️ Refactoring (no functional changes)
  • ⚡ Performance improvements
  • ✅ Test additions or updates
  • 🔧 Chore (maintenance, dependencies, CI, etc.)

Affected Components

  • CLI (cli/)
  • Schema (calm/)
  • CALM AI (calm-ai/)
  • CALM Hub (calm-hub/)
  • CALM Hub UI (calm-hub-ui/)
  • CALM Server (calm-server/)
  • CALM Widgets (calm-widgets/)
  • Documentation (docs/)
  • Shared (shared/)
  • VS Code Extension (calm-plugins/vscode/)
  • Dependencies
  • CI/CD

Commit Message Format ✅

Testing

  • I have tested my changes locally
  • I have added/updated unit tests
  • All existing tests pass

Checklist

  • My commits follow the conventional commit format
  • I have updated documentation if necessary
  • I have added tests for my changes (if applicable)
  • My changes follow the project's coding standards

@github-actions github-actions Bot added cli Affects `cli` code shared labels Aug 22, 2026
@jimthompson5802 jimthompson5802 changed the title Iss2975 idp directurl Enhance directUrlDocumentLoader support for End User Specific IDP Aug 22, 2026
@jimthompson5802

Copy link
Copy Markdown
Contributor Author

@markscott-ms @rocketstack-matt @jpgough-ms @willosborne @byrash here is the PR that implements authenticated access for the DirectUrlDocumentLoader.

To view how an end user organization integrates with the changes in this PR, see this repo: https://github.com/jimthompson5802/calm-web-repo. The README describes the three test scenarios used for testing the PR. There are sample code and configurations that show how the end user provided authentication module is used and integrates with calm cli DirectUrlDocumentLoader.

@rocketstack-matt rocketstack-matt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two non-blocking findings on the verbose error-cause logging in validate.ts. Build/lint/full test suite all pass locally.

const stack = err instanceof Error ? err.stack : undefined;
logger.error('An error occurred while validating: ' + message);
if (options.verbose) {
for (const causeMessage of formatErrorCauseChain(err)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This walks error.cause and logs each nested .message under --verbose. For the direct-URL auth failure path, cause is whatever the user's own auth-plugin code threw (direct-url-document-loader.ts AUTHENTICATION_FAILED branches) — unlike the redacted debug-request logging elsewhere in this PR, nothing sanitizes that message before it's printed. Worth redacting/suppressing the cause chain specifically for AUTHENTICATION_FAILED errors, consistent with the header-redaction already done for debug request logs.

let currentCause = getErrorCause(error);
let isFirst = true;

while (currentCause instanceof Error) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No cycle guard here: if error.cause ever forms a cycle (self-referential, or A→B→A) — plausible from a buggy or malicious user-supplied auth module, which is exactly the trust boundary this feature introduces — this loop never terminates and hangs the process under --verbose.

Suggested change
while (currentCause instanceof Error) {
const seen = new Set<Error>();
while (currentCause instanceof Error && !seen.has(currentCause)) {
seen.add(currentCause);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli Affects `cli` code shared

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Authentication Support for Direct URL Document Loading

2 participants