Skip to content

feat(cli): publish narrative documents from workspaces - #3015

Draft
101Steeps wants to merge 14 commits into
finos:mainfrom
101Steeps:feat/workspace-document-publish-2983
Draft

feat(cli): publish narrative documents from workspaces#3015
101Steeps wants to merge 14 commits into
finos:mainfrom
101Steeps:feat/workspace-document-publish-2983

Conversation

@101Steeps

@101Steeps 101Steeps commented Aug 22, 2026

Copy link
Copy Markdown

Description

Implements #2983: publish versioned Markdown narrative documents from a CALM workspace to CALM Hub.

  • Adds workspace support for sad and knowledge Markdown documents.
  • Tracks document identity and published Hub state in the workspace manifest.
  • Extends workspace check, bump, and push for narrative documents.
  • Validates persisted narrative identity before Hub calls.
  • Adds CLI/shared unit tests, a smoke scenario, and CLI README guidance.

Depends on the narrative-document API in #3002 / #2982. The combined local POC branch was used only for integration testing; this PR remains focused on the CLI and shared client changes.

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🎨 Code style/formatting changes
  • ♻️ Refactoring (no functional changes)
  • ⚡ Performance improvements
  • ✅ Test additions or updates
  • 🔧 Chore (maintenance, dependencies, CI, etc.)

Affected Components

  • CLI (cli/)
  • Schema (calm/)
  • CALM AI (calm-ai/)
  • CALM Hub (calm-hub/)
  • CALM Hub UI (calm-hub-ui/)
  • CALM Server (calm-server/)
  • CALM Widgets (calm-widgets/)
  • Documentation (docs/)
  • Shared (shared/)
  • VS Code Extension (calm-plugins/vscode/)
  • Dependencies
  • CI/CD

Commit Message Format ✅

All commits follow Conventional Commits.

Testing

  • I have tested my changes locally

  • I have added/updated unit tests

  • All existing tests pass

  • npm test passed on Linux: all 17 workspace summaries passed (including shared: 90 files / 1,084 tests; CLI: 28 files / 682 tests).

  • npm run lint completed with 0 errors; 11 existing warnings remain outside this change.

  • npm run build:cli passed.

  • Combined local POC: First-class Documents in CALM Hub  #2981 POC, with Add a first-class Document endpoint to CALM Hub #2982 merged locally:

    • cd calm-hub && ../mvnw verify — 2,772 tests passed; JaCoCo checks passed.
    • cd calm-hub && ../mvnw -P integration verify — 537 tests passed; JaCoCo checks passed.
    • bash scripts/build-hub-smoke-image.sh — passed and built calm-hub:smoke.
    • npm run test:smoke --workspace cli — 7 test files / 28 tests passed.

The smoke test exercises the full narrative-document workflow against a disposable Docker Calm Hub instance: create a workspace, publish architecture and Markdown SAD 1.0.0, retrieve it, detect an edited-but-unbumped document, bump to 1.1.0, republish it, and retrieve the updated Markdown.

A standalone manual POC repeated the same flow against local Calm Hub. The final workspace check reported the workspace up to date and all documents valid.

Swagger Evidence
Endpoint list showing the five narrative document routes
image

Version API used by workspace push
image

Retrieval endpoint used to verify the stored markdown
image

Checklist

  • My commits follow the conventional commit format
  • I have updated documentation if necessary
  • I have added tests for my changes (if applicable)
  • My changes follow the project's coding standards

@github-actions github-actions Bot added cli Affects `cli` code shared labels Aug 22, 2026
Comment thread shared/src/hub/calm-hub-client.ts Outdated

export type ResourceChangeType = 'MAJOR' | 'MINOR' | 'PATCH';

export const NARRATIVE_DOCUMENT_TYPES = ['knowledge', 'sad'] as const;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are structurally identical to CALM_DOCUMENT_TYPES_LIST / CalmDocumentType / isValidCalmDocumentType in calm-models/types. Co-locating them there would give a single source of truth and clean up the union in bundle.ts and the import in workspace-document-loader.ts. Hub API DTOs should stay here.

throw new Error('Narrative document Hub identity is incomplete. Re-add the document to repair it.');
}
const identity = {
namespace: entry.namespace ?? '',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ?? '' is misleading — validateNarrativeIdentity rejects empty strings anyway. Same at bump.ts.

const location = await client.createNarrativeDocumentVersion(
identity.namespace, identity.type, identity.calmHubDocumentId!, version, narrative.request
);
parseNarrativeDocumentLocation(location, identity);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return value discarded — pure validation. A // validate Location comment would prevent someone removing this.


vi.mock('@finos/calm-shared/src/hub/calm-hub-client', () => ({
CalmHubClient: mocks.CalmHubClient,
NARRATIVE_DOCUMENT_TYPES: ['knowledge', 'sad'],

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will drift if a type is added to the canonical constant. Consider vi.importActual.

copy: options.copy,
type,
namespace: options.namespace.trim(),
version: '1.0.0',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always 1.0.0, no --version override. Re-adding after manifest removal will reset regardless of Hub state.

for (const [id, entry] of Object.entries(manifest)) {
const filePath = resolveFilePath(bundlePath, entry.path);
if (!existsSync(filePath)) {
if (isNarrativeDocumentType(entry.type)) throw new Error(`Narrative document '${id}' file not found: ${filePath}`);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throws immediately here, while push.ts accumulates. Defensible, but a one-line rationale comment would help.

import { existsSync, readFileSync } from 'fs';
import { SchemaDirectory } from '../schema-directory';
import path from 'path';
import { NARRATIVE_DOCUMENT_TYPES, type NarrativeDocumentType } from '../hub/calm-hub-client';

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If types move to calm-models (see comment on calm-hub-client.ts), this can import from @finos/calm-models/types — matching the existing CalmDocumentType import.

@LeighFinegold

Copy link
Copy Markdown
Member

@101Steeps — POC is coming along nicely 👍. Just the CalmHub visuals left now.
A few inline suggestions, mainly around moving the narrative type definitions into calm-models alongside the existing document types.

urlToLocalPathMapping?: Map<string, string>;
}

const YAML_FRONTMATTER_PATTERN = /^---\r?\n([\s\S]*?)\r?\n---(?=\r?\n|$)/;

@LeighFinegold LeighFinegold Aug 23, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we build this out, I want to see what it takes to converge this with docify's parseFrontMatterFromContent just below. For example, docify expects the closing --- (with optional trailing whitespace via .trim()), but this regex is stricter. Part of the POC goal is to figure out where the two can share code. I'll pick that up when I loop back to flow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli Affects `cli` code shared

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants