feat(cli): publish narrative documents from workspaces

cli/README.md

@@ -803,7 +803,7 @@ calm workspace init my-system
 
 #### `calm workspace add <file>`
 
-Register a CALM document with the active workspace. By default the file is referenced at its current location on disk (no copying). Prompts interactively for document type and (manifest) name if they cannot be determined automatically.
+Register a CALM JSON document or narrative Markdown document with the active workspace. By default the file is referenced at its current location on disk (no copying). Prompts interactively for document type and (manifest) name if they cannot be determined automatically.
 
 ```
 calm workspace add <file> [--id <id>] [--type <type>] [--namespace <namespace>] [--copy]
@@ -812,23 +812,28 @@ calm workspace add <file> [--id <id>] [--type <type>] [--namespace <namespace>]
 | Option | Description |
 |--------|-------------|
 | `--id <id>` | Explicit manifest registration id. Overrides automatic resolution. |
-| `--type <type>` | Document type. If omitted, an interactive dropdown is shown. One of: `pattern`, `architecture`, `interface`, `flow`, `control`, `schema`, `timeline`, `adr`. |
-| `--namespace <namespace>` | CalmHub namespace to record in the manifest. If omitted, it is derived from the document `$id`. |
+| `--type <type>` | Document type. If omitted, an interactive dropdown is shown. One of: `pattern`, `architecture`, `interface`, `flow`, `control`, `schema`, `timeline`, `adr`, `knowledge`, `sad`. |
+| `--namespace <namespace>` | CalmHub namespace to record in the manifest. It is required for narrative Markdown and otherwise derived from the document `$id` when omitted. |
 | `--copy` | Copy the file into the bundle's `files/` directory instead of referencing it in place. |
 
-**Document `$id` handling.** `add` inspects the file's CalmHub `$id`:
+**Narrative Markdown documents.** Use `--type knowledge` or `sad`. `add` reads YAML frontmatter. A non-empty `title` becomes the manifest name unless you supply `--id`. `--namespace` is required. The initial manifest version is `1.0.0`. Markdown has no CALM `$id` and is never rewritten.
+
+**JSON document `$id` handling.** For JSON mapping documents, `add` inspects the file's CalmHub `$id`:
 - **No `$id`** → you are prompted interactively to build one from its components (see below); the `$id` is written into the file and the document is added.
 - **Conformant `$id`** → left untouched; the manifest namespace is derived from it.
 - **Non-conformant `$id`** → left as-is; a warning is printed and the document is still tracked, but it cannot be pushed to CalmHub until the `$id` is fixed (silently rewriting it would lose data for types that don't use CalmHub URLs, e.g. `flow`, `adr`, `timeline`).
 
-**Manifest name resolution** (when `--id` is not given): the `title` field from the JSON file, else an interactive prompt.
+**Manifest name resolution** (when `--id` is not given): the `title` field from the JSON file or Markdown frontmatter, else an interactive prompt.
 
 ```shell
 # Interactive — prompts for type, builds the $id if needed, then the manifest name
 calm workspace add ./architectures/payment-service.json
 
 # Reference an already-conformant document without copying
 calm workspace add ./architectures/payment-service.json --type architecture
+
+# Register a narrative Markdown document; the frontmatter title becomes its manifest name
+calm workspace add ./docs/payments-sad.md --type sad --namespace finos
 ```
 
 #### `calm workspace new [type] [name] [template]`
@@ -863,7 +868,9 @@ where `$TYPE` is one of `patterns`, `architectures`, `standards`, `interfaces`.
 
 #### `calm workspace push`
 
-Push every document in the workspace manifest to a CalmHub instance. Each document's identity — namespace, type, mapping id and **version** — comes from its `$id` (of the form `$BASE_URL/calm/namespaces/$NAMESPACE/$TYPE/$MAPPING_ID/versions/$VERSION`). Push **does not auto-bump**: it creates exactly the version each document declares. Documents without a well-formed mapping `$id` (or whose type has no CalmHub resource type) are skipped with a warning.
+Push every document in the workspace manifest to a CalmHub instance. JSON mapping documents derive their identity — namespace, type, mapping id and **version** — from `$id` (of the form `$BASE_URL/calm/namespaces/$NAMESPACE/$TYPE/$MAPPING_ID/versions/$VERSION`). Push **does not auto-bump**: it creates exactly the version each document declares. Documents without a well-formed mapping `$id` (or whose type has no CalmHub resource type) are skipped with a warning.
+
+Narrative Markdown documents use `--type knowledge` or `--type sad`. They require YAML frontmatter with a `title` and `--namespace`. The first push stores the Hub numeric document id, location, and version (`1.0.0`) in `workspace-manifest.json`. Later changes require `workspace bump`; the command updates the manifest version without rewriting the Markdown.
 
 ```
 calm workspace push [--calm-hub-url <url>] [--fail-if-modified]
@@ -887,6 +894,17 @@ calm workspace push --calm-hub-url https://calmhub.example.com
 calm workspace push --fail-if-modified           # strict merge-time mode
 ```
 
+```shell
+# First-class document POC: add, publish, inspect, edit, bump, and publish again
+calm workspace add ./docs/payments-sad.md --type sad --namespace finos
+calm workspace push --calm-hub-url http://localhost:8080
+calm workspace show                              # shows the published Hub location
+calm workspace check --calm-hub-url http://localhost:8080
+# Edit ./docs/payments-sad.md, then bump and publish the new version
+calm workspace bump --minor --calm-hub-url http://localhost:8080
+calm workspace push --calm-hub-url http://localhost:8080
+```
+
 #### `calm workspace check`
 
 Check whether any tracked document has changed on disk relative to CalmHub but has **not** been version-bumped. Intended as a CI/PR gate — it **exits non-zero** when a bump is required, so a PR cannot merge with unversioned changes.

cli/smoke/harness/hub-api.ts

@@ -39,5 +39,10 @@ export function hubApi(baseUrl: string = SMOKE_HUB_URL) {
                 `${baseUrl}/calm/namespaces/${namespace}/${type}/${mapping}/versions/${version}`
             );
         },
+        async getNarrativeDocument(namespace: string, type: string, id: number, version: string): Promise<string> {
+            const body = await getJson(`${baseUrl}/api/calm/namespaces/${namespace}/documents/${type}/${id}/versions/${version}`);
+            if (typeof body.documentMarkdown !== 'string') throw new Error('Narrative document response has no documentMarkdown');
+            return body.documentMarkdown;
+        },
     };
 }

cli/smoke/workspace-documents.smoke.spec.ts

@@ -0,0 +1,62 @@
+import path from 'path';
+import * as fs from 'fs';
+import { execSync } from 'child_process';
+import { afterAll, beforeAll, describe, expect, test } from 'vitest';
+import { installPackedCli, type CliInstall } from '../src/test_helpers/cli-runner';
+import { SMOKE_HUB_URL } from './global-setup';
+import { hubApi } from './harness/hub-api';
+import { hubDocId } from './harness/fixtures';
+
+const CLI_ROOT = path.resolve(__dirname, '..');
+const NS = 'smoke-workspace-documents';
+const api = hubApi();
+
+describe('workspace narrative-document POC', () => {
+    let cli: CliInstall;
+    let wsDir: string;
+    let documentPath: string;
+    let architecturePath: string;
+    let documentId: number;
+    const initial = '---\ntitle: Payments SAD\ndescription: Smoke document\n---\n# Payments\n';
+
+    async function run(args: string[]) {
+        return cli.run(args, { cwd: wsDir });
+    }
+
+    beforeAll(async () => {
+        cli = installPackedCli(CLI_ROOT, 'calm-smoke-workspace-documents');
+        wsDir = path.join(cli.tempDir, 'repo');
+        fs.mkdirSync(wsDir, { recursive: true });
+        execSync('git init', { cwd: wsDir, stdio: 'inherit' });
+        documentPath = path.join(wsDir, 'payments-sad.md');
+        fs.writeFileSync(documentPath, initial);
+        architecturePath = path.join(wsDir, 'payments.architecture.json');
+        fs.writeFileSync(architecturePath, JSON.stringify({
+            $schema: 'https://calm.finos.org/release/1.0/meta/calm.json',
+            $id: hubDocId(NS, 'architectures', 'payments', '1.0.0'),
+            title: 'Payments', nodes: [], relationships: [],
+        }, null, 2));
+        await cli.run(['hub', 'create', 'namespace', '--name', NS, '--description', 'workspace documents smoke', '-c', SMOKE_HUB_URL]);
+    }, 120_000);
+
+    afterAll(() => cli?.cleanup());
+
+    test('publishes, retrieves, bumps, and republishes a Markdown document', async () => {
+        await run(['workspace', 'init', 'documents']);
+        await run(['workspace', 'add', architecturePath, '--type', 'architecture', '--namespace', NS]);
+        await run(['workspace', 'add', documentPath, '--type', 'sad', '--namespace', NS]);
+        await run(['workspace', 'push', '--calm-hub-url', SMOKE_HUB_URL]);
+        expect(await api.listVersions(NS, 'architectures', 'payments')).toContain('1.0.0');
+
+        const manifestPath = path.join(wsDir, '.calm-workspace', 'bundles', 'documents', 'workspace-manifest.json');
+        const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8')) as Record<string, { calmHubDocumentId: number }>;
+        documentId = manifest['Payments SAD'].calmHubDocumentId;
+        expect(await api.getNarrativeDocument(NS, 'sad', documentId, '1.0.0')).toBe(initial);
+
+        fs.writeFileSync(documentPath, initial.replace('# Payments', '# Updated payments'));
+        await expect(run(['workspace', 'check', '--calm-hub-url', SMOKE_HUB_URL])).rejects.toHaveProperty('exitCode', 1);
+        await run(['workspace', 'bump', '--minor', '--calm-hub-url', SMOKE_HUB_URL]);
+        await run(['workspace', 'push', '--calm-hub-url', SMOKE_HUB_URL]);
+        expect(await api.getNarrativeDocument(NS, 'sad', documentId, '1.1.0')).toContain('# Updated payments');
+    });
+});

cli/src/command-helpers/workspace/bump.spec.ts

@@ -1,6 +1,6 @@
 import { describe, it, expect, beforeAll, afterAll, beforeEach, vi } from 'vitest';
 import { detectChangedResources, bumpWorkspace, canonicalEqual, maxIncrement } from './bump';
-import { saveManifest } from './bundle';
+import { loadManifest, saveManifest } from './bundle';
 import { CalmHubClient, ResourceChangeType } from '@finos/calm-shared/src/hub/calm-hub-client';
 import { mkdir, writeFile, rm, readFile } from 'fs/promises';
 import path from 'path';
@@ -16,10 +16,14 @@ const idAt = (resource: string, version: string, type = 'architectures', ns = 'c
 interface ClientOpts {
     versions?: Record<string, string[]>;
     remote?: Record<string, object>;
+    narrativeVersions?: string[];
+    narrativeMarkdown?: string;
 }
 const makeClient = (opts: ClientOpts = {}): CalmHubClient => ({
     getMappedResourceVersions: vi.fn(async (_ns: string, mappingId: string) => opts.versions?.[mappingId] ?? []),
     getMappedResourceByVersion: vi.fn(async (_ns: string, mappingId: string, version: string) => opts.remote?.[`${mappingId}@${version}`] ?? {}),
+    getNarrativeDocumentVersions: vi.fn(async () => opts.narrativeVersions ?? []),
+    getNarrativeDocumentVersion: vi.fn(async () => ({ documentMarkdown: opts.narrativeMarkdown ?? '' })),
 }) as unknown as CalmHubClient;
 
 describe('bump', () => {
@@ -48,6 +52,111 @@ describe('bump', () => {
     });
 
     describe('detectChangedResources', () => {
+        it('treats new, already-bumped, and unchanged narrative documents as clean', async () => {
+            const markdown = '---\ntitle: Payments SAD\n---\n# Published\n';
+            await writeFile(path.join(filesPath, 'payments.md'), markdown);
+            const baseEntry = {
+                path: 'files/payments.md', type: 'sad' as const, namespace: 'com.example', version: '1.0.0',
+                calmHubDocumentId: 42, calmHubId: '/api/calm/namespaces/com.example/documents/sad/42/versions/1.0.0',
+            };
+
+            await saveManifest(bundlePath, { payments: { ...baseEntry, calmHubDocumentId: undefined, calmHubId: undefined } });
+            expect(await detectChangedResources(bundlePath, makeClient())).toEqual([]);
+
+            await saveManifest(bundlePath, { payments: { ...baseEntry, version: '1.1.0' } });
+            expect(await detectChangedResources(bundlePath, makeClient({ narrativeVersions: ['1.0.0'] }))).toEqual([]);
+
+            await saveManifest(bundlePath, { payments: baseEntry });
+            expect(await detectChangedResources(bundlePath, makeClient({ narrativeVersions: ['1.0.0'], narrativeMarkdown: markdown }))).toEqual([]);
+        });
+
+        it('fails narrative checks with incomplete identity or missing source', async () => {
+            await saveManifest(bundlePath, {
+                partial: { path: 'files/missing.md', type: 'sad', namespace: 'com.example', version: '1.0.0', calmHubId: '/partial' },
+            });
+            await expect(detectChangedResources(bundlePath, makeClient())).rejects.toThrow(/file not found/);
+
+            await writeFile(path.join(filesPath, 'partial.md'), '---\ntitle: Partial\n---\n# Partial');
+            await saveManifest(bundlePath, {
+                partial: { path: 'files/partial.md', type: 'sad', namespace: 'com.example', version: '1.0.0', calmHubId: '/partial' },
+            });
+            await expect(detectChangedResources(bundlePath, makeClient())).rejects.toThrow(/incomplete Hub identity/);
+        });
+
+        it('fails narrative checks when Hub version retrieval fails', async () => {
+            await writeFile(path.join(filesPath, 'payments.md'), '---\ntitle: Payments SAD\n---\n# Payments');
+            await saveManifest(bundlePath, {
+                payments: {
+                    path: 'files/payments.md', type: 'sad', namespace: 'com.example', version: '1.0.0',
+                    calmHubDocumentId: 42, calmHubId: '/api/calm/namespaces/com.example/documents/sad/42/versions/1.0.0',
+                },
+            });
+            const client = makeClient({ narrativeVersions: ['1.0.0'] });
+            (client.getNarrativeDocumentVersion as ReturnType<typeof vi.fn>).mockRejectedValueOnce(new Error('Hub unavailable'));
+
+            await expect(detectChangedResources(bundlePath, client)).rejects.toThrow(/Hub unavailable/);
+        });
+
+        it('treats a document with no Hub versions as new and rejects missing manifest versions', async () => {
+            await writeFile(path.join(filesPath, 'payments.md'), '---\ntitle: Payments SAD\n---\n# Payments');
+            const entry = {
+                path: 'files/payments.md', type: 'sad' as const, namespace: 'com.example', version: '1.0.0',
+                calmHubDocumentId: 42, calmHubId: '/api/calm/namespaces/com.example/documents/sad/42/versions/1.0.0',
+            };
+            await saveManifest(bundlePath, { payments: entry });
+            expect(await detectChangedResources(bundlePath, makeClient({ narrativeVersions: [] }))).toEqual([]);
+
+            await saveManifest(bundlePath, { payments: { ...entry, version: undefined } });
+            await expect(detectChangedResources(bundlePath, makeClient())).rejects.toThrow(/no manifest version/);
+        });
+
+        it('fails narrative checks when a tracked path cannot be read', async () => {
+            await saveManifest(bundlePath, {
+                unreadable: { path: 'files', type: 'sad', namespace: 'com.example', version: '1.0.0' },
+            });
+            await expect(detectChangedResources(bundlePath, makeClient())).rejects.toThrow(/could not be read/);
+        });
+
+        it('detects and bumps changed narrative Markdown without rewriting it', async () => {
+            const markdown = '---\ntitle: Payments SAD\n---\n# Changed\n';
+            await writeFile(path.join(filesPath, 'payments.md'), markdown);
+            await saveManifest(bundlePath, {
+                payments: {
+                    path: 'files/payments.md', type: 'sad', namespace: 'com.example',
+                    version: '1.0.0', calmHubDocumentId: 42,
+                    calmHubId: '/api/calm/namespaces/com.example/documents/sad/42/versions/1.0.0',
+                },
+            });
+            const client = makeClient({ narrativeVersions: ['1.0.0'], narrativeMarkdown: markdown.replace('Changed', 'Published') });
+
+            const changed = await detectChangedResources(bundlePath, client);
+            expect(changed).toHaveLength(1);
+            await bumpWorkspace(bundlePath, client, { increment: 'MINOR', preDetectedChanges: changed });
+
+            expect((await loadManifest(bundlePath)).payments.version).toBe('1.1.0');
+            expect(await readFile(path.join(filesPath, 'payments.md'), 'utf8')).toBe(markdown);
+        });
+
+        it.each([
+            ['MAJOR', '2.0.0'],
+            ['PATCH', '1.0.1'],
+        ] as const)('applies a %s bump to a changed narrative document', async (increment, version) => {
+            const markdown = '---\ntitle: Payments SAD\n---\n# Changed\n';
+            await writeFile(path.join(filesPath, 'payments.md'), markdown);
+            await saveManifest(bundlePath, {
+                payments: {
+                    path: 'files/payments.md', type: 'sad', namespace: 'com.example', version: '1.0.0', calmHubDocumentId: 42,
+                    calmHubId: '/api/calm/namespaces/com.example/documents/sad/42/versions/1.0.0',
+                },
+            });
+            const client = makeClient({ narrativeVersions: ['1.0.0'], narrativeMarkdown: markdown.replace('Changed', 'Published') });
+
+            await bumpWorkspace(bundlePath, client, { increment });
+            expect((await loadManifest(bundlePath)).payments.version).toBe(version);
+
+            expect(await bumpWorkspace(bundlePath, client, { increment })).toMatchObject({ bumped: [] });
+        });
+
         it('skips a brand-new resource with no versions in CalmHub', async () => {
             await write('a.json', { $id: idAt('a', '1.0.0'), title: 'A' });
             await saveManifest(bundlePath, { 'a': { path: 'files/a.json', type: 'architecture' } });