Make Threadnote 4 self-contained

[Kashkovsky]

Summary

Threadnote 4 replaces the OpenViking/Python/Node.js runtime chain with a self-contained Bun distribution, moves
canonical product state under ~/.threadnote, and adds a zero-plugin Obsidian bridge. Users install one Threadnote
distribution and do not need Node.js, npm, Bun, Python, OpenViking, a database server, or a background daemon.

• own canonical Markdown, indexes, model files, locks, logs, migration receipts, and sharing metadata under
  ~/.threadnote
• run CLI, MCP, Manager, storage, recall, local inference, sharing, migration, installation, and updates through one
  embedded Bun application
• keep one root Effect runtime and scope per process, with Effect-owned filesystem, SQLite, HTTP, process, locking, and
  lifecycle boundaries
• store lexical data in SQLite and immutable packed vector generations under ~/.threadnote/indexes
• automatically provision, verify, select, and preserve the pinned BGE Small embedding model for in-process
  node-llama-cpp inference
• combine lexical, semantic, field, scope, lifecycle, authority, graph, temporal, and feedback signals in explainable
  hybrid recall
• add frozen recall baselines, quality gates, model bake-offs, microbenchmarks, and SQLite/vector regression coverage
• harden installation, upgrades, repair, migration receipts, command shims, MCP/instruction recovery, cross-process
  locks, Windows support, and large-repository traversal
• capture privacy-safe rotating CLI/MCP diagnostics and create explicitly approved GitHub support reports without transcripts
• add allowlisted Obsidian vault recall and explicitly selected Threadnote memory projections
• document the implemented architecture, migration, release, signing, rollback, and Obsidian contracts

Why

Threadnote 3 depended on a separately managed OpenViking/Python service and npm/Node.js installation. Installation,
upgrades, model preservation, recovery, and cross-platform behavior therefore depended on several external runtimes
and background state.

Threadnote 4 makes canonical data and all normal recall/write paths product-owned and local. Derived indexes can be
discarded and rebuilt without risking canonical memories. The installed application embeds Bun and runs inference in
process; users do not install or manage an application runtime.

Bun runtime and distribution contract

• Bun 1.3.14 is pinned for builds and embedded into every release executable.
• Bun bytecode compilation, minification, and linked source maps are enabled for the standalone entrypoint.
• src/standalone.ts owns the single BunRuntime.runMain process boundary. Application services compose Effects
  without nested runtimes.
• Runtime adapters use @effect/platform-bun and @effect/sql-sqlite-bun; SQLite WAL files remain disposable derived
  state.
• CLI and MCP use the same executable. Compatibility launchers select normal CLI or mcp-server mode.
• The Manager runs a foreground, loopback-only HTTP session. No persistent daemon is installed.
• node-llama-cpp is isolated behind one adapter and may load only the compatible prebuilt native payload shipped
  beside Threadnote. It never silently compiles llama.cpp or invokes Python.
• CI bytecode-compiles all eight accepted Bun base targets: macOS arm64/x64, Linux arm64/x64 glibc and musl, and
  Windows arm64/x64.
• The current beta publishes macOS and glibc Linux archives for arm64 and x64. Musl remains a compile gate, and Windows
  publication remains disabled until Authenticode signing is approved and verified.

Runtime and storage contract

• ~/.threadnote is the sole product-owned home.
• Canonical Markdown and stable threadnote:// URIs remain authoritative.
• SQLite lexical indexes and packed vector generations are derived state and can be verified, repaired, or rebuilt.
• Install and repair automatically download and checksum the 36.7 MB core embedding model and build indexes with
  progress feedback.
• Recall fails open to deterministic lexical search when native inference is temporarily unavailable; doctor
  reports the degraded capability.
• Install/update mutations use a shared lock, side-by-side version directories, atomic launcher promotion, rollback
  preservation, and process-identity leases so running releases are not pruned.
• The archive extractor validates checksums and paths, streams bounded gzip/tar input, and rejects malformed,
  truncated, escaping, or oversized payloads before activation.

Production diagnostics and issue reporting

• write owner-only JSONL under ~/.threadnote/logs, rotating a 1 MiB active file through five retained generations
• coordinate concurrent agents with reference-counted Effect semaphores and bounded cross-process leases
• reject unsafe filesystem entries, verify file identity before append/read, and never follow active or rotated log symlinks
• record only version, runtime, platform, command or MCP tool name, duration, outcome, and typed failure name
• exclude command arguments, environment values, memory content, queries/results, MCP payloads, exception messages, usernames, and local paths
• keep logging best-effort; MCP lifecycle writes have a short fail-open latency bound and cannot fail or rerun tool effects
• expose threadnote logs for locating and reviewing retained support files
• make threadnote report-issue preview-only by default, with credential blocking, path redaction, terminal-control rejection, strict log-schema parsing, and bounded newest-entry excerpts
• bind submission to the exact reviewed title/body/diagnostics using a SHA-256 approval digest
• submit only through gh api --hostname github.com, passing public bytes in an owner-only temporary JSON file rather than process arguments
• explain platform-appropriate GitHub CLI installation and gh auth login without installing or authenticating automatically
• instruct agents to request explicit approval for issue creation and for including privacy-safe logs

Obsidian bridge

The Obsidian integration is deliberately asymmetric so a vault can participate in recall without replacing
Threadnote's authority and lifecycle model.

Vault → Threadnote recall

• configure one or more vault sources with explicit include/exclude boundaries; no Obsidian plugin is required
• automatically refresh every enabled source before both CLI recall and MCP recall_context
• index sanitized Markdown snapshots under threadnote://resources/external/obsidian/...
• preserve authority: external and trust: untrusted regardless of note frontmatter
• warn and continue against the last successful snapshot when a source refresh fails
• reject boundary escapes, symlinks, likely credentials, and local path leaks before committing the sanitized copy
• keep manual inventory, preview, status, and applied sync commands for inspection and troubleshooting

Threadnote → vault projection

• configure a generated Obsidian folder and publish only explicitly selected canonical memory URIs
• never export the whole memory corpus merely because a projection is connected
• generate deterministic Markdown, stable identity/lifecycle metadata, relation links, source hashes, and Obsidian
  Bases
• secret-scan rendered output before writing
• preserve unmanaged files and user-edited generated notes; report drift unless regeneration is explicitly forced
• support shared memories subject to projection policy and opening projected memories through Obsidian CLI/URI
  navigation

Vault Inbox → candidate review

• scan only explicitly marked direct Markdown children of a configured Inbox
• form review candidates for decisions, preferences, invariants, and handoff state
• use the existing duplicate/contradiction/replacement comparison workflow
• require agent/user review before any candidate becomes durable memory

Source indexing never creates durable memories, and editing a generated projection never updates Threadnote.
Threadnote remains authoritative for identity, provenance, trust, lifecycle, recall, and sharing.

Migration and compatibility

• existing 3.x content is migrated non-destructively through staged copies, hash validation, atomic promotion, and
  validated receipts
• the legacy ~/.openviking source is preserved unchanged for rollback and completed migrations are not repeatedly
  advertised
• beta recovery merges into a previously created 4.0 target without overwriting different canonical content
• generated update-check.json state plus transient share FETCH_HEAD, COMMIT_EDITMSG, gc.log, and Git-generated lockfiles cannot block migration
• the current managed-share Git index and its staging state take precedence during recovery; the legacy index remains untouched as rollback evidence
• unpublished local share branch refs, Git objects, configuration, and worktree content remain preserved
• the installer removes only verified package-managed Threadnote/OpenViking tools and stale npm launchers; it does not
  remove legacy memories
• Windows installers, path handling, seeding, redaction, and local-binary workflows have dedicated coverage
• no Obsidian-specific post-update migration is required because source indexes are derived and projections have
  compatibility fallbacks
• prototype all-matching projections remain in legacy mode until explicitly reconfigured for selected-memory
  publishing

Release and update process

Threadnote 4 is distributed through immutable GitHub Releases, not npm.

1. Bump package.json and add .github/release-notes/vX.Y.Z.md; the file must begin with ## What's new.
2. Ensure source CI, recall-quality gates, all-target bytecode compilation, and self-contained E2E checks are green.
3. Optionally run Publish standalone release manually against a selected branch to build, sign, and notarize both
   macOS targets without publishing.
4. Push the exact matching vX.Y.Z tag. The tag-triggered workflow rejects version or release-note mismatches before
   building.
5. Every enabled release runner builds the final payload and produces a real embedding before archiving.
6. macOS signs every nested Mach-O payload and the Bun executable with hardened runtime, submits the exact payload to
   Apple notarization, and waits for acceptance. Linux uses the immutable GitHub release plus SHA-256 checksums as its
   trust root.
7. The workflow creates the prerelease only after all four enabled archives and adjacent checksum files are ready,
   prepends the curated notes to GitHub's generated changelog, and verifies the release is immutable.

Pull-request and ordinary CI builds stay unsigned. Windows build/signing jobs remain present but hard-disabled until
an Authenticode provider and bundled-native-code policy are approved.

There is intentionally no npm transition update from Threadnote 3 to 4 because that boundary cannot be tested safely.
Existing v3 users install v4 fresh; the installer preserves ~/.openviking for migration and rollback. Once v4 is
installed, threadnote update and threadnote update --beta use immutable GitHub Releases with checksum verification,
side-by-side installation, and atomic activation.

Root causes addressed

• monolithic JSON lexical indexes caused excessive memory and file-size growth
• legacy post-update bookkeeping could advertise already completed migrations
• stale global npm launchers could point at packages removed from another Node installation
• unbounded traversal and post-filter ignore handling could exhaust memory on large monorepos
• Windows path redaction could mistake Git Bash and WSL examples for macOS home paths
• cross-process share and resource operations could wait on stale or contended locks without bounded recovery
• CLI and MCP recall could diverge by bypassing the same ranking/runtime path
• automatic memory creation needed reviewable candidates rather than silent durable writes

Validation

• Prettier, oxlint, TypeScript, diff checks, self-contained source scan, and production builds
• 85 Vitest files and 739 tests
• frozen recall quality/non-regression gates, model evaluation artifacts, and SQLite/vector benchmarks
• all eight Bun target compile gates
• native self-contained E2E on Ubuntu, macOS, and Windows, including CLI, MCP, local embeddings, Obsidian, sharing,
  installers, concurrent production logging, and digest-approved support reporting; local packaged E2E currently passes
  17 tests with one platform-specific skip
• release runners produce a real embedding from the exact payload before signing or archiving
• signed and notarized macOS arm64/x64 release verification

Release status

v4.0.0-beta.13 is published as an immutable Apple/Linux prerelease. It contains signed/notarized macOS arm64/x64 and Linux arm64/x64 archives with adjacent SHA-256 files. Beta.13 completes interrupted local-share recovery by preserving the current managed-share Git index and staging state while recovering disjoint legacy memories and unpublished Git objects. It still fails closed on conflicting refs, configuration, reflogs, operation state, and other durable Git content, and leaves the legacy rollback source untouched. PR source CI and the complete release workflow passed, and the release tag tree exactly matches this branch.

Windows artifacts are not published yet; Windows remains a compile and self-contained E2E gate until Authenticode is
ready.