feat(types): pyright zero-diagnostic gate — 568-diagnostic ratchet burned to zero, warnings_strict flips to Met (#197, final)

.bestpractices.json

@@ -99,7 +99,7 @@
   "build_floss_tools_justification": "pip, uv, hatchling, pytest, ruff and pyright are all FLOSS, and the build runs on ubuntu-latest and windows-latest GitHub runners.",
 
   "test_status": "Met",
-  "test_justification": "An automated test suite of 6275 tests lives under tests_py/ (measured 2026-07-28 with 'pytest --collect-only -q'), covering core, handlers, infrastructure, integration, invariants and architecture: https://github.com/cdeust/Cortex/tree/main/tests_py",
+  "test_justification": "An automated test suite of 6297 tests lives under tests_py/ (measured 2026-07-28 with 'pytest --collect-only -q'), covering core, handlers, infrastructure, integration, invariants and architecture: https://github.com/cdeust/Cortex/tree/main/tests_py",
 
   "test_invocation_status": "Met",
   "test_invocation_justification": "The whole suite runs with a single 'pytest' command, documented in CONTRIBUTING.md under Testing along with per-layer subsets: https://github.com/cdeust/Cortex/blob/main/CONTRIBUTING.md",
@@ -114,7 +114,7 @@
   "test_policy_justification": "CONTRIBUTING.md carries an explicit, mandatory testing policy: new functionality ships with tests in the automated suite, a bug fix carries a regression test that fails on the pre-fix code, and each failure path asserts its observable effect including the signal it emits \u2014 https://github.com/cdeust/Cortex/blob/main/CONTRIBUTING.md#the-testing-policy-mandatory. The per-tool checklist ('Add a unit test', 'Add an integration test if the tool touches the database') and the five-element mechanism checklist restate it per change type.",
 
   "tests_are_added_status": "Met",
-  "tests_are_added_justification": "New functionality ships with its tests; the suite has grown to 6275 tests alongside the v4.x feature series, and CI runs it on every pull request.",
+  "tests_are_added_justification": "New functionality ships with its tests; the suite has grown to 6297 tests alongside the v4.x feature series, and CI runs it on every pull request.",
 
   "tests_documented_added_status": "Met",
   "tests_documented_added_justification": "The requirement is written into the documented instructions for change proposals: https://github.com/cdeust/Cortex/blob/main/CONTRIBUTING.md#the-testing-policy-mandatory states that every change adding or altering observable behaviour must arrive with tests in the same PR, the per-tool and per-mechanism checklists repeat it as a concrete step, and .github/PULL_REQUEST_TEMPLATE.md requires a Test plan section in every pull request.",
@@ -125,8 +125,8 @@
   "warnings_fixed_status": "Met",
   "warnings_fixed_justification": "The lint job fails the build on any ruff finding, so warnings cannot accumulate on main; the typecheck job enforces a ratchet that fails if the pyright diagnostic count rises above its floor.",
 
-  "warnings_strict_status": "Unmet",
-  "warnings_strict_justification": "Not yet maximally strict, and reported as such. Ruff runs its default rule set rather than a broad selection (a run with `--select ALL` minus docstring/annotation/comma rules reports 3,545 findings, measured 2026-07-27), and pyright runs typeCheckingMode `basic` behind a per-rule ratchet whose committed floors still total 568 diagnostics. Both gates do block regression today \u2014 CI fails on any ruff finding and on any rule exceeding its floor \u2014 so warnings cannot accumulate, but the strictness level itself is not maximal. Tightening it rule by rule is tracked with acceptance criteria in https://github.com/cdeust/Cortex/issues/197 and is item 1 on the roadmap.",
+  "warnings_strict_status": "Met",
+  "warnings_strict_justification": "Maximally strict, by configuration rather than promise. Ruff runs an explicit broadened rule set \u2014 select = [E4, E7, E9, F, S110, BLE001, PLR2004, E501, PLC0415, S608] (pyproject.toml [tool.ruff.lint]) \u2014 with every production finding fixed or carrying a per-site noqa naming its mechanism, and CI fails on any finding. Pyright (pinned 1.1.410) runs typeCheckingMode `standard` over mcp_server/ with ZERO diagnostics (568-diagnostic ratchet backlog burned to zero, measured 2026-07-28, issue #197); the per-rule ratchet is retired and CI fails on ANY pyright diagnostic via its exit code. `strict` mode is not practical: it reports 10,231 errors, ~9,300 of them the Unknown-type annotation-coverage family (measured 2026-07-27). See .github/workflows/ci.yml jobs `lint` and `typecheck`.",
 
   "know_secure_design_status": "Met",
   "know_secure_design_justification": "The design is local-first with an explicitly documented trust boundary: PRIVACY.md states what is processed, where it is stored, and precisely what leaves the machine (model downloads only, opt-in OTLP), and SECURITY.md documents what Cortex accesses and its supply-chain assurance: https://github.com/cdeust/Cortex/blob/main/PRIVACY.md",
@@ -189,7 +189,7 @@
   "static_analysis_often_justification": "CodeQL default setup analyses each push and pull request and additionally runs on a weekly schedule, so analysis happens per change rather than per release.",
 
   "dynamic_analysis_status": "Unmet",
-  "dynamic_analysis_justification": "No dynamic analysis tool in the badge's sense (fuzzer, sanitizer, or scanner) is applied. The project runs a 6275-test suite with coverage on every change, but a test suite is not a dynamic analysis tool and is not claimed as one here.",
+  "dynamic_analysis_justification": "No dynamic analysis tool in the badge's sense (fuzzer, sanitizer, or scanner) is applied. The project runs a 6297-test suite with coverage on every change, but a test suite is not a dynamic analysis tool and is not claimed as one here.",
 
   "dynamic_analysis_unsafe_status": "N/A",
   "dynamic_analysis_unsafe_justification": "Cortex is written in Python, a memory-safe language, so the memory-safety tooling this criterion asks about (ASan, Valgrind) does not apply.",
@@ -261,7 +261,7 @@
   "coding_standards_justification": "https://github.com/cdeust/Cortex/blob/main/CONTRIBUTING.md#coding-standards-excerpt names the standard for the primary language (Python): ruff formatting and linting as the style baseline, plus the project's own load-bearing rules (no `Any` in production code, sourced constants, no bare `except`, file and function size limits), with the full text at https://github.com/cdeust/zetetic-team-subagents/blob/main/rules/coding-standards.md. Compliance is required of contributions, not suggested.",
 
   "coding_standards_enforced_status": "Met",
-  "coding_standards_enforced_justification": "Automatically enforced in CI on every push and pull request: `ruff format --check .` and `ruff check .` with ruff pinned at 0.15.20, plus a pyright per-rule ratchet (scripts/check_pyright_ratchet.py) that fails the build if any rule's diagnostic count rises above its committed floor. Style exceptions are per-line `# noqa` with the rule code named at the site. See .github/workflows/ci.yml (jobs `lint` and `typecheck`).",
+  "coding_standards_enforced_justification": "Automatically enforced in CI on every push and pull request: `ruff format --check .` and `ruff check .` with ruff pinned at 0.15.20, plus pyright (pinned 1.1.410, typeCheckingMode `standard`) failing the build on any diagnostic over mcp_server/ (zero-diagnostic tree since issue #197, 2026-07-28). Style exceptions are per-line `# noqa` with the rule code named at the site. See .github/workflows/ci.yml (jobs `lint` and `typecheck`).",
 
   "build_standard_variables_status": "N/A",
   "build_standard_variables_justification": "No native binaries are produced. Cortex is pure Python packaged with hatchling; there is no compiler or linker invocation for CC/CFLAGS/LDFLAGS to reach.",
@@ -297,7 +297,7 @@
   "interfaces_current_justification": "The stack targets currently supported runtimes and APIs: Python 3.10 through 3.13, all four in the CI matrix, with pgvector 0.3+/PostgreSQL 17 and current major versions of FastMCP, Pydantic v2, numpy and sentence-transformers. Deprecated interfaces are removed rather than wrapped \u2014 the standing rule is one-shot migrations with no back-compat shims (CLAUDE.md), and CI runs on the newest released Python so a deprecation surfaces as a warning in the build rather than as a surprise at end of life.",
 
   "automated_integration_testing_status": "Met",
-  "automated_integration_testing_justification": "The full suite runs on every push and pull request to main via .github/workflows/ci.yml and reports success or failure per job: 6275 tests on Python 3.10-3.13 against PostgreSQL + pgvector, the same suite against the SQLite backend, the suite on Windows, and a Docker smoke job that boots the bare container and exercises the DB-less contract. tests_py/integration/ holds the database-backed integration tests specifically.",
+  "automated_integration_testing_justification": "The full suite runs on every push and pull request to main via .github/workflows/ci.yml and reports success or failure per job: 6297 tests on Python 3.10-3.13 against PostgreSQL + pgvector, the same suite against the SQLite backend, the suite on Windows, and a Docker smoke job that boots the bare container and exercises the DB-less contract. tests_py/integration/ holds the database-backed integration tests specifically.",
 
   "regression_tests_added50_status": "Met",
   "regression_tests_added50_justification": "Measured on 2026-07-27 over the merged pull requests titled as fixes in the preceding six months (2026-01-27 onward): 23 of 30 \u2014 76.7% \u2014 changed the test tree in the same PR, against the 50% this criterion asks for. (The measure counts a PR as carrying tests when it touches tests_py/ or tests_js/, which is a proxy for 'added a regression test'; the policy behind it is written down at https://github.com/cdeust/Cortex/blob/main/CONTRIBUTING.md#the-testing-policy-mandatory \u2014 a bug fix carries a regression test that fails on the pre-fix code.)",

.github/workflows/ci.yml

@@ -381,30 +381,28 @@ jobs:
       # Pyright resolves third-party imports from ./.venv (pinned in
       # pyrightconfig.json: venvPath="."/venv=".venv"). The full stub set MUST
       # be installed or every import collapses to Unknown — and Unknown
-      # SUPPRESSES downstream type errors, making the baseline meaningless.
+      # SUPPRESSES downstream type errors, silently masking real defects.
       # source: measured 2026-06-18 — an unresolved env reports 566 errors, a
       # fully-resolved env reports 593 (Unknown was masking 27+ real errors).
-      # flashrank (core reranker) + sqlite-vec live outside dev/postgresql/codebase.
+      # flashrank (core reranker) + sqlite-vec live outside dev/postgresql/
+      # codebase; [otel] resolves the opentelemetry exporter imports.
       - name: Create .venv with the full type-check environment
         run: |
           python -m venv .venv
           .venv/bin/pip install -U pip
-          .venv/bin/pip install -e ".[dev,postgresql,sqlite,codebase]" flashrank
-          # Pin pyright — diagnostic output drifts between releases, so the
-          # committed baseline is only comparable against the pinned version.
+          .venv/bin/pip install -e ".[dev,postgresql,sqlite,codebase,otel]" flashrank
+          # Pin pyright — diagnostic output drifts between releases, so a
+          # zero-diagnostic tree is only comparable against the pinned version.
           .venv/bin/pip install pyright==1.1.410
 
-      # The ratchet IS the gate. pyright always exits non-zero while the
-      # 568-error backlog stands, so its own exit is ignored (|| true); the
-      # build fails only when a --blocking rule regresses past its committed
-      # baseline (0). Add rules to --blocking as each batch drives them to
-      # their floor — see docs/provenance/pyright-remediation-plan.md.
-      - name: Run pyright + per-rule ratchet gate
-        run: |
-          .venv/bin/python -m pyright --outputjson mcp_server/ > pyright-current.json || true
-          .venv/bin/python scripts/check_pyright_ratchet.py \
-            pyright-current.json typecheck-baseline.json \
-            --blocking reportOptionalMemberAccess reportOptionalSubscript
+      # pyright IS the gate: the backlog was burned to zero (issue #197,
+      # 568 baselined diagnostics -> 0 measured 2026-07-28), so ANY
+      # diagnostic — any rule, any severity-error — fails the build via
+      # pyright's own exit code. The former per-rule ratchet
+      # (scripts/check_pyright_ratchet.py + typecheck-baseline.json) is
+      # retired: a floor file only exists to tolerate a backlog.
+      - name: Run pyright (zero-diagnostic gate)
+        run: .venv/bin/python -m pyright mcp_server/
 
   build:
     name: Build Package

CHANGELOG.md

@@ -7,6 +7,7 @@ adheres to [Semantic Versioning](https://semver.org/).
 ## [Unreleased]
 
 ### Added
+- **Pyright is now a zero-diagnostic blocking gate** (#197, final family of the maximal-strictness program). The 568-diagnostic per-rule ratchet backlog was burned to **zero** at `typeCheckingMode: "standard"` (pyright 1.1.410, measured 2026-07-28): no rule disabled, no floor raised; the single per-site suppression is the unpublished optional `cortex_beam_abstain` import whose `except ImportError` arm is the documented degraded mode. The ratchet machinery (`typecheck-baseline.json` + `scripts/check_pyright_ratchet.py`) is retired — CI fails on ANY diagnostic via pyright's own exit code, and the CI type-check env installs the `[otel]` extra so the exporter imports resolve. The burn-down was fixes, not annotations-to-match: a typed host contract for the eight `PgMemoryStore` mixins (`pg_store_host.PgStoreHost` + `MaterializedCursor`, whose honest `DictRow` typing surfaced ten unchecked `INSERT..RETURNING` sites, now `one()` with a real error), a cross-backend `StoreConnection` union for the 16 shared query modules (the psycopg-only annotation had switched checking off for every SQLite call path), and **SQLite store parity for eight methods callers already used unconditionally** — `acquire_interactive`/`acquire_batch`, `_execute`, `search_newer_neighbors`, `update_forgetting_pressure_accum`, `get_memories_by_tag`, `iter_memories_for_decay`, `find_co_accessed_pairs` — each of which previously raised `AttributeError` on the SQLite backend and was swallowed into silent degradation by broad stage boundaries. Latent bugs fixed en route, each with a regression test: the compat cursor lacked `executemany` (SQLite wiki page-sources writes crashed), `lastrowid` honesty (insert paths now raise on a broken row-id contract instead of masking it with a stale `type: ignore`), the pipeline installer accepted a success result carrying no cargo path (None flowed into the build argv), `update_style_ema(None, None)` returned `None` against a `dict` signature, `encode_session` died with a bare `TypeError` on a direction-less feature (now refuses loudly, naming it), `get_causal_chain` could return `reason=None`, and active forgetting sent a `None` timestamp into the store. `.bestpractices.json` flips `warnings_strict` to **Met**, citing the ruff select list, the pyright mode, and the measured zero. Suite grows 6275 → 6297.
 - **ruff `PLC0415` (import-outside-top-level) and `S608` (string-built SQL) are now blocking lint gates** (#197, fourth rule family of the maximal-strictness program). All 520 production `PLC0415` findings (407 `mcp_server/`, 57 `benchmarks/`, 56 `scripts/`) were triaged one by one: **360 lazy imports moved to module top** — so the import graph is static and a broken module fails at boot, not mid-operation — and the 160 that remain each carry a per-site `# noqa: PLC0415 — <reason>` naming one of six sanctioned justifications: an optional dependency behind an extra, an internal module whose top-level closure hard-imports one (hoisting would break `[sqlite]`-only installs at import time), an ImportError-probe boundary where the except arm IS the degraded mode, an import cycle (partner named; the pre-existing #233 family), the hook latency boundary (per-event hook processes boot in ~0.05 s vs ~0.6 s for the registry closure, measured 2026-07-28 — hoisting the handler/store stack into a hook would multiply every hook event's cost), or a deferral the module itself documents. The hoist is behavior-preserving: the per-module import sweep matches the pre-change baseline exactly (515 modules, the same 6 pre-existing cycle failures), and warm import timings are unchanged. All 44 production `S608` sites carry a per-site `# noqa: S608 — <mechanism>` naming the exact reason the interpolation is safe (two-literal ternaries, generated placeholder lists, module-level `WHERE` literals, or allowlist-gated identifiers per `docs/ASSURANCE-CASE.md` §5), so any NEW string-built SQL fails CI until it states its mechanism. `tests_py/**` adds both rules to its written per-file ignore (function-level imports in tests are the fixture mechanism; SQL built in tests is fixture setup against a throwaway database).
 - **ruff `PLR2004` (magic-value comparison) and `E501` (line-too-long) are now blocking lint gates** (#197, third rule family of the maximal-strictness program). All 420 production `PLR2004` findings (339 `mcp_server/`, 57 `benchmarks/`, 14 `scripts/`, 10 `video/`) were fixed with **zero `# noqa: PLR2004`**: every compared literal became a named constant carrying a `# source:` comment — a real citation where the module documents one (Frey & Morris 1997 / Kandel 2001 / Tse 2007 cascade thresholds, RFC 9110 status bands, FIPS 180-4 digest lengths, issue-quoted gates), a structural rationale for arities (split-parts, tuple lengths), and an explicit `pre-existing tuned value, extracted unchanged; provenance not recorded at introduction` where none is discoverable — never an invented source. `tests_py/**` carries a written per-file ignore (the compared literal in an assertion IS the expected value under test — the spec itself). All 470 `E501` findings (263 `mcp_server/`, 121 `tests_py/`, 48 `benchmarks/`, 37 `scripts/`, 1 `_pipeline`) were fixed by **rewrapping at the unchanged 88-column formatter limit** — string content kept byte-identical via implicit concatenation at existing whitespace (SQL and regex literals machine-verified byte-for-byte) — with exactly two per-site `# noqa: E501 — <reason>` for unsplittable absolute-path tokens; E501 has **no** tests ignore. Two drift risks were closed at the source: `handlers/consolidation/transfer.py` re-declared as bare literals the canonical constants of `core/two_stage_transfer.py` (whose own comment forbids redefinition) and now imports them; `benchmarks/beam/ablation.py`'s copies are named per-module without value drift.
 - **ruff `BLE001` (blind-except) is now a blocking lint gate** (#197, second rule family of the maximal-strictness program). All 351 broad `except Exception` sites were triaged one by one, none blanket-ignored: (a) sites whose failure class is precisely known were **narrowed to typed excepts** — `json.loads` tag decoders to `ValueError`, lazy imports to `ImportError`, `subprocess` probes to `(OSError, SubprocessError)`, SQLite store guards to `sqlite3.Error`, PG connection/read guards to `psycopg.Error`, URL probes to `(OSError, ValueError, HTTPException)`, file I/O to `OSError` — so an unexpected programming error now **propagates instead of being absorbed** by a tolerant fallback; (b) genuine last-resort boundaries (degraded-mechanism wrappers, per-item batch isolation, hook/CLI entry points, diagnostic probes) stay broad and each carries a per-site `# noqa: BLE001 — <reason>` naming the signal it emits; (c) ~50 previously **silent** broad handlers now emit an observable signal — `silent_failure.note()` under 38 new stable component names (spreading-activation, wiki classifier user rules, candidate scans, memify reweight/derive, ingest tag lookups, prospective-trigger injection, source attribution, wiki pointer memories, AP-bridge/groomer config reads, …) or the hook log (`session_start` banner fetches, cached-graph lookups); (d) `mcp_client` connection failures re-raise with `from e`, preserving the causal chain. `tests_py/**` keeps a written per-file ignore (broad excepts in tests are deliberate teardown/optional-path handling). Every new signal is asserted by a test (54 added).