Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -253,3 +253,6 @@ data/memory.db*
# demo stack supervisor: pids, git sha and source fingerprints of the
# processes this checkout started. Machine-local by definition.
.demo/

# written beside taxonomy.py when scripts/dev/tests runs; machine-local
scripts/dev/tests/per_test.json
7 changes: 5 additions & 2 deletions CALENDAR_QUERY_LOCATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@
> access it uses instead. `admonisher/base.py` and `admonisher/calendar.py`
> have no caller in `src/` or `scripts/`; their tests were removed in the
> 2026-09 test-suite prune (`tests/README.md`, "What's out of `tests/unit/`").
> Everything below is left as written, as a record of what was true then —
> it is not a guide to the current architecture.
> `McpCalendarClient` (`fateforger.agents.timeboxing.mcp_clients`) is also
> gone now — it left with the legacy timeboxing agent on 2026-09-09;
> `src/tmbx/calendar/` (`port.py`, `gcal.py`, `fake.py`) is the calendar port
> on the surviving path. Everything below is left as written, as a record of
> what was true then — it is not a guide to the current architecture.

## 🎯 Summary

Expand Down
8 changes: 6 additions & 2 deletions MIGRATION_ARCHIVE_TO_CALENDAR_HAUNTER.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,12 @@
> `src/fateforger/haunt/reconcile.py` and `src/fateforger/haunt/service.py`
> are where calendar reconciliation and reminder orchestration live now. The
> `CalendarHaunter` tests were removed in the 2026-09 test-suite prune
> (`tests/README.md`, "What's out of `tests/unit/`"). Everything below is
> left as written, as a record of what was true then.
> (`tests/README.md`, "What's out of `tests/unit/`"). `McpCalendarClient`
> (`fateforger.agents.timeboxing.mcp_clients`) is also gone now — it left
> with the legacy timeboxing agent on 2026-09-09; `src/tmbx/calendar/`
> (`port.py`, `gcal.py`, `fake.py`) is the calendar port on the surviving
> path. Everything below is left as written, as a record of what was true
> then.

If you have code using the old `CalendarMcpClient` from the archive, here's how to migrate to the mature `CalendarHaunter` implementation.

Expand Down
91 changes: 40 additions & 51 deletions docs/architecture/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,66 +2,55 @@
title: Agents
---

## TimeboxingFlowAgent
> **Retired (2026-09-09).** `TimeboxingFlowAgent` (`agents/timeboxing/agent.py`),
> `ConstraintRetriever` (`agents/timeboxing/constraint_retriever.py`), and the
> Notion-backed `ConstraintExtractorAgent` are deleted with the coordinator
> that owned them — `refactor: retire TimeboxingFlowAgent and the 34 modules
> only it reached` (commit `67489cd`). The sections below described those
> three; see `docs/indices/agents_timeboxing.md` for the file-by-file
> retirement note. What follows is the three components that actually plan
> and write a day now.

## Adaptive timeboxing kernel

Artifact-led planning-session orchestration: the Stage 1 elicitation loop
(coverage matrix, arithmetic gate, three judges) plus the newer
artifact-led session state (`session_contracts.py`, `readiness.py`,
`required_blocks.py`, `day_frame.py`, `feedback.py`). The kernel decides
what a planning turn does but takes timezone, calendar, and constraint-store
access as ports; it is driven from the Slack host, which supplies those
ports and does the actual Slack routing.

Primary day-planning agent that runs the GraphFlow timeboxing workflow and coordinates:
- Stage-gated planning for day schedule drafts (typed JSON contexts per stage)
- Patch-based refinement (`TimeboxPatcher`)
- Constraint extraction + persistence (background, non-blocking)

Code: `src/fateforger/agents/timeboxing/agent.py`
Code:
- `src/fateforger/agents/timeboxing/adaptive_timeboxing.py`
- `src/fateforger/slack_bot/timeboxing_host.py` (the host that supplies the kernel's ports and calls it per Stage 1 turn)

Related docs:

- `docs/indices/agents_timeboxing.md`
- `docs/architecture/timeboxing_refactor.md`
- `docs/architecture/constraint-flow.md`
- `docs/architecture/proposal_object_contract.md`
- `docs/superpowers/specs/2026-09-05-stage1-elicitation-loop-design.md`

## ConstraintExtractorAgent (Notion-backed)
## Harness planner (DeepSeek)

Extractor agent that turns user preference corrections into a deterministic constraint record and
upserts it into Notion for durable future reuse.

- Output schema: `ConstraintExtractionOutput` (JSON, structured)
- Persistence: `NotionConstraintStore.upsert_constraint(...)` + `TB Constraint Events` audit log
- Timeboxing agents can call the tool `extract_and_upsert_constraint` (Agent-as-Tool under the hood).
- Notion access is via the constraint-memory MCP server (`scripts/constraint_mcp_server.py`).
Host-owned context boundary for adaptive planning turns: refreshes the
constraint and calendar read models for the locked day and hands one
complete brief to a fresh harness run. Reads durable constraints via
`kg_constraint_client.py`, the read-only client onto the standalone memory
server's own store (`data/memory.db`), speaking the `DurableConstraintStore`
protocol `durable_constraint_store.py` defines.

Code:
- `src/fateforger/agents/timeboxing/notion_constraint_extractor.py`
- `src/fateforger/adapters/notion/timeboxing_preferences.py`

## ConstraintRetriever
- `src/fateforger/slack_bot/deepseek_timebox_planner.py`
- `src/fateforger/agents/timeboxing/kg_constraint_client.py`
- `src/fateforger/agents/timeboxing/durable_constraint_store.py`

Gap-driven retriever for durable constraints that:
- derives a small query plan from stage + day context (gaps/blocks/immovables)
- uses `constraint_query_types` to select relevant `type_id`s
- then queries constraints via `constraint_query_constraints` with those `type_id`s
## tmbx server (calendar writes)

Code:
- `src/fateforger/agents/timeboxing/constraint_retriever.py`
- `src/fateforger/agents/timeboxing/mcp_clients.py`
- `src/fateforger/agents/timeboxing/agent.py`
MCP server exposing the level 1 timebox tools (`plan_read`, `plan_apply`,
`plan_commit`, `plan_undo`, `plan_history`) that read and write the day's
Google Calendar events. A write path can refuse (reported as a normal JSON
result with a `"reason"` code, never raised as an exception) rather than
silently applying a stale or conflicting patch.

## (Next) ConstraintRetriever Improvements
Code: `src/tmbx/server.py`

Planned improvements:
- loads global/profile constraints first (high precedence)
- then queries only what is needed for remaining planning gaps ("degrees of freedom")
- uses structured Notion properties (no embeddings requirement)

Status: partially implemented; tracked in `lattice_ticket.md`.

## SlackBot Router + Review

Slack-facing routing + constraint review UI:
- Extracted constraints can be reviewed and accepted/declined via a Slack modal.
- Current implementation updates the local SQLite-backed constraint statuses.
- Proposal interactions should follow the shared contract in
`docs/architecture/proposal_object_contract.md`:
UI actions and NL replies must converge to the same typed intent + submit path.

Code:
- `src/fateforger/slack_bot/handlers.py`
- `src/fateforger/slack_bot/constraint_review.py`
Related docs: `src/tmbx/` module docstrings; `tickets/` entries under `tmbx`.
8 changes: 8 additions & 0 deletions docs/architecture/timeboxing_refactor.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ title: Timeboxing Refactor

# Timeboxing Refactor

> **Superseded (2026-09-09).** This page describes `TimeboxingFlowAgent`'s
> coordinator + stage-gating design, retired with the agent itself (commit
> `67489cd`); every code pointer below except `mcp_clients.py` names a
> deleted file. Of the tests listed below only
> `tests/e2e/test_slack_timebox_command.py` survives. See
> `docs/architecture/agents.md` for the live components and
> `docs/indices/agents_timeboxing.md` for the retirement note and file index.

This page summarizes the “prompt-splitting + typed stage contexts + background constraints” refactor for timeboxing.

For the detailed repo-level report, see `TIMEBOXING_REFACTOR_REPORT.md`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,21 +166,19 @@ in the spike: **51 source files, 17,584 lines**, in four groups.
`scheduler_prefetch_capability`, `task_marshalling_capability`,
`tool_result_presenter`, `toon_views`, `llm/toon`, `shared/handoff_policy`).
- **Found by the stores spike, not by reachability:**
- `mcp_clients.py` whole. `ConstraintMemoryClient` is imported by
`tasks/defaults_memory.py` but never constructed (`TASKS_DEFAULTS_MEMORY_BACKEND=disabled`
returns first); `McpCalendarClient`'s only importer is `agent.py`. Cut the
`defaults_memory.py:21` import with it.
- `McpCalendarClient` and `CalendarDaySnapshot` leave `mcp_clients.py`;
`ConstraintMemoryClient` stays — it is the default
`tasks_defaults_memory_backend` with tests of its own.
- `preferences.ConstraintStore`, `ensure_constraint_schema`,
`handlers._update_constraints` / `_maybe_update_timeboxing_thread_constraints`,
and the `constraint_review.py` handler set. Their only writers are the
legacy agent and a review modal only legacy posts; after the cut the
table is permanently empty and the harness-side reads are no-ops. Keep
the `Constraint` / `ConstraintStatus` / `ConstraintScope` types —
`messages.py` types against them.
- `settings.timeboxing_memory_backend` and its validator (`config.py:223,
300-309`): read only by `agent.py:1089`. The harness hardcodes
`KGConstraintMemoryClient(settings.memory_db_path)`. Remove the setting
and the `TIMEBOXING_MEMORY_BACKEND` line from `.env.template`.
- `settings.timeboxing_memory_backend`: Kept. It is also read by
`runtime.py`'s graphiti startup checks and by tasks' defaults memory;
only `agent.py`'s branch on it goes.
- The already-dead set PR #396 flagged: `admonisher/{base,calendar,commitment}`,
`schedular/diffing_agent`, `timeboxing/{flow,prompts,state,notebook_entrypoints}`,
`slack_bot/{relay_agent,topics}`, `tools_config/`. **Not deletable as
Expand Down Expand Up @@ -364,8 +362,11 @@ The rule from #396 stands: a double used by two modules lives here.
The draft proposed a `workbench=` keyword on four constructors. The spike
implemented it, mutation-tested it, and found:

- `ConstraintMemoryClient` and `McpCalendarClient` — the two where the seam
would have earned its keep — **are deleted in project 1**.
- `McpCalendarClient` — one of the two where the seam would have earned its
keep — **is deleted in project 1**.
- `ConstraintMemoryClient` — the other one — survives; its `workbench=` seam
is the one the spike found earns its keep (0.5s → 17.9s without it) —
project 2 adds it.
- `TickTickMcpClient` and `NotionMcpClient` **have no workbench**. Their
constructor is already pure (URL validation is parsing; the network probe
is in `probe()`, which every test monkeypatches). Their three `__new__`
Expand Down Expand Up @@ -395,7 +396,9 @@ found five times in one converted file and which a rule scoped to production
classes would miss — a double's state belongs in its `__init__`. An allowlist
exists for the legitimate case, each entry naming why; `TBPlan.__new__` is
the first entry. Lands in project 1's PR with whatever allowlist the cut
leaves, then shrinks.
leaves, then shrinks. Measured starting point (project 1's guard, per-file
rather than per-line): 34 files, 149 offences (24 `__new__`, 125 private
writes).

### `tests/contracts/`

Expand Down Expand Up @@ -423,7 +426,9 @@ directory.

One PR, after project 1 merges: the builders and the conversion of the
literal-heavy files, the four gratuitous `__new__` conversions, the
`tests/contracts/` moves, and the allowlist shrunk to `TBPlan`.
`tests/contracts/` moves, and the allowlist shrunk from its measured
starting point (34 files, 149 offences) toward `TBPlan`, the one entry the
table above says stays legitimate.

### Gate

Expand Down
101 changes: 101 additions & 0 deletions scripts/dev/tests/covdiff.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
"""Per-line coverage diff between two `coverage json` reports.

Usage:
python covdiff.py <before.json> <after.json> <source-root>

Run from the directory holding the two report files -- paths are taken
literally (relative or absolute), so a basename resolves against the
caller's cwd. The reports themselves store source paths relative to
wherever pytest ran (typically the repo root), which is rarely the same
place as the two report files when this is run as a one-off diagnostic
from a scratch directory -- so the required third argument names the
root those paths resolve against for the on-disk existence check. There
is no cwd-based default: a report's paths resolving against the wrong
root doesn't fail, it silently resolves every path to "does not exist"
and misfiles every regression under "deleted" instead of "surviving" --
the exact way this script was once run wrong. To guard against a root
that is merely a *different* wrong directory (also silent), refuse to
run when fewer than half the before-report's paths exist under it.

For every file the *before* report measured, this compares the set of
covered (`executed_lines`) line numbers against the same file in the
*after* report. A file that dropped out of the after-report entirely
counts as having lost every line it had covered. The verdict that
matters is whether the file still exists on disk: a deleted file losing
its covered lines is expected (nothing new can exercise code that is
gone); a *surviving* file losing covered lines is a regression -- some
line that used to run is no longer reached by any test.

A surviving file's line diff can also be a false positive if that file
was itself edited between the two coverage captures: an edit shifts
every line after it, so a line merely moved reads as a line lost. This
script cannot tell that apart from a genuine regression -- check `git
diff` on any surviving file this reports before trusting the number.
"""

from __future__ import annotations

import json
import pathlib
import sys


def _executed(report: dict, path: str) -> set[int]:
entry = report.get("files", {}).get(path)
if not entry:
return set()
return set(entry.get("executed_lines", []))


def main() -> None:
if len(sys.argv) != 4:
print(
f"usage: {sys.argv[0]} <before.json> <after.json> <source-root>",
file=sys.stderr,
)
raise SystemExit(2)

before = json.loads(pathlib.Path(sys.argv[1]).read_text())
after = json.loads(pathlib.Path(sys.argv[2]).read_text())
root = pathlib.Path(sys.argv[3])

before_paths = sorted(before.get("files", {}))
if before_paths:
resolved = sum(1 for p in before_paths if (root / p).exists())
if resolved < len(before_paths) / 2:
print(
f"refusing: only {resolved}/{len(before_paths)} of the before-report's "
f"paths exist under source-root {root} -- wrong root, every regression "
"would be misfiled as 'deleted'",
file=sys.stderr,
)
raise SystemExit(1)

deleted: list[tuple[str, list[int]]] = []
surviving: list[tuple[str, list[int]]] = []

for path in before_paths:
lost = sorted(_executed(before, path) - _executed(after, path))
if not lost:
continue
target = (deleted if not (root / path).exists() else surviving)
target.append((path, lost))

print(f"before: {len(before.get('files', {}))} files measured")
print(f"after: {len(after.get('files', {}))} files measured")
print()
print(f"=== deleted files that lost covered lines ({len(deleted)}) ===")
for path, lost in deleted:
print(f" {path} (-{len(lost)} lines)")
print()
print(f"=== SURVIVING files that lost covered lines ({len(surviving)}) ===")
if not surviving:
print(" none")
for path, lost in surviving:
shown = ", ".join(str(n) for n in lost[:20])
more = f" (+{len(lost) - 20} more)" if len(lost) > 20 else ""
print(f" {path}:{shown}{more}")


if __name__ == "__main__":
main()
62 changes: 62 additions & 0 deletions scripts/dev/tests/dangling.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
"""Every surviving src file that still imports a module I deleted.

Usage:
python dangling.py # deleted files staged (--cached)
python dangling.py 42d9eb2..HEAD # deleted files over a revision range
"""
import ast, pathlib, subprocess, sys

if len(sys.argv) > 1:
diff_cmd = ["git", "diff", "--diff-filter=D", "--name-only", sys.argv[1]]
else:
diff_cmd = ["git", "diff", "--cached", "--name-only", "--diff-filter=D"]

deleted_files = subprocess.run(
diff_cmd, capture_output=True, text=True).stdout.split()
gone = set()
for f in deleted_files:
if not f.startswith("src/") or not f.endswith(".py"):
continue
parts = f[len("src/"):].removesuffix(".py").split("/")
if parts[-1] == "__init__":
parts = parts[:-1]
gone.add(".".join(parts))

alive = [p for p in pathlib.Path("src").rglob("*.py") if "__pycache__" not in str(p)]

def selfmod(p):
parts = list(p.relative_to("src").with_suffix("").parts)
if parts[-1] == "__init__":
parts = parts[:-1]
return ".".join(parts)

hits = []
for p in alive:
me = selfmod(p)
ispkg = p.name == "__init__.py"
base = me if ispkg else (me.rsplit(".", 1)[0] if "." in me else me)
try:
tree = ast.parse(p.read_text())
except SyntaxError:
continue
for n in ast.walk(tree):
targets = []
if isinstance(n, ast.Import):
targets = [a.name for a in n.names]
elif isinstance(n, ast.ImportFrom):
if n.level:
parts = base.split(".")
up = parts[: len(parts) - (n.level - 1)] if n.level > 1 else parts
m = ".".join(up + ([n.module] if n.module else []))
else:
m = n.module or ""
targets = [m] + [m + "." + a.name for a in n.names]
for t in targets:
if t in gone:
hits.append((str(p), n.lineno, t))
break

print(f"deleted modules: {len(gone)}")
print(f"surviving src files still importing one: {len(set(h[0] for h in hits))}\n")
for f, line, t in sorted(hits):
print(f" {f}:{line} -> {t}")
Loading