From 23cad1cdfab8522f6ce68d0b0838f1127256f151 Mon Sep 17 00:00:00 2001 From: hugocool Date: Tue, 8 Sep 2026 19:48:09 +0200 Subject: [PATCH 01/20] docs(referents): the referent catalog design, measured on two real incidents (#345) A message with no structural owner gets one judgement over the user's standing things, and delivery goes to the referent's own surface. The trigger: on 2026-09-05 13:51 "can you replan today so the gym is before dinner?" opened a fresh five-stage session for a day committed at 01:41, and the request text was discarded on the way in. `standing_for` had known the day was committed all morning; the door never asked it. The design is measured rather than argued. Frozen fixtures built from real `timeboxing_session_states` rows, 8 draws per case on the flash pin at `reasoning: minimal`, ~0.4s per call: - one call over all candidates beats per-candidate scoring, which collapses to 0/8 on most cases because a candidate judged alone has no contrast; - one prompt distinction carries most of the quality, 74/112 -> 95/112; - how the never-used fact is *carried* is inside the noise floor (~2/112), but dropping that row costs 8/8 -> 2/8 on "cancel that session", picking the wrong day; - the plan gist (block titles WITH times) is the largest single effect, 28/56 -> 53/56 on the two-parallel-sessions incident; - and it does not invent matches: four probes naming blocks in neither plan answer `none` 8/8 with the gist, against 0/8-6/8 without. A content-free descriptor is not the cautious option -- it fails both ways. Two design points are enforcement rather than prose. The resolver returns a referent and never an action, so resolve-then-act is a return type; and the catalog is a callable seam taking an owner and an explicit as-of, so #352's door and the eval are consumers alongside the routing rung. Boundary agreed directly with #352's owner (admonish-1-56 [6988fa]), whose blind review corrected two fixture labels and supplied the false-positive probe that inverted the risk it was meant to confirm. The spike rides along as the primary source; it carries its fixtures inline and reproduces without the ledger, which matters because the ledger moves under a run. Co-Authored-By: Claude Fable 5.1 --- .../2026-09-08-referent-catalog-design.md | 283 +++++++++ scripts/spikes/referent_resolver_spike.py | 542 ++++++++++++++++++ 2 files changed, 825 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-08-referent-catalog-design.md create mode 100644 scripts/spikes/referent_resolver_spike.py diff --git a/docs/superpowers/specs/2026-09-08-referent-catalog-design.md b/docs/superpowers/specs/2026-09-08-referent-catalog-design.md new file mode 100644 index 00000000..f79c0c60 --- /dev/null +++ b/docs/superpowers/specs/2026-09-08-referent-catalog-design.md @@ -0,0 +1,283 @@ +# The referent catalog — standing things as offered options + +**Ticket:** #345 (map #333, *One grammar for the seams*). **Consumer:** #352. +**Status:** design approved by Hugo 2026-09-06; written 2026-09-08. + +## The question + +A message arrives with no structural owner. Which of the user's *standing things* — if any — is +it about? + +Today nothing asks. A top-level message in `#plan-sessions` is hardcoded to mean "open a new +session", so on 2026-09-05 13:51 *"can you replan today so the gym is before dinner?"* opened a +fifth-stage session for a day committed at 01:41, and the request text was discarded on the way +in. The session store knew the day was committed. `standing_for` had answered that question all +morning for the nudger. The door never asked it. + +This is the handoff seam's first clause made concrete: *every route is a judgement*, and a +default is an offered option rather than a verdict. + +## Destination + +One catalog of standing things and one judgement over it, reachable from more than one door, +with the judgement returning **a referent and never an action**. + +Hugo's scoping ruling (2026-09-06): build the resolver and the provider interface now; the task +marshal joins later as a second provider against the same interface. *"While it gets implemented +we use it to inform the interfaces and components for the marshal — keep it compounding as much +as possible."* So an **interface-strain log** is a deliverable of this ticket, not a side effect +(see [Compounding](#compounding)). + +## Evidence + +All figures on the flash pin at `reasoning: minimal`, 8 draws per case, against frozen fixtures +built from real `timeboxing_session_states` rows. Spike: `scripts/spikes/referent_resolver_spike.py`. + +**Noise floor first.** Two runs of one identical arm scored 95/112 and 93/112. So ~2 draws in 112 +is noise on this fixture, and any comparison inside that band is not a finding. + +| finding | measurement | +|---|---| +| One call over all candidates beats per-candidate scoring | per-candidate: **0/8** on most cases — a candidate judged alone has no contrast and affirms nearly everything | +| Prompt wording carries most of the gain | 74/112 → **95/112** from one distinction (see [Prompt](#prompt)) | +| How the never-used fact is *carried* does not matter | prose 93, structured field 94, field+sentence 91 — all inside noise | +| Whether it is carried **does** | dropping the row: 88/112, and "cancel that session" collapses 8/8 → 2/8, picking the wrong day | +| The gist is the largest single effect | no gist 28/56 → gist **53/56** on the two-parallel-sessions fixture | +| The gist does not invent matches — it prevents false ambiguity | four probes naming blocks in neither plan: no gist 0/8–6/8, gist **8/8 `none` on all four** | +| Real ambiguity survives the gist | "move the gym to the morning", where both plans hold a gym: **ambiguous 8/8** | +| Cost | p50 **0.38–0.48s**, one round trip | + +The false-positive result inverts the intuition and is the reason the gist is safe: a +content-free descriptor is **not** the cautious option. It fails in both directions — false +`none` on a block that exists, and false `ambiguous` on a block that does not. *"Push the standup +to 11"* was `ambiguous` 8/8 without a gist, which is not hedging; it is a confident claim that +the message concerns one of these sessions. + +Two of the fixture's labels were corrected during blind review by #352's owner, in the same +direction both times: the model was reading state and the label was reading an assumption. +Labels for the eval must be reviewed by someone who has not seen the prompt. + +## Design + +### Module + +`src/fateforger/referents/` — new, and deliberately outside `slack_bot/handlers.py`, which is +5,085 lines and slated for deletion under #157/#165. Nothing in this package imports Slack. + +| file | holds | +|---|---| +| `descriptor.py` | `StandingThing`, `Referent` — the typed contract | +| `provider.py` | `ReferentProvider` protocol | +| `catalog.py` | `build_catalog()` — gathers providers concurrently, mints ids | +| `resolver.py` | `Resolution` union, `resolve()`, the prompt | +| `timeboxing.py` | `TimeboxingReferentProvider` over the session store | + +### The descriptor + +Every field is minted by this system. Nothing here is the user's prose. + +```python +class StandingThing(BaseModel): # what a provider returns + model_config = ConfigDict(extra="forbid", frozen=True) + + key: str # delivery key (a session key today) + agent_type: str # who owns it + kind: str # "a plan for one day" + day: date | None # None when no day is locked yet + status: str # open | committed + never_used: bool # auto-opened, revision <= 1 + last_activity: datetime + accepts: tuple[str, ...] # what this state permits, in words + gist: tuple[str, ...] = () # capped block titles WITH times + channel_id: str | None = None # for the human-facing link, with: + thread_ts: str | None = None # …and for is_current_surface + +class Referent(StandingThing): # what the catalog hands the model + ref_id: str # host-minted; providers never set it + is_current_surface: bool = False # this candidate IS where the message arrived +``` + +`channel_id` / `thread_ts` are the only Slack-shaped fields, and they are opaque strings the +provider copies rather than interprets. A provider whose standing things live elsewhere leaves +them `None` and loses only the link and `is_current_surface`. If a second provider ever needs a +different locator shape, that is the first entry in the strain log rather than a guess now. + +Four points that are load-bearing rather than incidental: + +- **`gist` carries times, not only titles.** Half the resolving power is clocks. A title without + one cannot answer *"push the investor call prep later"*. Capped at 12 entries; the plan's + free-text notes are excluded. +- **Block titles reach the model to be *read*, never to be matched.** They are content, and + content only ever goes to a judge. A test guards that no code in this package compares, + sorts, or filters on `gist`. +- **`never_used` is a field, not a phrase.** The measurement says the model does not care; the + contract does. A fact carried as a sentence cannot be filtered, sorted or tested on by a + consumer. #352's door sees this row as its *common* case, because autostart pre-warms a + session at every planning event. +- **`is_current_surface` is structural.** A thread id compared to a thread id, so it costs no + judgement. It exists because *"cancel **that** session"* points away from where the speaker + is, and nothing else in the descriptor can express that. + +### The provider protocol + +```python +class ReferentProvider(Protocol): + agent_type: str + async def standing( + self, *, owner_user_id: str, as_of: datetime + ) -> Sequence[StandingThing]: ... +``` + +The whole seam. A provider takes an owner and a clock and returns descriptors — no Slack event, +no channel, no focus manager, no route in scope. Three consumers already: the routing rung, +#352's door, and the eval. The third is what keeps the shape honest. + +**Providers do not mint ids.** `build_catalog` assigns `ref_id`, so identity stays with the host +exactly as it does on every other surface in this repo. + +**`as_of` is explicit and required.** The store keeps no history, so a descriptor otherwise reads +*current* status while claiming to describe an earlier moment. This bit during the spike: two +runs 40 minutes apart drew different candidate sets because a peer committed a plan mid-run. + +### The resolver + +```python +class _Outcome(BaseModel): + catalog_complete: bool # every provider answered + +class Resolved(_Outcome): referent: Referent +class Ambiguous(_Outcome): candidates: tuple[Referent, ...] +class NoReferent(_Outcome): pass + +Resolution = Resolved | NoReferent | Ambiguous +``` + +`Ambiguous` carries the candidates that tied, so a consumer can name them on a card without +rebuilding the catalog. `catalog_complete` sits on all three because a `none` — and equally a +confident `Resolved` — drawn from a partial catalog is weaker evidence than one drawn from a +whole one. + +**None of the three has a field for an action, and that is the enforcement**: resolve-then-act is +a return type, not a paragraph someone skims. The consumer runs its own second judgement over the +resolved state's `allowed_decisions` (#352, measured separately in `action_judge_spike.py`). + +The call is the `SurfaceIntentInterpreter` shape already used everywhere else: the host mints the +candidate ids, the schema is narrowed to exactly those ids plus `none` and `ambiguous`, and a +returned id is validated against the minted set before it is believed. + +### Prompt + +The wording that moved 74/112 → 95/112 turns on one distinction, and it should not be edited +without re-running the eval: + +> A message is about a standing thing when it continues, changes, questions or ends **that day's** +> plan — including a question about what that plan says. It is about none of them when it asks +> for something new that no listed plan covers. **Wanting something scheduled is not the same as +> continuing an existing plan for a day.** + +### Where it is called + +**The rung**, in `route_slack_event`'s ordered resolvers: after every structural claim (the +planning card's own thread, a session's own thread) and **before** the channel default. Structural +ownership is a fact and always beats a judgement — that ordering is #310's contract and this adds +a rung to it rather than reordering it. + +The rung must run **before** anything is created, so the catalog never contains a row the message +itself minted. In the spike this is a `created_at < as_of` predicate inside the provider's query; +it is kept, but the real constraint is ordering and it gets a test, because a timestamp comparison +that holds by construction at two doors answers wrong at a third without raising. + +## What stays code + +Per the routing clause: *does this decide what the user meant* → model; *what the system may do* → +code with a test. + +| guarantee | mechanism | +|---|---| +| Which rows stand | one indexed query over `owner_user_id`, `status`, `planning_date`, `updated_at` — the snapshot JSON is never read | +| `none` never authorises creation | at any door that can create, `NoReferent` is accepted only when a structural query independently agrees nothing stands. A row exists + `none` is a **contradiction**, and the guarantee wins: the door asks. (Owned by the writing consumer, #352.) | +| Nothing writes against a stale view | the door re-reads its target row at the moment of the write | +| A forward is offered, never performed | the card names the session and waits for a press. This is the only protection against a *confidently wrong* referent, which no resolver can detect. | + +Together these give four ways to be wrong and **none of them writes**: a false `none` is caught by +the query contradicting the model; a false `ambiguous` becomes a card that asks; a wrong confident +referent becomes a card naming a session the user does not press; a target that moved is refused at +press time. The gist's contribution is turning needless questions into correct answers — the safety +never rested on the model. + +## Error handling + +- **A provider raises.** Its rows are omitted, the failure is logged and counted + (`record_error`), and the catalog is marked incomplete. Every `Resolution` carries + `catalog_complete` through, because a `none` drawn from a partial catalog is not evidence that + nothing stands. A door that can create must treat `NoReferent(catalog_complete=False)` as + "ask", never as "create" — which is the same rule as `none`-never-creates, reached by a second + route. +- **The model call fails.** It raises. The rung falls through to the receptionist, which is the + existing `none` path and is safe. There is no pattern fallback — two behaviours with the wrong + one silent is the shape this repo's first rule exists to stop. +- **The model returns an id the host did not mint.** Rejected as a schema violation, same as every + other surface. + +## Testing + +**Unit (stubbed model, offline).** That the right question was asked and the answer applied: ids +are minted by the catalog and not by providers; an unminted id is rejected; a failing provider +does not fail the catalog but does clear `complete`; `as_of` is threaded through; a row created +after `as_of` never appears; `Resolution` cannot express an action. + +**Eval (`tests/evals/`, real model, slow).** The two frozen fixtures, n=8, asserting rates: + +- *incident* — 2026-09-05 13:51, three candidates including the never-used row. +- *two-parallel-sessions* — the #275 incident, 2026-09-03, both open for one Friday, with each + session's real gist, plus the four false-positive probes. + +Both fixtures are **inline and frozen, with the as-of named in the docstring, and the docstring +says why** — it is exactly the ceremony the next person deletes. Re-reading the live store inside +a scored run measures the ledger's drift rather than the model. + +Labels get a blind review by someone who has not seen the prompt. Three were wrong on first +writing, all in the same direction. + +**Guard test.** No comparison, sort, or filter over `gist` anywhere in the package. + +## Compounding + +The map's test is *does the second provider get cheaper because the first exists?* The artefact +that decides it is the protocol, so its shape is a deliverable here. + +During implementation, keep `docs/superpowers/notes/2026-09-referent-interface-strain.md` (via the +`implementation-notes` skill), logging every place the interface strained to fit timeboxing: + +- what the descriptor needed that the protocol did not offer; +- what the resolver wanted that a provider could not supply; +- which of those are timeboxing-specific and which are general. + +That log is the input to the task marshal's own provider and to #160's decision about what a GTD +session's standing state even is. It is the reason this ticket is built before the marshal rather +than alongside it. + +## Out of scope + +- **The task marshal provider.** A later provider against this interface, once #160 settles what + its standing state is. Not fog — a known, deliberately deferred consumer. +- **The second judgement** (what the message *does* to the referent) — #352. +- **`none` → escalate** as a general rung — #337. +- **The `/timebox` day-proposal door** — #346, landed 2026-09-05 (`cb4d9e5`). +- **Focus manager and receptionist changes.** The rung reads focus as context; it does not + rewrite it. +- **Recycling the empty auto-opened session** rather than inserting a new row. Deterministic store + housekeeping, invisible to the user, and on the code side of the clause. Keeping it out of the + resolver is the point: asking a model to link *"plan tomorrow"* to a day-less shell is a + resource question dressed as a reference question. +- **Session facts dropped at `ConfirmPlanningDay`** (`timeboxing_intents.py:482` carries only the + date). A binder bug, filed separately; routing does not fix it. + +## Open, and deliberately not closed here + +- No fixture case resolves *to* the day-less row, so "should a bare planning request go there?" + cannot be settled empirically. It is answered by ruling above, not by measurement. +- No case arrives inside a candidate's own thread, so `is_current_surface` is reasoned rather than + measured. +- No fixture can express a candidate moving between the read and the act. That needs a test that + mutates the row mid-flight, and it belongs to the door that performs the write (#352). diff --git a/scripts/spikes/referent_resolver_spike.py b/scripts/spikes/referent_resolver_spike.py new file mode 100644 index 00000000..c22808bd --- /dev/null +++ b/scripts/spikes/referent_resolver_spike.py @@ -0,0 +1,542 @@ +"""PROTOTYPE -- throwaway. Referent resolver spike (2026-09-05). + +The question +------------ +A message arrives with no structural owner (not in a session's thread, not +under a planning card). The host enumerates the user's *standing things* -- +here, timeboxing sessions from the real ledger -- as system-minted +descriptors, and one model judgement says which of them, if any, the message +concerns. Delivery then goes to that referent's own surface. + +How reliably does that judgement land on real candidates, and which of two +shapes is more robust? + + Shape A -- one call: candidates + message -> {candidate id | none | ambiguous} + Shape C -- one yes/no call *per candidate*, concurrently, then arithmetic: + exactly one yes -> that one; zero -> none; several -> ambiguous. + +Both are the surface-interpreter shape: the host mints the ids, the model +chooses, the host validates the id came from its own list. Nothing compares +the user's words to anything. + +Resampled 8x per case (CLAUDE.md: one passing draw tests luck). Run: + + PYTHONPATH=src ./.venv/bin/python scripts/spikes/referent_resolver_spike.py + ... --say "your message" # one ad-hoc case + ... --draws 4 # fewer draws + +Reads data/admonish.db read-only for the candidate rows. Writes nothing. +""" + +from __future__ import annotations + +import argparse +import asyncio +import json +import os +import sqlite3 +import sys +import time +from dataclasses import dataclass +from datetime import date, datetime, timedelta, timezone +from zoneinfo import ZoneInfo + +import httpx +from dotenv import load_dotenv + +load_dotenv() + +#: The flash pin is the decision record (CLAUDE.md); never a literal model id. +MODEL = os.environ["OPENROUTER_DEFAULT_MODEL_FLASH"] +TZ = ZoneInfo("Europe/Amsterdam") +#: The moment the real message arrived (Slack ts 1788609097.908269). +NOW = datetime.fromtimestamp(1788609097.908269, tz=TZ) +OWNER = "U095637NL8P" +#: `standing_for` counts an open session as under way for one hour. That +#: bound is for the nudger; for the catalog it is too tight (the Monday +#: session was last saved 69 minutes before this message). Widened here to +#: see what a realistic set looks like; the right bound is a design question. +OPEN_RECENCY = timedelta(hours=12) +HORIZON = timedelta(days=7) + +# --------------------------------------------------------------------------- +# The portable part: catalog + resolver shapes. No I/O except the model call. +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class Referent: + """One standing thing, as the model may see it. Every field is minted by + the system; nothing here came from the user's words.""" + + ref_id: str # host-minted, opaque to the model + key: str # delivery key (session key) + agent: str + kind: str + day: date | None + status: str + last_activity: datetime + accepts: tuple[str, ...] + #: Whether the row was opened by autostart and never touched by the user. + #: Held apart from `status` on purpose: run 4 varied a *phrase inside* + #: `status` ("open" vs "open, opened automatically, never used") and that is + #: not the same claim as a structured field. #352's door sees this row as + #: the common case, so the shape of this datum is load-bearing there. + never_used: bool = False + #: A short, capped list of the plan's own block titles and times. Present so + #: a message naming a block inside a plan can be resolved at all: without it + #: the model is shown a day and a status and correctly answers `none`, which + #: at a door that can create a session is how #275's duplicate is minted. + gist: tuple[str, ...] = () + + def describe(self, now: datetime, *, mode: str = "prose") -> dict: + """``mode``: how the never-used fact is carried to the model. + + prose -- a phrase inside `status` (what run 4 actually varied) + structured -- a plain status plus a boolean field + both -- the boolean field AND a rendered sentence + """ + ago = now - self.last_activity + hours = round(ago.total_seconds() / 3600, 1) + out = { + "ref_id": self.ref_id, + "kind": self.kind, + "day": ( + f"{self.day.isoformat()} ({self.day.strftime('%A')})" + if self.day + else "no day locked yet" + ), + "status": self.status if mode in ("structured", "gist") else self.prose_status, + "last_activity": f"{hours}h ago", + "accepts": list(self.accepts), + } + if mode in ("structured", "both", "gist"): + out["opened_automatically_never_used"] = self.never_used + if mode == "gist" and self.gist: + out["plan_contains"] = list(self.gist[:12]) + return out + + @property + def prose_status(self) -> str: + """Run 4's arm: the fact carried as a phrase inside the status string.""" + return ( + "open, opened automatically, never used" if self.never_used else self.status + ) + + +def load_referents(db_path: str, *, owner: str, now: datetime) -> list[Referent]: + """What `standing_for` would return, widened to every qualifying row. + + Same predicate family as the ledger: open and saved recently, or + committed with a day inside the horizon. Cancelled never appears. + """ + since = (now - OPEN_RECENCY).astimezone(timezone.utc).replace(tzinfo=None) + conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) + rows = conn.execute( + """ + select session_key, status, planning_date, updated_at, revision + from timeboxing_session_states + where owner_user_id = ? + and created_at < ? + and ( + (status = 'open' and updated_at >= ?) + or (status = 'committed' and planning_date between ? and ?) + ) + order by updated_at desc + """, + ( + owner, + NOW.astimezone(timezone.utc).replace(tzinfo=None).isoformat(sep=" "), + since.isoformat(sep=" "), + now.date().isoformat(), + (now + HORIZON).date().isoformat(), + ), + ).fetchall() + conn.close() + out: list[Referent] = [] + for i, (key, status, pdate, updated, revision) in enumerate(rows): + last = datetime.fromisoformat(updated).replace(tzinfo=timezone.utc) + # Run 1: an auto-opened, never-touched, day-less DM session drew + # "plan tomorrow" 7/8 under shape C. Revision 1 is the opening turn + # (session_start.UNTOUCHED_REVISION); say so instead of hiding it. + untouched = status == "open" and revision <= 1 + out.append( + Referent( + ref_id=f"r{i + 1}", + key=key, + agent="timeboxing_agent", + kind="timeboxing session (a plan for one day)", + day=date.fromisoformat(pdate) if pdate else None, + status=status, + never_used=untouched, + last_activity=last, + accepts=( + ("revise the committed plan", "add a fact about the day") + if status == "committed" + else ("continue planning", "answer the open question", "cancel") + ), + ) + ) + return out + + +#: The ledger AS OF the message. Run 3 drew a different set than run 2 because a +#: peer committed the Monday plan mid-run and `describe` reads *current* status, +#: not status as of `now`. Any eval over live session rows measures the ledger's +#: drift unless it is frozen. Rows below are `timeboxing_session_states` at +#: 2026-09-05 13:51, verbatim. +FROZEN: tuple[tuple[str, str, str | None, str, int], ...] = ( + ("C0AA6HC1RJL:1788603379.318719", "open", "2026-09-07", "2026-09-05 10:42:53", 7), + ("D09A0RE9P7G:dm", "open", None, "2026-09-05 01:43:00", 1), + ("C0AA6HC1RJL:1788571682.407949", "committed", "2026-09-05", "2026-09-05 01:41:01", 7), +) + + +def frozen_referents( + *, drop_untouched: bool, rows: tuple = (), now: datetime | None = None +) -> list["Referent"]: + """The fixture, through the same descriptor code the live path uses. + + ``drop_untouched`` is the catalog-policy question run 3 raised: an + auto-opened session nobody has touched, with no day locked, matched + "add a dentist appointment on tuesday" 7/8. It is an empty room -- there is + nothing about it a user could be referring to. Is it a referent at all? + """ + out: list[Referent] = [] + for key, status, pdate, updated, revision in (rows or FROZEN): + untouched = status == "open" and revision <= 1 + if untouched and drop_untouched: + continue + out.append( + Referent( + ref_id=f"r{len(out) + 1}", + key=key, + agent="timeboxing_agent", + kind="timeboxing session (a plan for one day)", + day=date.fromisoformat(pdate) if pdate else None, + status=status, + never_used=untouched, + gist=AMBIG_GIST.get(key, ()), + last_activity=datetime.fromisoformat(updated).replace(tzinfo=timezone.utc), + accepts=( + ("revise the committed plan", "add a fact about the day") + if status == "committed" + else ("continue planning", "answer the open question", "cancel") + ), + ) + ) + return out + + +#: The #275 incident, from production rather than construction (supplied by +#: #352's owner). On 2026-09-03 two sessions for Hugo and Friday 2026-09-04 ran +#: in parallel in #plan-sessions; one committed over the other's morning blocks +#: (journal id 186). Two open sessions, one day, no structural tie-break: this +#: is what `ambiguous` exists for, and #352's door cannot be built correctly if +#: the resolver cannot return it here. +#: +#: Caveat: `updated_at` and `revision` are the rows' CURRENT values, not their +#: values at 12:15 -- the store keeps no history, which is the same as-of gap +#: this spike flagged elsewhere. It does not affect the question asked, since +#: both rows are same-day open sessions either way. +AMBIG_NOW = datetime(2026, 9, 3, 12, 15, tzinfo=TZ) +AMBIG: tuple[tuple[str, str, str | None, str, int], ...] = ( + ("C0AA6HC1RJL:1788429283.534419", "open", "2026-09-04", "2026-09-03 10:17:54", 10), + ("C0AA6HC1RJL:1788429809.317849", "open", "2026-09-04", "2026-09-03 10:14:14", 8), +) + +#: Every message here is about Friday, and Friday has two sessions. Only the +#: last is expected to resolve: nothing stands for Sunday. +#: Labels are ground truth given full knowledge of both plans, which is the +#: point: the no-gist arm cannot know it and should visibly fail the first three. +AMBIG_CASES: list[tuple[str, str]] = [ + ("move PR1 later", "s1"), # PR1 is the first session's own handle + ("move the finances block later", "s1"), # only the first plan has Finances + ("push the investor call prep later", "s2"), # only the second has it + ("move the gym to the morning", "ambiguous"), # both plans have a gym + ("cancel that session", "ambiguous"), + ("make it start at 10", "ambiguous"), + ("plan sunday", "none"), + # False-positive probes (#352's owner, blind review): the gist added content, + # and the risk content brings is a model inventing a match. Nothing below + # exists in EITHER plan, so a gist arm that resolves one of these is worse + # than no gist -- it would be confidently naming a session over a block that + # is not in it. `none` is the only correct answer. + ("move the dentist earlier", "none"), + ("push the standup to 11", "none"), + ("can you shorten the school run", "none"), + ("move the physio appointment to friday morning", "none"), +] + + +#: What each incident session's plan actually holds, from its own artifacts. +#: The two plans differ almost entirely, which is what makes them a fair test: +#: a gist should resolve a message naming a block only one of them contains, +#: and must still answer ambiguous for a block both contain (both have a gym). +AMBIG_GIST: dict[str, tuple[str, ...]] = { + "C0AA6HC1RJL:1788429283.534419": ( + "PR1 Serious C2F work 10:30-12:00", + "EVT2 Kapper 12:00-12:30 (foreign, fixed)", + "LN1 Lunch 12:30-13:00", + "DW1 Validate agent demos 13:00-13:45", + "INV1 Finances 13:45-14:30", + "OAT1 Oats 16:00-16:15", + "GYM1 Gym (chest) 18:00-19:00", + "DIN1 Dinner 19:15-20:00", + "SHD1 Evening shutdown ritual 20:00-21:00", + ), + "C0AA6HC1RJL:1788429809.317849": ( + "PR review - stage-UX, ends 11:30", + "Kapper 12:00-12:30 (calendar event, foreign)", + "Lunch ~12:30", + "Deep work - constraint memory design, 90 minutes", + "Prepare the Monday investor call 15:00-16:00", + "Oats 16:00", + "Gym 18:00 (chest)", + "Dinner ~19:30", + "Evening shutdown ritual", + "Sleep 23:00", + ), +} + + +class Judge: + def __init__(self) -> None: + self.base = os.environ["OPENROUTER_BASE_URL"].rstrip("/") + self.key = os.environ["OPENROUTER_API_KEY"] + self.client = httpx.AsyncClient(timeout=httpx.Timeout(60.0, connect=10.0)) + self.calls = 0 + self.latency: list[float] = [] + + async def ask(self, system: str, user: str) -> dict: + t0 = time.perf_counter() + for attempt in range(3): + try: + r = await self.client.post( + f"{self.base}/chat/completions", + headers={"Authorization": f"Bearer {self.key}"}, + json={ + "model": MODEL, + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": user}, + ], + "reasoning": {"effort": "minimal"}, + "response_format": {"type": "json_object"}, + }, + ) + r.raise_for_status() + body = r.json() + if "choices" not in body: + raise RuntimeError(body.get("error", body)) + self.calls += 1 + self.latency.append(time.perf_counter() - t0) + return json.loads(body["choices"][0]["message"]["content"]) + except (httpx.TimeoutException, RuntimeError, json.JSONDecodeError) as e: + if attempt == 2: + raise + await asyncio.sleep(1.5 * (attempt + 1)) + raise AssertionError + + +SHARED_PREAMBLE = """You route one message a user just typed to an assistant. +The user has some *standing things*: conversations or plans that already exist and can be continued. +Decide whether the message is about one of them, or is a new request that concerns none of them. +Judge by meaning. A message that continues, changes, questions, or cancels a standing thing is about it. +A message that asks for something none of the standing things covers is about none of them. +Never invent identifiers. Return only JSON.""" + +SHAPE_A_PROMPT = SHARED_PREAMBLE + """ +Answer with {"decision": "" | "none" | "ambiguous", "why": ""}. +Use "ambiguous" only when the message is about a standing thing but you cannot tell which of two or more.""" + +SHARP_PROMPT = """You route one message a user just typed to a scheduling assistant. +The user has some *standing things*: plans for a particular day that already exist and can be continued. +Decide which standing thing, if any, this message is about. + +A message is about a standing thing when it continues, changes, questions, or ends THAT day's plan -- +including a question about what that plan says. +A message is about NONE of them when it asks for something new that no listed plan covers: a fact, +an errand, a reminder, or planning a day that is not listed. Wanting something scheduled is not the +same as continuing an existing plan for a day. +Choose "ambiguous" only when the message is clearly about one of the listed plans but two or more fit equally. + +Judge by meaning. Never invent identifiers. Return only JSON. +Answer with {"decision": "" | "none" | "ambiguous", "why": ""}.""" + +SHAPE_C_PROMPT = SHARED_PREAMBLE + """ +You are shown exactly ONE standing thing. Answer whether the message is about it. +{"about_this": true | false, "why": ""}""" + + +async def resolve_shape_a( + judge: Judge, + refs: list[Referent], + text: str, + now: datetime, + prompt: str = SHAPE_A_PROMPT, + mode: str = "prose", +) -> str: + payload = { + "now": now.strftime("%Y-%m-%d %H:%M (%A)"), + "standing_things": [r.describe(now, mode=mode) for r in refs], + "message": text, + } + answer = await judge.ask(prompt, json.dumps(payload, ensure_ascii=False)) + decision = str(answer.get("decision", "")) + valid = {r.ref_id for r in refs} | {"none", "ambiguous"} + return decision if decision in valid else f"INVALID({decision})" + + +async def resolve_shape_c(judge: Judge, refs: list[Referent], text: str, now: datetime) -> str: + async def one(r: Referent) -> bool: + payload = { + "now": now.strftime("%Y-%m-%d %H:%M (%A)"), + "standing_thing": r.describe(now, mode="prose"), + "message": text, + } + answer = await judge.ask(SHAPE_C_PROMPT, json.dumps(payload, ensure_ascii=False)) + return bool(answer.get("about_this") is True) + + votes = await asyncio.gather(*(one(r) for r in refs)) + yes = [r.ref_id for r, v in zip(refs, votes) if v] + if len(yes) == 1: + return yes[0] + return "none" if not yes else "ambiguous" + + +# --------------------------------------------------------------------------- +# The throwaway shell: cases, resampling, report. +# --------------------------------------------------------------------------- + +#: (message, expected). Expected names a day (mapped to a ref_id at runtime), +#: or "none" / "ambiguous". Labels are my expectation of the right routing, +#: to be argued with. +CASES: list[tuple[str, str]] = [ + ("can you replan today so the gym is before dinner?", "2026-09-05"), + ("plan saturday", "2026-09-05"), # #275: a second opening for a planned day + ("let's plan monday", "2026-09-07"), # #275: a second opening for an open day + ("actually make monday start at 10", "2026-09-07"), + ("I'll wake up at 11 on monday", "2026-09-07"), + ("move the gym to the morning", "ambiguous"), # two days, no day named + ("cancel that session", "ambiguous"), + ("plan tomorrow", "none"), # Sunday: nothing stands + ("what's the weather tomorrow", "none"), + ("add a dentist appointment on tuesday", "none"), + ("remind me to pay taxes", "none"), + ("what did we decide about dinner?", "2026-09-05"), + ("I want to finish the finance ticket in the first shallow block", "2026-09-05"), + ("is it planned?", "none"), # a planning card resolves this structurally; here nothing does +] + + +#: The ambiguity fixture's two rows, in fixture order, so a label can name one. +_AMBIG_ORDER = {"s1": 0, "s2": 1} + + +def expected_id(expected: str, refs: list[Referent]) -> str: + if expected in _AMBIG_ORDER: + i = _AMBIG_ORDER[expected] + return refs[i].ref_id if i < len(refs) else f"MISSING({expected})" + if expected in ("none", "ambiguous"): + return expected + # Prefer the committed session for a day: it is the plan that stands. + for r in sorted(refs, key=lambda r: r.status != "committed"): + if r.day and r.day.isoformat() == expected: + return r.ref_id + return f"MISSING({expected})" + + +async def run(draws: int, say: str | None, scenario: str = "incident") -> None: + """Four arms on one frozen fixture, so only one thing varies at a time. + + Shape C is not run any more: runs 2 and 3 both retired it (a candidate shown + alone has no contrast, so the model affirms nearly everything). What is left + to settle is the catalog policy and the wording, on the pin. + """ + # Run 5 holds the prompt at the winner and varies ONLY how the never-used + # fact is carried, which is what run 4 failed to isolate. The fourth arm is + # #352's fallback: a structured field the provider renders into a sentence, + # so the store keeps a real field and the model still reads prose. + _moment_hdr = AMBIG_NOW if scenario == "ambiguity" else NOW + arms = ( + [ + ("no gist: day + status only", SHARP_PROMPT, False, "structured"), + ("gist: + block titles and times", SHARP_PROMPT, False, "gist"), + ] + if scenario == "ambiguity" + else [ + ("prose status (run 4's arm3)", SHARP_PROMPT, False, "prose"), + ("structured boolean field", SHARP_PROMPT, False, "structured"), + ("structured + rendered sentence", SHARP_PROMPT, False, "both"), + ("never-used row dropped", SHARP_PROMPT, True, "prose"), + ] + ) + print(f"\x1b[1mmodel\x1b[0m {MODEL} \x1b[1mscenario\x1b[0m {scenario} \x1b[1mnow\x1b[0m {_moment_hdr:%Y-%m-%d %H:%M %A} \x1b[1mdraws\x1b[0m {draws} \x1b[2m(frozen fixture)\x1b[0m") + _rows = AMBIG if scenario == "ambiguity" else FROZEN + _moment = AMBIG_NOW if scenario == "ambiguity" else NOW + for label, _, drop, _mode in arms[:1]: + refs = frozen_referents(drop_untouched=drop, rows=_rows, now=_moment) + print(f" \x1b[1m{'without' if drop else 'with'} the never-used session\x1b[0m: " + " | ".join( + f"{r.ref_id} {r.day or 'no day'} {r.status.split(',')[0]}" for r in refs)) + print() + + judge = Judge() + if scenario == "ambiguity": + cases, rows, moment = AMBIG_CASES, AMBIG, AMBIG_NOW + else: + cases, rows, moment = CASES, FROZEN, NOW + if say is not None: + cases = [(say, "?")] + + async def one_arm(prompt: str, drop: bool, mode: str): + refs = frozen_referents(drop_untouched=drop, rows=rows, now=moment) + out = [] + for text, expected in cases: + exp = expected_id(expected, refs) if expected != "?" else "?" + got = list(await asyncio.gather( + *(resolve_shape_a(judge, refs, text, moment, prompt, mode) for _ in range(draws)) + )) + out.append((text, exp, got)) + return out + + results = await asyncio.gather(*(one_arm(pr, dr, md) for _, pr, dr, md in arms)) + + def cell(got: list[str], exp: str) -> str: + hit = sum(1 for g in got if g == exp) + colour = "\x1b[32m" if hit == draws else ("\x1b[33m" if hit >= draws * 0.75 else "\x1b[31m") + top = max(set(got), key=got.count) + extra = "" if hit == draws else f" \x1b[2m{top}:{got.count(top)}\x1b[0m" + return f"{colour}{hit}/{draws}\x1b[0m{extra}" + + header = "message".ljust(46) + "expect".ljust(9) + "".join( + f"arm{i + 1}".ljust(18) for i in range(len(arms))) + print("\x1b[1m" + header + "\x1b[0m") + for row, (text, exp, _) in enumerate(results[0]): + line = text[:44].ljust(46) + exp.ljust(9) + for arm in results: + _, e, got = arm[row] + line += cell(got, e).ljust(30) + print(line) + + print() + for i, (label, _, _, _mode) in enumerate(arms): + total = sum(sum(1 for g in got if g == exp) for _, exp, got in results[i]) + possible = len(results[i]) * draws + clean = sum(1 for _, exp, got in results[i] if all(g == exp for g in got)) + print(f" \x1b[1marm{i + 1}\x1b[0m {label:<38} {total}/{possible} draws {clean}/{len(results[i])} cases unanimous") + + lat = sorted(judge.latency) + print(f"\n\x1b[2m{judge.calls} calls, latency p50 {lat[len(lat) // 2]:.2f}s p90 {lat[int(len(lat) * 0.9)]:.2f}s\x1b[0m") + await judge.client.aclose() + + +if __name__ == "__main__": + ap = argparse.ArgumentParser() + ap.add_argument("--draws", type=int, default=8) + ap.add_argument("--say", type=str, default=None) + ap.add_argument("--scenario", choices=("incident", "ambiguity"), default="incident") + ns = ap.parse_args() + if not os.environ.get("OPENROUTER_API_KEY"): + sys.exit("OPENROUTER_API_KEY missing (.env)") + asyncio.run(run(ns.draws, ns.say, ns.scenario)) From d439598fca4f8e57a96817a541edaf1f4580e514 Mon Sep 17 00:00:00 2001 From: hugocool Date: Tue, 8 Sep 2026 19:54:07 +0200 Subject: [PATCH 02/20] docs(referents): the implementation plan, nine tasks against the design (#345) TDD throughout, one test cycle per task, each ending in a committable deliverable. The order is descriptor, provider seam, resolver, guard, provider, query, rung, eval, strain log -- so the model call is testable with a stub by task three and the rung, which is the only task touching the 5,085-line handlers.py, arrives last with everything under it already proven. Three things the plan pins down rather than leaving to taste: - the eval's fixtures are inline and dated, with the docstring saying WHY, since reading the live ledger measures its drift and the store keeps no history; - the thresholds come from the measured runs (>=85% of draws, and the four false-positive probes unanimous, because a wrong answer there becomes a duplicate session at a door that creates); - task 4 is an AST guard that block titles are read by a judge and never compared by code, and task 8 step 3 breaks the prompt on purpose to prove the eval is not vacuous. Task 9 is the compounding deliverable Hugo asked for by name: a strain log written during the build, recording what the interface had to grow to fit timeboxing and which of it is general, as the input to the task marshal's own provider. Co-Authored-By: Claude Fable 5.1 --- .../plans/2026-09-08-referent-catalog.md | 2186 +++++++++++++++++ 1 file changed, 2186 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-08-referent-catalog.md diff --git a/docs/superpowers/plans/2026-09-08-referent-catalog.md b/docs/superpowers/plans/2026-09-08-referent-catalog.md new file mode 100644 index 00000000..8df4ba3e --- /dev/null +++ b/docs/superpowers/plans/2026-09-08-referent-catalog.md @@ -0,0 +1,2186 @@ +# Referent Catalog Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a catalog of the user's *standing things* and one judgement over it, so a message +with no structural owner reaches the session it is about instead of minting a new one. + +**Architecture:** A new `fateforger.referents` package with no Slack imports. Providers answer +`standing(owner_user_id, as_of)` with descriptors; the catalog mints ids and gathers providers +concurrently; the resolver makes one model call over all candidates and returns a referent, never +an action. Timeboxing sessions are the first provider. Task 7 wires it as a rung in +`route_slack_event` after every structural resolver. + +**Tech Stack:** Python 3.11, Pydantic v2, SQLAlchemy async, AutoGen `OpenAIChatCompletionClient` +via `fateforger.llm.build_autogen_chat_client`, pytest + pytest-asyncio (`asyncio_mode = "auto"`). + +**Spec:** `docs/superpowers/specs/2026-09-08-referent-catalog-design.md` — read it first. +**Spike (primary source, already committed):** `scripts/spikes/referent_resolver_spike.py`. + +## Global Constraints + +- **No `re`, no keyword lists, no substring tests over user content.** Anywhere. Any judgement + about what the user meant goes to a model (`CLAUDE.md`). +- **String ops on identifiers this system minted are fine** — session keys, thread ids, statuses, + ref ids. Comparing two uids is allowed; comparing two of the user's sentences is not. +- **Never hardcode a model id.** Build clients with + `build_autogen_chat_client("timeboxing_judge")` — that resolves to the flash pin + (`OPENROUTER_DEFAULT_MODEL_FLASH`) at `reasoning: minimal`. A `google/` id anywhere is a + regression. +- **Independent model calls go out concurrently**, never in sequence. +- **Evals resample.** n=8 per case, assert on the rate, never pin `temperature: 0`. +- **Run everything with `PYTHONPATH=src`.** +- **Test command:** `PYTHONPATH=src .venv/bin/python -m pytest tests -m "not slow" -q` +- Commit after every task. Never `git commit -a` (other sessions share this checkout). +- Work in this worktree: `.claude/worktrees/referent-catalog`, branch `feat/referent-catalog`. + +## File Structure + +| file | responsibility | +|---|---| +| `src/fateforger/referents/__init__.py` | public exports | +| `src/fateforger/referents/descriptor.py` | `StandingThing`, `Referent` | +| `src/fateforger/referents/provider.py` | `ReferentProvider` protocol | +| `src/fateforger/referents/catalog.py` | `build_catalog()` — concurrent gather, mints ids | +| `src/fateforger/referents/resolver.py` | `Resolution` union, `resolve()`, the prompt | +| `src/fateforger/referents/timeboxing.py` | `TimeboxingReferentProvider` | +| `tests/unit/test_referent_descriptor.py` | descriptor + id minting | +| `tests/unit/test_referent_catalog.py` | gather, failure isolation, as-of | +| `tests/unit/test_referent_resolver.py` | plumbing with a stubbed model | +| `tests/unit/test_referent_timeboxing_provider.py` | the SQL provider | +| `tests/unit/test_referents_never_match_gist.py` | guard test | +| `tests/evals/test_eval_referent_resolver.py` | quality, real model, `-m slow` | +| `src/fateforger/slack_bot/handlers.py` | the rung (Task 7 only) | + +--- + +### Task 1: The descriptor + +**Files:** +- Create: `src/fateforger/referents/__init__.py` +- Create: `src/fateforger/referents/descriptor.py` +- Test: `tests/unit/test_referent_descriptor.py` + +**Interfaces:** +- Consumes: nothing. +- Produces: `StandingThing` (frozen Pydantic model, fields below), `Referent(StandingThing)` adding + `ref_id: str` and `is_current_surface: bool`, and `Referent.describe(now: datetime) -> dict`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_referent_descriptor.py +from datetime import UTC, date, datetime + +import pytest +from pydantic import ValidationError + +from fateforger.referents import Referent, StandingThing + +NOW = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +def _thing(**over) -> StandingThing: + base = dict( + key="C1:123.456", + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + ) + base.update(over) + return StandingThing(**base) + + +def test_describe_names_the_weekday_because_a_bare_date_is_not_a_day_to_a_reader(): + ref = Referent(**_thing().model_dump(), ref_id="r1") + described = ref.describe(NOW) + assert described["day"] == "2026-09-05 (Saturday)" + assert described["ref_id"] == "r1" + + +def test_describe_reports_age_relative_to_the_asked_moment_not_the_wall_clock(): + ref = Referent(**_thing().model_dump(), ref_id="r1") + assert ref.describe(NOW)["last_activity"] == "10.2h ago" + + +def test_a_day_less_row_says_so_rather_than_omitting_the_field(): + ref = Referent(**_thing(day=None).model_dump(), ref_id="r1") + assert ref.describe(NOW)["day"] == "no day locked yet" + + +def test_never_used_is_a_field_the_model_sees_not_a_phrase_inside_status(): + # Measured: how it is carried is inside the noise floor, but a consumer + # must be able to filter and test on it. #352's door sees this as its + # common case because autostart pre-warms a session per planning event. + ref = Referent(**_thing(status="open", never_used=True).model_dump(), ref_id="r1") + described = ref.describe(NOW) + assert described["status"] == "open" + assert described["opened_automatically_never_used"] is True + + +def test_the_gist_is_capped_so_a_long_plan_cannot_dominate_the_prompt(): + ref = Referent( + **_thing(gist=tuple(f"B{i} block {i} 0{i}:00-0{i}:30" for i in range(20))).model_dump(), + ref_id="r1", + ) + assert len(ref.describe(NOW)["plan_contains"]) == 12 + + +def test_an_empty_gist_omits_the_key_rather_than_showing_an_empty_list(): + ref = Referent(**_thing().model_dump(), ref_id="r1") + assert "plan_contains" not in ref.describe(NOW) + + +def test_the_descriptor_is_frozen_and_refuses_unknown_fields(): + with pytest.raises(ValidationError): + StandingThing(**_thing().model_dump(), surprise="no") +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_descriptor.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'fateforger.referents'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# src/fateforger/referents/descriptor.py +"""What one standing thing looks like to the judge. + +Every field here is minted by this system -- a session key, a status, a date, a +block title the planner wrote. Nothing is the user's prose, which is why the +whole descriptor may be handed to a model without any of it being *matched*. +""" + +from __future__ import annotations + +from datetime import date, datetime + +from pydantic import BaseModel, ConfigDict + +#: How many gist entries reach the model. A cap, not a ranking: the provider +#: supplies the plan's own order and the tail is dropped, because a twenty-block +#: day would otherwise crowd out the other candidates. +GIST_LIMIT = 12 + + +class StandingThing(BaseModel): + """One thing a provider says is standing, before the host names it.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + key: str + agent_type: str + kind: str + day: date | None + status: str + never_used: bool + last_activity: datetime + accepts: tuple[str, ...] + #: The plan's own block titles WITH their times. Half the resolving power is + #: the clock: a title alone cannot answer "push the investor call prep + #: later". Present to be READ by a model and never matched, sorted or + #: filtered on by code -- `test_referents_never_match_gist` guards that. + gist: tuple[str, ...] = () + #: Opaque locators the provider copies rather than interprets. A provider + #: whose things live elsewhere leaves them None and loses only the link and + #: `is_current_surface`. + channel_id: str | None = None + thread_ts: str | None = None + + +class Referent(StandingThing): + """A standing thing the catalog has named, as the judge sees it.""" + + #: Host-minted. Providers never set this; `build_catalog` does, so identity + #: stays with the host exactly as on every other surface in this repo. + ref_id: str + #: This candidate IS the surface the message arrived in. Structural -- a + #: thread id compared to a thread id -- and it exists because "cancel THAT + #: session" points away from where the speaker is. + is_current_surface: bool = False + + def describe(self, now: datetime) -> dict: + """The candidate as JSON for the prompt. Reproducible from `now`.""" + hours = round((now - self.last_activity).total_seconds() / 3600, 1) + described: dict = { + "ref_id": self.ref_id, + "kind": self.kind, + "day": ( + f"{self.day.isoformat()} ({self.day.strftime('%A')})" + if self.day is not None + else "no day locked yet" + ), + "status": self.status, + "opened_automatically_never_used": self.never_used, + "last_activity": f"{hours}h ago", + "accepts": list(self.accepts), + "is_the_conversation_this_message_arrived_in": self.is_current_surface, + } + if self.gist: + described["plan_contains"] = list(self.gist[:GIST_LIMIT]) + return described +``` + +```python +# src/fateforger/referents/__init__.py +"""Standing things, offered as options to one judgement. + +Nothing in this package imports Slack. A provider takes an owner and a clock; +the catalog names what comes back; the resolver picks one, or none, or says it +cannot tell. It never says what to *do* -- that is the consumer's own judgement. +""" + +from .descriptor import GIST_LIMIT, Referent, StandingThing + +__all__ = ["GIST_LIMIT", "Referent", "StandingThing"] +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_descriptor.py -q` +Expected: PASS (7 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/fateforger/referents/__init__.py src/fateforger/referents/descriptor.py tests/unit/test_referent_descriptor.py +git commit -m "feat(referents): the descriptor a judge sees, with the plan's own blocks and times" +``` + +--- + +### Task 2: The provider protocol and the catalog + +**Files:** +- Create: `src/fateforger/referents/provider.py` +- Create: `src/fateforger/referents/catalog.py` +- Modify: `src/fateforger/referents/__init__.py` +- Test: `tests/unit/test_referent_catalog.py` + +**Interfaces:** +- Consumes: `StandingThing`, `Referent` from Task 1. +- Produces: + - `ReferentProvider` — Protocol with `agent_type: str` and + `async def standing(*, owner_user_id: str, as_of: datetime) -> Sequence[StandingThing]`. + - `Catalog` — frozen model with `referents: tuple[Referent, ...]` and `complete: bool`. + - `async def build_catalog(providers, *, owner_user_id, as_of, current_thread=None) -> Catalog` + where `current_thread: tuple[str, str] | None` is `(channel_id, thread_ts)`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_referent_catalog.py +from datetime import UTC, date, datetime + +from fateforger.referents import StandingThing, build_catalog + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +def _thing(key: str, **over) -> StandingThing: + base = dict( + key=key, + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="open", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("continue planning",), + ) + base.update(over) + return StandingThing(**base) + + +class _Provider: + def __init__(self, agent_type, things=(), raises=None): + self.agent_type = agent_type + self._things = list(things) + self._raises = raises + self.calls = [] + + async def standing(self, *, owner_user_id: str, as_of: datetime): + self.calls.append((owner_user_id, as_of)) + if self._raises is not None: + raise self._raises + return self._things + + +async def test_the_host_mints_the_ids_because_a_provider_may_not_name_identity(): + catalog = await build_catalog( + [_Provider("a", [_thing("k1"), _thing("k2")])], + owner_user_id="U1", + as_of=AS_OF, + ) + assert [r.ref_id for r in catalog.referents] == ["r1", "r2"] + assert [r.key for r in catalog.referents] == ["k1", "k2"] + + +async def test_ids_stay_unique_across_providers(): + catalog = await build_catalog( + [_Provider("a", [_thing("k1")]), _Provider("b", [_thing("k2")])], + owner_user_id="U1", + as_of=AS_OF, + ) + assert sorted(r.ref_id for r in catalog.referents) == ["r1", "r2"] + + +async def test_every_provider_is_asked_the_same_owner_and_moment(): + p1, p2 = _Provider("a", [_thing("k1")]), _Provider("b", [_thing("k2")]) + await build_catalog([p1, p2], owner_user_id="U1", as_of=AS_OF) + assert p1.calls == [("U1", AS_OF)] and p2.calls == [("U1", AS_OF)] + + +async def test_a_failing_provider_does_not_lose_the_others_but_does_clear_complete(): + # A `none` drawn from a partial catalog is not evidence that nothing + # stands, so the flag has to travel with the answer. + good = _Provider("a", [_thing("k1")]) + catalog = await build_catalog( + [good, _Provider("b", raises=RuntimeError("store down"))], + owner_user_id="U1", + as_of=AS_OF, + ) + assert [r.key for r in catalog.referents] == ["k1"] + assert catalog.complete is False + + +async def test_a_whole_catalog_is_complete(): + catalog = await build_catalog( + [_Provider("a", [_thing("k1")])], owner_user_id="U1", as_of=AS_OF + ) + assert catalog.complete is True + + +async def test_the_arriving_thread_is_marked_so_that_can_be_told_from_this(): + catalog = await build_catalog( + [ + _Provider( + "a", + [ + _thing("k1", channel_id="C1", thread_ts="111.0"), + _thing("k2", channel_id="C1", thread_ts="222.0"), + ], + ) + ], + owner_user_id="U1", + as_of=AS_OF, + current_thread=("C1", "222.0"), + ) + marked = {r.key: r.is_current_surface for r in catalog.referents} + assert marked == {"k1": False, "k2": True} + + +async def test_no_providers_is_an_empty_complete_catalog_not_a_failure(): + catalog = await build_catalog([], owner_user_id="U1", as_of=AS_OF) + assert catalog.referents == () and catalog.complete is True + + +async def test_providers_are_gathered_concurrently(): + import asyncio + + order = [] + + class _Slow(_Provider): + def __init__(self, agent_type, delay, key): + super().__init__(agent_type, [_thing(key)]) + self._delay = delay + + async def standing(self, *, owner_user_id, as_of): + await asyncio.sleep(self._delay) + order.append(self.agent_type) + return self._things + + await build_catalog( + [_Slow("slow", 0.05, "k1"), _Slow("fast", 0.0, "k2")], + owner_user_id="U1", + as_of=AS_OF, + ) + # Sequential awaits would finish slow-then-fast; concurrent finishes fast first. + assert order == ["fast", "slow"] +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_catalog.py -q` +Expected: FAIL — `ImportError: cannot import name 'build_catalog'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# src/fateforger/referents/provider.py +"""The whole seam: an owner, a clock, and what stands. + +One method. No Slack event, no channel, no focus manager, no route in scope -- +which is what lets the routing rung, a door that creates sessions, and the eval +all be consumers of the same catalog. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from datetime import datetime +from typing import Protocol, runtime_checkable + +from .descriptor import StandingThing + + +@runtime_checkable +class ReferentProvider(Protocol): + """Something that knows what one user currently has standing.""" + + #: Whose things these are. Travels onto every descriptor. + agent_type: str + + async def standing( + self, *, owner_user_id: str, as_of: datetime + ) -> Sequence[StandingThing]: + """What stands for this owner AT `as_of`. + + `as_of` is explicit and required rather than read from a clock inside. + The store keeps no history, so a descriptor otherwise reads *current* + state while claiming to describe an earlier moment -- which is how two + runs of one spike drew different candidate sets forty minutes apart. + """ + ... +``` + +```python +# src/fateforger/referents/catalog.py +"""Gather every provider, name what comes back, say whether it is whole.""" + +from __future__ import annotations + +import asyncio +import logging +from collections.abc import Sequence +from datetime import datetime + +from pydantic import BaseModel, ConfigDict + +from fateforger.core.logging_config import record_error + +from .descriptor import Referent, StandingThing +from .provider import ReferentProvider + +logger = logging.getLogger(__name__) + + +class Catalog(BaseModel): + """What stands, named, plus whether anyone failed to answer.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + referents: tuple[Referent, ...] = () + #: False when a provider raised. A `none` drawn from a partial catalog is + #: not evidence that nothing stands, so a door that can create must ask + #: rather than create when this is False. + complete: bool = True + + +async def build_catalog( + providers: Sequence[ReferentProvider], + *, + owner_user_id: str, + as_of: datetime, + current_thread: tuple[str, str] | None = None, +) -> Catalog: + """Ask every provider concurrently and name the results `r1`, `r2`, ... + + Ids are minted here and never by a provider, so identity stays with the + host. `current_thread` is `(channel_id, thread_ts)` for the conversation the + message arrived in, compared as identifiers this system minted. + """ + results = await asyncio.gather( + *(p.standing(owner_user_id=owner_user_id, as_of=as_of) for p in providers), + return_exceptions=True, + ) + + referents: list[Referent] = [] + complete = True + for provider, result in zip(providers, results): + if isinstance(result, BaseException): + complete = False + logger.exception( + "referent provider %s failed for %s", + provider.agent_type, + owner_user_id, + exc_info=result, + ) + record_error(component="referent_catalog", error_type="provider_failure") + continue + for thing in result: + referents.append( + Referent( + **thing.model_dump(), + ref_id=f"r{len(referents) + 1}", + is_current_surface=( + current_thread is not None + and thing.channel_id == current_thread[0] + and thing.thread_ts == current_thread[1] + ), + ) + ) + return Catalog(referents=tuple(referents), complete=complete) +``` + +```python +# src/fateforger/referents/__init__.py (replace the file) +"""Standing things, offered as options to one judgement. + +Nothing in this package imports Slack. A provider takes an owner and a clock; +the catalog names what comes back; the resolver picks one, or none, or says it +cannot tell. It never says what to *do* -- that is the consumer's own judgement. +""" + +from .catalog import Catalog, build_catalog +from .descriptor import GIST_LIMIT, Referent, StandingThing +from .provider import ReferentProvider + +__all__ = [ + "GIST_LIMIT", + "Catalog", + "Referent", + "ReferentProvider", + "StandingThing", + "build_catalog", +] +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_catalog.py -q` +Expected: PASS (8 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/fateforger/referents/provider.py src/fateforger/referents/catalog.py src/fateforger/referents/__init__.py tests/unit/test_referent_catalog.py +git commit -m "feat(referents): one provider seam, gathered concurrently, ids minted by the host" +``` + +--- + +### Task 3: The resolver + +**Files:** +- Create: `src/fateforger/referents/resolver.py` +- Modify: `src/fateforger/referents/__init__.py` +- Test: `tests/unit/test_referent_resolver.py` + +**Interfaces:** +- Consumes: `Catalog`, `Referent` from Tasks 1–2. +- Produces: + - `Resolved(catalog_complete: bool, referent: Referent)`, + `Ambiguous(catalog_complete: bool, candidates: tuple[Referent, ...])`, + `NoReferent(catalog_complete: bool)`; `Resolution = Resolved | Ambiguous | NoReferent`. + - `class ReferentResolver: def __init__(self, model_client); async def resolve(*, catalog: Catalog, message: str, as_of: datetime) -> Resolution` + - `RESOLVER_PROMPT: str`, `ReferentResolutionError(RuntimeError, ValueError)`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_referent_resolver.py +import json +from datetime import UTC, date, datetime +from types import SimpleNamespace + +import pytest + +from fateforger.referents import Catalog, Referent +from fateforger.referents.resolver import ( + Ambiguous, + NoReferent, + ReferentResolutionError, + ReferentResolver, + Resolved, +) + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +def _ref(ref_id: str, key: str, **over) -> Referent: + base = dict( + key=key, + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + ) + base.update(over) + return Referent(**base, ref_id=ref_id) + + +CATALOG = Catalog(referents=(_ref("r1", "k1"), _ref("r2", "k2"))) + + +class _Model: + """Records the prompt it was given and replays canned content.""" + + def __init__(self, content: str): + self._content = content + self.calls = [] + + async def create(self, messages, **kwargs): + self.calls.append((messages, kwargs)) + return SimpleNamespace(content=self._content) + + +async def test_a_named_candidate_comes_back_as_that_referent(): + model = _Model(json.dumps({"decision": "r2", "why": "names Saturday"})) + outcome = await ReferentResolver(model).resolve( + catalog=CATALOG, message="replan today", as_of=AS_OF + ) + assert isinstance(outcome, Resolved) + assert outcome.referent.key == "k2" + + +async def test_none_is_its_own_outcome_rather_than_a_null_referent(): + model = _Model(json.dumps({"decision": "none", "why": "new request"})) + outcome = await ReferentResolver(model).resolve( + catalog=CATALOG, message="what's the weather", as_of=AS_OF + ) + assert isinstance(outcome, NoReferent) + + +async def test_ambiguous_carries_the_candidates_so_a_card_need_not_rebuild_them(): + model = _Model(json.dumps({"decision": "ambiguous", "why": "two fit"})) + outcome = await ReferentResolver(model).resolve( + catalog=CATALOG, message="move the gym", as_of=AS_OF + ) + assert isinstance(outcome, Ambiguous) + assert [c.ref_id for c in outcome.candidates] == ["r1", "r2"] + + +async def test_an_id_the_host_never_minted_is_refused_rather_than_believed(): + model = _Model(json.dumps({"decision": "r9", "why": "invented"})) + with pytest.raises(ReferentResolutionError): + await ReferentResolver(model).resolve( + catalog=CATALOG, message="hello", as_of=AS_OF + ) + + +async def test_content_that_is_not_json_raises_rather_than_degrading(): + # Two behaviours with the wrong one silent is the shape CLAUDE.md's first + # rule exists to stop. + model = _Model("sorry, I can't help with that") + with pytest.raises(ReferentResolutionError): + await ReferentResolver(model).resolve( + catalog=CATALOG, message="hello", as_of=AS_OF + ) + + +async def test_an_empty_catalog_is_answered_without_asking_a_model_at_all(): + model = _Model(json.dumps({"decision": "none", "why": ""})) + outcome = await ReferentResolver(model).resolve( + catalog=Catalog(), message="plan tomorrow", as_of=AS_OF + ) + assert isinstance(outcome, NoReferent) + assert model.calls == [] + + +async def test_the_incomplete_flag_travels_onto_the_outcome(): + model = _Model(json.dumps({"decision": "none", "why": ""})) + partial = Catalog(referents=(_ref("r1", "k1"),), complete=False) + outcome = await ReferentResolver(model).resolve( + catalog=partial, message="hello", as_of=AS_OF + ) + assert outcome.catalog_complete is False + + +async def test_the_prompt_carries_every_candidate_and_the_users_words_verbatim(): + model = _Model(json.dumps({"decision": "r1", "why": ""})) + await ReferentResolver(model).resolve( + catalog=CATALOG, message="move the gym to the morning", as_of=AS_OF + ) + payload = json.loads(model.calls[0][0][1].content) + assert [c["ref_id"] for c in payload["standing_things"]] == ["r1", "r2"] + assert payload["message"] == "move the gym to the morning" + + +async def test_the_schema_offered_to_the_model_is_narrowed_to_the_minted_ids(): + model = _Model(json.dumps({"decision": "r1", "why": ""})) + await ReferentResolver(model).resolve( + catalog=CATALOG, message="hi", as_of=AS_OF + ) + schema = model.calls[0][1]["json_output"] + allowed = schema.model_fields["decision"].annotation + assert set(getattr(allowed, "__args__", ())) == {"r1", "r2", "none", "ambiguous"} + + +def test_no_outcome_can_express_an_action(): + # Resolve-then-act is enforced by the return type, not by prose: a consumer + # cannot fold the two judgements together without changing this. + for outcome in (Resolved, Ambiguous, NoReferent): + assert "action" not in outcome.model_fields + assert "decision" not in outcome.model_fields +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_resolver.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'fateforger.referents.resolver'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# src/fateforger/referents/resolver.py +"""One judgement: which standing thing, if any, is this message about. + +It never says what to *do* with the answer. That is a second judgement, run by +whichever consumer holds the state's own allowed decisions -- and keeping them +apart is what stops a resolution becoming a write with no human in between. +The enforcement is the return type: no outcome here has a field for an action. +""" + +from __future__ import annotations + +import json +from datetime import datetime +from typing import Literal + +from autogen_core.models import ChatCompletionClient, SystemMessage, UserMessage +from pydantic import BaseModel, ConfigDict, ValidationError, create_model + +from fateforger.core.llm_attribution import llm_attribution + +from .catalog import Catalog +from .descriptor import Referent + + +class ReferentResolutionError(RuntimeError, ValueError): + """Reading one message against one catalog failed. + + Also a ValueError because the schema violations it wraps already were one. + """ + + +class _Outcome(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + #: False when a provider failed. Carried on every outcome because a `none` + #: -- and equally a confident `Resolved` -- drawn from a partial catalog is + #: weaker evidence than one drawn from a whole one. + catalog_complete: bool = True + + +class Resolved(_Outcome): + referent: Referent + + +class Ambiguous(_Outcome): + candidates: tuple[Referent, ...] + + +class NoReferent(_Outcome): + pass + + +Resolution = Resolved | Ambiguous | NoReferent + + +#: The wording is measured. It moved 74/112 -> 95/112 draws on the frozen +#: fixture, entirely on the last sentence, which is the distinction between +#: continuing a day's plan and wanting something new scheduled. Changing this +#: text means re-running tests/evals/test_eval_referent_resolver.py. +RESOLVER_PROMPT = """You route one message a user just typed to a scheduling assistant. +The user has some *standing things*: plans for a particular day that already exist and can be continued. +Decide which standing thing, if any, this message is about. + +A message is about a standing thing when it continues, changes, questions, or ends THAT day's plan -- +including a question about what that plan says. +A message is about NONE of them when it asks for something new that no listed plan covers: a fact, +an errand, a reminder, or planning a day that is not listed. Wanting something scheduled is not the +same as continuing an existing plan for a day. +Choose "ambiguous" only when the message is clearly about one of the listed plans but two or more fit equally. + +Judge by meaning. Never invent identifiers. Return only JSON. +Answer with {"decision": "" | "none" | "ambiguous", "why": ""}.""" + + +class _Answer(BaseModel): + model_config = ConfigDict(extra="forbid") + + decision: str + why: str = "" + + +def _narrowed(catalog: Catalog) -> type[_Answer]: + """Offer the model exactly the ids the host minted, and nothing else.""" + ids = tuple(r.ref_id for r in catalog.referents) + return create_model( # type: ignore[call-overload] + "_NarrowedAnswer", + __base__=_Answer, + decision=(Literal[(*ids, "none", "ambiguous")], ...), + ) + + +class ReferentResolver: + def __init__(self, model_client: ChatCompletionClient) -> None: + self._model_client = model_client + + async def resolve( + self, *, catalog: Catalog, message: str, as_of: datetime + ) -> Resolution: + if not catalog.referents: + # Nothing to choose between. Asking anyway would spend a round trip + # to be told what the query already said. + return NoReferent(catalog_complete=catalog.complete) + + payload = { + "now": as_of.strftime("%Y-%m-%d %H:%M (%A)"), + "standing_things": [r.describe(as_of) for r in catalog.referents], + "message": message, + } + schema = _narrowed(catalog) + try: + with llm_attribution( + agent="referent_resolver", call_label="resolve", key="referents" + ): + result = await self._model_client.create( + [ + SystemMessage(content=RESOLVER_PROMPT), + UserMessage( + content=json.dumps(payload, ensure_ascii=False), + source="user", + ), + ], + json_output=schema, + ) + content = getattr(result, "content", None) + if not isinstance(content, str): + raise ReferentResolutionError( + "the resolver model returned no schema-bound JSON content" + ) + answer = schema.model_validate_json(content) + except ReferentResolutionError: + raise + except ValidationError as exc: + raise ReferentResolutionError( + f"the resolver model answered outside its schema: {exc}" + ) from exc + + if answer.decision == "none": + return NoReferent(catalog_complete=catalog.complete) + if answer.decision == "ambiguous": + return Ambiguous( + catalog_complete=catalog.complete, candidates=catalog.referents + ) + chosen = next( + (r for r in catalog.referents if r.ref_id == answer.decision), None + ) + if chosen is None: # pragma: no cover - the schema should prevent it + raise ReferentResolutionError( + f"the resolver named an id the host did not mint: {answer.decision!r}" + ) + return Resolved(catalog_complete=catalog.complete, referent=chosen) +``` + +Append to `src/fateforger/referents/__init__.py`'s imports and `__all__`: + +```python +from .resolver import ( + RESOLVER_PROMPT, + Ambiguous, + NoReferent, + ReferentResolutionError, + ReferentResolver, + Resolution, + Resolved, +) +``` + +and add `"RESOLVER_PROMPT", "Ambiguous", "NoReferent", "ReferentResolutionError", +"ReferentResolver", "Resolution", "Resolved"` to `__all__`. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_resolver.py -q` +Expected: PASS (10 tests) + +> If `test_an_id_the_host_never_minted_is_refused_rather_than_believed` passes for the wrong +> reason (Pydantic rejects `"r9"` against the narrowed `Literal` before your own check runs), +> that is correct behaviour — the schema is the first guard and the explicit check is the second. +> The test asserts the error type, not which guard fired. + +- [ ] **Step 5: Commit** + +```bash +git add src/fateforger/referents/resolver.py src/fateforger/referents/__init__.py tests/unit/test_referent_resolver.py +git commit -m "feat(referents): one call over the minted candidates, returning a referent and never an action" +``` + +--- + +### Task 4: The guard test + +**Files:** +- Create: `tests/unit/test_referents_never_match_gist.py` + +**Interfaces:** +- Consumes: the `fateforger.referents` package as written in Tasks 1–3. +- Produces: nothing importable. + +This is the standing guarantee that block titles are read by a model and never compared by code. +It is an AST test in the same spirit as the memory server's read-path guard. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_referents_never_match_gist.py +"""The gist is content, and content is only ever read by a judge. + +Block titles are the one field in the descriptor that came from a plan rather +than from a column, so they are the one field someone might be tempted to +compare, sort or filter on. CLAUDE.md forbids it, and a wrong pattern does not +raise -- it quietly returns the wrong answer forever. So it is asserted. +""" + +from __future__ import annotations + +import ast +from pathlib import Path + +import fateforger.referents as referents_pkg + +PACKAGE = Path(referents_pkg.__file__).parent + +#: Names that would mean code is reading the gist's *meaning* rather than +#: passing it along. `in`/`sorted`/`.lower()` over a title is the shape. +FORBIDDEN_METHODS = {"lower", "upper", "casefold", "strip", "split", "startswith", "endswith", "find", "index", "replace"} + + +def _gist_attribute_names(tree: ast.AST) -> list[ast.AST]: + return [ + node + for node in ast.walk(tree) + if isinstance(node, ast.Attribute) and node.attr == "gist" + ] + + +def test_no_module_in_the_package_imports_re(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + if isinstance(node, ast.Import): + assert all(a.name != "re" for a in node.names), path + if isinstance(node, ast.ImportFrom): + assert node.module != "re", path + + +def test_no_string_method_is_called_on_anything_reached_through_gist(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for call in (n for n in ast.walk(tree) if isinstance(n, ast.Call)): + func = call.func + if not isinstance(func, ast.Attribute): + continue + if func.attr not in FORBIDDEN_METHODS: + continue + # Walk the receiver looking for `.gist` + assert not _gist_attribute_names(func.value), f"{path}: {func.attr} on gist" + + +def test_the_gist_is_never_a_comparison_operand(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + if isinstance(node, ast.Compare): + operands = [node.left, *node.comparators] + for operand in operands: + assert not _gist_attribute_names(operand), f"{path}: gist compared" + + +def test_the_package_imports_no_slack(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + module = ( + node.module + if isinstance(node, ast.ImportFrom) + else None + ) + if module: + assert "slack" not in module.split("."), f"{path}: imports {module}" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Deliberately break it first, so the test is known not to be vacuous — several tests in this repo +were written this way and one was found hollow. Temporarily add to `descriptor.py`'s `describe`: + +```python + if self.gist and self.gist[0].lower() == "x": # TEMPORARY + pass +``` + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referents_never_match_gist.py -q` +Expected: FAIL on `test_no_string_method_is_called_on_anything_reached_through_gist`. +Then **remove those two lines**. + +- [ ] **Step 3: No implementation needed** + +The package as written in Tasks 1–3 already satisfies the guard. Re-run with the temporary lines +removed. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referents_never_match_gist.py -q` +Expected: PASS (4 tests) + +- [ ] **Step 5: Commit** + +```bash +git add tests/unit/test_referents_never_match_gist.py +git commit -m "test(referents): the gist is read by a judge and never compared by code" +``` + +--- + +### Task 5: The timeboxing provider + +**Files:** +- Create: `src/fateforger/referents/timeboxing.py` +- Modify: `src/fateforger/referents/__init__.py` +- Modify: `src/fateforger/slack_bot/timeboxing_session_store.py` (add one query method) +- Test: `tests/unit/test_referent_timeboxing_provider.py` + +**Interfaces:** +- Consumes: `StandingThing`, `ReferentProvider` from Tasks 1–2; + `SqlAlchemyTimeboxingSessionRepository`. +- Produces: + - `SqlAlchemyTimeboxingSessionRepository.standing_rows(*, owner_user_id, as_of, open_within, horizon) -> list[StandingSessionRow]` + where `StandingSessionRow` is a Pydantic model with + `session_key, status, planning_date, updated_at, revision, gist`. + - `TimeboxingReferentProvider(repository, *, open_within=timedelta(hours=12), horizon=timedelta(days=7))` + with `agent_type = "timeboxing_agent"`. + +**Design notes for the implementer:** + +- The predicate is the same family as the existing `standing_for` (line ~207): `open` and saved + recently, or `committed` with a day inside the horizon. **Cancelled never appears.** +- Add `created_at < as_of` so the catalog can never contain a row the current message minted. The + real constraint is ordering and Task 7 tests it; this predicate is the belt. +- `never_used` is `status == "open" and revision <= 1` — revision 1 is the opening turn + (`session_start.UNTOUCHED_REVISION`). +- **`updated_at` is written naive UTC by `save`**, so compare in the same basis: + `as_of.astimezone(UTC).replace(tzinfo=None)`. +- The gist comes from the snapshot's latest `validated_candidate` (its `rendered` block table) or + else the `skeleton` markdown headings. Reading it means loading `snapshot_json` for the rows + that qualify — acceptable because the row set is small (single digits). Return `()` when neither + artifact exists. +- `accepts` is `("revise the committed plan", "add a fact about the day")` for `committed`, and + `("continue planning", "answer the open question", "cancel")` for `open` — the same words the + session's own `_display_context` offers. +- The session key is `"{channel_id}:{thread_ts}"`, except a DM key ends `":dm"` and names the whole + DM rather than a thread. Split on the **last** `":"`; when the tail is `"dm"`, set `channel_id` + and leave `thread_ts` as `None`, because a DM key cannot mark a current surface. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_referent_timeboxing_provider.py +from datetime import UTC, date, datetime, timedelta + +from pydantic import BaseModel + +from fateforger.referents.timeboxing import TimeboxingReferentProvider + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +class _Row(BaseModel): + session_key: str + status: str + planning_date: date | None + updated_at: datetime + revision: int + gist: tuple[str, ...] = () + + +class _Repo: + def __init__(self, rows): + self._rows = rows + self.calls = [] + + async def standing_rows(self, *, owner_user_id, as_of, open_within, horizon): + self.calls.append((owner_user_id, as_of, open_within, horizon)) + return list(self._rows) + + +def _row(key, status, day, hours_ago=10.0, revision=7, gist=()): + return _Row( + session_key=key, + status=status, + planning_date=day, + updated_at=(AS_OF - timedelta(hours=hours_ago)).replace(tzinfo=None), + revision=revision, + gist=gist, + ) + + +async def test_a_committed_row_offers_revision_and_a_fact(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "committed", date(2026, 9, 5))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.status == "committed" + assert thing.accepts == ("revise the committed plan", "add a fact about the day") + + +async def test_an_open_row_offers_continuing_answering_and_cancelling(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "open", date(2026, 9, 7))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.accepts == ( + "continue planning", + "answer the open question", + "cancel", + ) + + +async def test_revision_one_is_the_opening_turn_so_the_row_is_never_used(): + provider = TimeboxingReferentProvider( + _Repo([_row("D1:dm", "open", None, revision=1)]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.never_used is True + + +async def test_a_worked_row_is_not_marked_never_used(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "open", date(2026, 9, 7), revision=7)]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.never_used is False + + +async def test_a_channel_key_splits_into_a_channel_and_a_thread(): + provider = TimeboxingReferentProvider( + _Repo([_row("C0AA6HC1RJL:1788571682.407949", "committed", date(2026, 9, 5))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.channel_id == "C0AA6HC1RJL" + assert thing.thread_ts == "1788571682.407949" + + +async def test_a_dm_key_names_the_whole_dm_so_it_has_no_thread(): + # `{channel}:dm` is thread-blind; treating "dm" as a thread_ts would let a + # DM row claim to be the surface a message arrived in. + provider = TimeboxingReferentProvider(_Repo([_row("D09A0RE9P7G:dm", "open", None)])) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.channel_id == "D09A0RE9P7G" + assert thing.thread_ts is None + + +async def test_the_gist_reaches_the_descriptor_unchanged(): + gist = ("PR1 Serious C2F work 10:30-12:00", "GYM1 Gym (chest) 18:00-19:00") + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "committed", date(2026, 9, 4), gist=gist)]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.gist == gist + + +async def test_the_asked_moment_and_the_windows_reach_the_repository(): + repo = _Repo([]) + provider = TimeboxingReferentProvider(repo) + await provider.standing(owner_user_id="U1", as_of=AS_OF) + owner, as_of, open_within, horizon = repo.calls[0] + assert owner == "U1" and as_of == AS_OF + assert open_within == timedelta(hours=12) and horizon == timedelta(days=7) + + +async def test_the_agent_type_travels_onto_every_descriptor(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "open", date(2026, 9, 7))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.agent_type == "timeboxing_agent" == provider.agent_type +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_timeboxing_provider.py -q` +Expected: FAIL — `ModuleNotFoundError: No module named 'fateforger.referents.timeboxing'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# src/fateforger/referents/timeboxing.py +"""Timeboxing sessions as standing things. + +The first provider. It answers the same question `standing_for` answers for the +nudger -- which sessions stand -- and returns descriptors instead of keys. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from datetime import datetime, timedelta +from typing import Protocol + +from .descriptor import StandingThing + +#: How long an untouched `open` session keeps counting as standing. The +#: nudger's own bound is one hour, which is right for "is a session under way" +#: and too tight for "what could this message be about": the Monday session was +#: last saved 69 minutes before the message that should have reached it. +DEFAULT_OPEN_WITHIN = timedelta(hours=12) + +#: How far ahead a committed day still counts. Matches the planning horizon. +DEFAULT_HORIZON = timedelta(days=7) + +#: The second half of a session key opened in a DM. It names the whole DM and +#: not a thread, so it can never be the surface a message arrived in. +DM_SUFFIX = "dm" + +_COMMITTED_ACCEPTS = ("revise the committed plan", "add a fact about the day") +_OPEN_ACCEPTS = ("continue planning", "answer the open question", "cancel") + +#: The opening turn's revision (`session_start.UNTOUCHED_REVISION`). Anything +#: above it is the user's own work. +UNTOUCHED_REVISION = 1 + + +class _StandingRows(Protocol): + async def standing_rows( + self, + *, + owner_user_id: str, + as_of: datetime, + open_within: timedelta, + horizon: timedelta, + ) -> Sequence: ... + + +class TimeboxingReferentProvider: + """Standing timeboxing sessions, as descriptors.""" + + agent_type = "timeboxing_agent" + + def __init__( + self, + repository: _StandingRows, + *, + open_within: timedelta = DEFAULT_OPEN_WITHIN, + horizon: timedelta = DEFAULT_HORIZON, + ) -> None: + self._repository = repository + self._open_within = open_within + self._horizon = horizon + + async def standing( + self, *, owner_user_id: str, as_of: datetime + ) -> Sequence[StandingThing]: + rows = await self._repository.standing_rows( + owner_user_id=owner_user_id, + as_of=as_of, + open_within=self._open_within, + horizon=self._horizon, + ) + return [self._describe(row, as_of=as_of) for row in rows] + + def _describe(self, row, *, as_of: datetime) -> StandingThing: + channel_id, thread_ts = _split_session_key(row.session_key) + committed = row.status == "committed" + return StandingThing( + key=row.session_key, + agent_type=self.agent_type, + kind="a plan for one day", + day=row.planning_date, + status=row.status, + never_used=( + row.status == "open" and row.revision <= UNTOUCHED_REVISION + ), + last_activity=row.updated_at.replace(tzinfo=as_of.tzinfo), + accepts=_COMMITTED_ACCEPTS if committed else _OPEN_ACCEPTS, + gist=tuple(row.gist), + channel_id=channel_id, + thread_ts=thread_ts, + ) + + +def _split_session_key(session_key: str) -> tuple[str | None, str | None]: + """`{channel}:{thread_ts}`, or `{channel}:dm` which names no thread. + + Identifiers this system minted, so splitting them is arithmetic and not a + reading of anything the user wrote. + """ + channel, _, tail = session_key.rpartition(":") + if not channel: + return None, None + if tail == DM_SUFFIX: + return channel, None + return channel, tail +``` + +Add to `src/fateforger/referents/__init__.py`: + +```python +from .timeboxing import TimeboxingReferentProvider +``` + +and `"TimeboxingReferentProvider"` to `__all__`. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_timeboxing_provider.py -q` +Expected: PASS (9 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/fateforger/referents/timeboxing.py src/fateforger/referents/__init__.py tests/unit/test_referent_timeboxing_provider.py +git commit -m "feat(referents): timeboxing sessions as the first provider" +``` + +--- + +### Task 6: `standing_rows` on the repository + +**Files:** +- Modify: `src/fateforger/slack_bot/timeboxing_session_store.py` +- Test: `tests/unit/test_standing_rows_query.py` (create) + +**Interfaces:** +- Consumes: `_TimeboxingSessionState`, `_StoredSessionEnvelope` (module-private, same file). +- Produces: `StandingSessionRow` (exported from the store module) and + `SqlAlchemyTimeboxingSessionRepository.standing_rows(...)` as declared in Task 5. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_standing_rows_query.py +"""The query half of the catalog: which sessions stand, as arithmetic. + +Per the routing clause, *which rows stand* is a guarantee and belongs in code +with a test beside it. Only *which standing one a message is about* is a +judgement. Keeping them apart is what stops someone replacing a correct query +with a classifier and calling it progress. +""" + +from datetime import UTC, date, datetime, timedelta + +import pytest +import pytest_asyncio +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from fateforger.slack_bot.timeboxing_session_store import ( + SqlAlchemyTimeboxingSessionRepository, + _Base, + _TimeboxingSessionState, +) + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) +NAIVE = AS_OF.replace(tzinfo=None) + + +@pytest_asyncio.fixture +async def repo(): + engine = create_async_engine("sqlite+aiosqlite:///:memory:") + async with engine.begin() as conn: + await conn.run_sync(_Base.metadata.create_all) + maker = async_sessionmaker(engine, expire_on_commit=False) + yield SqlAlchemyTimeboxingSessionRepository(maker), maker + await engine.dispose() + + +async def _insert(maker, **over): + row = dict( + session_key="C1:1.0", + owner_user_id="U1", + revision=7, + status="open", + planning_date=date(2026, 9, 5), + snapshot_json="{}", + created_at=NAIVE - timedelta(hours=20), + updated_at=NAIVE - timedelta(hours=1), + ) + row.update(over) + async with maker() as session: + session.add(_TimeboxingSessionState(**row)) + await session.commit() + + +async def test_a_recently_saved_open_session_stands(repo): + repository, maker = repo + await _insert(maker) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert [r.session_key for r in rows] == ["C1:1.0"] + + +async def test_a_stale_open_session_does_not(repo): + repository, maker = repo + await _insert(maker, updated_at=NAIVE - timedelta(hours=30)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_a_committed_day_inside_the_horizon_stands(repo): + repository, maker = repo + await _insert(maker, status="committed", updated_at=NAIVE - timedelta(hours=30)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert [r.status for r in rows] == ["committed"] + + +async def test_a_committed_day_in_the_past_does_not(repo): + repository, maker = repo + await _insert(maker, status="committed", planning_date=date(2026, 9, 1)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_a_cancelled_session_never_stands(repo): + repository, maker = repo + await _insert(maker, status="cancelled") + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_another_users_session_never_stands(repo): + repository, maker = repo + await _insert(maker, owner_user_id="U2") + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_a_row_created_after_the_asked_moment_is_excluded(repo): + # The catalog must never contain the row the current message minted. Task 7 + # guarantees the ordering; this is the belt. + repository, maker = repo + await _insert(maker, created_at=NAIVE + timedelta(minutes=1)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_the_gist_comes_from_the_candidates_rendered_blocks(repo): + repository, maker = repo + snapshot = { + "envelope_version": 1, + "snapshot": { + "artifacts": [ + { + "kind": "validated_candidate", + "revision": 1, + "payload": { + "rendered": ( + "blocks[2]{H,own,type,summary,ST,ET,mode,dur}:\n" + "PR1,tmbx,C,Serious C2F work,10:30,12:00,fs,PT1H30M\n" + "GYM1,tmbx,H,Gym (chest),18:00,19:00,fs,PT1H" + ) + }, + } + ] + }, + "outcomes": {}, + } + import json + + await _insert(maker, snapshot_json=json.dumps(snapshot)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == ( + "Serious C2F work 10:30-12:00", + "Gym (chest) 18:00-19:00", + ) + + +async def test_a_session_with_no_plan_yet_has_an_empty_gist(repo): + repository, maker = repo + await _insert(maker, snapshot_json="{}") + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == () +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_standing_rows_query.py -q` +Expected: FAIL — `ImportError: cannot import name 'StandingSessionRow'` / +`AttributeError: 'SqlAlchemyTimeboxingSessionRepository' object has no attribute 'standing_rows'` + +- [ ] **Step 3: Write minimal implementation** + +Add to `src/fateforger/slack_bot/timeboxing_session_store.py`: + +```python +class StandingSessionRow(BaseModel): + """One session that stands, from the indexed columns plus its plan's gist. + + `gist` is the only part that reads `snapshot_json`, and it is read to be + *shown to a judge*, never compared. The row set is single digits, so the + cost of loading those snapshots is bounded. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + session_key: str + status: str + planning_date: date | None + updated_at: datetime + revision: int + gist: tuple[str, ...] = () +``` + +and the method on `SqlAlchemyTimeboxingSessionRepository`: + +```python + async def standing_rows( + self, + *, + owner_user_id: str, + as_of: datetime, + open_within: timedelta, + horizon: timedelta, + ) -> list[StandingSessionRow]: + """Which sessions stand for this owner AT `as_of`. + + Same predicate family as `standing_for`, which answers this for the + nudger and returns keys. This returns rows a catalog can describe. + + `created_at < as_of` keeps a row the current message minted out of its + own catalog. `updated_at` is written naive UTC by `save`, so both bounds + are compared in that basis. + """ + moment = as_of.astimezone(UTC).replace(tzinfo=None) + since = moment - open_within + async with self._sessionmaker() as session: + result = await session.execute( + select( + _TimeboxingSessionState.session_key, + _TimeboxingSessionState.status, + _TimeboxingSessionState.planning_date, + _TimeboxingSessionState.updated_at, + _TimeboxingSessionState.revision, + _TimeboxingSessionState.snapshot_json, + ) + .where( + _TimeboxingSessionState.owner_user_id == owner_user_id, + _TimeboxingSessionState.created_at < moment, + or_( + (_TimeboxingSessionState.status == "open") + & (_TimeboxingSessionState.updated_at >= since), + (_TimeboxingSessionState.status == "committed") + & (_TimeboxingSessionState.planning_date >= moment.date()) + & ( + _TimeboxingSessionState.planning_date + <= (moment + horizon).date() + ), + ), + ) + .order_by(_TimeboxingSessionState.updated_at.desc()) + ) + rows = result.all() + return [ + StandingSessionRow( + session_key=key, + status=status, + planning_date=planning_date, + updated_at=updated_at, + revision=revision, + gist=_plan_gist(snapshot_json), + ) + for key, status, planning_date, updated_at, revision, snapshot_json in rows + ] +``` + +and the module-level helper: + +```python +def _plan_gist(snapshot_json: str) -> tuple[str, ...]: + """A few of the plan's own block titles, with their times. + + Read from the latest validated candidate's rendered block table, whose + columns are `H,own,type,summary,ST,ET,mode,dur` -- a table this system + generated, so taking the summary and the two clocks out of it is arithmetic + over our own format and not a reading of anything the user wrote. Anything + unparseable yields no gist rather than a guess. + """ + try: + envelope = json.loads(snapshot_json) + artifacts = envelope["snapshot"]["artifacts"] + except (ValueError, KeyError, TypeError): + return () + rendered = next( + ( + artifact.get("payload", {}).get("rendered") + for artifact in reversed(artifacts) + if artifact.get("kind") == "validated_candidate" + ), + None, + ) + if not isinstance(rendered, str): + return () + entries: list[str] = [] + for line in rendered.splitlines()[1:]: # first line is the column header + fields = line.split(",") + if len(fields) < 6: + continue + summary, start, end = fields[3], fields[4], fields[5] + entries.append(f"{summary} {start}-{end}") + return tuple(entries) +``` + +Add `import json` and `timedelta` to the module's imports if absent, and `ConfigDict` to the +Pydantic import. + +> **Note on `_plan_gist` and the no-parsing rule.** `rendered` is a table this system generated +> with a fixed column order; splitting it is arithmetic over our own format, exactly like reading +> a JSON key. It never decides what any of it *means*. The AST guard in Task 4 covers the +> `referents` package; this helper lives in the store, and its own test above is the guard. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_standing_rows_query.py -q` +Expected: PASS (9 tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/fateforger/slack_bot/timeboxing_session_store.py tests/unit/test_standing_rows_query.py +git commit -m "feat(timeboxing): standing_rows answers which sessions stand, with each plan's gist" +``` + +--- + +### Task 7: The routing rung + +**Files:** +- Modify: `src/fateforger/slack_bot/handlers.py` (the ordered resolvers inside + `route_slack_event`, ~line 2630–2712, and the `would_alias_root` branch ~line 3168) +- Test: `tests/unit/test_referent_rung_routing.py` (create) + +**Interfaces:** +- Consumes: `build_catalog`, `ReferentResolver`, `Resolved`, `Ambiguous`, `NoReferent`, + `TimeboxingReferentProvider`. +- Produces: nothing importable; behaviour only. + +**Where it goes, exactly.** In `route_slack_event`, the resolver block currently runs +`planning.owns_thread` first, then the session-store lookup, both only `if thread_ts`. The rung is +a new step **after** that whole block and **before** `would_alias_root` decides to open a session. +It only runs when no structural resolver has claimed the message — that is, `binding is None` and +the block above did not set `agent_type` to `timeboxing_agent`. + +**What it does with each outcome:** + +| outcome | behaviour | +|---|---| +| `Resolved` | set `agent_type` to the referent's `agent_type`, set a focus redirect to the referent's channel and thread, and post a one-line pointer where the user typed. The referent's own surface handles the message. | +| `Ambiguous` | post a card in the origin naming the candidates and stop. No session is opened. | +| `NoReferent` | fall through unchanged — today's behaviour, receptionist or channel default. | +| resolver raises | log, `record_error`, fall through unchanged. No pattern fallback. | + +**The ordering guarantee.** The rung must run before `_begin_timeboxing_session_surface`, so the +catalog can never contain the row this message minted. Test it directly rather than trusting +`created_at < as_of`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/unit/test_referent_rung_routing.py +"""The rung: a message with no structural owner reaches the session it is about. + +The incident this closes: 2026-09-05 13:51, "can you replan today so the gym is +before dinner?" typed top-level in #plan-sessions opened a fresh five-stage +session for a day committed at 01:41. +""" + +from datetime import UTC, date, datetime +from types import SimpleNamespace + +import pytest + +pytest.importorskip("autogen_agentchat") + +from fateforger.referents import Catalog, Referent +from fateforger.referents.resolver import Ambiguous, NoReferent, Resolved + + +def _ref(ref_id="r1", key="C0AA6HC1RJL:1788571682.407949"): + return Referent( + key=key, + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + channel_id="C0AA6HC1RJL", + thread_ts="1788571682.407949", + ref_id=ref_id, + ) + + +class _Resolver: + def __init__(self, outcome): + self._outcome = outcome + self.calls = [] + + async def resolve(self, *, catalog, message, as_of): + self.calls.append((catalog, message, as_of)) + if isinstance(self._outcome, BaseException): + raise self._outcome + return self._outcome + + +async def test_a_committed_day_is_reached_instead_of_a_second_session_being_opened( + routing_harness, +): + harness = routing_harness(resolver=_Resolver(Resolved(referent=_ref()))) + await harness.route_top_level("can you replan today so the gym is before dinner?") + assert harness.sessions_opened == [] + assert harness.delivered_to == "C0AA6HC1RJL:1788571682.407949" + + +async def test_the_origin_gets_a_pointer_to_the_thread_that_took_it(routing_harness): + harness = routing_harness(resolver=_Resolver(Resolved(referent=_ref()))) + await harness.route_top_level("replan today") + assert any("1788571682" in text for text in harness.origin_messages) + + +async def test_ambiguity_asks_and_opens_nothing(routing_harness): + harness = routing_harness( + resolver=_Resolver(Ambiguous(candidates=(_ref("r1"), _ref("r2", "C1:2.0")))) + ) + await harness.route_top_level("move the gym to the morning") + assert harness.sessions_opened == [] + assert harness.delivered_to is None + assert harness.origin_messages, "the user must be asked which one" + + +async def test_none_falls_through_to_todays_behaviour(routing_harness): + harness = routing_harness(resolver=_Resolver(NoReferent())) + await harness.route_top_level("plan tomorrow") + assert harness.sessions_opened == ["C0AA6HC1RJL"] + + +async def test_a_resolver_failure_falls_through_rather_than_guessing(routing_harness): + harness = routing_harness(resolver=_Resolver(RuntimeError("model down"))) + await harness.route_top_level("replan today") + assert harness.sessions_opened == ["C0AA6HC1RJL"] + + +async def test_the_catalog_is_built_before_any_session_is_opened(routing_harness): + # The ordering IS the guarantee; `created_at < as_of` is only the belt. + harness = routing_harness(resolver=_Resolver(NoReferent())) + await harness.route_top_level("plan tomorrow") + assert harness.event_order.index("catalog") < harness.event_order.index("open") + + +async def test_a_thread_a_structural_resolver_already_claimed_never_reaches_the_rung( + routing_harness, +): + # Structural ownership is a fact and always beats a judgement (#310). + resolver = _Resolver(Resolved(referent=_ref())) + harness = routing_harness(resolver=resolver, planning_owns_thread=True) + await harness.route_thread_reply("is it planned?") + assert resolver.calls == [] +``` + +The harness fixture belongs in `tests/unit/conftest.py`; model it on the fakes already in +`tests/unit/test_slack_timeboxing_routing.py` (`_FakeRuntime`, `_FakeClient`, +`_PlanningReplyHandler`), extended to record `sessions_opened`, `delivered_to`, +`origin_messages` and `event_order`. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_rung_routing.py -q` +Expected: FAIL — fixture missing, then assertion failures once the harness exists. + +- [ ] **Step 3: Write minimal implementation** + +Wire the rung into `route_slack_event` as described in the table above. Keep it to one helper, +`_resolve_referent(...)`, defined near the other resolver helpers, so the route body gains a call +rather than a block. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests/unit/test_referent_rung_routing.py tests/unit/test_slack_timeboxing_routing.py -q` +Expected: PASS, including the existing routing tests unchanged. + +- [ ] **Step 5: Commit** + +```bash +git add src/fateforger/slack_bot/handlers.py tests/unit/test_referent_rung_routing.py tests/unit/conftest.py +git commit -m "feat(slack): a follow-up reaches the session it is about, instead of minting a second one (#345)" +``` + +--- + +### Task 8: The eval + +**Files:** +- Create: `tests/evals/test_eval_referent_resolver.py` + +**Interfaces:** +- Consumes: everything above, plus `build_autogen_chat_client("timeboxing_judge")`. +- Produces: nothing importable. + +Two frozen fixtures, both lifted from `scripts/spikes/referent_resolver_spike.py`, which already +holds the rows and the gists verbatim. Copy them; do not re-read the live store. + +**Thresholds, from the measured runs.** The best arm scored 95/112 and 93/112 on two runs of the +identical configuration, so the noise floor is about 2 draws in 112. Gate at a level those runs +clear comfortably and a real regression does not: **≥ 85% of draws correct overall**, and +**every `none` probe unanimous** (all four scored 8/8 with the gist, and they are the cases where +a wrong answer becomes a duplicate session at a creating door). + +- [ ] **Step 1: Write the eval** + +```python +# tests/evals/test_eval_referent_resolver.py +"""Referent resolution quality on the pin, against two frozen incidents. + +**Why frozen.** Reading `timeboxing_session_states` live measures the ledger's +drift, not the model: two runs of one spike forty minutes apart drew different +candidate sets because a peer committed a plan mid-run, and the store keeps no +history, so a descriptor built from it reads *current* status while claiming to +describe an earlier moment. The rows below are inline and dated. Do not replace +them with a query -- that is the ceremony this docstring exists to protect. + +**Labels were reviewed blind.** Three were wrong on first writing, all in the +same direction: the model reading state and the label reading an assumption. +"is it planned?" resolves to the standing session (what happens next is the +consumer's judgement, not this one's), and "move the gym to the morning" +resolves to the only plan that contains a gym. + +n = 8 draws per case, asserted on the rate. + + set -a; source .env; set +a + PYTHONPATH=src .venv/bin/python -m pytest tests/evals/test_eval_referent_resolver.py -m slow -q +""" + +from __future__ import annotations + +import asyncio +from datetime import UTC, date, datetime, timedelta + +import pytest + +from fateforger.llm import build_autogen_chat_client +from fateforger.referents import Catalog, Referent +from fateforger.referents.resolver import ( + Ambiguous, + NoReferent, + ReferentResolver, + Resolved, +) + +pytestmark = [pytest.mark.slow, pytest.mark.asyncio] + +DRAWS = 8 +TZ = UTC + +# --- fixture one: the incident, 2026-09-05 13:51 -------------------------- +INCIDENT_AT = datetime(2026, 9, 5, 11, 51, tzinfo=TZ) +INCIDENT = ( + # (ref_id, day, status, never_used, hours_ago, gist) + ("r1", date(2026, 9, 7), "open", False, 1.1, ()), + ("r2", None, "open", True, 10.1, ()), + ( + "r3", + date(2026, 9, 5), + "committed", + False, + 10.2, + ( + "Wake up 11:00-11:00", + "Breakfast (oats) 11:00-11:30", + "Buy a new white shirt 11:30-12:30", + "Gym session 13:00-14:00", + "Pay taxes 14:15-15:15", + "Lunch 15:15-15:45", + "Dinner 19:30-20:30", + "Evening shutdown ritual 20:30-21:30", + "Sleep 23:00-23:00", + ), + ), +) +INCIDENT_CASES = [ + ("can you replan today so the gym is before dinner?", "r3"), + ("plan saturday", "r3"), + ("let's plan monday", "r1"), + ("actually make monday start at 10", "r1"), + ("I'll wake up at 11 on monday", "r1"), + ("what did we decide about dinner?", "r3"), + ("is it planned?", "r3"), + ("plan tomorrow", "none"), + ("what's the weather tomorrow", "none"), + ("add a dentist appointment on tuesday", "none"), + ("remind me to pay taxes", "none"), +] + +# --- fixture two: #275, two sessions for one Friday, 2026-09-03 12:15 ----- +PARALLEL_AT = datetime(2026, 9, 3, 10, 15, tzinfo=TZ) +PARALLEL = ( + ( + "r1", + date(2026, 9, 4), + "open", + False, + 0.0, + ( + "Serious C2F work 10:30-12:00", + "Kapper 12:00-12:30", + "Lunch 12:30-13:00", + "Validate agent demos 13:00-13:45", + "Finances 13:45-14:30", + "Oats 16:00-16:15", + "Gym (chest) 18:00-19:00", + "Dinner 19:15-20:00", + ), + ), + ( + "r2", + date(2026, 9, 4), + "open", + False, + 0.1, + ( + "PR review - stage-UX, ends 11:30", + "Kapper 12:00-12:30", + "Lunch ~12:30", + "Deep work - constraint memory design, 90 minutes", + "Prepare the Monday investor call 15:00-16:00", + "Oats 16:00", + "Gym 18:00 (chest)", + "Dinner ~19:30", + ), + ), +) +PARALLEL_CASES = [ + ("move PR1 later", "r1"), + ("move the finances block later", "r1"), + ("push the investor call prep later", "r2"), + ("move the gym to the morning", "ambiguous"), + ("cancel that session", "ambiguous"), + ("plan sunday", "none"), + # False-positive probes: none of these exist in either plan. A wrong answer + # here is a duplicate session at a door that creates, so they are gated + # unanimously. + ("move the dentist earlier", "none"), + ("push the standup to 11", "none"), + ("can you shorten the school run", "none"), + ("move the physio appointment to friday morning", "none"), +] +PROBES = { + "move the dentist earlier", + "push the standup to 11", + "can you shorten the school run", + "move the physio appointment to friday morning", +} + + +def _catalog(rows, at: datetime) -> Catalog: + return Catalog( + referents=tuple( + Referent( + key=f"C1:{ref_id}", + agent_type="timeboxing_agent", + kind="a plan for one day", + day=day, + status=status, + never_used=never_used, + last_activity=at - timedelta(hours=hours_ago), + accepts=( + ("revise the committed plan", "add a fact about the day") + if status == "committed" + else ("continue planning", "answer the open question", "cancel") + ), + gist=gist, + ref_id=ref_id, + ) + for ref_id, day, status, never_used, hours_ago, gist in rows + ) + ) + + +def _label(outcome) -> str: + if isinstance(outcome, Resolved): + return outcome.referent.ref_id + if isinstance(outcome, Ambiguous): + return "ambiguous" + if isinstance(outcome, NoReferent): + return "none" + raise AssertionError(outcome) + + +async def _draws(resolver, catalog, message, at) -> list[str]: + outcomes = await asyncio.gather( + *( + resolver.resolve(catalog=catalog, message=message, as_of=at) + for _ in range(DRAWS) + ) + ) + return [_label(o) for o in outcomes] + + +@pytest.mark.parametrize( + "rows, cases, at", + [(INCIDENT, INCIDENT_CASES, INCIDENT_AT), (PARALLEL, PARALLEL_CASES, PARALLEL_AT)], + ids=["incident-2026-09-05", "two-parallel-sessions-2026-09-04"], +) +async def test_resolution_quality(rows, cases, at): + resolver = ReferentResolver(build_autogen_chat_client("timeboxing_judge")) + catalog = _catalog(rows, at) + + results = await asyncio.gather( + *(_draws(resolver, catalog, message, at) for message, _ in cases) + ) + + hits = 0 + failures = [] + for (message, expected), drawn in zip(cases, results): + correct = sum(1 for d in drawn if d == expected) + hits += correct + print(f" {correct}/{DRAWS} {expected:<10} {message}") + if message in PROBES and correct != DRAWS: + failures.append( + f"probe {message!r} expected {expected} unanimously, drew {drawn}" + ) + + total = len(cases) * DRAWS + rate = hits / total + print(f" == {hits}/{total} draws ({rate:.0%})") + assert not failures, "\n".join(failures) + assert rate >= 0.85, f"{hits}/{total} draws correct; the measured arm scored ~0.85+" +``` + +- [ ] **Step 2: Run it** + +Run: `set -a; source .env; set +a; PYTHONPATH=src .venv/bin/python -m pytest tests/evals/test_eval_referent_resolver.py -m slow -q -s` +Expected: PASS, and read the printed per-case counts — they are the numbers to compare against the +next prompt change. + +- [ ] **Step 3: Confirm it is not vacuous** + +Break the prompt on purpose: delete the last sentence of `RESOLVER_PROMPT` (*"Wanting something +scheduled is not the same as continuing an existing plan for a day."*) and re-run. That sentence +is what moved 74/112 to 95/112, so the eval must fail. Restore it. + +- [ ] **Step 4: Confirm the fast suite is unaffected** + +Run: `PYTHONPATH=src .venv/bin/python -m pytest tests -m "not slow" -q` +Expected: PASS, with the eval deselected. + +- [ ] **Step 5: Commit** + +```bash +git add tests/evals/test_eval_referent_resolver.py +git commit -m "test(referents): resolution quality on the pin, two frozen incidents, n=8" +``` + +--- + +### Task 9: The strain log and the docs ticket + +**Files:** +- Create: `docs/superpowers/notes/2026-09-referent-interface-strain.md` +- Modify: `src/fateforger/referents/__init__.py` (module docstring pointer only) + +**Interfaces:** none. + +This is the ticket's compounding deliverable, and Hugo asked for it by name: *"while it gets +implemented we use it to inform the interfaces and components for the marshal."* The log is +written **during** Tasks 1–8, not reconstructed afterwards — use the `implementation-notes` skill. + +- [ ] **Step 1: Write the log as you go** + +One entry per strain, each answering three things: + +```markdown +### + +**Where:** Task N, `path/to/file.py` +**What timeboxing needed that the protocol did not offer:** … +**What the resolver wanted that a provider could not supply:** … +**Timeboxing-specific, or general?** … (and why) +``` + +Seed it with the four already known from the spike and the design: + +- `channel_id` / `thread_ts` are Slack-shaped and sit on a protocol that claims not to know about + Slack. General question: does a locator belong on the descriptor at all, or should a provider + render its own link? +- `gist` is a list of strings, which suits blocks-in-a-day. A GTD session's standing content is a + list of tasks with due dates and projects — same slot, different shape. Does `gist` stay + `tuple[str, ...]` rendered by the provider, or become structured? +- `accepts` duplicates the state machine's own `_display_context` table in prose. Two places that + must agree. +- `day` is on the descriptor because a timeboxing session *is* a day. A marshal's standing thing + may have no day at all, and `day: None` currently means "no day locked yet" rather than "days do + not apply here". + +- [ ] **Step 2: Raise the docs ticket** + +Per `CLAUDE.md`, every implementation round ends with a ticket for the docs, picked up by a Sonnet +subagent and merged with the PR. Open it against #345 naming: `docs/architecture/` needs the +provider contract, and `src/fateforger/referents/AGENTS.md` needs writing. + +- [ ] **Step 3: Commit** + +```bash +git add docs/superpowers/notes/2026-09-referent-interface-strain.md src/fateforger/referents/__init__.py +git commit -m "docs(referents): the interface-strain log the marshal's provider inherits" +``` + +--- + +## Self-Review + +**Spec coverage.** Descriptor → Task 1. Provider protocol and catalog, including concurrency, +failure isolation and `is_current_surface` → Task 2. Resolver, the union, the narrowed schema and +the prompt → Task 3. "Titles are read, never matched" → Task 4. Timeboxing provider and the +`as_of`/`created_at` handling → Tasks 5–6. "Which rows stand is a query" → Task 6. The rung, the +ordering guarantee and structural-ownership-first → Task 7. Both frozen fixtures, n=8, the +false-positive probes and the frozen-fixture rationale → Task 8. Compounding → Task 9. + +Not covered by any task, deliberately, and all listed as out of scope in the spec: `none`-never- +creates (owned by the writing consumer, #352), the second judgement (#352), `none`→escalate +(#337), the dropped facts at `ConfirmPlanningDay` (separate issue). + +**Type consistency.** `StandingThing` fields are identical in Tasks 1, 2 and 5. +`build_catalog(providers, *, owner_user_id, as_of, current_thread)` is used with those exact +keywords in Tasks 2 and 7. `standing(*, owner_user_id, as_of)` matches between Tasks 2 and 5. +`standing_rows(*, owner_user_id, as_of, open_within, horizon)` is declared in Task 5 and +implemented in Task 6 with the same signature. `Resolution` members carry `catalog_complete` in +Tasks 3 and 7. + +**Known risk in Task 7.** It is the only task touching a 5,000-line file, and the routing tests +there are the ones most likely to surprise. If the harness fixture proves awkward, that is a +signal to extract the resolver block into its own function first rather than to weaken the test. From d51f245dcc698cce32ccdfc793f35778d47ea7bf Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:18:00 +0200 Subject: [PATCH 03/20] feat(referents): the descriptor a judge sees, with the plan's own blocks and times Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/referents/__init__.py | 10 ++++ src/fateforger/referents/descriptor.py | 75 ++++++++++++++++++++++++++ tests/unit/test_referent_descriptor.py | 68 +++++++++++++++++++++++ 3 files changed, 153 insertions(+) create mode 100644 src/fateforger/referents/__init__.py create mode 100644 src/fateforger/referents/descriptor.py create mode 100644 tests/unit/test_referent_descriptor.py diff --git a/src/fateforger/referents/__init__.py b/src/fateforger/referents/__init__.py new file mode 100644 index 00000000..ba9711c8 --- /dev/null +++ b/src/fateforger/referents/__init__.py @@ -0,0 +1,10 @@ +"""Standing things, offered as options to one judgement. + +Nothing in this package imports Slack. A provider takes an owner and a clock; +the catalog names what comes back; the resolver picks one, or none, or says it +cannot tell. It never says what to *do* -- that is the consumer's own judgement. +""" + +from .descriptor import GIST_LIMIT, Referent, StandingThing + +__all__ = ["GIST_LIMIT", "Referent", "StandingThing"] diff --git a/src/fateforger/referents/descriptor.py b/src/fateforger/referents/descriptor.py new file mode 100644 index 00000000..e9da780f --- /dev/null +++ b/src/fateforger/referents/descriptor.py @@ -0,0 +1,75 @@ +"""What one standing thing looks like to the judge. + +Every field here is minted by this system -- a session key, a status, a date, a +block title the planner wrote. Nothing is the user's prose, which is why the +whole descriptor may be handed to a model without any of it being *matched*. +""" + +from __future__ import annotations + +from datetime import date, datetime + +from pydantic import BaseModel, ConfigDict + +#: How many gist entries reach the model. A cap, not a ranking: the provider +#: supplies the plan's own order and the tail is dropped, because a twenty-block +#: day would otherwise crowd out the other candidates. +GIST_LIMIT = 12 + + +class StandingThing(BaseModel): + """One thing a provider says is standing, before the host names it.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + key: str + agent_type: str + kind: str + day: date | None + status: str + never_used: bool + last_activity: datetime + accepts: tuple[str, ...] + #: The plan's own block titles WITH their times. Half the resolving power is + #: the clock: a title alone cannot answer "push the investor call prep + #: later". Present to be READ by a model and never matched, sorted or + #: filtered on by code -- `test_referents_never_match_gist` guards that. + gist: tuple[str, ...] = () + #: Opaque locators the provider copies rather than interprets. A provider + #: whose things live elsewhere leaves them None and loses only the link and + #: `is_current_surface`. + channel_id: str | None = None + thread_ts: str | None = None + + +class Referent(StandingThing): + """A standing thing the catalog has named, as the judge sees it.""" + + #: Host-minted. Providers never set this; `build_catalog` does, so identity + #: stays with the host exactly as on every other surface in this repo. + ref_id: str + #: This candidate IS the surface the message arrived in. Structural -- a + #: thread id compared to a thread id -- and it exists because "cancel THAT + #: session" points away from where the speaker is. + is_current_surface: bool = False + + def describe(self, now: datetime) -> dict: + """The candidate as JSON for the prompt. Reproducible from `now`.""" + hours = round((now - self.last_activity).total_seconds() / 3600, 1) + described: dict = { + "ref_id": self.ref_id, + "kind": self.kind, + "day": ( + f"{self.day.isoformat()} ({self.day.strftime('%A')})" + if self.day is not None + else "no day locked yet" + ), + "status": self.status, + "opened_automatically_never_used": self.never_used, + "last_activity": f"{hours}h ago", + "accepts": list(self.accepts), + "is_the_conversation_this_message_arrived_in": self.is_current_surface, + } + if self.gist: + described["plan_contains"] = list(self.gist[:GIST_LIMIT]) + return described diff --git a/tests/unit/test_referent_descriptor.py b/tests/unit/test_referent_descriptor.py new file mode 100644 index 00000000..99424c30 --- /dev/null +++ b/tests/unit/test_referent_descriptor.py @@ -0,0 +1,68 @@ +from datetime import UTC, date, datetime + +import pytest +from pydantic import ValidationError + +from fateforger.referents import Referent, StandingThing + +NOW = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +def _thing(**over) -> StandingThing: + base = dict( + key="C1:123.456", + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + ) + base.update(over) + return StandingThing(**base) + + +def test_describe_names_the_weekday_because_a_bare_date_is_not_a_day_to_a_reader(): + ref = Referent(**_thing().model_dump(), ref_id="r1") + described = ref.describe(NOW) + assert described["day"] == "2026-09-05 (Saturday)" + assert described["ref_id"] == "r1" + + +def test_describe_reports_age_relative_to_the_asked_moment_not_the_wall_clock(): + ref = Referent(**_thing().model_dump(), ref_id="r1") + assert ref.describe(NOW)["last_activity"] == "10.2h ago" + + +def test_a_day_less_row_says_so_rather_than_omitting_the_field(): + ref = Referent(**_thing(day=None).model_dump(), ref_id="r1") + assert ref.describe(NOW)["day"] == "no day locked yet" + + +def test_never_used_is_a_field_the_model_sees_not_a_phrase_inside_status(): + # Measured: how it is carried is inside the noise floor, but a consumer + # must be able to filter and test on it. #352's door sees this as its + # common case because autostart pre-warms a session per planning event. + ref = Referent(**_thing(status="open", never_used=True).model_dump(), ref_id="r1") + described = ref.describe(NOW) + assert described["status"] == "open" + assert described["opened_automatically_never_used"] is True + + +def test_the_gist_is_capped_so_a_long_plan_cannot_dominate_the_prompt(): + ref = Referent( + **_thing(gist=tuple(f"B{i} block {i} 0{i}:00-0{i}:30" for i in range(20))).model_dump(), + ref_id="r1", + ) + assert len(ref.describe(NOW)["plan_contains"]) == 12 + + +def test_an_empty_gist_omits_the_key_rather_than_showing_an_empty_list(): + ref = Referent(**_thing().model_dump(), ref_id="r1") + assert "plan_contains" not in ref.describe(NOW) + + +def test_the_descriptor_is_frozen_and_refuses_unknown_fields(): + with pytest.raises(ValidationError): + StandingThing(**_thing().model_dump(), surprise="no") From 380973b800d3ef0e89ba1bbc8b6333a817f3c77d Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:22:38 +0200 Subject: [PATCH 04/20] feat(referents): one provider seam, gathered concurrently, ids minted by the host Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/referents/__init__.py | 11 ++- src/fateforger/referents/catalog.py | 75 ++++++++++++++++ src/fateforger/referents/provider.py | 34 +++++++ tests/unit/test_referent_catalog.py | 127 +++++++++++++++++++++++++++ 4 files changed, 246 insertions(+), 1 deletion(-) create mode 100644 src/fateforger/referents/catalog.py create mode 100644 src/fateforger/referents/provider.py create mode 100644 tests/unit/test_referent_catalog.py diff --git a/src/fateforger/referents/__init__.py b/src/fateforger/referents/__init__.py index ba9711c8..36ffc0ba 100644 --- a/src/fateforger/referents/__init__.py +++ b/src/fateforger/referents/__init__.py @@ -5,6 +5,15 @@ cannot tell. It never says what to *do* -- that is the consumer's own judgement. """ +from .catalog import Catalog, build_catalog from .descriptor import GIST_LIMIT, Referent, StandingThing +from .provider import ReferentProvider -__all__ = ["GIST_LIMIT", "Referent", "StandingThing"] +__all__ = [ + "GIST_LIMIT", + "Catalog", + "Referent", + "ReferentProvider", + "StandingThing", + "build_catalog", +] diff --git a/src/fateforger/referents/catalog.py b/src/fateforger/referents/catalog.py new file mode 100644 index 00000000..1568146a --- /dev/null +++ b/src/fateforger/referents/catalog.py @@ -0,0 +1,75 @@ +"""Gather every provider, name what comes back, say whether it is whole.""" + +from __future__ import annotations + +import asyncio +import logging +from collections.abc import Sequence +from datetime import datetime + +from pydantic import BaseModel, ConfigDict + +from fateforger.core.logging_config import record_error + +from .descriptor import Referent, StandingThing +from .provider import ReferentProvider + +logger = logging.getLogger(__name__) + + +class Catalog(BaseModel): + """What stands, named, plus whether anyone failed to answer.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + referents: tuple[Referent, ...] = () + #: False when a provider raised. A `none` drawn from a partial catalog is + #: not evidence that nothing stands, so a door that can create must ask + #: rather than create when this is False. + complete: bool = True + + +async def build_catalog( + providers: Sequence[ReferentProvider], + *, + owner_user_id: str, + as_of: datetime, + current_thread: tuple[str, str] | None = None, +) -> Catalog: + """Ask every provider concurrently and name the results `r1`, `r2`, ... + + Ids are minted here and never by a provider, so identity stays with the + host. `current_thread` is `(channel_id, thread_ts)` for the conversation the + message arrived in, compared as identifiers this system minted. + """ + results = await asyncio.gather( + *(p.standing(owner_user_id=owner_user_id, as_of=as_of) for p in providers), + return_exceptions=True, + ) + + referents: list[Referent] = [] + complete = True + for provider, result in zip(providers, results): + if isinstance(result, BaseException): + complete = False + logger.exception( + "referent provider %s failed for %s", + provider.agent_type, + owner_user_id, + exc_info=result, + ) + record_error(component="referent_catalog", error_type="provider_failure") + continue + for thing in result: + referents.append( + Referent( + **thing.model_dump(), + ref_id=f"r{len(referents) + 1}", + is_current_surface=( + current_thread is not None + and thing.channel_id == current_thread[0] + and thing.thread_ts == current_thread[1] + ), + ) + ) + return Catalog(referents=tuple(referents), complete=complete) diff --git a/src/fateforger/referents/provider.py b/src/fateforger/referents/provider.py new file mode 100644 index 00000000..d69fb0e8 --- /dev/null +++ b/src/fateforger/referents/provider.py @@ -0,0 +1,34 @@ +"""The whole seam: an owner, a clock, and what stands. + +One method. No Slack event, no channel, no focus manager, no route in scope -- +which is what lets the routing rung, a door that creates sessions, and the eval +all be consumers of the same catalog. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from datetime import datetime +from typing import Protocol, runtime_checkable + +from .descriptor import StandingThing + + +@runtime_checkable +class ReferentProvider(Protocol): + """Something that knows what one user currently has standing.""" + + #: Whose things these are. Travels onto every descriptor. + agent_type: str + + async def standing( + self, *, owner_user_id: str, as_of: datetime + ) -> Sequence[StandingThing]: + """What stands for this owner AT `as_of`. + + `as_of` is explicit and required rather than read from a clock inside. + The store keeps no history, so a descriptor otherwise reads *current* + state while claiming to describe an earlier moment -- which is how two + runs of one spike drew different candidate sets forty minutes apart. + """ + ... diff --git a/tests/unit/test_referent_catalog.py b/tests/unit/test_referent_catalog.py new file mode 100644 index 00000000..1a75ed73 --- /dev/null +++ b/tests/unit/test_referent_catalog.py @@ -0,0 +1,127 @@ +from datetime import UTC, date, datetime + +from fateforger.referents import StandingThing, build_catalog + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +def _thing(key: str, **over) -> StandingThing: + base = dict( + key=key, + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="open", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("continue planning",), + ) + base.update(over) + return StandingThing(**base) + + +class _Provider: + def __init__(self, agent_type, things=(), raises=None): + self.agent_type = agent_type + self._things = list(things) + self._raises = raises + self.calls = [] + + async def standing(self, *, owner_user_id: str, as_of: datetime): + self.calls.append((owner_user_id, as_of)) + if self._raises is not None: + raise self._raises + return self._things + + +async def test_the_host_mints_the_ids_because_a_provider_may_not_name_identity(): + catalog = await build_catalog( + [_Provider("a", [_thing("k1"), _thing("k2")])], + owner_user_id="U1", + as_of=AS_OF, + ) + assert [r.ref_id for r in catalog.referents] == ["r1", "r2"] + assert [r.key for r in catalog.referents] == ["k1", "k2"] + + +async def test_ids_stay_unique_across_providers(): + catalog = await build_catalog( + [_Provider("a", [_thing("k1")]), _Provider("b", [_thing("k2")])], + owner_user_id="U1", + as_of=AS_OF, + ) + assert sorted(r.ref_id for r in catalog.referents) == ["r1", "r2"] + + +async def test_every_provider_is_asked_the_same_owner_and_moment(): + p1, p2 = _Provider("a", [_thing("k1")]), _Provider("b", [_thing("k2")]) + await build_catalog([p1, p2], owner_user_id="U1", as_of=AS_OF) + assert p1.calls == [("U1", AS_OF)] and p2.calls == [("U1", AS_OF)] + + +async def test_a_failing_provider_does_not_lose_the_others_but_does_clear_complete(): + # A `none` drawn from a partial catalog is not evidence that nothing + # stands, so the flag has to travel with the answer. + good = _Provider("a", [_thing("k1")]) + catalog = await build_catalog( + [good, _Provider("b", raises=RuntimeError("store down"))], + owner_user_id="U1", + as_of=AS_OF, + ) + assert [r.key for r in catalog.referents] == ["k1"] + assert catalog.complete is False + + +async def test_a_whole_catalog_is_complete(): + catalog = await build_catalog( + [_Provider("a", [_thing("k1")])], owner_user_id="U1", as_of=AS_OF + ) + assert catalog.complete is True + + +async def test_the_arriving_thread_is_marked_so_that_can_be_told_from_this(): + catalog = await build_catalog( + [ + _Provider( + "a", + [ + _thing("k1", channel_id="C1", thread_ts="111.0"), + _thing("k2", channel_id="C1", thread_ts="222.0"), + ], + ) + ], + owner_user_id="U1", + as_of=AS_OF, + current_thread=("C1", "222.0"), + ) + marked = {r.key: r.is_current_surface for r in catalog.referents} + assert marked == {"k1": False, "k2": True} + + +async def test_no_providers_is_an_empty_complete_catalog_not_a_failure(): + catalog = await build_catalog([], owner_user_id="U1", as_of=AS_OF) + assert catalog.referents == () and catalog.complete is True + + +async def test_providers_are_gathered_concurrently(): + import asyncio + + order = [] + + class _Slow(_Provider): + def __init__(self, agent_type, delay, key): + super().__init__(agent_type, [_thing(key)]) + self._delay = delay + + async def standing(self, *, owner_user_id, as_of): + await asyncio.sleep(self._delay) + order.append(self.agent_type) + return self._things + + await build_catalog( + [_Slow("slow", 0.05, "k1"), _Slow("fast", 0.0, "k2")], + owner_user_id="U1", + as_of=AS_OF, + ) + # Sequential awaits would finish slow-then-fast; concurrent finishes fast first. + assert order == ["fast", "slow"] From 7e146885e921428158cccc955ea87f2a82baf1c8 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:28:53 +0200 Subject: [PATCH 05/20] feat(referents): one call over the minted candidates, returning a referent and never an action Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/referents/__init__.py | 16 +++ src/fateforger/referents/resolver.py | 149 +++++++++++++++++++++++++++ tests/unit/test_referent_resolver.py | 136 ++++++++++++++++++++++++ 3 files changed, 301 insertions(+) create mode 100644 src/fateforger/referents/resolver.py create mode 100644 tests/unit/test_referent_resolver.py diff --git a/src/fateforger/referents/__init__.py b/src/fateforger/referents/__init__.py index 36ffc0ba..3ded3654 100644 --- a/src/fateforger/referents/__init__.py +++ b/src/fateforger/referents/__init__.py @@ -8,12 +8,28 @@ from .catalog import Catalog, build_catalog from .descriptor import GIST_LIMIT, Referent, StandingThing from .provider import ReferentProvider +from .resolver import ( + RESOLVER_PROMPT, + Ambiguous, + NoReferent, + ReferentResolutionError, + ReferentResolver, + Resolution, + Resolved, +) __all__ = [ "GIST_LIMIT", + "RESOLVER_PROMPT", + "Ambiguous", "Catalog", + "NoReferent", "Referent", "ReferentProvider", + "ReferentResolutionError", + "ReferentResolver", + "Resolution", + "Resolved", "StandingThing", "build_catalog", ] diff --git a/src/fateforger/referents/resolver.py b/src/fateforger/referents/resolver.py new file mode 100644 index 00000000..41da1891 --- /dev/null +++ b/src/fateforger/referents/resolver.py @@ -0,0 +1,149 @@ +"""One judgement: which standing thing, if any, is this message about. + +It never says what to *do* with the answer. That is a second judgement, run by +whichever consumer holds the state's own allowed decisions -- and keeping them +apart is what stops a resolution becoming a write with no human in between. +The enforcement is the return type: no outcome here has a field for an action. +""" + +from __future__ import annotations + +import json +from datetime import datetime +from typing import Literal + +from autogen_core.models import ChatCompletionClient, SystemMessage, UserMessage +from pydantic import BaseModel, ConfigDict, ValidationError, create_model + +from fateforger.core.llm_attribution import llm_attribution + +from .catalog import Catalog +from .descriptor import Referent + + +class ReferentResolutionError(RuntimeError, ValueError): + """Reading one message against one catalog failed. + + Also a ValueError because the schema violations it wraps already were one. + """ + + +class _Outcome(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + #: False when a provider failed. Carried on every outcome because a `none` + #: -- and equally a confident `Resolved` -- drawn from a partial catalog is + #: weaker evidence than one drawn from a whole one. + catalog_complete: bool = True + + +class Resolved(_Outcome): + referent: Referent + + +class Ambiguous(_Outcome): + candidates: tuple[Referent, ...] + + +class NoReferent(_Outcome): + pass + + +Resolution = Resolved | Ambiguous | NoReferent + + +#: The wording is measured. It moved 74/112 -> 95/112 draws on the frozen +#: fixture, entirely on the last sentence, which is the distinction between +#: continuing a day's plan and wanting something new scheduled. Changing this +#: text means re-running tests/evals/test_eval_referent_resolver.py. +RESOLVER_PROMPT = """You route one message a user just typed to a scheduling assistant. +The user has some *standing things*: plans for a particular day that already exist and can be continued. +Decide which standing thing, if any, this message is about. + +A message is about a standing thing when it continues, changes, questions, or ends THAT day's plan -- +including a question about what that plan says. +A message is about NONE of them when it asks for something new that no listed plan covers: a fact, +an errand, a reminder, or planning a day that is not listed. Wanting something scheduled is not the +same as continuing an existing plan for a day. +Choose "ambiguous" only when the message is clearly about one of the listed plans but two or more fit equally. + +Judge by meaning. Never invent identifiers. Return only JSON. +Answer with {"decision": "" | "none" | "ambiguous", "why": ""}.""" + + +class _Answer(BaseModel): + model_config = ConfigDict(extra="forbid") + + decision: str + why: str = "" + + +def _narrowed(catalog: Catalog) -> type[_Answer]: + """Offer the model exactly the ids the host minted, and nothing else.""" + ids = tuple(r.ref_id for r in catalog.referents) + return create_model( # type: ignore[call-overload] + "_NarrowedAnswer", + __base__=_Answer, + decision=(Literal[(*ids, "none", "ambiguous")], ...), + ) + + +class ReferentResolver: + def __init__(self, model_client: ChatCompletionClient) -> None: + self._model_client = model_client + + async def resolve( + self, *, catalog: Catalog, message: str, as_of: datetime + ) -> Resolution: + if not catalog.referents: + # Nothing to choose between. Asking anyway would spend a round trip + # to be told what the query already said. + return NoReferent(catalog_complete=catalog.complete) + + payload = { + "now": as_of.strftime("%Y-%m-%d %H:%M (%A)"), + "standing_things": [r.describe(as_of) for r in catalog.referents], + "message": message, + } + schema = _narrowed(catalog) + try: + with llm_attribution( + agent="referent_resolver", call_label="resolve", key="referents" + ): + result = await self._model_client.create( + [ + SystemMessage(content=RESOLVER_PROMPT), + UserMessage( + content=json.dumps(payload, ensure_ascii=False), + source="user", + ), + ], + json_output=schema, + ) + content = getattr(result, "content", None) + if not isinstance(content, str): + raise ReferentResolutionError( + "the resolver model returned no schema-bound JSON content" + ) + answer = schema.model_validate_json(content) + except ReferentResolutionError: + raise + except ValidationError as exc: + raise ReferentResolutionError( + f"the resolver model answered outside its schema: {exc}" + ) from exc + + if answer.decision == "none": + return NoReferent(catalog_complete=catalog.complete) + if answer.decision == "ambiguous": + return Ambiguous( + catalog_complete=catalog.complete, candidates=catalog.referents + ) + chosen = next( + (r for r in catalog.referents if r.ref_id == answer.decision), None + ) + if chosen is None: # pragma: no cover - the schema should prevent it + raise ReferentResolutionError( + f"the resolver named an id the host did not mint: {answer.decision!r}" + ) + return Resolved(catalog_complete=catalog.complete, referent=chosen) diff --git a/tests/unit/test_referent_resolver.py b/tests/unit/test_referent_resolver.py new file mode 100644 index 00000000..d92b0ae6 --- /dev/null +++ b/tests/unit/test_referent_resolver.py @@ -0,0 +1,136 @@ +import json +from datetime import UTC, date, datetime +from types import SimpleNamespace + +import pytest + +from fateforger.referents import Catalog, Referent +from fateforger.referents.resolver import ( + Ambiguous, + NoReferent, + ReferentResolutionError, + ReferentResolver, + Resolved, +) + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +def _ref(ref_id: str, key: str, **over) -> Referent: + base = dict( + key=key, + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + ) + base.update(over) + return Referent(**base, ref_id=ref_id) + + +CATALOG = Catalog(referents=(_ref("r1", "k1"), _ref("r2", "k2"))) + + +class _Model: + """Records the prompt it was given and replays canned content.""" + + def __init__(self, content: str): + self._content = content + self.calls = [] + + async def create(self, messages, **kwargs): + self.calls.append((messages, kwargs)) + return SimpleNamespace(content=self._content) + + +async def test_a_named_candidate_comes_back_as_that_referent(): + model = _Model(json.dumps({"decision": "r2", "why": "names Saturday"})) + outcome = await ReferentResolver(model).resolve( + catalog=CATALOG, message="replan today", as_of=AS_OF + ) + assert isinstance(outcome, Resolved) + assert outcome.referent.key == "k2" + + +async def test_none_is_its_own_outcome_rather_than_a_null_referent(): + model = _Model(json.dumps({"decision": "none", "why": "new request"})) + outcome = await ReferentResolver(model).resolve( + catalog=CATALOG, message="what's the weather", as_of=AS_OF + ) + assert isinstance(outcome, NoReferent) + + +async def test_ambiguous_carries_the_candidates_so_a_card_need_not_rebuild_them(): + model = _Model(json.dumps({"decision": "ambiguous", "why": "two fit"})) + outcome = await ReferentResolver(model).resolve( + catalog=CATALOG, message="move the gym", as_of=AS_OF + ) + assert isinstance(outcome, Ambiguous) + assert [c.ref_id for c in outcome.candidates] == ["r1", "r2"] + + +async def test_an_id_the_host_never_minted_is_refused_rather_than_believed(): + model = _Model(json.dumps({"decision": "r9", "why": "invented"})) + with pytest.raises(ReferentResolutionError): + await ReferentResolver(model).resolve( + catalog=CATALOG, message="hello", as_of=AS_OF + ) + + +async def test_content_that_is_not_json_raises_rather_than_degrading(): + # Two behaviours with the wrong one silent is the shape CLAUDE.md's first + # rule exists to stop. + model = _Model("sorry, I can't help with that") + with pytest.raises(ReferentResolutionError): + await ReferentResolver(model).resolve( + catalog=CATALOG, message="hello", as_of=AS_OF + ) + + +async def test_an_empty_catalog_is_answered_without_asking_a_model_at_all(): + model = _Model(json.dumps({"decision": "none", "why": ""})) + outcome = await ReferentResolver(model).resolve( + catalog=Catalog(), message="plan tomorrow", as_of=AS_OF + ) + assert isinstance(outcome, NoReferent) + assert model.calls == [] + + +async def test_the_incomplete_flag_travels_onto_the_outcome(): + model = _Model(json.dumps({"decision": "none", "why": ""})) + partial = Catalog(referents=(_ref("r1", "k1"),), complete=False) + outcome = await ReferentResolver(model).resolve( + catalog=partial, message="hello", as_of=AS_OF + ) + assert outcome.catalog_complete is False + + +async def test_the_prompt_carries_every_candidate_and_the_users_words_verbatim(): + model = _Model(json.dumps({"decision": "r1", "why": ""})) + await ReferentResolver(model).resolve( + catalog=CATALOG, message="move the gym to the morning", as_of=AS_OF + ) + payload = json.loads(model.calls[0][0][1].content) + assert [c["ref_id"] for c in payload["standing_things"]] == ["r1", "r2"] + assert payload["message"] == "move the gym to the morning" + + +async def test_the_schema_offered_to_the_model_is_narrowed_to_the_minted_ids(): + model = _Model(json.dumps({"decision": "r1", "why": ""})) + await ReferentResolver(model).resolve( + catalog=CATALOG, message="hi", as_of=AS_OF + ) + schema = model.calls[0][1]["json_output"] + allowed = schema.model_fields["decision"].annotation + assert set(getattr(allowed, "__args__", ())) == {"r1", "r2", "none", "ambiguous"} + + +def test_no_outcome_can_express_an_action(): + # Resolve-then-act is enforced by the return type, not by prose: a consumer + # cannot fold the two judgements together without changing this. + for outcome in (Resolved, Ambiguous, NoReferent): + assert "action" not in outcome.model_fields + assert "decision" not in outcome.model_fields From 97e0fc4e8449e7a618b2206ed1a27c69951edec0 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:32:59 +0200 Subject: [PATCH 06/20] test(referents): the gist is read by a judge and never compared by code --- tests/unit/test_referents_never_match_gist.py | 75 +++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 tests/unit/test_referents_never_match_gist.py diff --git a/tests/unit/test_referents_never_match_gist.py b/tests/unit/test_referents_never_match_gist.py new file mode 100644 index 00000000..b9fe552d --- /dev/null +++ b/tests/unit/test_referents_never_match_gist.py @@ -0,0 +1,75 @@ +# tests/unit/test_referents_never_match_gist.py +"""The gist is content, and content is only ever read by a judge. + +Block titles are the one field in the descriptor that came from a plan rather +than from a column, so they are the one field someone might be tempted to +compare, sort or filter on. CLAUDE.md forbids it, and a wrong pattern does not +raise -- it quietly returns the wrong answer forever. So it is asserted. +""" + +from __future__ import annotations + +import ast +from pathlib import Path + +import fateforger.referents as referents_pkg + +PACKAGE = Path(referents_pkg.__file__).parent + +#: Names that would mean code is reading the gist's *meaning* rather than +#: passing it along. `in`/`sorted`/`.lower()` over a title is the shape. +FORBIDDEN_METHODS = {"lower", "upper", "casefold", "strip", "split", "startswith", "endswith", "find", "index", "replace"} + + +def _gist_attribute_names(tree: ast.AST) -> list[ast.AST]: + return [ + node + for node in ast.walk(tree) + if isinstance(node, ast.Attribute) and node.attr == "gist" + ] + + +def test_no_module_in_the_package_imports_re(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + if isinstance(node, ast.Import): + assert all(a.name != "re" for a in node.names), path + if isinstance(node, ast.ImportFrom): + assert node.module != "re", path + + +def test_no_string_method_is_called_on_anything_reached_through_gist(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for call in (n for n in ast.walk(tree) if isinstance(n, ast.Call)): + func = call.func + if not isinstance(func, ast.Attribute): + continue + if func.attr not in FORBIDDEN_METHODS: + continue + # Walk the receiver looking for `.gist` + assert not _gist_attribute_names(func.value), f"{path}: {func.attr} on gist" + + +def test_the_gist_is_never_a_comparison_operand(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + if isinstance(node, ast.Compare): + operands = [node.left, *node.comparators] + for operand in operands: + assert not _gist_attribute_names(operand), f"{path}: gist compared" + + +def test_the_package_imports_no_slack(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for node in ast.walk(tree): + module = ( + node.module + if isinstance(node, ast.ImportFrom) + else None + ) + if module: + assert "slack" not in module.split("."), f"{path}: imports {module}" From de152e4a6c09f260c977405e9d125eb9b9b31909 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:40:21 +0200 Subject: [PATCH 07/20] test(referents): extend gist guard with slack import and builtin function checks - Add check for both ast.Import and ast.ImportFrom for slack imports - Add test for forbidden builtin functions (sorted, any, all, max, min, sum, set) called on gist to prevent content reordering or filtering Co-Authored-By: Claude Opus 5 (1M context) --- tests/unit/test_referents_never_match_gist.py | 33 +++++++++++++++---- 1 file changed, 26 insertions(+), 7 deletions(-) diff --git a/tests/unit/test_referents_never_match_gist.py b/tests/unit/test_referents_never_match_gist.py index b9fe552d..af8ad540 100644 --- a/tests/unit/test_referents_never_match_gist.py +++ b/tests/unit/test_referents_never_match_gist.py @@ -20,6 +20,10 @@ #: passing it along. `in`/`sorted`/`.lower()` over a title is the shape. FORBIDDEN_METHODS = {"lower", "upper", "casefold", "strip", "split", "startswith", "endswith", "find", "index", "replace"} +#: Built-in functions that make decisions about content and must not be called on gist. +#: Excluded: list, tuple, iter (these are plumbing that pass values through unchanged). +FORBIDDEN_BUILTINS = {"sorted", "any", "all", "max", "min", "sum", "set"} + def _gist_attribute_names(tree: ast.AST) -> list[ast.AST]: return [ @@ -52,6 +56,20 @@ def test_no_string_method_is_called_on_anything_reached_through_gist(): assert not _gist_attribute_names(func.value), f"{path}: {func.attr} on gist" +def test_no_builtin_with_decision_logic_is_called_on_gist(): + for path in PACKAGE.rglob("*.py"): + tree = ast.parse(path.read_text()) + for call in (n for n in ast.walk(tree) if isinstance(n, ast.Call)): + func = call.func + if not isinstance(func, ast.Name): + continue + if func.id not in FORBIDDEN_BUILTINS: + continue + # Check if any argument reaches `.gist` + for arg in call.args: + assert not _gist_attribute_names(arg), f"{path}: {func.id}() on gist" + + def test_the_gist_is_never_a_comparison_operand(): for path in PACKAGE.rglob("*.py"): tree = ast.parse(path.read_text()) @@ -66,10 +84,11 @@ def test_the_package_imports_no_slack(): for path in PACKAGE.rglob("*.py"): tree = ast.parse(path.read_text()) for node in ast.walk(tree): - module = ( - node.module - if isinstance(node, ast.ImportFrom) - else None - ) - if module: - assert "slack" not in module.split("."), f"{path}: imports {module}" + if isinstance(node, ast.Import): + # Check plain imports like `import slack_sdk` or `import slack.client` + for alias in node.names: + assert "slack" not in alias.name, f"{path}: imports {alias.name}" + if isinstance(node, ast.ImportFrom): + # Check from imports like `from slack_sdk import ...` + if node.module: + assert "slack" not in node.module, f"{path}: imports from {node.module}" From f4cba3b38e14b34ddcb402f4b61b6edda4485c79 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:45:55 +0200 Subject: [PATCH 08/20] test(referents): narrow builtin guard to flag only direct gist references Fixes regression where nested gist usage like min(len(self.gist), GIST_LIMIT) was incorrectly flagged. The guard now only fails on direct calls like sorted(self.gist) or set(self.gist), allowing structural operations that don't inspect content meaning. Co-Authored-By: Claude Opus 5 (1M context) --- tests/unit/test_referents_never_match_gist.py | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/tests/unit/test_referents_never_match_gist.py b/tests/unit/test_referents_never_match_gist.py index af8ad540..bf0ce2c6 100644 --- a/tests/unit/test_referents_never_match_gist.py +++ b/tests/unit/test_referents_never_match_gist.py @@ -33,6 +33,18 @@ def _gist_attribute_names(tree: ast.AST) -> list[ast.AST]: ] +def _is_direct_gist_reference(node: ast.AST) -> bool: + """Check if node is directly self.gist or a slice of self.gist (e.g., self.gist[:N]). + + Returns False if .gist appears only deeper in the expression tree. + """ + if isinstance(node, ast.Attribute) and node.attr == "gist": + return True + if isinstance(node, ast.Subscript) and isinstance(node.value, ast.Attribute) and node.value.attr == "gist": + return True + return False + + def test_no_module_in_the_package_imports_re(): for path in PACKAGE.rglob("*.py"): tree = ast.parse(path.read_text()) @@ -65,9 +77,10 @@ def test_no_builtin_with_decision_logic_is_called_on_gist(): continue if func.id not in FORBIDDEN_BUILTINS: continue - # Check if any argument reaches `.gist` + # Check if any argument directly reaches `.gist` or a slice of it + # (not when .gist appears only deeper in nested expressions) for arg in call.args: - assert not _gist_attribute_names(arg), f"{path}: {func.id}() on gist" + assert not _is_direct_gist_reference(arg), f"{path}: {func.id}() on gist" def test_the_gist_is_never_a_comparison_operand(): From 3d9c91df50951ef7db78f694fb314d5615c1aeb0 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:50:51 +0200 Subject: [PATCH 09/20] feat(referents): timeboxing sessions as the first provider Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/referents/__init__.py | 2 + src/fateforger/referents/timeboxing.py | 105 ++++++++++++++++ .../unit/test_referent_timeboxing_provider.py | 118 ++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 src/fateforger/referents/timeboxing.py create mode 100644 tests/unit/test_referent_timeboxing_provider.py diff --git a/src/fateforger/referents/__init__.py b/src/fateforger/referents/__init__.py index 3ded3654..123c71de 100644 --- a/src/fateforger/referents/__init__.py +++ b/src/fateforger/referents/__init__.py @@ -17,6 +17,7 @@ Resolution, Resolved, ) +from .timeboxing import TimeboxingReferentProvider __all__ = [ "GIST_LIMIT", @@ -31,5 +32,6 @@ "Resolution", "Resolved", "StandingThing", + "TimeboxingReferentProvider", "build_catalog", ] diff --git a/src/fateforger/referents/timeboxing.py b/src/fateforger/referents/timeboxing.py new file mode 100644 index 00000000..d3888caa --- /dev/null +++ b/src/fateforger/referents/timeboxing.py @@ -0,0 +1,105 @@ +"""Timeboxing sessions as standing things. + +The first provider. It answers the same question `standing_for` answers for the +nudger -- which sessions stand -- and returns descriptors instead of keys. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from datetime import datetime, timedelta +from typing import Protocol + +from .descriptor import StandingThing + +#: How long an untouched `open` session keeps counting as standing. The +#: nudger's own bound is one hour, which is right for "is a session under way" +#: and too tight for "what could this message be about": the Monday session was +#: last saved 69 minutes before the message that should have reached it. +DEFAULT_OPEN_WITHIN = timedelta(hours=12) + +#: How far ahead a committed day still counts. Matches the planning horizon. +DEFAULT_HORIZON = timedelta(days=7) + +#: The second half of a session key opened in a DM. It names the whole DM and +#: not a thread, so it can never be the surface a message arrived in. +DM_SUFFIX = "dm" + +_COMMITTED_ACCEPTS = ("revise the committed plan", "add a fact about the day") +_OPEN_ACCEPTS = ("continue planning", "answer the open question", "cancel") + +#: The opening turn's revision (`session_start.UNTOUCHED_REVISION`). Anything +#: above it is the user's own work. +UNTOUCHED_REVISION = 1 + + +class _StandingRows(Protocol): + async def standing_rows( + self, + *, + owner_user_id: str, + as_of: datetime, + open_within: timedelta, + horizon: timedelta, + ) -> Sequence: ... + + +class TimeboxingReferentProvider: + """Standing timeboxing sessions, as descriptors.""" + + agent_type = "timeboxing_agent" + + def __init__( + self, + repository: _StandingRows, + *, + open_within: timedelta = DEFAULT_OPEN_WITHIN, + horizon: timedelta = DEFAULT_HORIZON, + ) -> None: + self._repository = repository + self._open_within = open_within + self._horizon = horizon + + async def standing( + self, *, owner_user_id: str, as_of: datetime + ) -> Sequence[StandingThing]: + rows = await self._repository.standing_rows( + owner_user_id=owner_user_id, + as_of=as_of, + open_within=self._open_within, + horizon=self._horizon, + ) + return [self._describe(row, as_of=as_of) for row in rows] + + def _describe(self, row, *, as_of: datetime) -> StandingThing: + channel_id, thread_ts = _split_session_key(row.session_key) + committed = row.status == "committed" + return StandingThing( + key=row.session_key, + agent_type=self.agent_type, + kind="a plan for one day", + day=row.planning_date, + status=row.status, + never_used=( + row.status == "open" and row.revision <= UNTOUCHED_REVISION + ), + last_activity=row.updated_at.replace(tzinfo=as_of.tzinfo), + accepts=_COMMITTED_ACCEPTS if committed else _OPEN_ACCEPTS, + gist=tuple(row.gist), + channel_id=channel_id, + thread_ts=thread_ts, + ) + + +def _split_session_key(session_key: str) -> tuple[str | None, str | None]: + """`{channel}:{thread_ts}`, or `{channel}:dm` which names no thread. + + Identifiers this system minted, so splitting them is arithmetic and not a + reading of anything the user wrote. + """ + channel, _, tail = session_key.rpartition(":") + if not channel: + return None, None + if tail == DM_SUFFIX: + return channel, None + return channel, tail diff --git a/tests/unit/test_referent_timeboxing_provider.py b/tests/unit/test_referent_timeboxing_provider.py new file mode 100644 index 00000000..d5346d0d --- /dev/null +++ b/tests/unit/test_referent_timeboxing_provider.py @@ -0,0 +1,118 @@ +from datetime import UTC, date, datetime, timedelta + +from pydantic import BaseModel + +from fateforger.referents.timeboxing import TimeboxingReferentProvider + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) + + +class _Row(BaseModel): + session_key: str + status: str + planning_date: date | None + updated_at: datetime + revision: int + gist: tuple[str, ...] = () + + +class _Repo: + def __init__(self, rows): + self._rows = rows + self.calls = [] + + async def standing_rows(self, *, owner_user_id, as_of, open_within, horizon): + self.calls.append((owner_user_id, as_of, open_within, horizon)) + return list(self._rows) + + +def _row(key, status, day, hours_ago=10.0, revision=7, gist=()): + return _Row( + session_key=key, + status=status, + planning_date=day, + updated_at=(AS_OF - timedelta(hours=hours_ago)).replace(tzinfo=None), + revision=revision, + gist=gist, + ) + + +async def test_a_committed_row_offers_revision_and_a_fact(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "committed", date(2026, 9, 5))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.status == "committed" + assert thing.accepts == ("revise the committed plan", "add a fact about the day") + + +async def test_an_open_row_offers_continuing_answering_and_cancelling(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "open", date(2026, 9, 7))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.accepts == ( + "continue planning", + "answer the open question", + "cancel", + ) + + +async def test_revision_one_is_the_opening_turn_so_the_row_is_never_used(): + provider = TimeboxingReferentProvider( + _Repo([_row("D1:dm", "open", None, revision=1)]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.never_used is True + + +async def test_a_worked_row_is_not_marked_never_used(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "open", date(2026, 9, 7), revision=7)]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.never_used is False + + +async def test_a_channel_key_splits_into_a_channel_and_a_thread(): + provider = TimeboxingReferentProvider( + _Repo([_row("C0AA6HC1RJL:1788571682.407949", "committed", date(2026, 9, 5))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.channel_id == "C0AA6HC1RJL" + assert thing.thread_ts == "1788571682.407949" + + +async def test_a_dm_key_names_the_whole_dm_so_it_has_no_thread(): + # `{channel}:dm` is thread-blind; treating "dm" as a thread_ts would let a + # DM row claim to be the surface a message arrived in. + provider = TimeboxingReferentProvider(_Repo([_row("D09A0RE9P7G:dm", "open", None)])) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.channel_id == "D09A0RE9P7G" + assert thing.thread_ts is None + + +async def test_the_gist_reaches_the_descriptor_unchanged(): + gist = ("PR1 Serious C2F work 10:30-12:00", "GYM1 Gym (chest) 18:00-19:00") + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "committed", date(2026, 9, 4), gist=gist)]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.gist == gist + + +async def test_the_asked_moment_and_the_windows_reach_the_repository(): + repo = _Repo([]) + provider = TimeboxingReferentProvider(repo) + await provider.standing(owner_user_id="U1", as_of=AS_OF) + owner, as_of, open_within, horizon = repo.calls[0] + assert owner == "U1" and as_of == AS_OF + assert open_within == timedelta(hours=12) and horizon == timedelta(days=7) + + +async def test_the_agent_type_travels_onto_every_descriptor(): + provider = TimeboxingReferentProvider( + _Repo([_row("C1:111.0", "open", date(2026, 9, 7))]) + ) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + assert thing.agent_type == "timeboxing_agent" == provider.agent_type From 9f69d6579e475eef8dcb98a4695feb9e7b7bd0d5 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 11:56:49 +0200 Subject: [PATCH 10/20] fix(referents): tag naive UTC updated_at with UTC, not as_of.tzinfo The session store writes updated_at as naive UTC. Tagging it with as_of.tzinfo was wrong when as_of was not in UTC, silently computing wrong elapsed times and biasing router judgements. Use UTC consistently with the store's convention. Add test that catches this when as_of is non-UTC (Europe/Amsterdam). Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/referents/timeboxing.py | 6 +++-- .../unit/test_referent_timeboxing_provider.py | 25 +++++++++++++++++++ 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/src/fateforger/referents/timeboxing.py b/src/fateforger/referents/timeboxing.py index d3888caa..f1f46aab 100644 --- a/src/fateforger/referents/timeboxing.py +++ b/src/fateforger/referents/timeboxing.py @@ -7,7 +7,7 @@ from __future__ import annotations from collections.abc import Sequence -from datetime import datetime, timedelta +from datetime import UTC, datetime, timedelta from typing import Protocol from .descriptor import StandingThing @@ -83,7 +83,9 @@ def _describe(self, row, *, as_of: datetime) -> StandingThing: never_used=( row.status == "open" and row.revision <= UNTOUCHED_REVISION ), - last_activity=row.updated_at.replace(tzinfo=as_of.tzinfo), + # row.updated_at is written naive UTC by the store's save method, + # so we tag it with UTC (not as_of.tzinfo) to preserve its true instant. + last_activity=row.updated_at.replace(tzinfo=UTC), accepts=_COMMITTED_ACCEPTS if committed else _OPEN_ACCEPTS, gist=tuple(row.gist), channel_id=channel_id, diff --git a/tests/unit/test_referent_timeboxing_provider.py b/tests/unit/test_referent_timeboxing_provider.py index d5346d0d..92608ef5 100644 --- a/tests/unit/test_referent_timeboxing_provider.py +++ b/tests/unit/test_referent_timeboxing_provider.py @@ -1,4 +1,5 @@ from datetime import UTC, date, datetime, timedelta +from zoneinfo import ZoneInfo from pydantic import BaseModel @@ -116,3 +117,27 @@ async def test_the_agent_type_travels_onto_every_descriptor(): ) (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) assert thing.agent_type == "timeboxing_agent" == provider.agent_type + + +async def test_last_activity_is_correctly_zoned_when_as_of_is_not_utc(): + # row.updated_at is naive UTC (10:51). as_of is 13:51+02:00 (which equals 11:51 UTC). + # True elapsed time is 1 hour. The bug was tagging the naive UTC value with as_of.tzinfo, + # making it 10:51+02:00 (which equals 08:51 UTC). That would compute 3 hours elapsed instead of 1. + naive_utc_time = datetime(2026, 9, 5, 10, 51) # naive UTC + amsterdam_tz = ZoneInfo("Europe/Amsterdam") + as_of_in_amsterdam = datetime(2026, 9, 5, 13, 51, tzinfo=amsterdam_tz) # 11:51 UTC + + row = _Row( + session_key="C1:111.0", + status="open", + planning_date=date(2026, 9, 5), + updated_at=naive_utc_time, + revision=7, + gist=(), + ) + provider = TimeboxingReferentProvider(_Repo([row])) + (thing,) = await provider.standing(owner_user_id="U1", as_of=as_of_in_amsterdam) + + # The elapsed time should be 1 hour (true UTC difference) + elapsed = as_of_in_amsterdam - thing.last_activity + assert abs(elapsed.total_seconds() - 3600) < 1 # Allow 1 second tolerance for rounding From ae637401abf8fe58a160bb0924807ce51c465f75 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 12:01:41 +0200 Subject: [PATCH 11/20] feat(timeboxing): standing_rows answers which sessions stand, with each plan's gist Co-Authored-By: Claude Opus 5 (1M context) --- .../slack_bot/timeboxing_session_store.py | 115 +++++++++++- tests/unit/test_standing_rows_query.py | 165 ++++++++++++++++++ 2 files changed, 278 insertions(+), 2 deletions(-) create mode 100644 tests/unit/test_standing_rows_query.py diff --git a/src/fateforger/slack_bot/timeboxing_session_store.py b/src/fateforger/slack_bot/timeboxing_session_store.py index de560c7a..c547d259 100644 --- a/src/fateforger/slack_bot/timeboxing_session_store.py +++ b/src/fateforger/slack_bot/timeboxing_session_store.py @@ -3,9 +3,10 @@ from __future__ import annotations import asyncio +import json from collections.abc import AsyncIterator from contextlib import asynccontextmanager -from datetime import UTC, date, datetime +from datetime import UTC, date, datetime, timedelta from typing import Literal from pydantic import BaseModel, ConfigDict @@ -57,6 +58,24 @@ class _StoredSessionEnvelope(BaseModel): outcomes: dict[str, TurnOutcome] +class StandingSessionRow(BaseModel): + """One session that stands, from the indexed columns plus its plan's gist. + + `gist` is the only part that reads `snapshot_json`, and it is read to be + *shown to a judge*, never compared. The row set is single digits, so the + cost of loading those snapshots is bounded. + """ + + model_config = ConfigDict(extra="forbid", frozen=True) + + session_key: str + status: str + planning_date: date | None + updated_at: datetime + revision: int + gist: tuple[str, ...] = () + + class SqlAlchemyTimeboxingSessionRepository(PlanningSessionRepository): """Persist adaptive sessions with SQL CAS and process-local coalescing.""" @@ -246,6 +265,64 @@ async def standing_for( open_session_key=open_key, committed_session_key=committed_key ) + async def standing_rows( + self, + *, + owner_user_id: str, + as_of: datetime, + open_within: timedelta, + horizon: timedelta, + ) -> list[StandingSessionRow]: + """Which sessions stand for this owner AT `as_of`. + + Same predicate family as `standing_for`, which answers this for the + nudger and returns keys. This returns rows a catalog can describe. + + `created_at < as_of` keeps a row the current message minted out of its + own catalog. `updated_at` is written naive UTC by `save`, so both bounds + are compared in that basis. + """ + moment = as_of.astimezone(UTC).replace(tzinfo=None) + since = moment - open_within + async with self._sessionmaker() as session: + result = await session.execute( + select( + _TimeboxingSessionState.session_key, + _TimeboxingSessionState.status, + _TimeboxingSessionState.planning_date, + _TimeboxingSessionState.updated_at, + _TimeboxingSessionState.revision, + _TimeboxingSessionState.snapshot_json, + ) + .where( + _TimeboxingSessionState.owner_user_id == owner_user_id, + _TimeboxingSessionState.created_at < moment, + or_( + (_TimeboxingSessionState.status == "open") + & (_TimeboxingSessionState.updated_at >= since), + (_TimeboxingSessionState.status == "committed") + & (_TimeboxingSessionState.planning_date >= moment.date()) + & ( + _TimeboxingSessionState.planning_date + <= (moment + horizon).date() + ), + ), + ) + .order_by(_TimeboxingSessionState.updated_at.desc()) + ) + rows = result.all() + return [ + StandingSessionRow( + session_key=key, + status=status, + planning_date=planning_date, + updated_at=updated_at, + revision=revision, + gist=_plan_gist(snapshot_json), + ) + for key, status, planning_date, updated_at, revision, snapshot_json in rows + ] + async def open_sessions(self, *, owner_user_id: str) -> list[OpenSessionRow]: """Every open session this user holds, newest save first. @@ -369,4 +446,38 @@ def _day_frame(snapshot: PlanningSessionSnapshot) -> dict | None: return None -__all__ = ["SqlAlchemyTimeboxingSessionRepository"] +def _plan_gist(snapshot_json: str) -> tuple[str, ...]: + """A few of the plan's own block titles, with their times. + + Read from the latest validated candidate's rendered block table, whose + columns are `H,own,type,summary,ST,ET,mode,dur` -- a table this system + generated, so taking the summary and the two clocks out of it is arithmetic + over our own format and not a reading of anything the user wrote. Anything + unparseable yields no gist rather than a guess. + """ + try: + envelope = json.loads(snapshot_json) + artifacts = envelope["snapshot"]["artifacts"] + except (ValueError, KeyError, TypeError): + return () + rendered = next( + ( + artifact.get("payload", {}).get("rendered") + for artifact in reversed(artifacts) + if artifact.get("kind") == "validated_candidate" + ), + None, + ) + if not isinstance(rendered, str): + return () + entries: list[str] = [] + for line in rendered.splitlines()[1:]: # first line is the column header + fields = line.split(",") + if len(fields) < 6: + continue + summary, start, end = fields[3], fields[4], fields[5] + entries.append(f"{summary} {start}-{end}") + return tuple(entries) + + +__all__ = ["SqlAlchemyTimeboxingSessionRepository", "StandingSessionRow"] diff --git a/tests/unit/test_standing_rows_query.py b/tests/unit/test_standing_rows_query.py new file mode 100644 index 00000000..0449a897 --- /dev/null +++ b/tests/unit/test_standing_rows_query.py @@ -0,0 +1,165 @@ +"""The query half of the catalog: which sessions stand, as arithmetic. + +Per the routing clause, *which rows stand* is a guarantee and belongs in code +with a test beside it. Only *which standing one a message is about* is a +judgement. Keeping them apart is what stops someone replacing a correct query +with a classifier and calling it progress. +""" + +from datetime import UTC, date, datetime, timedelta + +import pytest +import pytest_asyncio +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from fateforger.slack_bot.timeboxing_session_store import ( + SqlAlchemyTimeboxingSessionRepository, + _Base, + _TimeboxingSessionState, +) + +AS_OF = datetime(2026, 9, 5, 11, 51, tzinfo=UTC) +NAIVE = AS_OF.replace(tzinfo=None) + + +@pytest_asyncio.fixture +async def repo(): + engine = create_async_engine("sqlite+aiosqlite:///:memory:") + async with engine.begin() as conn: + await conn.run_sync(_Base.metadata.create_all) + maker = async_sessionmaker(engine, expire_on_commit=False) + yield SqlAlchemyTimeboxingSessionRepository(maker), maker + await engine.dispose() + + +async def _insert(maker, **over): + row = dict( + session_key="C1:1.0", + owner_user_id="U1", + revision=7, + status="open", + planning_date=date(2026, 9, 5), + snapshot_json="{}", + created_at=NAIVE - timedelta(hours=20), + updated_at=NAIVE - timedelta(hours=1), + ) + row.update(over) + async with maker() as session: + session.add(_TimeboxingSessionState(**row)) + await session.commit() + + +async def test_a_recently_saved_open_session_stands(repo): + repository, maker = repo + await _insert(maker) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert [r.session_key for r in rows] == ["C1:1.0"] + + +async def test_a_stale_open_session_does_not(repo): + repository, maker = repo + await _insert(maker, updated_at=NAIVE - timedelta(hours=30)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_a_committed_day_inside_the_horizon_stands(repo): + repository, maker = repo + await _insert(maker, status="committed", updated_at=NAIVE - timedelta(hours=30)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert [r.status for r in rows] == ["committed"] + + +async def test_a_committed_day_in_the_past_does_not(repo): + repository, maker = repo + await _insert(maker, status="committed", planning_date=date(2026, 9, 1)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_a_cancelled_session_never_stands(repo): + repository, maker = repo + await _insert(maker, status="cancelled") + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_another_users_session_never_stands(repo): + repository, maker = repo + await _insert(maker, owner_user_id="U2") + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_a_row_created_after_the_asked_moment_is_excluded(repo): + # The catalog must never contain the row the current message minted. Task 7 + # guarantees the ordering; this is the belt. + repository, maker = repo + await _insert(maker, created_at=NAIVE + timedelta(minutes=1)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_the_gist_comes_from_the_candidates_rendered_blocks(repo): + repository, maker = repo + snapshot = { + "envelope_version": 1, + "snapshot": { + "artifacts": [ + { + "kind": "validated_candidate", + "revision": 1, + "payload": { + "rendered": ( + "blocks[2]{H,own,type,summary,ST,ET,mode,dur}:\n" + "PR1,tmbx,C,Serious C2F work,10:30,12:00,fs,PT1H30M\n" + "GYM1,tmbx,H,Gym (chest),18:00,19:00,fs,PT1H" + ) + }, + } + ] + }, + "outcomes": {}, + } + import json + + await _insert(maker, snapshot_json=json.dumps(snapshot)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == ( + "Serious C2F work 10:30-12:00", + "Gym (chest) 18:00-19:00", + ) + + +async def test_a_session_with_no_plan_yet_has_an_empty_gist(repo): + repository, maker = repo + await _insert(maker, snapshot_json="{}") + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == () From c2d86b3a3635d071f7a26c446139008da057935f Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 12:11:02 +0200 Subject: [PATCH 12/20] fix(timeboxing): parse rendered plan rows with csv, not a naive comma split A summary containing a comma is CSV-quoted by render_plan's _escape, and line.split(",") broke a quoted field into two pieces, shifting every column after it -- the end time came back as the start time and the summary carried a stray quote. csv.reader respects that quoting, so the gist shown to the referent judge no longer silently misdescribes the plan. A genuinely unparseable row still fails closed to an empty gist. Adds two tests that fail against the old split(",") parser and pass against csv.reader, plus a committed-day-beyond-horizon exclusion test. Co-Authored-By: Claude Opus 5 (1M context) --- .../slack_bot/timeboxing_session_store.py | 16 +++- tests/unit/test_standing_rows_query.py | 84 +++++++++++++++++++ 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/src/fateforger/slack_bot/timeboxing_session_store.py b/src/fateforger/slack_bot/timeboxing_session_store.py index c547d259..d545a2e3 100644 --- a/src/fateforger/slack_bot/timeboxing_session_store.py +++ b/src/fateforger/slack_bot/timeboxing_session_store.py @@ -3,6 +3,7 @@ from __future__ import annotations import asyncio +import csv import json from collections.abc import AsyncIterator from contextlib import asynccontextmanager @@ -454,6 +455,14 @@ def _plan_gist(snapshot_json: str) -> tuple[str, ...]: generated, so taking the summary and the two clocks out of it is arithmetic over our own format and not a reading of anything the user wrote. Anything unparseable yields no gist rather than a guess. + + Parsed with `csv.reader`, not `line.split(",")`: `render_plan`'s `_escape` + CSV-quotes a summary that contains the table's own delimiter (its + docstring's own example is `"Sprint, planning"`), and a naive split breaks + a quoted field into two, shifting every column after it -- the end time + comes back as the start time, and the summary carries a stray quote. Using + `csv.reader` is still reading our own format by its own rules, just more + faithfully than a hand-rolled split. """ try: envelope = json.loads(snapshot_json) @@ -471,8 +480,11 @@ def _plan_gist(snapshot_json: str) -> tuple[str, ...]: if not isinstance(rendered, str): return () entries: list[str] = [] - for line in rendered.splitlines()[1:]: # first line is the column header - fields = line.split(",") + try: + data_rows = list(csv.reader(rendered.splitlines()[1:])) + except csv.Error: + return () + for fields in data_rows: # first line was already dropped: column header if len(fields) < 6: continue summary, start, end = fields[3], fields[4], fields[5] diff --git a/tests/unit/test_standing_rows_query.py b/tests/unit/test_standing_rows_query.py index 0449a897..67163fdb 100644 --- a/tests/unit/test_standing_rows_query.py +++ b/tests/unit/test_standing_rows_query.py @@ -163,3 +163,87 @@ async def test_a_session_with_no_plan_yet_has_an_empty_gist(repo): open_within=timedelta(hours=12), horizon=timedelta(days=7), ) assert rows[0].gist == () + + +async def test_a_committed_day_beyond_the_horizon_does_not_stand(repo): + repository, maker = repo + await _insert( + maker, + status="committed", + planning_date=date(2026, 9, 13), # AS_OF.date() + 8 days > 7-day horizon + updated_at=NAIVE - timedelta(hours=30), + ) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows == [] + + +async def test_the_gist_keeps_a_comma_inside_a_quoted_summary_whole(repo): + # render.py's `_escape` CSV-quotes a summary containing the table's own + # delimiter -- its own docstring uses "Sprint, planning" as the example. + # A naive `line.split(",")` breaks the quoted field into two pieces, + # shifting every column after it: the "end" time comes back as what was + # really the start time, and the summary carries a stray quote character. + repository, maker = repo + snapshot = { + "envelope_version": 1, + "snapshot": { + "artifacts": [ + { + "kind": "validated_candidate", + "revision": 1, + "payload": { + "rendered": ( + "blocks[1]{H,own,type,summary,ST,ET,mode,dur}:\n" + 'PR1,tmbx,C,"Serious C2F work, prep",10:30,12:00,fs,PT1H30M' + ) + }, + } + ] + }, + "outcomes": {}, + } + import json + + await _insert(maker, snapshot_json=json.dumps(snapshot)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == ("Serious C2F work, prep 10:30-12:00",) + + +async def test_a_malformed_row_yields_no_gist_entry(repo): + # An unterminated quote is not a shape `_escape` ever emits, but a naive + # `line.split(",")` still finds >= 6 comma-separated pieces in it and + # returns a garbled partial entry instead of recognizing the row as + # unparseable. Fail closed: no entry, not a guess. + repository, maker = repo + snapshot = { + "envelope_version": 1, + "snapshot": { + "artifacts": [ + { + "kind": "validated_candidate", + "revision": 1, + "payload": { + "rendered": ( + "blocks[1]{H,own,type,summary,ST,ET,mode,dur}:\n" + 'PR1,tmbx,C,"Unterminated summary,10:30,12:00,fs,PT1H30M' + ) + }, + } + ] + }, + "outcomes": {}, + } + import json + + await _insert(maker, snapshot_json=json.dumps(snapshot)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == () From 909838bdd1e80630d00d031193cf65af2c847cfd Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 15:28:15 +0200 Subject: [PATCH 13/20] feat(slack): a follow-up reaches the session it is about, instead of minting a second one (#345) The 2026-09-05 13:51 incident: "can you replan today so the gym is before dinner?" typed top-level in #plan-sessions opened a fresh five-stage session for a day that had been committed at 01:41. No structural resolver owned that message -- it was top-level, so there was no thread to own -- and the channel default sent it straight through the door that creates. The referent rung goes last among the resolvers and ahead of every door that can create. It asks one judgement -- which standing thing is this about -- and nothing more: what to do about the answer belongs to the surface that owns the state. Resolved redirects the turn and points at it from where the user typed; ambiguous asks and opens nothing; none, and every way the judgement can decline, falls through to exactly today's behaviour. There is no pattern fallback. The catalog is drawn before any session is opened, so it can never offer the row this very message would mint. That ordering is the guarantee and it is tested directly, not inferred from timestamps. Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/core/runtime.py | 10 + src/fateforger/slack_bot/handlers.py | 198 ++++++++++++++++ tests/unit/conftest.py | 288 +++++++++++++++++++++++ tests/unit/test_referent_rung_routing.py | 99 ++++++++ 4 files changed, 595 insertions(+) create mode 100644 tests/unit/test_referent_rung_routing.py diff --git a/src/fateforger/core/runtime.py b/src/fateforger/core/runtime.py index 7cd731dd..850125b0 100644 --- a/src/fateforger/core/runtime.py +++ b/src/fateforger/core/runtime.py @@ -34,6 +34,7 @@ ) from fateforger.agents.timeboxing.kg_constraint_client import KGConstraintMemoryClient from fateforger.core.config import settings +from fateforger.referents import ReferentResolver from fateforger.haunt.agents import HauntingAgent, UserChannelAgent from fateforger.haunt.delivery import deliver_user_facing from fateforger.haunt.event_draft_store import ( @@ -881,6 +882,15 @@ async def dispatch_planning(reminder: PlanningReminder) -> None: setattr(runtime, "planning_session_store", planning_session_store) setattr(runtime, "event_draft_store", event_draft_store) setattr(runtime, "timeboxing_session_store", timeboxing_session_store) + # The referent rung's judge (#345). It rides the same flash pin as the + # other Stage 1 judgements; the route reads this attribute and, finding + # nothing, simply does not ask -- which is what keeps a host that never + # wired it on exactly today's behaviour. + setattr( + runtime, + "referent_resolver", + ReferentResolver(timeboxing_judge_model_client), + ) setattr(runtime, "timeboxing_constraint_store", timeboxing_constraint_store) # The dispatcher revalidates every required-block rung against this rule # before posting it (R3); without it here, those reminders are dropped. diff --git a/src/fateforger/slack_bot/handlers.py b/src/fateforger/slack_bot/handlers.py index 94fc0259..d728b845 100644 --- a/src/fateforger/slack_bot/handlers.py +++ b/src/fateforger/slack_bot/handlers.py @@ -52,6 +52,13 @@ ) from fateforger.core.config import settings from fateforger.core.logging_config import observe_stage_duration, record_error +from fateforger.referents import ( + Ambiguous, + Referent, + Resolved, + TimeboxingReferentProvider, + build_catalog, +) from fateforger.slack_bot.bootstrap import ensure_workspace_ready from fateforger.slack_bot.constraint_review import ( CONSTRAINT_REVIEW_VIEW_CALLBACK_ID, @@ -335,6 +342,53 @@ def _extract_thread_state(result) -> str | None: return None +def _referent_label(referent: Referent) -> str: + """Name one referent out of its own minted fields, never out of prose. + + Day, kind and status are all written by this system, so composing them into + a line a human reads is arithmetic on identifiers and not a reading of + anything the user typed. + """ + day = referent.day.strftime("%A %d %B") if referent.day else "no day locked yet" + return f"{referent.kind} for {day} ({referent.status})" + + +async def _resolve_referent( + *, + runtime, + owner_user_id: str, + message: str, + as_of: datetime, + current_thread: tuple[str, str] | None, +): + """Which standing thing is this message about? `None` when nobody answered. + + `None` means "fall through unchanged" and covers every way this can decline: + a runtime with no resolver wired, no session store to draw a catalog from, + or a judgement that failed. There is deliberately no pattern fallback -- a + resolver this route cannot reach leaves today's behaviour exactly as it was, + loudly metered rather than quietly guessed. + """ + resolver = getattr(runtime, "referent_resolver", None) + if resolver is None: + return None + session_store = getattr(runtime, "timeboxing_session_store", None) + if session_store is None: + return None + try: + catalog = await build_catalog( + [TimeboxingReferentProvider(session_store)], + owner_user_id=owner_user_id, + as_of=as_of, + current_thread=current_thread, + ) + return await resolver.resolve(catalog=catalog, message=message, as_of=as_of) + except Exception: + logger.exception("referent resolution failed for %s", owner_user_id) + record_error(component="surface_intent", error_type="referent_failure") + return None + + async def _maybe_update_timeboxing_thread_constraints( *, client: AsyncWebClient, @@ -2655,6 +2709,11 @@ async def _update_constraints(thread_key: str) -> None: # resolver, so one DM's sticky `user_focus` swallowed the planning card's # own thread: on 2026-09-05 03:43 "Is it planned?" under a planning card # opened a fresh 5-stage session instead of being answered. + # Did a structural resolver claim this message? A fact beats a judgement + # (#310), so the referent rung below asks only while this stays False. + # `agent_type` alone cannot answer that: in the planning channel it already + # reads `timeboxing_agent` from the channel default, which nobody claimed. + structurally_claimed = False if thread_ts: # The DM session key names the whole DM, not this thread, so a live # session would otherwise claim the planning card's own thread. @@ -2674,6 +2733,7 @@ async def _update_constraints(thread_key: str) -> None: component="surface_intent", error_type="resolver_failure" ) if claimed_by_planning: + structurally_claimed = True # `user_focus` is a DM-wide guess at what the user is doing; this # thread hanging off a planning card is a fact, and the fact wins. # An explicit per-thread binding does not lose here -- that one the @@ -2707,6 +2767,75 @@ async def _update_constraints(thread_key: str) -> None: ) else: agent_type = "timeboxing_agent" + structurally_claimed = True + + # The referent rung (#345). Last among the resolvers, and deliberately + # ahead of every door that can create: the catalog is drawn here, before + # any session exists, so it can never offer the row this very message + # would mint. That ordering is the guarantee; `as_of` is only the belt. + # + # It resolves and nothing more. What to do about the answer is a second + # judgement, and it belongs to the surface that owns the state -- so this + # only picks *which* standing thing, then hands the turn to it. + referent: Referent | None = None + ambiguous_candidates: tuple[Referent, ...] = () + if binding is None and not structurally_claimed and text.strip(): + resolution = await _resolve_referent( + runtime=runtime, + owner_user_id=user, + message=_strip_bot_mention(text, bot_user_id), + as_of=datetime.now(UTC), + current_thread=(channel, thread_ts) if thread_ts else None, + ) + if isinstance(resolution, Resolved): + referent = resolution.referent + elif isinstance(resolution, Ambiguous): + ambiguous_candidates = resolution.candidates + + if referent is not None and referent.is_current_surface: + # It is about the conversation it arrived in. There is nowhere to send + # it and nothing to point at; the surface just needs the right agent. + try: + focus.set_focus(origin_key, referent.agent_type, by_user=user, note="referent") + except ValueError: + logger.warning("focus refused the referent agent %s", referent.agent_type) + else: + agent_type = referent.agent_type + referent = None + elif referent is not None: + if not (referent.channel_id and referent.thread_ts): + # A session whose key names no thread (a DM) cannot be a redirect + # target: the redirect is addressed by channel and thread. + logger.info("referent %s has no thread to redirect to", referent.key) + referent = None + else: + try: + redirect_to_referent = focus.set_redirect( + origin_key, + target_channel=referent.channel_id, + target_thread_ts=referent.thread_ts, + agent_type=referent.agent_type, + by_user=user, + note="referent", + ) + except ValueError: + logger.warning( + "focus refused the referent agent %s", referent.agent_type + ) + referent = None + else: + agent_type = referent.agent_type + # Bind the thread that takes the turn, so its own follow-ups + # resolve structurally instead of asking the judge again. + try: + focus.set_focus( + redirect_to_referent.target_key, + referent.agent_type, + by_user=user, + note="referent", + ) + except ValueError: + pass cleaned_text = _strip_bot_mention(text, bot_user_id) # The seam below may prefix `cleaned_text` with card context meant for @@ -3081,6 +3210,75 @@ async def _begin_timeboxing_session_surface( # Whoever answers now knows what the card is. cleaned_text = f"{reply.context}\n\nThe user's reply:\n{cleaned_text}" + if ambiguous_candidates: + # Two standing plans fit equally well. Ask. Opening one on a coin flip + # is exactly the failure this rung exists to stop, and so is opening a + # third because neither could be picked. + lines: list[str] = [] + candidate_blocks: list[dict] = [ + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": ":thinking_face: I'm not sure which plan you mean.", + }, + } + ] + for candidate in ambiguous_candidates: + label = _referent_label(candidate) + candidate_link = ( + await _permalink(candidate.channel_id, candidate.thread_ts) + if candidate.channel_id and candidate.thread_ts + else None + ) + lines.append(f"- {label}" + (f" ({candidate_link})" if candidate_link else "")) + candidate_blocks.append( + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": ( + f"- <{candidate_link}|{label}>" + if candidate_link + else f"- {label}" + ), + }, + } + ) + candidate_blocks.append( + { + "type": "context", + "elements": [ + {"type": "mrkdwn", "text": "Reply in the one you mean."} + ], + } + ) + await _origin_update( + text=( + ":thinking_face: I'm not sure which plan you mean:\n" + + "\n".join(lines) + + "\nReply in the one you mean." + ), + blocks=candidate_blocks, + ) + return + + if referent is not None: + # One line where the user typed, so the turn never happens somewhere + # they were not told about. + referent_link = await _permalink(referent.channel_id, referent.thread_ts) + label = _referent_label(referent) + await _origin_update( + text=( + f":left_right_arrow: Reading that as {label} -- " + + ( + f"continuing there: {referent_link}" + if referent_link + else "continuing in that thread." + ) + ) + ) + redirect = focus.get_redirect(origin_key) if redirect and agent_type == redirect.agent_type: focus.set_user_focus(user, redirect.agent_type) diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index c2aa79a0..b4dada00 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -35,3 +35,291 @@ def _refuse(): monkeypatch.setattr( "fateforger.agents.tasks.board.TaskBoard.from_settings", staticmethod(_refuse) ) + + +# -------------------------------------------------------------------------- +# The routing harness: one door, driven end to end. +# +# `route_slack_event` is the only way to observe the referent rung, because +# where it sits between the structural resolvers and the session-opening +# branch IS the behaviour under test. This drives the real route with fake +# Slack, a fake runtime and a fake resolver, and records the four things the +# rung's tests ask about: what was opened, where the turn was delivered, what +# the origin was told, and in which order the catalog and the opening happened. +# +# Modelled on the fakes in `tests/unit/test_slack_timeboxing_routing.py`. +# -------------------------------------------------------------------------- + +from datetime import date, datetime, timedelta # noqa: E402 +from types import SimpleNamespace # noqa: E402 + +#: The plan-sessions channel from the 2026-09-05 incident. Its route is the +#: hardcoded "open a new session" door the rung has to get in front of. +PLAN_SESSIONS_CHANNEL = "C0AA6HC1RJL" + + +class _HarnessClient: + """Enough Slack to run the route, with every message recorded.""" + + def __init__(self) -> None: + self.messages: list[dict] = [] + self._ts = 0 + + def _next_ts(self) -> str: + self._ts += 1 + return f"p{self._ts}" + + async def chat_postMessage(self, **payload): + ts = self._next_ts() + self.messages.append( + { + "channel": payload.get("channel"), + "ts": ts, + "text": payload.get("text") or "", + "blocks": payload.get("blocks"), + "thread_ts": payload.get("thread_ts"), + } + ) + return {"channel": payload.get("channel"), "ts": ts} + + async def chat_update(self, **payload): + self.messages.append( + { + "channel": payload.get("channel"), + "ts": payload.get("ts"), + "text": payload.get("text") or "", + "blocks": payload.get("blocks"), + "thread_ts": None, + } + ) + return {"ok": True} + + async def chat_getPermalink(self, *, channel: str, message_ts: str): + # The shape Slack returns: the dot is dropped and a `p` prefixed. + return { + "permalink": ( + f"https://example.slack.com/archives/{channel}/" + f"p{message_ts.replace('.', '')}" + ) + } + + async def conversations_open(self, **_payload): + return {"channel": {"id": "D_HUGO"}} + + +class _HarnessRuntime: + """The adapter bag the route reads, plus a record of every delivery.""" + + def __init__(self, *, session_store, resolver) -> None: + self.timeboxing_session_store = session_store + self.referent_resolver = resolver + self.calls: list[tuple[object, object]] = [] + + async def send_message(self, message, recipient): + self.calls.append((message, recipient)) + from autogen_agentchat.messages import TextMessage + + return SimpleNamespace(chat_message=TextMessage(content="ok", source="bot")) + + +class _HarnessSessionStore: + """`load` for the structural resolver, `standing_rows` for the catalog.""" + + def __init__(self, *, rows, sessions, event_order) -> None: + self._rows = rows + self._sessions = sessions + self._event_order = event_order + self.asked: list[str] = [] + + async def load(self, session_key: str): + self.asked.append(session_key) + return self._sessions.get(session_key) + + async def standing_rows( + self, *, owner_user_id: str, as_of, open_within, horizon + ): + self._event_order.append("catalog") + return list(self._rows) + + +class _HarnessPlanning: + def __init__(self, *, owns: bool) -> None: + self._owns = owns + self.ownership_calls: list[tuple[str, str]] = [] + + async def owns_thread(self, *, channel_id: str, thread_ts: str) -> bool: + self.ownership_calls.append((channel_id, thread_ts)) + return self._owns + + async def maybe_handle_thread_reply( + self, *, channel_id: str, thread_ts: str, text: str, thread_respond + ): + from fateforger.slack_bot.planning import ThreadReply, ThreadReplyOutcome + + return ThreadReply(ThreadReplyOutcome.NOT_A_SURFACE) + + +def _standing_row() -> SimpleNamespace: + """One committed day, exactly the row the incident's store already held.""" + + return SimpleNamespace( + session_key=f"{PLAN_SESSIONS_CHANNEL}:1788571682.407949", + status="committed", + planning_date=date(2026, 9, 5), + updated_at=datetime(2026, 9, 5, 1, 41), + revision=7, + gist=("07:00 Oats", "18:00 Gym", "19:30 Dinner"), + ) + + +class _RoutingHarness: + def __init__(self, *, focus, runtime, client, planning, event_order, opened): + self.focus = focus + self.runtime = runtime + self.client = client + self.planning = planning + self.event_order = event_order + self.sessions_opened = opened + self._origin_ts: str | None = None + + @property + def delivered_to(self) -> str | None: + if not self.runtime.calls: + return None + return self.runtime.calls[-1][1].key + + @property + def origin_messages(self) -> list[str]: + """Everything said in the message the user's own words landed on.""" + + return [ + m["text"] + for m in self.client.messages + if self._origin_ts is not None and m["ts"] == self._origin_ts + ] + + async def _route(self, event: dict) -> None: + from fateforger.slack_bot.handlers import route_slack_event + + before = len(self.client.messages) + await route_slack_event( + runtime=self.runtime, + focus=self.focus, + default_agent="receptionist_agent", + event=event, + bot_user_id=None, + say=_harness_unused_say, + client=self.client, + planning=self.planning, + ) + posted = self.client.messages[before:] + if posted and self._origin_ts is None: + self._origin_ts = posted[0]["ts"] + + async def route_top_level(self, text: str) -> None: + await self._route( + { + "channel": PLAN_SESSIONS_CHANNEL, + "user": "U_HUGO", + "text": text, + "ts": "1788600060.000100", + } + ) + + async def route_thread_reply(self, text: str) -> None: + await self._route( + { + "channel": PLAN_SESSIONS_CHANNEL, + "user": "U_HUGO", + "text": text, + "thread_ts": "1788599000.000100", + "ts": "1788600060.000200", + } + ) + + +async def _harness_unused_say(**_kwargs): + return {"channel": PLAN_SESSIONS_CHANNEL, "ts": "unused"} + + +@pytest.fixture() +def routing_harness(monkeypatch: pytest.MonkeyPatch): + """Build a harness around the real `route_slack_event`.""" + + import fateforger.slack_bot.handlers as handlers_mod + from fateforger.slack_bot.focus import FocusManager + + def _build( + *, + resolver, + planning_owns_thread: bool = False, + rows=None, + sessions=None, + ) -> _RoutingHarness: + event_order: list[str] = [] + opened: list[str] = [] + + focus = FocusManager( + ttl_seconds=60, + allowed_agents=["receptionist_agent", "timeboxing_agent"], + ) + store = _HarnessSessionStore( + rows=[_standing_row()] if rows is None else rows, + sessions=sessions or {}, + event_order=event_order, + ) + runtime = _HarnessRuntime(session_store=store, resolver=resolver) + client = _HarnessClient() + planning = _HarnessPlanning(owns=planning_owns_thread) + + real_open = handlers_mod.open_session_surface + + async def _recording_open(*args, **kwargs): + event_order.append("open") + opened.append(kwargs["target_channel"]) + return await real_open(*args, **kwargs) + + async def _fake_turn(**_kwargs): + from fateforger.slack_bot.messages import SlackBlockMessage + + return SlackBlockMessage(text="turn ran", blocks=[]) + + # The route fires `remember` as a background task. With no memory + # store configured -- a worktree, CI -- it posts its own loud warning + # into the origin, and whether it wins the race with the route's own + # first message then depends on the environment rather than on the + # rung. Isolated here for the same reason the task board is above. + async def _no_thread_memory(**_kwargs): + return None + + monkeypatch.setattr( + "fateforger.slack_bot.thread_memory.remember", _no_thread_memory + ) + monkeypatch.setattr(handlers_mod, "open_session_surface", _recording_open) + monkeypatch.setattr(handlers_mod, "_run_adaptive_timebox_turn", _fake_turn) + monkeypatch.setattr(handlers_mod, "_timebox_backend", lambda: "harness") + monkeypatch.setattr( + handlers_mod, + "_agent_for_channel", + lambda channel_id: ( + "timeboxing_agent" if channel_id == PLAN_SESSIONS_CHANNEL else None + ), + ) + monkeypatch.setattr( + handlers_mod, + "_channel_for_agent", + lambda agent_type: ( + PLAN_SESSIONS_CHANNEL if agent_type == "timeboxing_agent" else None + ), + ) + + return _RoutingHarness( + focus=focus, + runtime=runtime, + client=client, + planning=planning, + event_order=event_order, + opened=opened, + ) + + return _build diff --git a/tests/unit/test_referent_rung_routing.py b/tests/unit/test_referent_rung_routing.py new file mode 100644 index 00000000..9e43c4ab --- /dev/null +++ b/tests/unit/test_referent_rung_routing.py @@ -0,0 +1,99 @@ +"""The rung: a message with no structural owner reaches the session it is about. + +The incident this closes: 2026-09-05 13:51, "can you replan today so the gym is +before dinner?" typed top-level in #plan-sessions opened a fresh five-stage +session for a day committed at 01:41. +""" + +from datetime import UTC, date, datetime + +import pytest + +pytest.importorskip("autogen_agentchat") + +from fateforger.referents import Referent +from fateforger.referents.resolver import Ambiguous, NoReferent, Resolved + + +def _ref(ref_id="r1", key="C0AA6HC1RJL:1788571682.407949"): + return Referent( + key=key, + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + channel_id=key.split(":")[0], + thread_ts=key.split(":")[1], + ref_id=ref_id, + ) + + +class _Resolver: + def __init__(self, outcome): + self._outcome = outcome + self.calls = [] + + async def resolve(self, *, catalog, message, as_of): + self.calls.append((catalog, message, as_of)) + if isinstance(self._outcome, BaseException): + raise self._outcome + return self._outcome + + +async def test_a_committed_day_is_reached_instead_of_a_second_session_being_opened( + routing_harness, +): + harness = routing_harness(resolver=_Resolver(Resolved(referent=_ref()))) + await harness.route_top_level("can you replan today so the gym is before dinner?") + assert harness.sessions_opened == [] + assert harness.delivered_to == "C0AA6HC1RJL:1788571682.407949" + + +async def test_the_origin_gets_a_pointer_to_the_thread_that_took_it(routing_harness): + harness = routing_harness(resolver=_Resolver(Resolved(referent=_ref()))) + await harness.route_top_level("replan today") + assert any("1788571682" in text for text in harness.origin_messages) + + +async def test_ambiguity_asks_and_opens_nothing(routing_harness): + harness = routing_harness( + resolver=_Resolver( + Ambiguous(candidates=(_ref("r1"), _ref("r2", "C1:1788500000.000200"))) + ) + ) + await harness.route_top_level("move the gym to the morning") + assert harness.sessions_opened == [] + assert harness.delivered_to is None + assert harness.origin_messages, "the user must be asked which one" + + +async def test_none_falls_through_to_todays_behaviour(routing_harness): + harness = routing_harness(resolver=_Resolver(NoReferent())) + await harness.route_top_level("plan tomorrow") + assert harness.sessions_opened == ["C0AA6HC1RJL"] + + +async def test_a_resolver_failure_falls_through_rather_than_guessing(routing_harness): + harness = routing_harness(resolver=_Resolver(RuntimeError("model down"))) + await harness.route_top_level("replan today") + assert harness.sessions_opened == ["C0AA6HC1RJL"] + + +async def test_the_catalog_is_built_before_any_session_is_opened(routing_harness): + # The ordering IS the guarantee; `created_at < as_of` is only the belt. + harness = routing_harness(resolver=_Resolver(NoReferent())) + await harness.route_top_level("plan tomorrow") + assert harness.event_order.index("catalog") < harness.event_order.index("open") + + +async def test_a_thread_a_structural_resolver_already_claimed_never_reaches_the_rung( + routing_harness, +): + # Structural ownership is a fact and always beats a judgement (#310). + resolver = _Resolver(Resolved(referent=_ref())) + harness = routing_harness(resolver=resolver, planning_owns_thread=True) + await harness.route_thread_reply("is it planned?") + assert resolver.calls == [] From 84661e12ea8f0cd3082a72d6329aafc571db4087 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 15:45:37 +0200 Subject: [PATCH 14/20] fix(slack): the rung asks rather than creates, and never outranks a live thread (#345) Four findings from review, all of them the same shape: a place where the rung could still end in a second session for a day that already had one. A partial catalog is no longer read as "nothing stands". `build_catalog` says so in `complete`, and this rung is the consumer that field was built for: in front of a door that creates, a `none` drawn from a catalog a provider failed to fill now asks instead of creating. Outside such a door it still falls through, because asking there is noise. A resolver that *raises* is a different fact and still falls through unchanged. A positive resolution the route cannot deliver -- a session key naming a whole DM, an agent focus refuses -- used to be dropped to a `logger.info` and then create anyway. That is the incident with the machinery built to catch it reporting success. It is now metered and said out loud, and it never continues into the creating branch. The session store is asked whatever `agent_type` already says. It used to be asked only when `agent_type` was not already `timeboxing_agent`, which in the planning channel is never -- the channel default says so before anyone has claimed anything. On the app_mention path, where focus is never auto-recovered, that left a live session's own thread with no structural claim, and after a restart a judgement could send "move the gym earlier" to a different day. Structural ownership is a fact and beats a judgement (#310). The rung also moves to after the acknowledgement. A model round trip before the first frame is silence a user cannot tell from a dropped message, and only the app_mention path had an ack to hide it. It still runs ahead of every door that can create, which is the ordering that matters. It also makes the origin's first message the route's own by construction rather than by racing the background memory task, so the harness needs no stub for that. Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/slack_bot/handlers.py | 315 ++++++++++++++--------- tests/unit/conftest.py | 11 - tests/unit/test_referent_rung_routing.py | 64 +++++ 3 files changed, 252 insertions(+), 138 deletions(-) diff --git a/src/fateforger/slack_bot/handlers.py b/src/fateforger/slack_bot/handlers.py index d728b845..ed394a32 100644 --- a/src/fateforger/slack_bot/handlers.py +++ b/src/fateforger/slack_bot/handlers.py @@ -54,6 +54,7 @@ from fateforger.core.logging_config import observe_stage_duration, record_error from fateforger.referents import ( Ambiguous, + NoReferent, Referent, Resolved, TimeboxingReferentProvider, @@ -2740,7 +2741,14 @@ async def _update_constraints(thread_key: str) -> None: # user asked for by name. if binding is None and agent_type == "timeboxing_agent": agent_type = channel_default_agent or default_agent - elif agent_type != "timeboxing_agent": + else: + # Asked whatever `agent_type` already says. It used to be asked only + # when `agent_type` was not already `timeboxing_agent`, which in the + # planning channel is never -- the channel default says so before + # anyone has claimed anything. That left a live session's own thread + # with no structural claim on the `app_mention` path, where + # `_auto_recover_timeboxing_focus_for_thread` never runs, and a + # judgement could then send the turn to a different day (#310). session_store = getattr(runtime, "timeboxing_session_store", None) if session_store is not None: session_key = f"{channel}:dm" if is_dm else f"{channel}:{thread_ts}" @@ -2769,74 +2777,6 @@ async def _update_constraints(thread_key: str) -> None: agent_type = "timeboxing_agent" structurally_claimed = True - # The referent rung (#345). Last among the resolvers, and deliberately - # ahead of every door that can create: the catalog is drawn here, before - # any session exists, so it can never offer the row this very message - # would mint. That ordering is the guarantee; `as_of` is only the belt. - # - # It resolves and nothing more. What to do about the answer is a second - # judgement, and it belongs to the surface that owns the state -- so this - # only picks *which* standing thing, then hands the turn to it. - referent: Referent | None = None - ambiguous_candidates: tuple[Referent, ...] = () - if binding is None and not structurally_claimed and text.strip(): - resolution = await _resolve_referent( - runtime=runtime, - owner_user_id=user, - message=_strip_bot_mention(text, bot_user_id), - as_of=datetime.now(UTC), - current_thread=(channel, thread_ts) if thread_ts else None, - ) - if isinstance(resolution, Resolved): - referent = resolution.referent - elif isinstance(resolution, Ambiguous): - ambiguous_candidates = resolution.candidates - - if referent is not None and referent.is_current_surface: - # It is about the conversation it arrived in. There is nowhere to send - # it and nothing to point at; the surface just needs the right agent. - try: - focus.set_focus(origin_key, referent.agent_type, by_user=user, note="referent") - except ValueError: - logger.warning("focus refused the referent agent %s", referent.agent_type) - else: - agent_type = referent.agent_type - referent = None - elif referent is not None: - if not (referent.channel_id and referent.thread_ts): - # A session whose key names no thread (a DM) cannot be a redirect - # target: the redirect is addressed by channel and thread. - logger.info("referent %s has no thread to redirect to", referent.key) - referent = None - else: - try: - redirect_to_referent = focus.set_redirect( - origin_key, - target_channel=referent.channel_id, - target_thread_ts=referent.thread_ts, - agent_type=referent.agent_type, - by_user=user, - note="referent", - ) - except ValueError: - logger.warning( - "focus refused the referent agent %s", referent.agent_type - ) - referent = None - else: - agent_type = referent.agent_type - # Bind the thread that takes the turn, so its own follow-ups - # resolve structurally instead of asking the judge again. - try: - focus.set_focus( - redirect_to_referent.target_key, - referent.agent_type, - by_user=user, - note="referent", - ) - except ValueError: - pass - cleaned_text = _strip_bot_mention(text, bot_user_id) # The seam below may prefix `cleaned_text` with card context meant for # whichever agent answers. `user_reply_text` stays the user's own words, @@ -3210,74 +3150,195 @@ async def _begin_timeboxing_session_surface( # Whoever answers now knows what the card is. cleaned_text = f"{reply.context}\n\nThe user's reply:\n{cleaned_text}" - if ambiguous_candidates: - # Two standing plans fit equally well. Ask. Opening one on a coin flip - # is exactly the failure this rung exists to stop, and so is opening a - # third because neither could be picked. - lines: list[str] = [] - candidate_blocks: list[dict] = [ - { - "type": "section", - "text": { - "type": "mrkdwn", - "text": ":thinking_face: I'm not sure which plan you mean.", - }, - } - ] - for candidate in ambiguous_candidates: - label = _referent_label(candidate) - candidate_link = ( - await _permalink(candidate.channel_id, candidate.thread_ts) - if candidate.channel_id and candidate.thread_ts - else None - ) - lines.append(f"- {label}" + (f" ({candidate_link})" if candidate_link else "")) - candidate_blocks.append( + # The referent rung (#345). Last among the resolvers: structural ownership + # is a fact and beats a judgement (#310), so this asks only while nothing + # above claimed the message. + # + # Two placements are load-bearing. It runs *after* the acknowledgement, + # because a model round trip before the first frame is silence a user + # cannot tell from a dropped message. And it runs *before* every door that + # can create, so the catalog can never offer the row this very message + # would mint -- that ordering is the guarantee, and `as_of` is only the + # belt. + # + # It resolves and nothing more. What to do about the answer is a second + # judgement, and it belongs to the surface that owns the state. + if binding is None and not structurally_claimed and text.strip(): + # The one place a wrong "nothing stands" is unrecoverable: a door that + # creates mints a second session for a day that already has one, which + # is the whole of the 2026-09-05 incident. Read before the rung can + # move `agent_type`. + would_start_a_session = agent_type == "timeboxing_agent" and not thread_ts + resolution = await _resolve_referent( + runtime=runtime, + owner_user_id=user, + message=_strip_bot_mention(text, bot_user_id), + as_of=datetime.now(UTC), + current_thread=(channel, thread_ts) if thread_ts else None, + ) + + if isinstance(resolution, Resolved): + referent = resolution.referent + label = _referent_label(referent) + if referent.is_current_surface: + # It is about the conversation it arrived in. Nowhere to send + # it and nothing to point at; the surface needs the right agent + # and no more. + try: + focus.set_focus( + origin_key, referent.agent_type, by_user=user, note="referent" + ) + except ValueError: + logger.warning( + "focus refused the referent agent %s", referent.agent_type + ) + else: + agent_type = referent.agent_type + else: + # A positive resolution that cannot be delivered may never fall + # through into a creating door: the judgement said this message + # is about a plan that already exists, and creating a second one + # anyway is the incident with the machinery built to catch it + # reporting success. Say what was understood instead. + reachable = bool(referent.channel_id and referent.thread_ts) + redirect_to_referent = None + if reachable: + try: + redirect_to_referent = focus.set_redirect( + origin_key, + target_channel=referent.channel_id, + target_thread_ts=referent.thread_ts, + agent_type=referent.agent_type, + by_user=user, + note="referent", + ) + except ValueError: + logger.warning( + "focus refused the referent agent %s", referent.agent_type + ) + if redirect_to_referent is None: + # A session key that names no thread (a DM's `{channel}:dm`) + # has no address the redirect can carry, and an agent focus + # refuses is not one this route may hand a turn to. + logger.info( + "resolved referent %s cannot be handed the turn", referent.key + ) + record_error( + component="surface_intent", error_type="referent_unreachable" + ) + await _origin_update( + text=( + f":left_right_arrow: That reads as {label}, and I " + "can't hand it to that conversation from here. Say " + "it there and I'll pick it up." + ) + ) + return + agent_type = referent.agent_type + # Bind the thread that takes the turn, so its own follow-ups + # resolve structurally instead of asking the judge again. + try: + focus.set_focus( + redirect_to_referent.target_key, + referent.agent_type, + by_user=user, + note="referent", + ) + except ValueError: + pass + # One line where the user typed, so the turn never happens + # somewhere they were not told about. + referent_link = await _permalink( + referent.channel_id, referent.thread_ts + ) + await _origin_update( + text=( + f":left_right_arrow: Reading that as {label} -- " + + ( + f"continuing there: {referent_link}" + if referent_link + else "continuing in that thread." + ) + ) + ) + + elif isinstance(resolution, Ambiguous): + # Two standing plans fit equally well. Ask. Opening one on a coin + # flip is exactly the failure this rung exists to stop, and so is + # opening a third because neither could be picked. + lines: list[str] = [] + candidate_blocks: list[dict] = [ { "type": "section", "text": { "type": "mrkdwn", - "text": ( - f"- <{candidate_link}|{label}>" - if candidate_link - else f"- {label}" - ), + "text": ":thinking_face: I'm not sure which plan you mean.", }, } + ] + for candidate in resolution.candidates: + candidate_label = _referent_label(candidate) + candidate_link = ( + await _permalink(candidate.channel_id, candidate.thread_ts) + if candidate.channel_id and candidate.thread_ts + else None + ) + lines.append( + f"- {candidate_label}" + + (f" ({candidate_link})" if candidate_link else "") + ) + candidate_blocks.append( + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": ( + f"- <{candidate_link}|{candidate_label}>" + if candidate_link + else f"- {candidate_label}" + ), + }, + } + ) + candidate_blocks.append( + { + "type": "context", + "elements": [ + {"type": "mrkdwn", "text": "Reply in the one you mean."} + ], + } ) - candidate_blocks.append( - { - "type": "context", - "elements": [ - {"type": "mrkdwn", "text": "Reply in the one you mean."} - ], - } - ) - await _origin_update( - text=( - ":thinking_face: I'm not sure which plan you mean:\n" - + "\n".join(lines) - + "\nReply in the one you mean." - ), - blocks=candidate_blocks, - ) - return + await _origin_update( + text=( + ":thinking_face: I'm not sure which plan you mean:\n" + + "\n".join(lines) + + "\nReply in the one you mean." + ), + blocks=candidate_blocks, + ) + return - if referent is not None: - # One line where the user typed, so the turn never happens somewhere - # they were not told about. - referent_link = await _permalink(referent.channel_id, referent.thread_ts) - label = _referent_label(referent) - await _origin_update( - text=( - f":left_right_arrow: Reading that as {label} -- " - + ( - f"continuing there: {referent_link}" - if referent_link - else "continuing in that thread." + elif ( + isinstance(resolution, NoReferent) + and not resolution.catalog_complete + and would_start_a_session + ): + # A provider failed, so `none` is not evidence that nothing stands + # (`referents/catalog.py`'s own contract). Everywhere else that is + # survivable; in front of a door that creates it is the incident, + # so ask rather than create. + record_error( + component="surface_intent", error_type="referent_catalog_partial" + ) + await _origin_update( + text=( + ":warning: I couldn't check what you already have planned, " + "so I won't start a second session over the top of it. Say " + "that again and I'll retry, or open one from the day's own " + "thread if there is one." ) ) - ) + return redirect = focus.get_redirect(origin_key) if redirect and agent_type == redirect.agent_type: diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index b4dada00..09ec2364 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -284,17 +284,6 @@ async def _fake_turn(**_kwargs): return SlackBlockMessage(text="turn ran", blocks=[]) - # The route fires `remember` as a background task. With no memory - # store configured -- a worktree, CI -- it posts its own loud warning - # into the origin, and whether it wins the race with the route's own - # first message then depends on the environment rather than on the - # rung. Isolated here for the same reason the task board is above. - async def _no_thread_memory(**_kwargs): - return None - - monkeypatch.setattr( - "fateforger.slack_bot.thread_memory.remember", _no_thread_memory - ) monkeypatch.setattr(handlers_mod, "open_session_surface", _recording_open) monkeypatch.setattr(handlers_mod, "_run_adaptive_timebox_turn", _fake_turn) monkeypatch.setattr(handlers_mod, "_timebox_backend", lambda: "harness") diff --git a/tests/unit/test_referent_rung_routing.py b/tests/unit/test_referent_rung_routing.py index 9e43c4ab..2b14ddb4 100644 --- a/tests/unit/test_referent_rung_routing.py +++ b/tests/unit/test_referent_rung_routing.py @@ -6,6 +6,7 @@ """ from datetime import UTC, date, datetime +from types import SimpleNamespace import pytest @@ -97,3 +98,66 @@ async def test_a_thread_a_structural_resolver_already_claimed_never_reaches_the_ harness = routing_harness(resolver=resolver, planning_owns_thread=True) await harness.route_thread_reply("is it planned?") assert resolver.calls == [] + + +def _dm_ref(): + """A session whose key names a whole DM, and therefore names no thread. + + `{channel}:dm` is a real shape the provider emits. It has no address a + redirect can carry, which is the case that used to be dropped silently. + """ + return Referent( + key="D_HUGO:dm", + agent_type="timeboxing_agent", + kind="a plan for one day", + day=date(2026, 9, 5), + status="committed", + never_used=False, + last_activity=datetime(2026, 9, 5, 1, 41, tzinfo=UTC), + accepts=("revise the committed plan",), + channel_id="D_HUGO", + thread_ts=None, + ref_id="r1", + ) + + +async def test_a_partial_catalog_never_creates_over_a_day_it_could_not_see( + routing_harness, +): + # `complete=False` says a provider failed, so `none` is not evidence that + # nothing stands. Everywhere else that is survivable; in front of a door + # that creates it is the 2026-09-05 incident with the machinery built to + # catch it reporting "nothing stands". + harness = routing_harness(resolver=_Resolver(NoReferent(catalog_complete=False))) + await harness.route_top_level("can you replan today so the gym is before dinner?") + assert harness.sessions_opened == [] + assert harness.origin_messages, "the user must be told what could not be checked" + + +async def test_a_resolution_it_cannot_deliver_still_never_creates(routing_harness): + # The judgement was positive and right; the route just has no way to hand + # the turn over. Creating a second session anyway is the incident again. + harness = routing_harness(resolver=_Resolver(Resolved(referent=_dm_ref()))) + await harness.route_top_level("move the gym earlier") + assert harness.sessions_opened == [] + assert harness.delivered_to is None + assert harness.origin_messages, "an undeliverable resolution must be said aloud" + + +async def test_a_live_sessions_own_thread_is_claimed_before_the_judge_is_asked( + routing_harness, +): + # #310, on the `app_mention` path: focus is never auto-recovered there, and + # the channel default already reads `timeboxing_agent`, so the session store + # has to be asked anyway. Otherwise a restart turns "move the gym earlier", + # typed inside Tuesday's live thread, into a turn on some other day. + resolver = _Resolver(Resolved(referent=_ref())) + harness = routing_harness( + resolver=resolver, + sessions={"C0AA6HC1RJL:1788599000.000100": SimpleNamespace(status="open")}, + ) + await harness.route_thread_reply("move the gym earlier") + assert harness.runtime.timeboxing_session_store.asked == [ + "C0AA6HC1RJL:1788599000.000100" + ] + assert resolver.calls == [] From 19299665445168981a965c83514d982b3fa2c6e2 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 16:00:52 +0200 Subject: [PATCH 15/20] fix(slack): one reading of "would this turn mint a session", and the handoff door obeys it (#345) The rung's own predicate for "is there a door that creates ahead of this message" was `agent_type == "timeboxing_agent" and not thread_ts`. It was a third reading of a condition two other sites already derived their own way, and it missed the door a plain "can you replan today so the gym is before dinner?" actually goes through: typed anywhere but the planning channel, the turn goes to the receptionist, nothing about it reads as timeboxing, and the receptionist hands it off into a session surface built over a day the catalog could not see. `_turn_can_mint_a_session` replaces all of it with one reading, anchored on `open_session_surface` -- the thing that mints -- and on the fact that any agent which is not the timeboxing one can hand the turn to it. It over-answers on purpose, because nobody can know before the turn whether the model will hand off, and the two mistakes are not symmetric: asking when nothing would have been created costs a retry, creating over a day that already stands is the incident. The door answers back. The check sits inside the sole caller of `open_session_surface`, so a third door cannot be added without inheriting it, and a mint the rung was told could not happen is logged and metered rather than inherited in silence. Also: the branch that reads `catalog_complete` now says why the other two outcomes need not, and a resolution that cannot be delivered names where the plan lives instead of only saying it cannot be reached. Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/slack_bot/handlers.py | 87 +++++++++++++++++++++--- tests/unit/conftest.py | 21 ++++-- tests/unit/test_referent_rung_routing.py | 20 ++++++ 3 files changed, 115 insertions(+), 13 deletions(-) diff --git a/src/fateforger/slack_bot/handlers.py b/src/fateforger/slack_bot/handlers.py index ed394a32..622ff6f5 100644 --- a/src/fateforger/slack_bot/handlers.py +++ b/src/fateforger/slack_bot/handlers.py @@ -354,6 +354,39 @@ def _referent_label(referent: Referent) -> str: return f"{referent.kind} for {day} ({referent.status})" +def _turn_can_mint_a_session( + *, agent_type: str, is_dm: bool, thread_ts: str | None +) -> bool: + """Could this turn still end at a door that mints a timeboxing session? + + One reading, asked once before the turn runs. There were three separate + re-derivations of this condition and they had already drifted apart; the + one the referent rung carried missed the handoff door entirely, which is + the door a plain "can you replan today so the gym is before dinner?" goes + through when it is typed anywhere but the planning channel. + + `open_session_surface` is the thing that actually mints, and it has two + callers -- the fresh-channel start and the handoff. The legacy backend + reaches the same place by sending `StartTimeboxing` instead. This answers + for all of them: + + * already on `timeboxing_agent`: a message with no thread to continue is a + fresh start whichever backend takes it. A DM reaches the session by its + `{channel}:dm` key instead, and mints nothing. + * on any other agent: that agent can hand the turn off, and a handoff to + timeboxing builds a surface of its own -- from a thread reply as readily + as from a root message, and in a DM as readily as in a channel. + + It over-answers on purpose. Nobody can know before the turn whether the + model will hand off, and the two mistakes are not symmetric: asking when + nothing would have been created costs a retry, while creating over a day + that already stands is the incident this whole rung exists to prevent. + """ + if agent_type == "timeboxing_agent": + return not is_dm and not thread_ts + return True + + async def _resolve_referent( *, runtime, @@ -2777,6 +2810,12 @@ async def _update_constraints(thread_key: str) -> None: agent_type = "timeboxing_agent" structurally_claimed = True + # Read once, before the rung can move `agent_type`, and consulted both by + # the rung and by the door that mints. One reading, two users, no drift. + turn_can_mint_a_session = _turn_can_mint_a_session( + agent_type=agent_type, is_dm=is_dm, thread_ts=thread_ts + ) + cleaned_text = _strip_bot_mention(text, bot_user_id) # The seam below may prefix `cleaned_text` with card context meant for # whichever agent answers. `user_reply_text` stays the user's own words, @@ -2906,6 +2945,24 @@ async def _begin_timeboxing_session_surface( one builder for every door. What stays here is what only a Slack event can supply: the origin link, the working card, and the turn. """ + if not turn_can_mint_a_session: + # The rung was told this turn could not mint one, and here it is + # minting one. That means `_turn_can_mint_a_session` has stopped + # covering every door, and a partial catalog can now reach this + # line believing nothing stands. Loud and metered at the site, + # rather than inherited in silence by whatever reads the store + # next -- which is exactly how the 2026-09-05 duplicate happened. + logger.error( + "a session is being minted at a door the rung did not foresee " + "(agent=%s is_dm=%s thread_ts=%s channel=%s)", + agent_type, + is_dm, + thread_ts, + channel, + ) + record_error( + component="surface_intent", error_type="unforeseen_session_mint" + ) surface = await open_session_surface( client, focus, @@ -3164,11 +3221,6 @@ async def _begin_timeboxing_session_surface( # It resolves and nothing more. What to do about the answer is a second # judgement, and it belongs to the surface that owns the state. if binding is None and not structurally_claimed and text.strip(): - # The one place a wrong "nothing stands" is unrecoverable: a door that - # creates mints a second session for a day that already has one, which - # is the whole of the 2026-09-05 incident. Read before the rung can - # move `agent_type`. - would_start_a_session = agent_type == "timeboxing_agent" and not thread_ts resolution = await _resolve_referent( runtime=runtime, owner_user_id=user, @@ -3226,11 +3278,21 @@ async def _begin_timeboxing_session_surface( record_error( component="surface_intent", error_type="referent_unreachable" ) + # Name where, at least. A DM channel id has no `<#...>` + # rendering, and telling someone their message went nowhere + # without saying where to put it is the whole dead end. + # (`D...` is Slack's own prefix, not a reading of anything + # the user wrote -- the same test the route opens with.) + where = ( + "our DM" + if str(referent.channel_id or "").startswith("D") + else f"<#{referent.channel_id}>" + ) await _origin_update( text=( - f":left_right_arrow: That reads as {label}, and I " - "can't hand it to that conversation from here. Say " - "it there and I'll pick it up." + f":left_right_arrow: That reads as {label}, which " + f"lives in {where} -- and I can't hand a message to " + "it from here. Say it there and I'll pick it up." ) ) return @@ -3321,12 +3383,19 @@ async def _begin_timeboxing_session_surface( elif ( isinstance(resolution, NoReferent) and not resolution.catalog_complete - and would_start_a_session + and turn_can_mint_a_session ): # A provider failed, so `none` is not evidence that nothing stands # (`referents/catalog.py`'s own contract). Everywhere else that is # survivable; in front of a door that creates it is the incident, # so ask rather than create. + # + # `catalog_complete` rides every outcome and is acted on only here, + # deliberately. The flag changes what a *weak* answer licenses, and + # the other two outcomes license nothing that a partial catalog + # could make dangerous: `Resolved` hands the turn to a session that + # demonstrably exists, and `Ambiguous` already asks. `NoReferent` is + # the only one whose answer is "so go ahead and create". record_error( component="surface_intent", error_type="referent_catalog_partial" ) diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index 09ec2364..7905003b 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -110,15 +110,25 @@ async def conversations_open(self, **_payload): class _HarnessRuntime: """The adapter bag the route reads, plus a record of every delivery.""" - def __init__(self, *, session_store, resolver) -> None: + def __init__(self, *, session_store, resolver, handoff_to=None) -> None: self.timeboxing_session_store = session_store self.referent_resolver = resolver self.calls: list[tuple[object, object]] = [] + self._handoff_to = handoff_to async def send_message(self, message, recipient): self.calls.append((message, recipient)) from autogen_agentchat.messages import TextMessage + if self._handoff_to is not None: + # `_extract_handoff_target` reads `.target`; a plain string is one + # of the shapes it accepts. This is how the receptionist reaches + # the door that mints a session without any channel default. + return SimpleNamespace( + chat_message=SimpleNamespace( + target=self._handoff_to, content="handing off", source="bot" + ) + ) return SimpleNamespace(chat_message=TextMessage(content="ok", source="bot")) @@ -216,10 +226,10 @@ async def _route(self, event: dict) -> None: if posted and self._origin_ts is None: self._origin_ts = posted[0]["ts"] - async def route_top_level(self, text: str) -> None: + async def route_top_level(self, text: str, *, channel: str | None = None) -> None: await self._route( { - "channel": PLAN_SESSIONS_CHANNEL, + "channel": channel or PLAN_SESSIONS_CHANNEL, "user": "U_HUGO", "text": text, "ts": "1788600060.000100", @@ -255,6 +265,7 @@ def _build( planning_owns_thread: bool = False, rows=None, sessions=None, + handoff_to: str | None = None, ) -> _RoutingHarness: event_order: list[str] = [] opened: list[str] = [] @@ -268,7 +279,9 @@ def _build( sessions=sessions or {}, event_order=event_order, ) - runtime = _HarnessRuntime(session_store=store, resolver=resolver) + runtime = _HarnessRuntime( + session_store=store, resolver=resolver, handoff_to=handoff_to + ) client = _HarnessClient() planning = _HarnessPlanning(owns=planning_owns_thread) diff --git a/tests/unit/test_referent_rung_routing.py b/tests/unit/test_referent_rung_routing.py index 2b14ddb4..bc1544a9 100644 --- a/tests/unit/test_referent_rung_routing.py +++ b/tests/unit/test_referent_rung_routing.py @@ -161,3 +161,23 @@ async def test_a_live_sessions_own_thread_is_claimed_before_the_judge_is_asked( "C0AA6HC1RJL:1788599000.000100" ] assert resolver.calls == [] + + +async def test_a_partial_catalog_never_creates_through_the_handoff_door( + routing_harness, +): + # The door the rung's first predicate could not see. Typed anywhere but the + # planning channel, the turn goes to the receptionist -- so nothing about it + # reads as timeboxing -- and the receptionist hands it off, and the handoff + # builds a session surface of its own over a day the catalog could not see. + harness = routing_harness( + resolver=_Resolver(NoReferent(catalog_complete=False)), + handoff_to="timeboxing_agent", + ) + await harness.route_top_level( + "can you replan today so the gym is before dinner?", channel="C_GENERAL" + ) + assert harness.sessions_opened == [] + assert harness.runtime.calls == [], "nothing may be delivered, so nothing hands off" + assert harness.origin_messages, "the user must be told what could not be checked" + From 3a4342a5e0d732b4c9034620b1cfd547e41c8a09 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 16:06:22 +0200 Subject: [PATCH 16/20] fix(slack): a store outage stops sessions being minted, not every conversation (#345) The rung refused up front whenever a door that could create lay ahead. That was every message reaching the receptionist, because the receptionist can hand the turn off -- so during a store outage someone asking what the weather is was told the planner could not check their calendar. A rare failure with the blast radius of every conversation. The direction was right and the place was wrong. "Would this turn mint a session" is not a yes or no: on `timeboxing_agent` with no thread to continue it is certain, and there the turn has nothing else to do, so refusing up front is right. Everywhere else it depends on a judgement nobody has made yet -- whether the model hands off -- so the turn now answers, carrying the fact that the catalog was short, and whatever door turns out to create refuses at that moment instead. Both paths say the same words. Which internal door the turn took is not something a user should have to model, and a hard refusal would have surfaced as "something went wrong", naming a bug rather than a store that could not be read. The two doors `_begin_timeboxing_session_surface` never sees -- the legacy fallback and the in-thread handoff -- ask `isinstance(msg, StartTimeboxing)` of the message the builder actually produced. That reads the decision rather than re-deriving it, so it cannot drift, and it closes the gap round 2 left. Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/slack_bot/handlers.py | 135 +++++++++++++++-------- tests/unit/test_referent_rung_routing.py | 22 +++- 2 files changed, 109 insertions(+), 48 deletions(-) diff --git a/src/fateforger/slack_bot/handlers.py b/src/fateforger/slack_bot/handlers.py index 622ff6f5..44c3a47a 100644 --- a/src/fateforger/slack_bot/handlers.py +++ b/src/fateforger/slack_bot/handlers.py @@ -354,37 +354,59 @@ def _referent_label(referent: Referent) -> str: return f"{referent.kind} for {day} ({referent.status})" -def _turn_can_mint_a_session( +#: This turn ends at a door that mints a session. There is nothing else it can +#: do with itself. +MINT_CERTAIN = "certain" +#: It might. The turn goes to some other agent, and that agent can hand it off +#: to timeboxing, which builds a session surface of its own. +MINT_POSSIBLE = "possible" +#: It cannot. There is a thread to continue, or a DM whose `{channel}:dm` key +#: names a session that already exists. +MINT_NO = "no" + +#: Said in every place a partial catalog stops a session being created: up front +#: when the turn was certainly headed for one, and at the door when a handoff +#: turned out to be. The same words in both, so which internal path the turn +#: took is not something a user has to model. +PARTIAL_CATALOG_ASK = ( + ":warning: I couldn't check what you already have planned, so I won't start " + "a second session over the top of it. Say that again and I'll retry, or open " + "one from the day's own thread if there is one." +) + + +def _session_mint_prospect( *, agent_type: str, is_dm: bool, thread_ts: str | None -) -> bool: - """Could this turn still end at a door that mints a timeboxing session? +) -> str: + """Will this turn mint a timeboxing session -- certainly, possibly, or not? One reading, asked once before the turn runs. There were three separate - re-derivations of this condition and they had already drifted apart; the - one the referent rung carried missed the handoff door entirely, which is - the door a plain "can you replan today so the gym is before dinner?" goes - through when it is typed anywhere but the planning channel. + re-derivations of this and they had already drifted apart; the one the + referent rung carried missed the handoff door entirely, which is the door a + plain "can you replan today so the gym is before dinner?" goes through when + it is typed anywhere but the planning channel. `open_session_surface` is the thing that actually mints, and it has two callers -- the fresh-channel start and the handoff. The legacy backend - reaches the same place by sending `StartTimeboxing` instead. This answers - for all of them: - - * already on `timeboxing_agent`: a message with no thread to continue is a - fresh start whichever backend takes it. A DM reaches the session by its - `{channel}:dm` key instead, and mints nothing. - * on any other agent: that agent can hand the turn off, and a handoff to - timeboxing builds a surface of its own -- from a thread reply as readily - as from a root message, and in a DM as readily as in a channel. - - It over-answers on purpose. Nobody can know before the turn whether the - model will hand off, and the two mistakes are not symmetric: asking when - nothing would have been created costs a retry, while creating over a day - that already stands is the incident this whole rung exists to prevent. + reaches the same place by sending `StartTimeboxing` instead. + + The distinction between `certain` and `possible` is what keeps a rare store + outage from becoming a bot-wide one. On `certain` the turn has nothing else + to do, so a caller may refuse up front. On `possible` it depends on a + judgement that has not been made yet -- whether the model hands off -- so a + caller must let the turn run and refuse at the door, if it comes to one. + Refusing up front there would answer "I couldn't check your calendar" to + someone asking what the weather is. """ if agent_type == "timeboxing_agent": - return not is_dm and not thread_ts - return True + # A message with no thread to continue is a fresh start whichever + # backend takes it. A DM reaches its session by the `{channel}:dm` key + # and mints nothing. + return MINT_CERTAIN if (not is_dm and not thread_ts) else MINT_NO + # Any other agent can hand the turn off, and a handoff to timeboxing builds + # a surface of its own -- from a thread reply as readily as from a root + # message, and in a DM as readily as in a channel. + return MINT_POSSIBLE async def _resolve_referent( @@ -2810,11 +2832,15 @@ async def _update_constraints(thread_key: str) -> None: agent_type = "timeboxing_agent" structurally_claimed = True - # Read once, before the rung can move `agent_type`, and consulted both by - # the rung and by the door that mints. One reading, two users, no drift. - turn_can_mint_a_session = _turn_can_mint_a_session( + # Read once, before the rung can move `agent_type`, and consulted by the + # rung and by every door that mints. One reading, several users, no drift. + mint_prospect = _session_mint_prospect( agent_type=agent_type, is_dm=is_dm, thread_ts=thread_ts ) + # Set when the rung found the catalog short and let the turn run anyway + # because nothing was certain to be created. Whatever door then turns out to + # create refuses instead, in the rung's own words. + catalog_was_partial = False cleaned_text = _strip_bot_mention(text, bot_user_id) # The seam below may prefix `cleaned_text` with card context meant for @@ -2905,6 +2931,13 @@ async def _permalink(channel_id: str, message_ts: str) -> str | None: except Exception: return None + async def _refuse_to_mint_over_a_partial_catalog() -> None: + """Say, in the rung's own words, why no session is being created.""" + record_error( + component="surface_intent", error_type="referent_catalog_partial" + ) + await _origin_update(text=PARTIAL_CATALOG_ASK) + async def _origin_link_to_thread( *, channel_id: str, thread_ts: str, agent_label: str ) -> None: @@ -2945,9 +2978,12 @@ async def _begin_timeboxing_session_surface( one builder for every door. What stays here is what only a Slack event can supply: the origin link, the working card, and the turn. """ - if not turn_can_mint_a_session: + if catalog_was_partial: + await _refuse_to_mint_over_a_partial_catalog() + return + if mint_prospect == MINT_NO: # The rung was told this turn could not mint one, and here it is - # minting one. That means `_turn_can_mint_a_session` has stopped + # minting one. That means `_session_mint_prospect` has stopped # covering every door, and a partial catalog can now reach this # line believing nothing stands. Loud and metered at the site, # rather than inherited in silence by whatever reads the store @@ -3380,11 +3416,7 @@ async def _begin_timeboxing_session_surface( ) return - elif ( - isinstance(resolution, NoReferent) - and not resolution.catalog_complete - and turn_can_mint_a_session - ): + elif isinstance(resolution, NoReferent) and not resolution.catalog_complete: # A provider failed, so `none` is not evidence that nothing stands # (`referents/catalog.py`'s own contract). Everywhere else that is # survivable; in front of a door that creates it is the incident, @@ -3396,18 +3428,16 @@ async def _begin_timeboxing_session_surface( # could make dangerous: `Resolved` hands the turn to a session that # demonstrably exists, and `Ambiguous` already asks. `NoReferent` is # the only one whose answer is "so go ahead and create". - record_error( - component="surface_intent", error_type="referent_catalog_partial" - ) - await _origin_update( - text=( - ":warning: I couldn't check what you already have planned, " - "so I won't start a second session over the top of it. Say " - "that again and I'll retry, or open one from the day's own " - "thread if there is one." - ) - ) - return + if mint_prospect == MINT_CERTAIN: + # Nothing else this turn could do. Refusing at the end would + # spend a model call and a card to say the same thing later. + await _refuse_to_mint_over_a_partial_catalog() + return + # It only *might* create, by a handoff nobody has decided on yet. + # Let it answer -- someone asking what the weather is must not be + # told the planner could not read their calendar -- and refuse at + # the door, if the turn reaches one. + catalog_was_partial = True redirect = focus.get_redirect(origin_key) if redirect and agent_type == redirect.agent_type: @@ -3536,6 +3566,13 @@ async def _begin_timeboxing_session_surface( else None ), ) + if catalog_was_partial and isinstance(msg, StartTimeboxing): + # The legacy backend's own creating door. Asked of the message that was + # actually built rather than of the event shape all over again -- + # `StartTimeboxing` *is* the decision to create, `TimeboxingUserReply` + # is not -- so this cannot drift from the builder that made it. + await _refuse_to_mint_over_a_partial_catalog() + return recipient_key = origin_key if forced_thread_root: recipient_key = f"{channel}:{forced_thread_root}" @@ -3802,6 +3839,14 @@ async def _turn_heartbeat() -> None: True if (is_dm and handoff_target == "timeboxing_agent") else None ), ) + if catalog_was_partial and isinstance(handoff_msg, StartTimeboxing): + # The door `_begin_timeboxing_session_surface` never sees: the + # in-thread fallback taken when the session channel is not + # reachable. Asked of the message that was actually built, not of + # the event shape all over again -- `StartTimeboxing` *is* the + # decision to create, and `TimeboxingUserReply` is not. + await _refuse_to_mint_over_a_partial_catalog() + return try: result = await runtime.send_message( handoff_msg, recipient=AgentId(handoff_target, key=origin_key) diff --git a/tests/unit/test_referent_rung_routing.py b/tests/unit/test_referent_rung_routing.py index bc1544a9..62bf5e1d 100644 --- a/tests/unit/test_referent_rung_routing.py +++ b/tests/unit/test_referent_rung_routing.py @@ -13,6 +13,7 @@ pytest.importorskip("autogen_agentchat") from fateforger.referents import Referent +from fateforger.slack_bot.handlers import PARTIAL_CATALOG_ASK from fateforger.referents.resolver import Ambiguous, NoReferent, Resolved @@ -163,13 +164,16 @@ async def test_a_live_sessions_own_thread_is_claimed_before_the_judge_is_asked( assert resolver.calls == [] -async def test_a_partial_catalog_never_creates_through_the_handoff_door( +async def test_a_partial_catalog_is_stopped_at_the_handoff_door_not_before_it( routing_harness, ): # The door the rung's first predicate could not see. Typed anywhere but the # planning channel, the turn goes to the receptionist -- so nothing about it # reads as timeboxing -- and the receptionist hands it off, and the handoff # builds a session surface of its own over a day the catalog could not see. + # + # The turn still runs: whether it hands off is a judgement nobody has made + # yet. It is stopped at the moment it tries to create, and told why. harness = routing_harness( resolver=_Resolver(NoReferent(catalog_complete=False)), handoff_to="timeboxing_agent", @@ -178,6 +182,18 @@ async def test_a_partial_catalog_never_creates_through_the_handoff_door( "can you replan today so the gym is before dinner?", channel="C_GENERAL" ) assert harness.sessions_opened == [] - assert harness.runtime.calls == [], "nothing may be delivered, so nothing hands off" - assert harness.origin_messages, "the user must be told what could not be checked" + assert len(harness.runtime.calls) == 1, "the turn runs; only the mint is refused" + assert PARTIAL_CATALOG_ASK in harness.origin_messages + + +async def test_a_partial_catalog_does_not_break_every_other_conversation( + routing_harness, +): + # A store outage must not become a bot-wide outage. Nothing about this + # message is headed for a session, so it is answered like any other. + harness = routing_harness(resolver=_Resolver(NoReferent(catalog_complete=False))) + await harness.route_top_level("what is the weather?", channel="C_GENERAL") + assert harness.sessions_opened == [] + assert len(harness.runtime.calls) == 1, "the receptionist must still answer" + assert PARTIAL_CATALOG_ASK not in harness.origin_messages From 20d9a547c5937eafe316861a10f490a6409deaed Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 16:33:06 +0200 Subject: [PATCH 17/20] fix(slack): a partial catalog is refused by the store, not by the message type (#345) Creation is decided by whether the session key is already known. Three rounds asked the event shape or the message type instead, and all three were wrong the same way: `StartTimeboxing` is not the only creating message. `TimeboxingUserReply` mints one through `on_user_reply`'s `_ensure_uncommitted_session`, and on the kernel backend the route sends neither message and mints at `load_or_create`. So `MINT_NO` -- "a thread to continue, or a DM whose key names a session that already exists" -- was a claim about the store made without asking the store, and a DM top-level message and a first-touch thread reply both wrote a row under it. `_session_mint_prospect` and its tri-state are gone, and so are both `isinstance(msg, StartTimeboxing)` guards. In their place, one question asked at every door that hands a turn to timeboxing: is the catalog short, and does the key this turn will use name nothing yet? It fails closed -- an unreadable store answers the same as an empty one, because that is the condition that made the catalog short in the first place. Four doors ask it now, up from two-and-a-half: the session surface builder (which mints the key it would ask about, so it refuses outright), the route's own turn on both backends, the in-thread handoff fallback, and the redirect branch, which nothing had guarded. The harness was the reason none of this showed. It stubbed the kernel turn and the runtime delivery with functions that wrote nothing, so the two doors that mint without `open_session_surface` were invisible and both guards could be replaced with `False` for a green suite. Both fakes now write the row the real thing writes, and the new tests assert on rows. Co-Authored-By: Claude Opus 5 (1M context) --- src/fateforger/slack_bot/handlers.py | 187 +++++++++++------------ tests/unit/conftest.py | 68 ++++++++- tests/unit/test_referent_rung_routing.py | 97 ++++++++++++ 3 files changed, 251 insertions(+), 101 deletions(-) diff --git a/src/fateforger/slack_bot/handlers.py b/src/fateforger/slack_bot/handlers.py index 44c3a47a..6dafd496 100644 --- a/src/fateforger/slack_bot/handlers.py +++ b/src/fateforger/slack_bot/handlers.py @@ -354,20 +354,8 @@ def _referent_label(referent: Referent) -> str: return f"{referent.kind} for {day} ({referent.status})" -#: This turn ends at a door that mints a session. There is nothing else it can -#: do with itself. -MINT_CERTAIN = "certain" -#: It might. The turn goes to some other agent, and that agent can hand it off -#: to timeboxing, which builds a session surface of its own. -MINT_POSSIBLE = "possible" -#: It cannot. There is a thread to continue, or a DM whose `{channel}:dm` key -#: names a session that already exists. -MINT_NO = "no" - -#: Said in every place a partial catalog stops a session being created: up front -#: when the turn was certainly headed for one, and at the door when a handoff -#: turned out to be. The same words in both, so which internal path the turn -#: took is not something a user has to model. +#: Said in every place a partial catalog stops a session being created, so which +#: internal door the turn reached is not something a user has to model. PARTIAL_CATALOG_ASK = ( ":warning: I couldn't check what you already have planned, so I won't start " "a second session over the top of it. Say that again and I'll retry, or open " @@ -375,40 +363,6 @@ def _referent_label(referent: Referent) -> str: ) -def _session_mint_prospect( - *, agent_type: str, is_dm: bool, thread_ts: str | None -) -> str: - """Will this turn mint a timeboxing session -- certainly, possibly, or not? - - One reading, asked once before the turn runs. There were three separate - re-derivations of this and they had already drifted apart; the one the - referent rung carried missed the handoff door entirely, which is the door a - plain "can you replan today so the gym is before dinner?" goes through when - it is typed anywhere but the planning channel. - - `open_session_surface` is the thing that actually mints, and it has two - callers -- the fresh-channel start and the handoff. The legacy backend - reaches the same place by sending `StartTimeboxing` instead. - - The distinction between `certain` and `possible` is what keeps a rare store - outage from becoming a bot-wide one. On `certain` the turn has nothing else - to do, so a caller may refuse up front. On `possible` it depends on a - judgement that has not been made yet -- whether the model hands off -- so a - caller must let the turn run and refuse at the door, if it comes to one. - Refusing up front there would answer "I couldn't check your calendar" to - someone asking what the weather is. - """ - if agent_type == "timeboxing_agent": - # A message with no thread to continue is a fresh start whichever - # backend takes it. A DM reaches its session by the `{channel}:dm` key - # and mints nothing. - return MINT_CERTAIN if (not is_dm and not thread_ts) else MINT_NO - # Any other agent can hand the turn off, and a handoff to timeboxing builds - # a surface of its own -- from a thread reply as readily as from a root - # message, and in a DM as readily as in a channel. - return MINT_POSSIBLE - - async def _resolve_referent( *, runtime, @@ -2832,14 +2786,10 @@ async def _update_constraints(thread_key: str) -> None: agent_type = "timeboxing_agent" structurally_claimed = True - # Read once, before the rung can move `agent_type`, and consulted by the - # rung and by every door that mints. One reading, several users, no drift. - mint_prospect = _session_mint_prospect( - agent_type=agent_type, is_dm=is_dm, thread_ts=thread_ts - ) - # Set when the rung found the catalog short and let the turn run anyway - # because nothing was certain to be created. Whatever door then turns out to - # create refuses instead, in the rung's own words. + # Set when the rung found the catalog short. The turn still runs -- whether + # it ever reaches timeboxing is a judgement nobody has made yet, and a store + # outage must not become a bot-wide one -- and every door that would hand + # the turn to timeboxing asks `_a_partial_catalog_forbids_this_turn` first. catalog_was_partial = False cleaned_text = _strip_bot_mention(text, bot_user_id) @@ -2938,6 +2888,52 @@ async def _refuse_to_mint_over_a_partial_catalog() -> None: ) await _origin_update(text=PARTIAL_CATALOG_ASK) + async def _a_session_already_stands(session_key: str) -> bool: + """Is there a session at this key already? Asked, never created. + + Fails closed. "I could not tell" and "there is nothing there" have to + answer the same way here, because the caller turns this answer into + permission to write, and a store that cannot be read is exactly the + condition that made the catalog short in the first place. + """ + store = getattr(runtime, "timeboxing_session_store", None) + if store is None: + return False + try: + return await store.load(session_key) is not None + except Exception: + logger.exception("session lookup failed for %s", session_key) + record_error(component="surface_intent", error_type="resolver_failure") + return False + + async def _a_partial_catalog_forbids_this_turn( + session_key: str, *, target_agent: str + ) -> bool: + """May this turn be handed to `target_agent` at `session_key`? Refuses aloud. + + One question, asked at every door that hands a turn to timeboxing, and + it is the store's question rather than the message builder's. + **Creation is decided by whether the session key is already known, not + by which message type the route happens to build.** Three separate + readings of "this turn creates" have now been wrong here: + `StartTimeboxing` is not the only creating message -- + `TimeboxingUserReply` mints one through `on_user_reply`'s + `_ensure_uncommitted_session` (`session_started_from_reply`) -- and on + the kernel backend the route sends no message at all, minting at + `load_or_create` instead. All three converge on one fact the store can + answer directly: does a row exist at the key this turn will use? + + Returns True when the caller must stop; the user has already been told. + """ + if not catalog_was_partial or target_agent != "timeboxing_agent": + return False + if await _a_session_already_stands(session_key): + # Continuing something that demonstrably stands is not creating a + # second one, so a short catalog says nothing against it. + return False + await _refuse_to_mint_over_a_partial_catalog() + return True + async def _origin_link_to_thread( *, channel_id: str, thread_ts: str, agent_label: str ) -> None: @@ -2978,27 +2974,11 @@ async def _begin_timeboxing_session_surface( one builder for every door. What stays here is what only a Slack event can supply: the origin link, the working card, and the turn. """ + # This door has no key to ask about: `open_session_surface` mints the + # root the key is made of, so it is a creation every single time. if catalog_was_partial: await _refuse_to_mint_over_a_partial_catalog() return - if mint_prospect == MINT_NO: - # The rung was told this turn could not mint one, and here it is - # minting one. That means `_session_mint_prospect` has stopped - # covering every door, and a partial catalog can now reach this - # line believing nothing stands. Loud and metered at the site, - # rather than inherited in silence by whatever reads the store - # next -- which is exactly how the 2026-09-05 duplicate happened. - logger.error( - "a session is being minted at a door the rung did not foresee " - "(agent=%s is_dm=%s thread_ts=%s channel=%s)", - agent_type, - is_dm, - thread_ts, - channel, - ) - record_error( - component="surface_intent", error_type="unforeseen_session_mint" - ) surface = await open_session_surface( client, focus, @@ -3428,19 +3408,26 @@ async def _begin_timeboxing_session_surface( # could make dangerous: `Resolved` hands the turn to a session that # demonstrably exists, and `Ambiguous` already asks. `NoReferent` is # the only one whose answer is "so go ahead and create". - if mint_prospect == MINT_CERTAIN: - # Nothing else this turn could do. Refusing at the end would - # spend a model call and a card to say the same thing later. - await _refuse_to_mint_over_a_partial_catalog() - return - # It only *might* create, by a handoff nobody has decided on yet. - # Let it answer -- someone asking what the weather is must not be - # told the planner could not read their calendar -- and refuse at - # the door, if the turn reaches one. + # + # Recorded here and nothing more. Whether this turn ever reaches + # timeboxing is a judgement nobody has made yet -- the model may or + # may not hand off -- so pre-empting here would answer "I couldn't + # check what you have planned" to someone asking what the weather + # is, and a rare store outage would become a bot-wide one. Every + # door that hands the turn to timeboxing refuses instead, and the + # ones that mint without a model call refuse before spending one. catalog_was_partial = True redirect = focus.get_redirect(origin_key) if redirect and agent_type == redirect.agent_type: + # A redirect outliving the focus binding it was set beside (`/ff-clear` + # drops one and not the other; they are separate TTL caches) puts this + # turn on a target key nothing has to have created yet -- and the reply + # built below is a `TimeboxingUserReply`, which mints one. + if await _a_partial_catalog_forbids_this_turn( + redirect.target_key, target_agent=redirect.agent_type + ): + return focus.set_user_focus(user, redirect.agent_type) persona = _persona_for_agent(redirect.agent_type) processing_payload = { @@ -3566,16 +3553,19 @@ async def _begin_timeboxing_session_surface( else None ), ) - if catalog_was_partial and isinstance(msg, StartTimeboxing): - # The legacy backend's own creating door. Asked of the message that was - # actually built rather than of the event shape all over again -- - # `StartTimeboxing` *is* the decision to create, `TimeboxingUserReply` - # is not -- so this cannot drift from the builder that made it. - await _refuse_to_mint_over_a_partial_catalog() - return recipient_key = origin_key if forced_thread_root: recipient_key = f"{channel}:{forced_thread_root}" + # The route's own door, and it creates on both backends: the kernel path + # below mints at `load_or_create(recipient_key)` without sending `msg` at + # all, and the legacy path's `TimeboxingUserReply` mints at + # `_ensure_uncommitted_session`. Asking the message type here answered + # "nothing to refuse" for a DM and for a first-touch thread reply, both of + # which write a row. + if await _a_partial_catalog_forbids_this_turn( + recipient_key, target_agent=agent_type + ): + return # The turn behind this call runs for tens of seconds -- measured at 43-54s # of graph on a real Refine turn, with prefetch and stage decision before # it. The ack posted above then sits unchanged for that whole minute, which @@ -3839,13 +3829,16 @@ async def _turn_heartbeat() -> None: True if (is_dm and handoff_target == "timeboxing_agent") else None ), ) - if catalog_was_partial and isinstance(handoff_msg, StartTimeboxing): - # The door `_begin_timeboxing_session_surface` never sees: the - # in-thread fallback taken when the session channel is not - # reachable. Asked of the message that was actually built, not of - # the event shape all over again -- `StartTimeboxing` *is* the - # decision to create, and `TimeboxingUserReply` is not. - await _refuse_to_mint_over_a_partial_catalog() + # The door `_begin_timeboxing_session_surface` never sees: the in-thread + # fallback, taken whenever `_channel_for_agent("timeboxing_agent")` is + # unset or is the channel the user is already in, and whenever the + # redirecting handoff above raised. A DM builds a `TimeboxingUserReply` + # here (`force_reply=True`) and so does any thread reply, and both mint + # a session at `origin_key` -- which is why reading the message type + # here let the exact 2026-09-05 shape straight through. + if await _a_partial_catalog_forbids_this_turn( + origin_key, target_agent=handoff_target + ): return try: result = await runtime.send_message( diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index 7905003b..9fbacefe 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -57,6 +57,10 @@ def _refuse(): #: hardcoded "open a new session" door the rung has to get in front of. PLAN_SESSIONS_CHANNEL = "C0AA6HC1RJL" +#: Hugo's DM with the bot. Its session key is `{channel}:dm`, and no structural +#: resolver ever runs there: the store lookup above is guarded by `thread_ts`. +DM_CHANNEL = "D_HUGO" + class _HarnessClient: """Enough Slack to run the route, with every message recorded.""" @@ -120,6 +124,16 @@ async def send_message(self, message, recipient): self.calls.append((message, recipient)) from autogen_agentchat.messages import TextMessage + if recipient.type == "timeboxing_agent": + # What the real agent does with either message it can be sent: + # `on_start` establishes a session, and `on_user_reply` establishes + # one too when the key names none (`_ensure_uncommitted_session`, + # debug event `session_started_from_reply`). Delivering to + # timeboxing IS minting, whichever message carried the turn. + await self.timeboxing_session_store.load_or_create( + recipient.key, owner_user_id="U_HUGO" + ) + if self._handoff_to is not None: # `_extract_handoff_target` reads `.target`; a plain string is one # of the shapes it accepts. This is how the receptionist reaches @@ -140,11 +154,24 @@ def __init__(self, *, rows, sessions, event_order) -> None: self._sessions = sessions self._event_order = event_order self.asked: list[str] = [] + #: Every key this store had to write a row for. The invariant under + #: test is about rows, not about Slack messages, so this is what the + #: partial-catalog tests assert on. + self.created: list[str] = [] async def load(self, session_key: str): self.asked.append(session_key) return self._sessions.get(session_key) + async def load_or_create(self, session_key: str, *, owner_user_id: str): + existing = self._sessions.get(session_key) + if existing is not None: + return existing + self.created.append(session_key) + session = SimpleNamespace(status="open", session_key=session_key) + self._sessions[session_key] = session + return session + async def standing_rows( self, *, owner_user_id: str, as_of, open_within, horizon ): @@ -192,6 +219,18 @@ def __init__(self, *, focus, runtime, client, planning, event_order, opened): self.sessions_opened = opened self._origin_ts: str | None = None + @property + def sessions_created(self) -> list[str]: + """Session keys a row was written for -- the thing the invariant is about. + + `sessions_opened` only sees `open_session_surface`. Two doors mint + without going anywhere near it: the kernel route writes at + `load_or_create`, and delivering either timeboxing message to the agent + writes at `_ensure_uncommitted_session`. + """ + + return self.runtime.timeboxing_session_store.created + @property def delivered_to(self) -> str | None: if not self.runtime.calls: @@ -236,10 +275,10 @@ async def route_top_level(self, text: str, *, channel: str | None = None) -> Non } ) - async def route_thread_reply(self, text: str) -> None: + async def route_thread_reply(self, text: str, *, channel: str | None = None) -> None: await self._route( { - "channel": PLAN_SESSIONS_CHANNEL, + "channel": channel or PLAN_SESSIONS_CHANNEL, "user": "U_HUGO", "text": text, "thread_ts": "1788599000.000100", @@ -247,6 +286,19 @@ async def route_thread_reply(self, text: str) -> None: } ) + async def route_dm(self, text: str) -> None: + """A DM, whose session key is `{channel}:dm` and never names a thread.""" + + await self._route( + { + "channel": DM_CHANNEL, + "channel_type": "im", + "user": "U_HUGO", + "text": text, + "ts": "1788600060.000300", + } + ) + async def _harness_unused_say(**_kwargs): return {"channel": PLAN_SESSIONS_CHANNEL, "ts": "unused"} @@ -266,6 +318,7 @@ def _build( rows=None, sessions=None, handoff_to: str | None = None, + timeboxing_channel: str | None = PLAN_SESSIONS_CHANNEL, ) -> _RoutingHarness: event_order: list[str] = [] opened: list[str] = [] @@ -292,9 +345,16 @@ async def _recording_open(*args, **kwargs): opened.append(kwargs["target_channel"]) return await real_open(*args, **kwargs) - async def _fake_turn(**_kwargs): + async def _fake_turn(**kwargs): from fateforger.slack_bot.messages import SlackBlockMessage + # The real turn opens with `repository.load_or_create(session_key)`, + # which writes a revision-zero row when the key names nothing. A + # stub that skipped that made the kernel route look like it never + # created anything, which is how this door stayed invisible. + await store.load_or_create( + kwargs["session_key"], owner_user_id=kwargs["actor_user_id"] + ) return SlackBlockMessage(text="turn ran", blocks=[]) monkeypatch.setattr(handlers_mod, "open_session_surface", _recording_open) @@ -311,7 +371,7 @@ async def _fake_turn(**_kwargs): handlers_mod, "_channel_for_agent", lambda agent_type: ( - PLAN_SESSIONS_CHANNEL if agent_type == "timeboxing_agent" else None + timeboxing_channel if agent_type == "timeboxing_agent" else None ), ) diff --git a/tests/unit/test_referent_rung_routing.py b/tests/unit/test_referent_rung_routing.py index 62bf5e1d..61a1e3ee 100644 --- a/tests/unit/test_referent_rung_routing.py +++ b/tests/unit/test_referent_rung_routing.py @@ -132,6 +132,7 @@ async def test_a_partial_catalog_never_creates_over_a_day_it_could_not_see( harness = routing_harness(resolver=_Resolver(NoReferent(catalog_complete=False))) await harness.route_top_level("can you replan today so the gym is before dinner?") assert harness.sessions_opened == [] + assert harness.sessions_created == [] assert harness.origin_messages, "the user must be told what could not be checked" @@ -182,6 +183,7 @@ async def test_a_partial_catalog_is_stopped_at_the_handoff_door_not_before_it( "can you replan today so the gym is before dinner?", channel="C_GENERAL" ) assert harness.sessions_opened == [] + assert harness.sessions_created == [] assert len(harness.runtime.calls) == 1, "the turn runs; only the mint is refused" assert PARTIAL_CATALOG_ASK in harness.origin_messages @@ -197,3 +199,98 @@ async def test_a_partial_catalog_does_not_break_every_other_conversation( assert len(harness.runtime.calls) == 1, "the receptionist must still answer" assert PARTIAL_CATALOG_ASK not in harness.origin_messages + + +# --------------------------------------------------------------------------- +# Creation is decided by whether the session key is already known. +# +# Not by the message type. `StartTimeboxing` is one creating message; so is +# `TimeboxingUserReply`, through `on_user_reply`'s `_ensure_uncommitted_session` +# (`agent.py`, debug event `session_started_from_reply`). And the kernel backend +# sends neither -- it mints at `timeboxing_session_store.load_or_create`. Each +# of the four doors below writes a row, and each was reachable with a catalog +# that could not see what already stood. +# --------------------------------------------------------------------------- + + +async def test_a_dm_on_the_planner_does_not_mint_over_a_day_it_could_not_see( + routing_harness, +): + # No structural resolver ever runs in a DM -- the store lookup is guarded by + # `thread_ts` -- so the rung is the only thing between this message and + # `load_or_create("D_HUGO:dm")`. The message built here is a + # `TimeboxingUserReply`, which is why reading the type said "nothing to + # refuse" while a row was being written. + harness = routing_harness(resolver=_Resolver(NoReferent(catalog_complete=False))) + harness.focus.set_user_focus("U_HUGO", "timeboxing_agent") + await harness.route_dm("can you replan today so the gym is before dinner?") + assert harness.sessions_created == [] + assert PARTIAL_CATALOG_ASK in harness.origin_messages + + +async def test_a_dm_that_continues_a_session_that_stands_is_not_refused( + routing_harness, +): + # The other half of the same guard, and the reason it asks the store rather + # than the flag: continuing something that demonstrably exists is not + # creating a second one, so a short catalog says nothing against it. + harness = routing_harness( + resolver=_Resolver(NoReferent(catalog_complete=False)), + sessions={"D_HUGO:dm": SimpleNamespace(status="open")}, + ) + harness.focus.set_user_focus("U_HUGO", "timeboxing_agent") + await harness.route_dm("move the gym earlier") + assert harness.sessions_created == [] + assert PARTIAL_CATALOG_ASK not in harness.origin_messages + + +async def test_a_first_touch_thread_reply_does_not_mint_over_a_partial_catalog( + routing_harness, +): + # A reply in a thread the store knows nothing about: the structural lookup + # answers `None`, so nothing claims it, and the turn goes to the kernel with + # this thread's own key -- where `load_or_create` writes the row. + harness = routing_harness(resolver=_Resolver(NoReferent(catalog_complete=False))) + await harness.route_thread_reply("can you move the gym before dinner?") + assert harness.sessions_created == [] + assert PARTIAL_CATALOG_ASK in harness.origin_messages + + +async def test_the_in_thread_handoff_fallback_does_not_mint_over_a_partial_catalog( + routing_harness, +): + # `_channel_for_agent("timeboxing_agent")` unset (or naming the channel the + # user is already in) makes `should_redirect` false, and the handoff falls + # through to the in-thread path -- the one door + # `_begin_timeboxing_session_surface` never sees. In a DM it builds a + # `TimeboxingUserReply`, so a guard reading the message type let it past. + harness = routing_harness( + resolver=_Resolver(NoReferent(catalog_complete=False)), + handoff_to="timeboxing_agent", + timeboxing_channel=None, + ) + await harness.route_dm("can you replan today so the gym is before dinner?") + assert harness.sessions_created == [] + assert len(harness.runtime.calls) == 1, "the receptionist turn still runs" + assert PARTIAL_CATALOG_ASK in harness.origin_messages + + +async def test_a_redirect_that_outlived_its_focus_does_not_mint_over_a_partial_catalog( + routing_harness, +): + # Focus and redirects are two TTL caches, and `/ff-clear` drops one without + # the other. With the binding gone the rung runs again, and the surviving + # redirect then carries this turn to a target key that need not have been + # created -- as a `TimeboxingUserReply`, which creates it. + harness = routing_harness(resolver=_Resolver(NoReferent(catalog_complete=False))) + harness.focus.set_user_focus("U_HUGO", "timeboxing_agent") + harness.focus.set_redirect( + "D_HUGO:dm", + target_channel="C0AA6HC1RJL", + target_thread_ts="1788599000.000100", + agent_type="timeboxing_agent", + by_user="U_HUGO", + ) + await harness.route_dm("move the gym earlier") + assert harness.sessions_created == [] + assert PARTIAL_CATALOG_ASK in harness.origin_messages From 77024eb38b0ba7b56a1f3c99f6ff4e16c5868389 Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 16:59:28 +0200 Subject: [PATCH 18/20] test(referents): resolution quality on the pin, two frozen incidents, n=8 Co-Authored-By: Claude Opus 5 (1M context) --- tests/evals/test_eval_referent_resolver.py | 216 +++++++++++++++++++++ 1 file changed, 216 insertions(+) create mode 100644 tests/evals/test_eval_referent_resolver.py diff --git a/tests/evals/test_eval_referent_resolver.py b/tests/evals/test_eval_referent_resolver.py new file mode 100644 index 00000000..76efe3c0 --- /dev/null +++ b/tests/evals/test_eval_referent_resolver.py @@ -0,0 +1,216 @@ +# tests/evals/test_eval_referent_resolver.py +"""Referent resolution quality on the pin, against two frozen incidents. + +**Why frozen.** Reading `timeboxing_session_states` live measures the ledger's +drift, not the model: two runs of one spike forty minutes apart drew different +candidate sets because a peer committed a plan mid-run, and the store keeps no +history, so a descriptor built from it reads *current* status while claiming to +describe an earlier moment. The rows below are inline and dated. Do not replace +them with a query -- that is the ceremony this docstring exists to protect. + +**Labels were reviewed blind.** Three were wrong on first writing, all in the +same direction: the model reading state and the label reading an assumption. +"is it planned?" resolves to the standing session (what happens next is the +consumer's judgement, not this one's), and "move the gym to the morning" +resolves to the only plan that contains a gym. + +n = 8 draws per case, asserted on the rate. + + set -a; source .env; set +a + PYTHONPATH=src .venv/bin/python -m pytest tests/evals/test_eval_referent_resolver.py -m slow -q +""" + +from __future__ import annotations + +import asyncio +from datetime import UTC, date, datetime, timedelta + +import pytest + +from fateforger.llm import build_autogen_chat_client +from fateforger.referents import Catalog, Referent +from fateforger.referents.resolver import ( + Ambiguous, + NoReferent, + ReferentResolver, + Resolved, +) + +pytestmark = [pytest.mark.slow, pytest.mark.asyncio] + +DRAWS = 8 +TZ = UTC + +# --- fixture one: the incident, 2026-09-05 13:51 -------------------------- +INCIDENT_AT = datetime(2026, 9, 5, 11, 51, tzinfo=TZ) +INCIDENT = ( + # (ref_id, day, status, never_used, hours_ago, gist) + ("r1", date(2026, 9, 7), "open", False, 1.1, ()), + ("r2", None, "open", True, 10.1, ()), + ( + "r3", + date(2026, 9, 5), + "committed", + False, + 10.2, + ( + "Wake up 11:00-11:00", + "Breakfast (oats) 11:00-11:30", + "Buy a new white shirt 11:30-12:30", + "Gym session 13:00-14:00", + "Pay taxes 14:15-15:15", + "Lunch 15:15-15:45", + "Dinner 19:30-20:30", + "Evening shutdown ritual 20:30-21:30", + "Sleep 23:00-23:00", + ), + ), +) +INCIDENT_CASES = [ + ("can you replan today so the gym is before dinner?", "r3"), + ("plan saturday", "r3"), + ("let's plan monday", "r1"), + ("actually make monday start at 10", "r1"), + ("I'll wake up at 11 on monday", "r1"), + ("what did we decide about dinner?", "r3"), + ("is it planned?", "r3"), + ("plan tomorrow", "none"), + ("what's the weather tomorrow", "none"), + ("add a dentist appointment on tuesday", "none"), + ("remind me to pay taxes", "none"), +] + +# --- fixture two: #275, two sessions for one Friday, 2026-09-03 12:15 ----- +PARALLEL_AT = datetime(2026, 9, 3, 10, 15, tzinfo=TZ) +PARALLEL = ( + ( + "r1", + date(2026, 9, 4), + "open", + False, + 0.0, + ( + "Serious C2F work 10:30-12:00", + "Kapper 12:00-12:30", + "Lunch 12:30-13:00", + "Validate agent demos 13:00-13:45", + "Finances 13:45-14:30", + "Oats 16:00-16:15", + "Gym (chest) 18:00-19:00", + "Dinner 19:15-20:00", + ), + ), + ( + "r2", + date(2026, 9, 4), + "open", + False, + 0.1, + ( + "PR review - stage-UX, ends 11:30", + "Kapper 12:00-12:30", + "Lunch ~12:30", + "Deep work - constraint memory design, 90 minutes", + "Prepare the Monday investor call 15:00-16:00", + "Oats 16:00", + "Gym 18:00 (chest)", + "Dinner ~19:30", + ), + ), +) +PARALLEL_CASES = [ + ("move PR1 later", "r1"), + ("move the finances block later", "r1"), + ("push the investor call prep later", "r2"), + ("move the gym to the morning", "ambiguous"), + ("cancel that session", "ambiguous"), + ("plan sunday", "none"), + # False-positive probes: none of these exist in either plan. A wrong answer + # here is a duplicate session at a door that creates, so they are gated + # unanimously. + ("move the dentist earlier", "none"), + ("push the standup to 11", "none"), + ("can you shorten the school run", "none"), + ("move the physio appointment to friday morning", "none"), +] +PROBES = { + "move the dentist earlier", + "push the standup to 11", + "can you shorten the school run", + "move the physio appointment to friday morning", +} + + +def _catalog(rows, at: datetime) -> Catalog: + return Catalog( + referents=tuple( + Referent( + key=f"C1:{ref_id}", + agent_type="timeboxing_agent", + kind="a plan for one day", + day=day, + status=status, + never_used=never_used, + last_activity=at - timedelta(hours=hours_ago), + accepts=( + ("revise the committed plan", "add a fact about the day") + if status == "committed" + else ("continue planning", "answer the open question", "cancel") + ), + gist=gist, + ref_id=ref_id, + ) + for ref_id, day, status, never_used, hours_ago, gist in rows + ) + ) + + +def _label(outcome) -> str: + if isinstance(outcome, Resolved): + return outcome.referent.ref_id + if isinstance(outcome, Ambiguous): + return "ambiguous" + if isinstance(outcome, NoReferent): + return "none" + raise AssertionError(outcome) + + +async def _draws(resolver, catalog, message, at) -> list[str]: + outcomes = await asyncio.gather( + *( + resolver.resolve(catalog=catalog, message=message, as_of=at) + for _ in range(DRAWS) + ) + ) + return [_label(o) for o in outcomes] + + +@pytest.mark.parametrize( + "rows, cases, at", + [(INCIDENT, INCIDENT_CASES, INCIDENT_AT), (PARALLEL, PARALLEL_CASES, PARALLEL_AT)], + ids=["incident-2026-09-05", "two-parallel-sessions-2026-09-04"], +) +async def test_resolution_quality(rows, cases, at): + resolver = ReferentResolver(build_autogen_chat_client("timeboxing_judge")) + catalog = _catalog(rows, at) + + results = await asyncio.gather( + *(_draws(resolver, catalog, message, at) for message, _ in cases) + ) + + hits = 0 + failures = [] + for (message, expected), drawn in zip(cases, results): + correct = sum(1 for d in drawn if d == expected) + hits += correct + print(f" {correct}/{DRAWS} {expected:<10} {message}") + if message in PROBES and correct != DRAWS: + failures.append( + f"probe {message!r} expected {expected} unanimously, drew {drawn}" + ) + + total = len(cases) * DRAWS + rate = hits / total + print(f" == {hits}/{total} draws ({rate:.0%})") + assert not failures, "\n".join(failures) + assert rate >= 0.85, f"{hits}/{total} draws correct; the measured arm scored ~0.85+" From f5de0ee072be3cd5a6b4b18f5b321f290dd12e4f Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 17:06:17 +0200 Subject: [PATCH 19/20] docs(referents): the interface-strain log the marshal's provider inherits Records every place the referent-catalog interface strained to fit timeboxing across Tasks 1-8, verified against the code as shipped at 77024eb rather than against the design doc's wording. Nine entries judged general (locator addressing, gist's forced flatten-to-string round trip, accepts duplicating the session state machine, day's overloaded None, what catalog_complete required of the consumer across Task 7's four fix rounds, the resolver's blind spot on non-text creating doors, the shared observability key, the AST guard's package-scoped blind spot, and as_of as a filtering horizon rather than a point-in-time read) plus a named list of what was judged Slack-routing or eval-fixture noise rather than interface strain, so the boundary is stated rather than left implicit. Also raises hugocool/FateForger#443 against #345 per CLAUDE.md's docs-ticket rule: docs/architecture/ needs the provider contract, and src/fateforger/referents/AGENTS.md does not exist yet. Co-Authored-By: Claude Opus 5 (1M context) --- .../2026-09-referent-interface-strain.md | 369 ++++++++++++++++++ 1 file changed, 369 insertions(+) create mode 100644 docs/superpowers/notes/2026-09-referent-interface-strain.md diff --git a/docs/superpowers/notes/2026-09-referent-interface-strain.md b/docs/superpowers/notes/2026-09-referent-interface-strain.md new file mode 100644 index 00000000..99ecd42e --- /dev/null +++ b/docs/superpowers/notes/2026-09-referent-interface-strain.md @@ -0,0 +1,369 @@ +# Referent-catalog interface strain log + +Written after the fact from `.superpowers/sdd/2026-09-08-referent-catalog/progress.md` and +`task-1-report.md` … `task-8-report.md`, checked against the code as it shipped at `77024eb` +rather than against the design doc's wording (`docs/superpowers/specs/2026-09-08-referent-catalog-design.md`). +Purpose: the task marshal (#160) is a second provider against `src/fateforger/referents/`'s +protocol. Every entry below is something that cost real implementation time against timeboxing, +the first and only provider that has existed so far — the marshal's implementer should not have +to rediscover any of them. + +Each entry cites the task and file where the strain showed up and says whether it is a property +of the protocol (general — the marshal will hit it too) or a fact about timeboxing/Slack that has +no bearing on a second provider. Getting that call wrong in either direction defeats the point of +the log, so entries that turned out to be ordinary bugs rather than interface strain are named at +the bottom instead of padded into the list. + +--- + +### A locator only works for things addressed by (channel, thread) + +**Where:** Task 1, `src/fateforger/referents/descriptor.py` (`channel_id`, `thread_ts`); Task 7, +`src/fateforger/slack_bot/handlers.py:3271-3341` (the redirect branch), `handlers.py:3245` +(`current_thread`). + +**What the interface could not express:** a referent whose home is not a Slack thread. A DM +session's key is `{channel}:dm`; `TimeboxingReferentProvider._split_session_key` (`timeboxing.py`) +correctly returns `thread_ts=None` for it, because a DM names no thread. But the consumer's +reachability test is `bool(referent.channel_id and referent.thread_ts)` — verified in the shipped +code, `handlers.py:3271` — so **any** referent with `thread_ts=None` is permanently undeliverable +through `focus.set_redirect`, which addresses a target by channel *and* thread and has no other +form. The design's own wording (`channel_id`/`thread_ts` are "the only Slack-shaped fields... a +provider whose standing things live elsewhere leaves them `None` and loses only the link and +`is_current_surface`") undersells the cost: losing `is_current_surface` for such a referent isn't +incidental, it's structural — `current_thread` is only ever built when `thread_ts` is truthy +(`handlers.py:3245`, `(channel, thread_ts) if thread_ts else None`), so a top-level (non-threaded) +message can never be recognised as *arriving inside* any referent's own surface, DM or not. + +Note for anyone reading the design doc's account of this: it says the DM branch fails because +`focus.set_redirect` rebuilds `{channel}:dm` and the redirect branch then posts +`chat_postMessage(thread_ts="dm")`, which Slack rejects. That was true after Task 7's first +round. By the version that shipped, the reachability check above stops the redirect from ever +being attempted for such a referent — it no longer crashes, it silently can never be reached. The +user is told (`"...which lives in {where}... I can't hand a message to it from here"`, +`handlers.py:3302-3313`), which is better than a Slack API error, but the underlying limit is +identical: the protocol has exactly one way to say "here is where this referent lives" and it +only covers things addressed by a thread. + +**How timeboxing worked around it:** it doesn't, structurally — it degrades gracefully. A +DM-keyed referent is resolved correctly (the resolver still sees it, can still pick it, can still +call it ambiguous) but can never be *delivered to*; the consumer detects that at the point of +delivery and tells the user to go say it there instead. + +**Timeboxing-specific, or general?** General. Any provider whose standing things aren't addressed +by a Slack (channel, thread) pair — which very plausibly includes a marshal's GTD session, if it +isn't itself a Slack thread — inherits exactly this gap: it can be *chosen* by the resolver but +never *reached* by this consumer. The design flags the open question correctly ("does a locator +belong on the descriptor at all, or should a provider render its own link?") but nothing in this +branch answers it; what shipped is a consumer-side special case keyed to the Slack shape +(`channel_id`/`thread_ts` both truthy), not a general "can I reach this referent" affordance on +the protocol. + +--- + +### `gist` as `tuple[str, ...]` forces every provider to invent its own serialize/parse round trip + +**Where:** Task 6, `src/fateforger/slack_bot/timeboxing_session_store.py` (`_plan_gist`, fix +round 1). + +**What the interface could not express:** anything about a standing thing's content beyond a +flat list of already-rendered strings. For timeboxing this meant `_plan_gist` had to take the +plan's rendered CSV-shaped block table (produced by `render_plan`/`_escape` in +`src/tmbx/core/render.py`) and re-parse it back into `"{summary} {start}-{end}"` strings just to +satisfy the descriptor's shape — flattening structured data (blocks with summaries and times) down +to text, purely so it could be handed to a model as text. That round trip was not free: a block +summary containing a comma (a shape `_escape` legitimately quotes for) broke the original +`line.split(",")` parser, silently producing a garbled entry (`'"Serious C2F work prep"-10:30'` +instead of `'Serious C2F work, prep 10:30-12:00'`) that would have been fed straight to the +resolver as if it were the plan's real content. Caught by review, not by the type system — nothing +about `gist: tuple[str, ...]` could have caught it, because the interface has no opinion on what a +gist entry is built from. + +**How timeboxing worked around it:** swapped the naive split for `csv.reader`, kept the existing +`len(fields) < 6` malformed-row guard (still correct against the new parser, verified by hand +against an unterminated-quote case), and made a parse failure fail closed to `()` rather than +guess. + +**Timeboxing-specific, or general?** The bug itself is timeboxing-specific (CSV rendering of one +particular plan format). The shape choice that produced the bug is general: a GTD session's +standing content is "tasks with due dates and projects" per the design's own framing — structured +data from the moment it's read out of whatever store the marshal uses. Forcing it through +`tuple[str, ...]` means the marshal's provider will face the identical tax on day one: either it +renders tasks to strings at read time (inventing its own serialization, with its own version of +this bug class available to it) or the descriptor stops being `tuple[str, ...]` and becomes +something the resolver's prompt-builder has to render generically instead. The design seeds this +exact question ("same slot, different shape") without resolving it; this is the first piece of +evidence that resolving it in favour of structure would have prevented a real, shipped defect. + +--- + +### `accepts` is a second, hand-written copy of the session state machine's own option table + +**Where:** Task 5, `src/fateforger/referents/timeboxing.py:28-29` +(`_COMMITTED_ACCEPTS`/`_OPEN_ACCEPTS`); compare `src/fateforger/slack_bot/timeboxing_intents.py:303` +(`_display_context`). + +**What the interface could not express:** a link between `accepts` and the actual decision +surface. `_display_context` derives `allowed_decisions` from roughly ten branches of session +state — stage, whether an artifact is pending and which kind, open constraints, a pending blocker, +outstanding assumptions — and emits decision ids like `confirm_planning_day`, `provide_facts`, +`approve`, `revise`, `back`, `cancel`, `choose_option`, `advance`, `restore`, `steer_not_today`, +`assume`, `deny`. `TimeboxingReferentProvider._describe` collapses all of that into one of two +fixed, two-to-three-item English tuples keyed only on `status in {open, committed}` +(`_COMMITTED_ACCEPTS = ("revise the committed plan", "add a fact about the day")`, +`_OPEN_ACCEPTS = ("continue planning", "answer the open question", "cancel")`). Grepped the +shipped code for any symbol shared between the two functions: none. They agree today only because +someone kept them in sync by hand, and there's no test that would catch them drifting — they aren't +even the same type (English prose versus decision ids), so no equality check could compare them +even if one were written. + +**How timeboxing worked around it:** didn't reconcile them. `accepts` is descriptive copy that +feeds the resolver's judgement of "what would replying here mean"; `allowed_decisions` remains the +actual enforcement surface for the second judgement (#352, out of scope here). The two are allowed +to be inconsistent without either test suite noticing. + +**Timeboxing-specific, or general?** General, and confirmed rather than merely predicted by the +design. Any provider that layers a coarse, human-readable `accepts` on top of a richer internal +state machine — which a GTD session with its own transitions plausibly is — inherits the same +"two places must agree, nothing enforces it" problem, and the referents protocol offers no +mechanism (shared vocabulary, derivation, or even a test hook) to keep them aligned. The concrete +lesson for the marshal: budget for `accepts` drifting from whatever its own state machine actually +allows, on day one, exactly as timeboxing's did from the start. + +--- + +### `day: None` means "not decided yet" for timeboxing; the marshal needs it to mean "not applicable" + +**Where:** Task 1, `src/fateforger/referents/descriptor.py` (`day: date | None`); Task 5, +`timeboxing.py:80` (`day=row.planning_date`). + +**What the interface could not express:** two different reasons a standing thing might have no +day. For timeboxing `day` is central to `kind` ("a plan for one day"), and `None` is a real, +load-bearing state along the session's own lifecycle (a planning session before `planning_date` is +confirmed) — `Referent.describe` even renders it as `"no day locked yet"` rather than omitting the +field, deliberately (Task 1 test: +`test_a_day_less_row_says_so_rather_than_omitting_the_field`). A GTD session per the design is not +day-scoped at all: `day` would be `None` on every row, forever, and nothing distinguishes that from +"still choosing." Nothing in the shipped code resolves this — it was never exercised, because +timeboxing never has a standing thing for which "day" is a meaningless dimension rather than an +undetermined one. + +**How timeboxing worked around it:** not applicable — the ambiguity was never triggered, so no +workaround exists to inherit. This entry stays open exactly as the design left it. + +**Timeboxing-specific, or general?** General, and specifically the marshal's problem to solve +first, since nothing downstream (the resolver prompt, `describe()`, the rung) currently +distinguishes the two meanings. The cheapest fix is probably: let a provider's own `kind` string +never mention a day, and stop trying to encode "N/A" in the `day` field at all — but that's a +convention, not something the type enforces, so it's worth stating in `AGENTS.md` explicitly +rather than relying on the marshal's implementer to notice `describe()`'s current +`"no day locked yet"` phrasing already assumes a day is coming. + +--- + +### `catalog_complete` is carried correctly; making it *mean* something at the write side cost four review rounds + +**Where:** Task 3, `src/fateforger/referents/resolver.py` (`_Outcome.catalog_complete`); Task 7, +`src/fateforger/slack_bot/handlers.py`, fix rounds 1-4 (`progress.md`, task-7-report.md). + +**What the interface could not express:** which of a consumer's own code paths are the ones that +need gating. The protocol does its part correctly — `catalog_complete` rides every `Resolution` +outcome (`resolver.py`), and it's true that only `NoReferent` licenses anything dangerous +(`Resolved` hands the turn to a session that demonstrably exists; `Ambiguous` already asks). But +turning "the catalog might be missing rows" into "therefore this specific write must not happen" +required the consumer to independently enumerate every place in a 5,000+-line routing file that +could mint a new session — something the protocol has no vocabulary for at all, because +`ReferentProvider.standing()` is read-only by design. + +Three successive attempts to derive "would this turn create a session" from the *message the +route was about to send* were all wrong: + +- round 1/2: `agent_type == "timeboxing_agent" and not thread_ts` — missed the handoff door + entirely (a non-timeboxing agent handing off can mint too). +- round 3: a tri-state (`MINT_CERTAIN`/`MINT_POSSIBLE`/`MINT_NO`) with two + `isinstance(msg, StartTimeboxing)` guards — wrong because `StartTimeboxing` is not the only + creating message: `TimeboxingUserReply` also creates, via `on_user_reply`'s + `_ensure_uncommitted_session`, and the kernel backend creates via `repository.load_or_create` + while sending *no* distinguishing message type at all. +- round 4: abandoned deriving intent from message shape and asked the store directly — + `_a_session_already_stands(session_key)` — at the four places (of four, found by grepping every + `AgentId(`, `send_message(`, `_run_adaptive_timebox_turn` and `open_session_surface` call inside + the routing function, not by reasoning about which "could" create) that actually write. That + version held; disabling each of the four guards individually, one at a time, produced exactly + one new test failure per guard. + +**How timeboxing worked around it:** `_a_session_already_stands` (fail-closed: an unreadable store +answers the same as "nothing there", because that's the exact condition that made the catalog +partial in the first place) plus `_a_partial_catalog_forbids_this_turn`, called at all four +creating doors by hand. + +**Timeboxing-specific, or general?** General, and probably the single most expensive lesson in +this branch for the marshal to inherit directly rather than re-derive. `catalog_complete` tells a +consumer "some providers may be lying by omission," but finding every place that omission could +turn into a bad write is entirely the consumer's own archaeology, proportional to how many ways +that consumer's routing code can create a standing thing — not to anything the referents package +exposes. A marshal wired into this same rung (or a marshal-specific rung) inherits the identical +obligation and should expect it to take multiple review passes, exactly as it did here, unless the +consumer-side "enumerate every creating door, gate each on the store" pattern from round 4 is +treated as required practice rather than rediscovered. + +--- + +### The resolver only fires on free text — doors that create without going through it are invisible to `catalog_complete` by construction + +**Where:** Task 7, `handlers.py:3239` (`if binding is None and not structurally_claimed and +text.strip():`), and the "A creating door nobody has named" section of round 4's report. + +**What the interface could not express:** that a standing thing can be created by something other +than a message the resolver was asked to judge. The rung's entire question is "which standing +thing is *this message* about," so anywhere a session gets minted without a message reaching that +question — an empty-text `/timebox` command (`text == ""` after `_route_command_as_message`, so +`text.strip()` is falsy and the whole rung, catalog included, never runs), or an explicit +`/ff-focus timeboxing` on a session-less thread (skips the rung via the pre-existing focus-binding +check, a *fact* that structurally outranks the judgement per #310) — is a door the referent +machinery has literally never been consulted about. Both were found empirically in round 4 by +driving the real route, not reasoned about in advance; both are correctly left un-refused, on the +argument that an explicit command to create is not an inferred follow-up. + +**How timeboxing worked around it:** it doesn't route these through the catalog at all. The two +other creating doors this branch found outside `route_slack_event` — `SessionStarter.start`'s own +`_blocked` (which fails closed on *any* store-read failure, stricter than the rung) and the action +handlers' card-keyed `load_or_create` (no message, no catalog, the key came off the card the user +pressed) — already have their own independent duplicate-guards, unrelated to referents. + +**Timeboxing-specific, or general?** General. A marshal's own doors that create via structured +input rather than free text — a button, a slash command, a scheduled digest — will be equally +invisible to any referent-resolution rung for the identical structural reason: the rung only runs +when there's a message to resolve. The lesson isn't "extend the resolver to cover these" (the +branch explicitly declines to, and the reasoning holds up) — it's that **the referent catalog is +not a general duplicate-prevention mechanism**, and every creation path that doesn't originate from +a free-text judgement needs its own guard regardless of whether the catalog/resolver exists at all. + +--- + +### A shared resolver has one hardcoded observability key across every call, every provider + +**Where:** Task 3, `src/fateforger/referents/resolver.py:113` (`llm_attribution(agent= +"referent_resolver", call_label="resolve", key="referents")`); flagged as a deferred minor in +`progress.md` after Task 3 ("Task 7 should reconsider a per-session key") and never revisited — +Task 7's report does not mention `llm_attribution` at all. + +**What the interface could not express:** which conversation, user, or provider a given resolver +call belonged to, in the system's own cost/quality attribution. `key="referents"` is a fixed +string; `SurfaceIntentInterpreter`'s sibling call site (`surface_intents.py`) uses a real +session/thread key (`"CHANID:thread_ts"`) for the same field. Every resolver call this branch +makes — regardless of which channel, which user, or (once a second provider exists) which +provider's rows dominated the catalog — lands in the same attribution bucket. + +**How timeboxing worked around it:** didn't; it was noted as a deferred minor after Task 3 and +never picked up in Task 7, which wires the resolver into the live route without touching this +call. + +**Timeboxing-specific, or general?** General, and directly relevant to the map's own success +metric ("does the second provider get cheaper because the first exists?") — that question is +answerable from cost data, and cost data keyed to one static string can't distinguish timeboxing's +calls from the marshal's once both are candidates in the same catalog. Worth fixing before or at +the point a second provider lands, not after. + +--- + +### The AST guard on `gist` is real, catches real violations, and is scoped to one package by construction + +**Where:** Task 4, `tests/unit/test_referents_never_match_gist.py`; deferred minors recorded in +`progress.md` after Task 4. + +**What the interface could not express:** an enforceable, semantic guarantee that "content reaches +a model and only a model." What's enforced instead is a syntactic AST walk over +`src/fateforger/referents/*.py` for method calls, comparisons, and a fixed set of "decision" builtins +(`sorted`, `any`, `all`, `max`, `min`, `sum`, `set`) touching `.gist`. It is not decorative — during +Task 4 it caught two real coverage gaps (a bare `import slack_sdk` past the `ImportFrom`-only slack +check; a builtin-function call past the method-call-only check) and one real false positive of its +own (`min(len(self.gist), GIST_LIMIT)` flagged as a violation because the walk found `.gist` +nested inside `len()`, fixed by narrowing to direct references only) — each confirmed by +deliberately injecting the violation and watching the guard catch or miss it. But by its own +documented limits it is: (a) defeatable by aliasing (`g = self.gist` then comparing `g`); (b) +bounded by a hand-written, finite list of forbidden methods/builtins, so a violation shape on +neither list passes silently; and (c) scoped to `PACKAGE.rglob("*.py")` under +`src/fateforger/referents/` only — a downstream consumer that reads `.gist` and pattern-matches on +it (nothing in `handlers.py` currently does; `_referent_label` deliberately reads only `kind`/`day`/ +`status`) would be invisible to it. + +**How timeboxing worked around it:** didn't need to — the consumer this branch built never reads +`gist` for anything but display, so the guard's blind spot at the package boundary was never +exercised. The guard's real job here was catching two authoring mistakes inside `descriptor.py` +during development, which it did. + +**Timeboxing-specific, or general?** General — this is the same style of guard the memory server +uses on its own read path (noted in the deferred-minor itself), so it's a known, accepted repo +pattern rather than something new here. The concrete inheritance for the marshal: re-run this exact +guard-style discipline (write the test, break it on purpose, confirm the break, fix, confirm clean) +against whatever free-text field the marshal's descriptor carries, and don't assume the +`referents`-package guard already covers a marshal-side consumer that reads that field — it +structurally can't. + +--- + +### `as_of` bounds a query; it does not reconstruct a moment + +**Where:** design (`docs/superpowers/specs/2026-09-08-referent-catalog-design.md`, "as_of is +explicit and required"); Task 6, `timeboxing_session_store.py:standing_rows` (lines 269-325); Task +5 fix round 1 (`timeboxing.py`, `last_activity` timezone bug). + +**What the interface could not express:** a genuine point-in-time read. `standing_rows` uses +`as_of` only to bound comparisons — `created_at < moment` (excludes a row the current message +itself just minted), `updated_at >= since` (the open-session freshness window), and the horizon +window on `planning_date` — every other field (`status`, `revision`, `gist`) is read as *currently* +stored, verified by reading the query directly. Calling `standing_rows` with an `as_of` an hour in +the past, after a row's status has since changed, returns today's status filtered by whether it +still structurally qualifies under those windows — not the status as it stood an hour ago. This +matches what the design's own spike already observed (two runs 40 minutes apart drew different +candidate sets because a peer committed mid-run) but is easy to over-read from the parameter's +name and type signature (an explicit, required `datetime`) as a stronger promise than it is. + +Separately, and concretely costly: the first cut of `TimeboxingReferentProvider._describe` tagged +the store's naive-UTC `updated_at` with `as_of.tzinfo` rather than `UTC` — plausible-looking code +(`row.updated_at.replace(tzinfo=as_of.tzinfo)`) that silently miscomputed `last_activity` by +exactly the caller's UTC offset (a 3-hour error against a caller at UTC+2, per the reproduction in +Task 5's report) whenever `as_of` wasn't already UTC. `last_activity` feeds the resolver's recency +judgement directly, so this would have quietly biased every routing decision made with a non-UTC +`as_of`. Caught only by a test written specifically to vary `as_of`'s timezone; nothing in the +protocol's types would have caught it, because the protocol says nothing about what timezone basis +a provider's own persisted timestamps are in. + +**How timeboxing worked around it:** fixed the tagging bug to always assume the stored value is +naive UTC (`row.updated_at.replace(tzinfo=UTC)`), and left the "as_of only bounds, doesn't +reconstruct" behaviour as-is, since it matches the design's own stated limitation ("the store keeps +no history"). + +**Timeboxing-specific, or general?** Both halves are general. Any provider backed by a +naive-datetime store will hit the identical tagging footgun the first time it's exercised with a +non-UTC `as_of`, and nothing type-checks it — only a test that varies the timezone catches it. And +any provider should expect `as_of` to be read the same way timeboxing's is: a filtering horizon +over current state, not a time machine. If the marshal's store keeps a history that timeboxing's +doesn't, this is exactly the place that distinction would show up — and the protocol's docstring +doesn't currently say which promise a provider is required to make. + +--- + +## Judged not to belong in this log (Slack-routing or eval-fixture issues, not interface strain) + +Named here rather than silently omitted, per the instruction to be honest about the boundary: + +- **The rung must run after the ack, not before it** (Task 7, round 1) — a Slack UX latency + concern (a silent model round-trip before the first frame) specific to how this bot acknowledges + messages. Nothing about the referents protocol forced this; it's purely about where in + `route_slack_event` the call happens. +- **The origin message shows the pointer, then gets overwritten by the "Continuing in" card** + (Task 7, concern carried through all four rounds) — cosmetic, Slack-message-update sequencing, + unrelated to what the protocol can express. +- **The comma-in-CSV `_plan_gist` bug itself** (Task 6) — the bug is timeboxing's own rendering + format; only the *shape choice that made such a bug possible* (flattening structured content to + `tuple[str, ...]`) is logged above as general. +- **`remind me to pay taxes` (3/8) and `move PR1 later` (0/8) scoring badly in the eval** (Task 8) + — flagged in that report as likely fixture-label problems (the label may not match what the + model is actually shown), not a resolver or protocol defect. +- **Eval JSON-truncation flakiness under concurrency** (Task 8) — diagnosed as an OpenRouter/host + concurrency and account-budget concern, reproduced independent of any prompt or protocol change. +- **`structurally_claimed`'s placement and the ordered-resolver invariant (#310)** (Task 7) — this + is pre-existing Slack routing architecture that the rung had to respect, not something the + referents package strained against. From 2af914515a9bdba6bc3316967f63d22b42427fba Mon Sep 17 00:00:00 2001 From: hugocool Date: Wed, 9 Sep 2026 17:36:24 +0200 Subject: [PATCH 20/20] fix(referents): the chosen day survives the turn, and a model outage stops minting (#345) The final wave before this branch is offered for merge. Eight findings, one pass. Two blockers in the rung. The line naming which referent was chosen was written into the origin message and then overwritten within the same turn by `_origin_link_to_thread`'s `chat_update` -- so it survived in a DM and was erased in #plan-sessions, the incident's own channel. Since the rung delivers rather than waiting for a press, that line is the whole remaining protection against a confidently wrong referent; it is now carried into the final message. And `_resolve_referent` returned `None` on every failure, so a model outage over a catalog that plainly contained the standing day fell through to the minting door -- the incident, with a test asserting it. It now separates "declined to ask" (no resolver, no store: today's behaviour exactly) from "asked and failed over a non-empty catalog", which is treated as a partial catalog and refuses. The resolver eval named the wrong instrument. Task 8 measured that the rate does not discriminate a known prompt regression (87-90% with the load-bearing sentence deleted); the probe-unanimity gate is what catches it, and it appeared only as an inline comment. The docstring now says so and the rate assertion reads as the collapse floor it is. The eval was not run. `_plan_gist` re-parsed the rendered table with the rows sitting beside it in the same payload -- the one thing `schedule_render.py` rules out and both `candidate_display_text` and `required_blocks.slugs_on_candidate` already follow. It reads `payload["rows"]` now; the table stays as a documented pre-rows fallback, reading its columns by name out of `tmbx.core.render.COLUMNS` instead of at 3/4/5, and the fixtures move to the real 11-column header. The rung's `none` could not undo the rung's own redirect: a DM's origin key is the stable `{channel}:dm`, so the redirect outlived the judgement that set it and silently carried the next message to the old session. It is cleared on `NoReferent`, and the design's out-of-scope line now describes what the code does. Also: an aware `updated_at` keeps its offset instead of being overwritten, the referents package points at the interface-strain note, and the operator section of docs/setup/development.md says why a session-store outage refuses every timeboxing turn including the ones that should continue. Co-Authored-By: Claude Opus 5 (1M context) --- docs/setup/development.md | 38 +++++ .../2026-09-08-referent-catalog-design.md | 11 +- src/fateforger/referents/__init__.py | 3 + src/fateforger/referents/timeboxing.py | 15 +- src/fateforger/slack_bot/handlers.py | 135 +++++++++++++----- .../slack_bot/timeboxing_session_store.py | 90 +++++++++--- tests/evals/test_eval_referent_resolver.py | 28 +++- tests/unit/test_referent_rung_routing.py | 92 +++++++++++- .../unit/test_referent_timeboxing_provider.py | 25 ++++ tests/unit/test_standing_rows_query.py | 122 ++++++++++++++-- 10 files changed, 490 insertions(+), 69 deletions(-) diff --git a/docs/setup/development.md b/docs/setup/development.md index 1c4887f4..6794ce5d 100644 --- a/docs/setup/development.md +++ b/docs/setup/development.md @@ -29,3 +29,41 @@ what that means; this section is only the operational side. material store no longer holds. The refusal names the block and the handle; clear the link on that block with an explicit null, or restore the ticket under the same handle, before the day can be committed again. + +## A session-store outage refuses every timeboxing turn {#session-store-outage} + +**Expected, and it will not look expected.** When the timeboxing session store +cannot be read, the referent rung (`src/fateforger/slack_bot/handlers.py`) +cannot draw the catalog of days that already stand, and every door that would +hand a turn to `timeboxing_agent` refuses with: + +> :warning: I couldn't check what you already have planned, so I won't start a +> second session over the top of it. Say that again and I'll retry, or open one +> from the day's own thread if there is one. + +That includes **continuing a session that really does stand**. The guard asks +the store whether a row exists at the key the turn would use +(`_a_partial_catalog_forbids_this_turn`), and the store is the thing that is +down, so "there is nothing there" and "I could not tell" have to answer the +same way. Fail-closed is deliberate: the alternative is a second five-stage +session opened over a day that was already committed, which is the 2026-09-05 +incident this rung exists to close. + +What it looks like on call, and what to do: + +- **Symptom.** Every planning message — new day, DM, a reply in a live session + thread — comes back with the line above. Nothing else in the bot is affected: + the receptionist still answers, and a message not headed for a session runs + normally. A bot answering some things and refusing all planning is the + signature; a total outage is not. +- **Confirm it is the store.** `record_error(component="surface_intent")` fires + with `error_type="referent_catalog_partial"` on every refusal, and + `component="referent_catalog"`, `error_type="provider_failure"` on the read + that failed. The exception itself is logged by `build_catalog`. +- **Fix.** The store is the sessions database behind + `SqlAlchemyTimeboxingSessionRepository`; restore it or its connection. The + refusal clears by itself on the next message once reads succeed — there is no + cache to bust and no flag to reset. +- **Do not** work around it by restarting the bot or clearing focus. Neither + touches the store, and `/ff-clear` drops a focus binding without dropping the + redirect beside it, which is a separate hazard the same guard covers. diff --git a/docs/superpowers/specs/2026-09-08-referent-catalog-design.md b/docs/superpowers/specs/2026-09-08-referent-catalog-design.md index f79c0c60..571643a5 100644 --- a/docs/superpowers/specs/2026-09-08-referent-catalog-design.md +++ b/docs/superpowers/specs/2026-09-08-referent-catalog-design.md @@ -264,8 +264,15 @@ than alongside it. - **The second judgement** (what the message *does* to the referent) — #352. - **`none` → escalate** as a general rung — #337. - **The `/timebox` day-proposal door** — #346, landed 2026-09-05 (`cb4d9e5`). -- **Focus manager and receptionist changes.** The rung reads focus as context; it does not - rewrite it. +- **Focus manager and receptionist changes.** No change to the focus manager itself, and none + to the receptionist. The rung does write focus state, which an earlier draft of this line + denied: on a positive resolution it sets a redirect on the origin key and binds the target + thread to the referent's agent, because delivering a turn is what a redirect *is*. It also + clears that redirect when it later answers `none` — without which its own hour-old pointer + silently outvotes it on the next message, and a DM's origin key (`{channel}:dm`) is stable + enough for that to happen for the whole focus TTL. What the rung never does is bind focus on + the origin key itself: every other redirect setter does, and a bound key never reaches the + rung, which is what keeps this clearing confined to the rung's own leftovers. - **Recycling the empty auto-opened session** rather than inserting a new row. Deterministic store housekeeping, invisible to the user, and on the code side of the clause. Keeping it out of the resolver is the point: asking a model to link *"plan tomorrow"* to a day-less shell is a diff --git a/src/fateforger/referents/__init__.py b/src/fateforger/referents/__init__.py index 123c71de..8ad933fa 100644 --- a/src/fateforger/referents/__init__.py +++ b/src/fateforger/referents/__init__.py @@ -3,6 +3,9 @@ Nothing in this package imports Slack. A provider takes an owner and a clock; the catalog names what comes back; the resolver picks one, or none, or says it cannot tell. It never says what to *do* -- that is the consumer's own judgement. + +Where this interface strains, and what the second provider will hit before it +writes a line: `docs/superpowers/notes/2026-09-referent-interface-strain.md`. """ from .catalog import Catalog, build_catalog diff --git a/src/fateforger/referents/timeboxing.py b/src/fateforger/referents/timeboxing.py index f1f46aab..e558eb13 100644 --- a/src/fateforger/referents/timeboxing.py +++ b/src/fateforger/referents/timeboxing.py @@ -83,9 +83,18 @@ def _describe(self, row, *, as_of: datetime) -> StandingThing: never_used=( row.status == "open" and row.revision <= UNTOUCHED_REVISION ), - # row.updated_at is written naive UTC by the store's save method, - # so we tag it with UTC (not as_of.tzinfo) to preserve its true instant. - last_activity=row.updated_at.replace(tzinfo=UTC), + # `row.updated_at` is written naive UTC by the store's `save`, so a + # naive value is tagged UTC (not `as_of.tzinfo`) to keep its true + # instant. An aware one is converted, never overwritten: a blanket + # `.replace(tzinfo=UTC)` would silently discard a real offset the + # day some repository starts returning one -- the same silent-offset + # class this repo has already paid for once. Pattern taken from + # `haunt/reconcile.py`'s `_is_recent_local_stored_session`. + last_activity=( + row.updated_at.replace(tzinfo=UTC) + if row.updated_at.tzinfo is None + else row.updated_at.astimezone(UTC) + ), accepts=_COMMITTED_ACCEPTS if committed else _OPEN_ACCEPTS, gist=tuple(row.gist), channel_id=channel_id, diff --git a/src/fateforger/slack_bot/handlers.py b/src/fateforger/slack_bot/handlers.py index 6dafd496..b44dd77e 100644 --- a/src/fateforger/slack_bot/handlers.py +++ b/src/fateforger/slack_bot/handlers.py @@ -371,13 +371,28 @@ async def _resolve_referent( as_of: datetime, current_thread: tuple[str, str] | None, ): - """Which standing thing is this message about? `None` when nobody answered. - - `None` means "fall through unchanged" and covers every way this can decline: - a runtime with no resolver wired, no session store to draw a catalog from, - or a judgement that failed. There is deliberately no pattern fallback -- a - resolver this route cannot reach leaves today's behaviour exactly as it was, - loudly metered rather than quietly guessed. + """Which standing thing is this message about? `None` when nobody was asked. + + Two failures, and they are not the same failure. + + **Declined to ask** -- no resolver wired on the runtime, no session store to + draw a catalog from -- returns `None`, meaning "fall through unchanged". + That is a host that never opted in, and it must keep today's behaviour + exactly (`core/runtime.py`: *"the route reads this attribute and, finding + nothing, simply does not ask"*). There is deliberately no pattern fallback. + + **Asked and failed over a catalog that was not empty** returns + `NoReferent(catalog_complete=False)`. The system demonstrably *saw* the + standing day and merely could not judge it, so the answer it did not get is + at least as weak as one drawn from a short catalog -- and `catalog_complete` + already models "too weak to license creating something". The asymmetry it + replaces was unprincipled: a provider failure refused while a model failure + minted, which is the 2026-09-05 incident with the outage in the middle. + + The empty-catalog short-circuit keeps the blast radius tight. With nothing + standing there is nothing a second session could be opened over, so a model + outage falls through as before and only refuses when something really does + stand. """ resolver = getattr(runtime, "referent_resolver", None) if resolver is None: @@ -392,11 +407,21 @@ async def _resolve_referent( as_of=as_of, current_thread=current_thread, ) + except Exception: + # `build_catalog` swallows a provider's own failure into + # `complete=False`; reaching here means nothing was drawn at all, so + # there is no standing thing this could be refusing on behalf of. + logger.exception("referent catalog failed for %s", owner_user_id) + record_error(component="surface_intent", error_type="referent_failure") + return None + try: return await resolver.resolve(catalog=catalog, message=message, as_of=as_of) except Exception: logger.exception("referent resolution failed for %s", owner_user_id) record_error(component="surface_intent", error_type="referent_failure") - return None + if not catalog.referents: + return None + return NoReferent(catalog_complete=False) async def _maybe_update_timeboxing_thread_constraints( @@ -2792,6 +2817,19 @@ async def _update_constraints(thread_key: str) -> None: # the turn to timeboxing asks `_a_partial_catalog_forbids_this_turn` first. catalog_was_partial = False + # Which standing thing the rung read this message as, in the rung's own + # words, kept for whatever message finally lands on the origin ts. + # + # The rung *delivers* -- it does not post a card and wait for a press -- so + # this label is the whole remaining protection against a confidently wrong + # referent: it is the only place the user is told which day was chosen, and + # it has to be readable after the turn, not during it. The redirect path + # below ends in `_origin_link_to_thread`, a `chat_update` on that same ts, + # so a label written by the rung and left there survives in a DM and is + # overwritten in a channel -- including the incident's own #plan-sessions. + # It is carried into that final line instead. + chosen_referent_label: str | None = None + cleaned_text = _strip_bot_mention(text, bot_user_id) # The seam below may prefix `cleaned_text` with card context meant for # whichever agent answers. `user_reply_text` stays the user's own words, @@ -2937,20 +2975,31 @@ async def _a_partial_catalog_forbids_this_turn( async def _origin_link_to_thread( *, channel_id: str, thread_ts: str, agent_label: str ) -> None: + """The last word on the origin ts, carrying the rung's reading with it. + + This is a `chat_update` on the message the rung already wrote to, so + anything it does not repeat is erased. When a referent was chosen, the + line that names it is repeated here -- otherwise the one protection + against a confidently wrong referent lives for part of one turn and + only in a DM. + """ + lead = ( + f"Reading that as {chosen_referent_label} -- continuing" + if chosen_referent_label + else "Continuing" + ) link = await _permalink(channel_id, thread_ts) if not link: - await _origin_update( - text=f":left_right_arrow: Continuing in <#{channel_id}>." - ) + await _origin_update(text=f":left_right_arrow: {lead} in <#{channel_id}>.") return blocks = open_link_blocks( - text=f":left_right_arrow: Continuing in <#{channel_id}> (agent: *{agent_label}*).", + text=f":left_right_arrow: {lead} in <#{channel_id}> (agent: *{agent_label}*).", url=link, button_text="Go to Thread", action_id="ff_open_thread", ) await _origin_update( - text=f":left_right_arrow: Continuing in <#{channel_id}>.", blocks=blocks + text=f":left_right_arrow: {lead} in <#{channel_id}>.", blocks=blocks ) async def _begin_timeboxing_session_surface( @@ -3248,6 +3297,9 @@ async def _begin_timeboxing_session_surface( if isinstance(resolution, Resolved): referent = resolution.referent label = _referent_label(referent) + # Named once, and it has to outlive the branch: the redirect path + # below rewrites this same origin message on its way out. + chosen_referent_label = label if referent.is_current_surface: # It is about the conversation it arrived in. Nowhere to send # it and nothing to point at; the surface needs the right agent @@ -3396,27 +3448,44 @@ async def _begin_timeboxing_session_surface( ) return - elif isinstance(resolution, NoReferent) and not resolution.catalog_complete: - # A provider failed, so `none` is not evidence that nothing stands - # (`referents/catalog.py`'s own contract). Everywhere else that is - # survivable; in front of a door that creates it is the incident, - # so ask rather than create. - # - # `catalog_complete` rides every outcome and is acted on only here, - # deliberately. The flag changes what a *weak* answer licenses, and - # the other two outcomes license nothing that a partial catalog - # could make dangerous: `Resolved` hands the turn to a session that - # demonstrably exists, and `Ambiguous` already asks. `NoReferent` is - # the only one whose answer is "so go ahead and create". + elif isinstance(resolution, NoReferent): + # The rung ran and pointed at nothing, so a redirect *the rung + # itself* set on an earlier turn must not quietly carry this turn + # anyway. It can: a DM's `origin_key` is the stable `{channel}:dm`, + # the rung sets a redirect on it and (unlike every other setter -- + # `open_session_surface`, the handoff) no focus binding beside it, + # so the rung runs again an hour later and `get_redirect` below + # still answers with the old session. Clearing here is what makes + # `none` mean `none`; without it the rung's judgement is overridden + # by its own stale pointer, and the design's "the rung reads focus + # as context" is only half true. # - # Recorded here and nothing more. Whether this turn ever reaches - # timeboxing is a judgement nobody has made yet -- the model may or - # may not hand off -- so pre-empting here would answer "I couldn't - # check what you have planned" to someone asking what the weather - # is, and a rare store outage would become a bot-wide one. Every - # door that hands the turn to timeboxing refuses instead, and the - # ones that mint without a model call refuse before spending one. - catalog_was_partial = True + # Nothing else loses a redirect to this: every other setter binds + # focus on the same key, and a bound key never reaches the rung. + focus.clear_redirect(origin_key) + + if not resolution.catalog_complete: + # A provider failed, or the judge could not be asked over a catalog + # that was not empty, so `none` is not evidence that nothing stands + # (`referents/catalog.py`'s own contract). Everywhere else that is + # survivable; in front of a door that creates it is the incident, + # so ask rather than create. + # + # `catalog_complete` rides every outcome and is acted on only here, + # deliberately. The flag changes what a *weak* answer licenses, and + # the other two outcomes license nothing that a partial catalog + # could make dangerous: `Resolved` hands the turn to a session that + # demonstrably exists, and `Ambiguous` already asks. `NoReferent` is + # the only one whose answer is "so go ahead and create". + # + # Recorded here and nothing more. Whether this turn ever reaches + # timeboxing is a judgement nobody has made yet -- the model may or + # may not hand off -- so pre-empting here would answer "I couldn't + # check what you have planned" to someone asking what the weather + # is, and a rare store outage would become a bot-wide one. Every + # door that hands the turn to timeboxing refuses instead, and the + # ones that mint without a model call refuse before spending one. + catalog_was_partial = True redirect = focus.get_redirect(origin_key) if redirect and agent_type == redirect.agent_type: diff --git a/src/fateforger/slack_bot/timeboxing_session_store.py b/src/fateforger/slack_bot/timeboxing_session_store.py index d545a2e3..a61fa4b8 100644 --- a/src/fateforger/slack_bot/timeboxing_session_store.py +++ b/src/fateforger/slack_bot/timeboxing_session_store.py @@ -28,6 +28,7 @@ PlanningSessionSnapshot, TurnOutcome, ) +from tmbx.core.render import COLUMNS class _Base(DeclarativeBase): @@ -450,45 +451,94 @@ def _day_frame(snapshot: PlanningSessionSnapshot) -> dict | None: def _plan_gist(snapshot_json: str) -> tuple[str, ...]: """A few of the plan's own block titles, with their times. - Read from the latest validated candidate's rendered block table, whose - columns are `H,own,type,summary,ST,ET,mode,dur` -- a table this system - generated, so taking the summary and the two clocks out of it is arithmetic - over our own format and not a reading of anything the user wrote. Anything - unparseable yields no gist rather than a guess. - - Parsed with `csv.reader`, not `line.split(",")`: `render_plan`'s `_escape` - CSV-quotes a summary that contains the table's own delimiter (its - docstring's own example is `"Sprint, planning"`), and a naive split breaks - a quoted field into two, shifting every column after it -- the end time - comes back as the start time, and the summary carries a stray quote. Using - `csv.reader` is still reading our own format by its own rules, just more - faithfully than a hand-rolled split. + **From the rows, never from the table when the rows are there.** That + ruling is already this repo's (`schedule_render.py`): *"A comma in a + summary, a block crossing midnight, a column renamed on the server -- each + is a way a parser here would go quietly wrong, and the rows already carry + every field the table does."* `candidate_display_text` follows it and so + does `required_blocks.slugs_on_candidate` (*"the authoritative record when + the capture has them"*). The rows sit in the same `validated_candidate` + payload as the rendered table, carrying `summary`, `start` and `end` as + fields (`validated_timebox_draft.py`), so this reads those. + + The table is the fallback and nothing else: an artifact captured before + `plan_apply` returned rows beside the table carries only `rendered`, and a + table is still better than no gist at all. It is a format this system + generated, so taking three fields out of it is arithmetic over our own + columns and not a reading of anything the user wrote -- but the columns are + located by name in `tmbx.core.render.COLUMNS` rather than by hardcoded + position, because a column inserted before `summary` would otherwise shift + every field silently. Parsed with `csv.reader`, not `line.split(",")`: + `render_plan`'s `_escape` CSV-quotes a summary containing the table's own + delimiter (its docstring's own example is `"Sprint, planning"`), and a naive + split breaks a quoted field into two, shifting every column after it. + Anything unparseable yields no gist rather than a guess. """ try: envelope = json.loads(snapshot_json) artifacts = envelope["snapshot"]["artifacts"] except (ValueError, KeyError, TypeError): return () - rendered = next( + payload = next( ( - artifact.get("payload", {}).get("rendered") + artifact.get("payload") for artifact in reversed(artifacts) - if artifact.get("kind") == "validated_candidate" + if isinstance(artifact, dict) + and artifact.get("kind") == "validated_candidate" ), None, ) - if not isinstance(rendered, str): + if not isinstance(payload, dict): return () + rows = payload.get("rows") + if isinstance(rows, list) and rows: + return _gist_from_rows(rows) + return _gist_from_rendered(payload.get("rendered")) + + +def _gist_from_rows(rows: list) -> tuple[str, ...]: + """The resolved rows as the model reads them, in the plan's own order.""" entries: list[str] = [] + for row in rows: + if not isinstance(row, dict): + continue + summary, start, end = ( + row.get("summary"), + row.get("start"), + row.get("end"), + ) + if not ( + isinstance(summary, str) + and isinstance(start, str) + and isinstance(end, str) + ): + continue + entries.append(f"{summary} {start}-{end}") + return tuple(entries) + + +def _gist_from_rendered(rendered: object) -> tuple[str, ...]: + """The pre-rows fallback: the handle table, read by column name.""" + if not isinstance(rendered, str): + return () + try: + summary_at = COLUMNS.index("summary") + start_at = COLUMNS.index("ST") + end_at = COLUMNS.index("ET") + except ValueError: # pragma: no cover - the render module renamed a column + return () + width = max(summary_at, start_at, end_at) + 1 try: data_rows = list(csv.reader(rendered.splitlines()[1:])) except csv.Error: return () + entries: list[str] = [] for fields in data_rows: # first line was already dropped: column header - if len(fields) < 6: + if len(fields) < width: continue - summary, start, end = fields[3], fields[4], fields[5] - entries.append(f"{summary} {start}-{end}") + entries.append( + f"{fields[summary_at]} {fields[start_at]}-{fields[end_at]}" + ) return tuple(entries) diff --git a/tests/evals/test_eval_referent_resolver.py b/tests/evals/test_eval_referent_resolver.py index 76efe3c0..9dd7780a 100644 --- a/tests/evals/test_eval_referent_resolver.py +++ b/tests/evals/test_eval_referent_resolver.py @@ -14,7 +14,23 @@ consumer's judgement, not this one's), and "move the gym to the morning" resolves to the only plan that contains a gym. -n = 8 draws per case, asserted on the rate. +**The probes are the instrument; the rate is a watch number.** Task 8 measured +this directly: with the load-bearing last sentence of `RESOLVER_PROMPT` deleted, +the two fixtures still scored 87.5% and 90% -- both above the 85% floor. A rate +over 21 mixed cases cannot discriminate a known prompt regression, so do not +edit `RESOLVER_PROMPT`, watch the percentage, and ship. What holds the line is +the unanimity gate on `PROBES`: four messages naming a block that exists in no +listed plan, each required to draw `none` on all 8 samples. A wrong answer to +one of those is a duplicate session at a door that creates, which is the failure +this whole rung exists to stop, so anything short of 8/8 fails the run outright. + +The rate assertion below is a floor against wholesale collapse and nothing more. +Two per-case results already draw badly with the prompt intact and are not +regressions: `remind me to pay taxes` (2-3/8; `r3`'s gist really does contain +`"Pay taxes 14:15-15:15"`) and `move PR1 later` (0/8; only `r2`'s gist mentions +a PR). Read the printed per-case counts, not the percentage. + +n = 8 draws per case. set -a; source .env; set +a PYTHONPATH=src .venv/bin/python -m pytest tests/evals/test_eval_referent_resolver.py -m slow -q @@ -212,5 +228,13 @@ async def test_resolution_quality(rows, cases, at): total = len(cases) * DRAWS rate = hits / total print(f" == {hits}/{total} draws ({rate:.0%})") + # The gate. A probe that is not unanimous is a session opened over a plan + # nobody named, whatever the rate says. assert not failures, "\n".join(failures) - assert rate >= 0.85, f"{hits}/{total} draws correct; the measured arm scored ~0.85+" + # A floor against wholesale collapse, not a prompt check: a deliberately + # broken prompt still scored 87.5-90% here (see the module docstring). + assert rate >= 0.85, ( + f"{hits}/{total} draws correct, below the 0.85 collapse floor; " + "read the per-case counts above -- this number does not detect a " + "prompt regression, the probe gate does" + ) diff --git a/tests/unit/test_referent_rung_routing.py b/tests/unit/test_referent_rung_routing.py index 61a1e3ee..c8102b26 100644 --- a/tests/unit/test_referent_rung_routing.py +++ b/tests/unit/test_referent_rung_routing.py @@ -60,6 +60,22 @@ async def test_the_origin_gets_a_pointer_to_the_thread_that_took_it(routing_harn assert any("1788571682" in text for text in harness.origin_messages) +async def test_the_line_naming_the_chosen_day_survives_the_turn_in_a_channel( + routing_harness, +): + # The rung delivers; it does not post a card and wait for a press. So the + # line naming which day was chosen is the whole remaining protection + # against a confidently wrong referent -- and it has to still be there when + # the user reads it. The redirect path ends in `_origin_link_to_thread`, a + # `chat_update` on the very message the rung wrote to, which used to + # replace it with a generic "Continuing in <#...>": the label survived in a + # DM and was erased in #plan-sessions, the incident's own channel. + harness = routing_harness(resolver=_Resolver(Resolved(referent=_ref()))) + await harness.route_top_level("can you replan today so the gym is before dinner?") + assert harness.origin_messages + assert "05 September" in harness.origin_messages[-1] + + async def test_ambiguity_asks_and_opens_nothing(routing_harness): harness = routing_harness( resolver=_Resolver( @@ -78,10 +94,45 @@ async def test_none_falls_through_to_todays_behaviour(routing_harness): assert harness.sessions_opened == ["C0AA6HC1RJL"] -async def test_a_resolver_failure_falls_through_rather_than_guessing(routing_harness): +async def test_a_resolver_failure_over_a_day_that_stands_refuses_rather_than_minting( + routing_harness, +): + # The committed 2026-09-05 row is in the catalog and the judge is down. The + # system demonstrably *saw* the day and only failed to judge it, so the + # answer it did not get is at least as weak as one drawn from a short + # catalog -- and falling through here lands on the minting door, which is + # the incident itself. A provider failure refusing while a model failure + # mints is an asymmetry with nothing behind it. harness = routing_harness(resolver=_Resolver(RuntimeError("model down"))) await harness.route_top_level("replan today") + assert harness.sessions_opened == [] + assert harness.sessions_created == [] + assert PARTIAL_CATALOG_ASK in harness.origin_messages + + +async def test_a_resolver_failure_with_nothing_standing_still_falls_through( + routing_harness, +): + # The short-circuit that keeps the blast radius tight: with an empty + # catalog there is nothing a second session could be opened over, so a + # model outage costs nobody a turn. + harness = routing_harness(resolver=_Resolver(RuntimeError("model down")), rows=[]) + await harness.route_top_level("replan today") assert harness.sessions_opened == ["C0AA6HC1RJL"] + assert PARTIAL_CATALOG_ASK not in harness.origin_messages + + +async def test_a_host_that_wired_no_resolver_keeps_todays_behaviour_exactly( + routing_harness, +): + # Declined to ask, which is not the same failure as asked-and-failed. The + # route reads `runtime.referent_resolver` and, finding nothing, does not + # ask (`core/runtime.py`); a host that never opted in must not start being + # refused turns. + harness = routing_harness(resolver=None) + await harness.route_top_level("replan today") + assert harness.sessions_opened == ["C0AA6HC1RJL"] + assert PARTIAL_CATALOG_ASK not in harness.origin_messages async def test_the_catalog_is_built_before_any_session_is_opened(routing_harness): @@ -294,3 +345,42 @@ async def test_a_redirect_that_outlived_its_focus_does_not_mint_over_a_partial_c await harness.route_dm("move the gym earlier") assert harness.sessions_created == [] assert PARTIAL_CATALOG_ASK in harness.origin_messages + + +class _EachTurn: + """A resolver whose answer changes from one turn to the next.""" + + def __init__(self, *outcomes): + self._outcomes = list(outcomes) + self.calls = [] + + async def resolve(self, *, catalog, message, as_of): + self.calls.append((catalog, message, as_of)) + index = min(len(self.calls) - 1, len(self._outcomes) - 1) + return self._outcomes[index] + + +async def test_a_later_none_in_a_dm_is_not_overridden_by_the_rungs_own_redirect( + routing_harness, +): + # A DM's `origin_key` is the stable `{channel}:dm`, not a one-shot + # `{channel}:{ts}`, so the redirect the rung sets there lives for the focus + # TTL (an hour by default). The rung sets no focus binding beside it, which + # is exactly why it runs again on the next DM message -- and answering + # `none` used to change nothing, because `get_redirect` further down still + # pointed at the session an hour-old judgement had chosen. The rung's own + # answer was silently overridden by the rung's own stale pointer. + session = "C0AA6HC1RJL:1788571682.407949" + resolver = _EachTurn(Resolved(referent=_ref()), NoReferent()) + harness = routing_harness(resolver=resolver) + + await harness.route_dm("can you replan today so the gym is before dinner?") + assert harness.delivered_to == session + + await harness.route_dm("what is the weather tomorrow?") + assert len(resolver.calls) == 2, "the rung must run again on the next DM turn" + assert harness.focus.get_redirect("D_HUGO:dm") is None + # The turn still happens -- it lands on the DM's own key, and only the + # first message ever reached the session the first judgement chose. + assert [call[1].key for call in harness.runtime.calls] == [session] + assert harness.sessions_created[-1] == "D_HUGO:dm" diff --git a/tests/unit/test_referent_timeboxing_provider.py b/tests/unit/test_referent_timeboxing_provider.py index 92608ef5..229b0e93 100644 --- a/tests/unit/test_referent_timeboxing_provider.py +++ b/tests/unit/test_referent_timeboxing_provider.py @@ -141,3 +141,28 @@ async def test_last_activity_is_correctly_zoned_when_as_of_is_not_utc(): # The elapsed time should be 1 hour (true UTC difference) elapsed = as_of_in_amsterdam - thing.last_activity assert abs(elapsed.total_seconds() - 3600) < 1 # Allow 1 second tolerance for rounding + + +async def test_an_aware_updated_at_keeps_its_own_offset(): + # The store writes naive UTC today, so this is the future case: a repository + # that starts returning an aware value. A blanket `.replace(tzinfo=UTC)` + # overwrites rather than asserts naivety, and the true instant is silently + # discarded -- 10:51+02:00 (08:51 UTC) would be read as 10:51 UTC, and a + # session last touched three hours ago would describe itself as one hour old. + amsterdam_tz = ZoneInfo("Europe/Amsterdam") + aware = datetime(2026, 9, 5, 10, 51, tzinfo=amsterdam_tz) # 08:51 UTC + + row = _Row( + session_key="C1:111.0", + status="open", + planning_date=date(2026, 9, 5), + updated_at=aware, + revision=7, + gist=(), + ) + provider = TimeboxingReferentProvider(_Repo([row])) + (thing,) = await provider.standing(owner_user_id="U1", as_of=AS_OF) + + assert thing.last_activity == aware + # AS_OF is 11:51 UTC, so the true gap is three hours, not one. + assert abs((AS_OF - thing.last_activity).total_seconds() - 3 * 3600) < 1 diff --git a/tests/unit/test_standing_rows_query.py b/tests/unit/test_standing_rows_query.py index 67163fdb..78e78f3d 100644 --- a/tests/unit/test_standing_rows_query.py +++ b/tests/unit/test_standing_rows_query.py @@ -121,7 +121,9 @@ async def test_a_row_created_after_the_asked_moment_is_excluded(repo): assert rows == [] -async def test_the_gist_comes_from_the_candidates_rendered_blocks(repo): +async def test_a_pre_rows_candidate_falls_back_to_the_rendered_table(repo): + # An artifact captured before `plan_apply` returned rows beside the table + # carries only `rendered`. A table is better than no gist at all. repository, maker = repo snapshot = { "envelope_version": 1, @@ -132,9 +134,11 @@ async def test_the_gist_comes_from_the_candidates_rendered_blocks(repo): "revision": 1, "payload": { "rendered": ( - "blocks[2]{H,own,type,summary,ST,ET,mode,dur}:\n" - "PR1,tmbx,C,Serious C2F work,10:30,12:00,fs,PT1H30M\n" - "GYM1,tmbx,H,Gym (chest),18:00,19:00,fs,PT1H" + "blocks[2]{H,own,type,summary,ST,ET,mode," + "dur,slug,link,link_label}:\n" + "PR1,tmbx,C,Serious C2F work,10:30,12:00,fs," + "PT1H30M,,,\n" + "GYM1,tmbx,H,Gym (chest),18:00,19:00,fs,PT1H,,," ) }, } @@ -196,8 +200,10 @@ async def test_the_gist_keeps_a_comma_inside_a_quoted_summary_whole(repo): "revision": 1, "payload": { "rendered": ( - "blocks[1]{H,own,type,summary,ST,ET,mode,dur}:\n" - 'PR1,tmbx,C,"Serious C2F work, prep",10:30,12:00,fs,PT1H30M' + "blocks[1]{H,own,type,summary,ST,ET,mode," + "dur,slug,link,link_label}:\n" + 'PR1,tmbx,C,"Serious C2F work, prep",10:30,' + "12:00,fs,PT1H30M,,," ) }, } @@ -230,8 +236,10 @@ async def test_a_malformed_row_yields_no_gist_entry(repo): "revision": 1, "payload": { "rendered": ( - "blocks[1]{H,own,type,summary,ST,ET,mode,dur}:\n" - 'PR1,tmbx,C,"Unterminated summary,10:30,12:00,fs,PT1H30M' + "blocks[1]{H,own,type,summary,ST,ET,mode," + "dur,slug,link,link_label}:\n" + 'PR1,tmbx,C,"Unterminated summary,10:30,' + "12:00,fs,PT1H30M,,," ) }, } @@ -247,3 +255,101 @@ async def test_a_malformed_row_yields_no_gist_entry(repo): open_within=timedelta(hours=12), horizon=timedelta(days=7), ) assert rows[0].gist == () + + +async def test_the_gist_comes_from_the_rows_and_not_the_table_beside_them(repo): + # `schedule_render.py`'s ruling, which `candidate_display_text` and + # `required_blocks.slugs_on_candidate` both already follow: never from the + # table when the rows are there. The two disagree here on purpose -- the + # table is what a stale or reshuffled render would say, the rows are what + # `plan_apply` resolved -- so which one the gist came from is visible. + repository, maker = repo + snapshot = { + "envelope_version": 1, + "snapshot": { + "artifacts": [ + { + "kind": "validated_candidate", + "revision": 1, + "payload": { + "rows": [ + { + "summary": "Serious C2F work", + "start": "10:30", + "end": "12:00", + }, + { + "summary": "Gym (chest)", + "start": "18:00", + "end": "19:00", + }, + ], + "rendered": ( + "blocks[1]{H,own,type,summary,ST,ET,mode," + "dur,slug,link,link_label}:\n" + "PR1,tmbx,C,what the table says,08:00,09:00,fs," + "PT1H,,," + ), + }, + } + ] + }, + "outcomes": {}, + } + import json + + await _insert(maker, snapshot_json=json.dumps(snapshot)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == ( + "Serious C2F work 10:30-12:00", + "Gym (chest) 18:00-19:00", + ) + + +async def test_the_fallback_locates_its_columns_by_name(repo): + # The columns are read out of `tmbx.core.render.COLUMNS`, not hardcoded at + # 3/4/5. This row is written positionally against the COLUMNS of the day, + # so inserting a column before `summary` breaks it loudly instead of + # shifting the gist by one field in silence. + repository, maker = repo + from tmbx.core.render import COLUMNS + + values = { + "H": "PR1", + "own": "tmbx", + "type": "C", + "summary": "Serious C2F work", + "ST": "10:30", + "ET": "12:00", + "mode": "fs", + "dur": "PT1H30M", + } + line = ",".join(values.get(column, "") for column in COLUMNS) + snapshot = { + "envelope_version": 1, + "snapshot": { + "artifacts": [ + { + "kind": "validated_candidate", + "revision": 1, + "payload": { + "rendered": ( + "blocks[1]{" + ",".join(COLUMNS) + "}:\n" + line + ) + }, + } + ] + }, + "outcomes": {}, + } + import json + + await _insert(maker, snapshot_json=json.dumps(snapshot)) + rows = await repository.standing_rows( + owner_user_id="U1", as_of=AS_OF, + open_within=timedelta(hours=12), horizon=timedelta(days=7), + ) + assert rows[0].gist == ("Serious C2F work 10:30-12:00",)