Skip to content

feat(referents): a follow-up reaches the session it is about (#345) - #445

Open
hugolytics wants to merge 20 commits into
mainfrom
feat/referent-catalog
Open

feat(referents): a follow-up reaches the session it is about (#345)#445
hugolytics wants to merge 20 commits into
mainfrom
feat/referent-catalog

Conversation

@hugolytics

Copy link
Copy Markdown
Collaborator

Closes #345. Boundary agreed with #352; docs follow-up is #443.

The incident

On 2026-09-05 at 13:51 Hugo typed, top-level in #plan-sessions:

can you replan today so the gym is before dinner?

That channel's route was hardcoded to mean open a new session. It started a
fresh five-stage session for a day committed at 01:41 that morning, and
discarded the request text on the way in (ConfirmPlanningDay carries only the
date). The session store knew the day was committed — standing_for had been
answering that question for the nudger all morning. The door never asked.

What this adds

A catalog of the user's standing things, one judgement over it, and a
rung in route_slack_event that delivers a follow-up into the session it
concerns.

  • src/fateforger/referents/ — a new package that imports no Slack. A provider
    answers standing(owner_user_id, as_of); the catalog names what comes back;
    the resolver picks one, or none, or says it cannot tell.
  • The resolver returns a referent and never an action. Resolve-then-act is
    enforced by the return type, not by prose: no outcome has a field that can
    carry a decision. What the message does to the referent is the surface's own
    judgement, which is how PR timeboxing: asked ≠ started — a question to the Schedular is answered, never turned into a session (#316–#320) #328's asked-≠-started rule survives a new door.
  • Timeboxing sessions are the first provider. The task marshal is intended as
    the second, against the same interface.

It is measured, not argued

Frozen fixtures built from real timeboxing_session_states rows, 8 draws per
case on the flash pin at reasoning: minimal, ~0.4s per call.

finding measurement
One call over all candidates beats per-candidate scoring per-candidate: 0/8 on most cases — a candidate judged alone has no contrast
One prompt sentence carries most of the quality 74/112 → 95/112
How the never-used fact is carried prose 93, field 94, both 91 — all inside a ~2/112 noise floor
Whether it is carried at all dropping the row: 88/112, and "cancel that session" collapses 8/8 → 2/8 onto the wrong day
The plan gist (titles with times) 28/56 → 53/56 on the two-parallel-sessions incident (#275)
The gist does not invent matches four probes naming blocks in neither plan: 8/8 none with it, 0/8–6/8 without
Real ambiguity survives "move the gym", where both plans have a gym: ambiguous 8/8

A content-free descriptor is not the cautious option — it fails both ways:
false none on a block that exists, false ambiguous on one that does not.

tests/evals/test_eval_referent_resolver.py gates at 152/168 draws (90.5%).
Read its docstring before touching RESOLVER_PROMPT: the probe-unanimity gate
is the instrument
, not the rate — a deliberately broken prompt still scored
87–90% against the 85% floor.

The invariant, and what it cost

No door may create a session over a day the system could not see. A none
drawn from a partial catalog is not evidence that nothing stands.

That took four fix rounds, and each round's claim of completeness was falsified
by the next review. The root cause was not the routing code: the test harness
was lying.
Its fakes wrote no session rows, so two of the four minting doors
were invisible to every test. Both fakes now write what production writes, and
sessions_created asserts on rows rather than on Slack messages.

The lesson that generalises: creation is decided by whether the session key is
already known, not by the message type.
Guards written as
isinstance(msg, StartTimeboxing) were false — TimeboxingUserReply creates
too, and the harness backend creates via load_or_create without sending
either.

Four ways to be wrong, and none of them writes: a false none is caught by the
structural query contradicting the model; a false ambiguous asks; a wrong
confident referent is named in a label the user sees; a target that moved is
refused at press time.

Known gaps, named rather than hidden

  • DM referents are unreachable. set_redirect addresses by channel+thread;
    a {channel}:dm key names no thread. The user is told, it is metered, and it
    never falls through into a creating door. Needs a key-addressed redirect.
  • Three no-catalog doors. /timebox with no argument (ruled acceptable: an
    explicit command is not an inferred follow-up), a file_share with no comment
    (not an explicit request — parked, real, mints regardless of catalog
    state), and an explicit /ff-focus binding. The right fix for all three is a
    sentinel at load_or_create, not a fourth guard.
  • During a real session-store outage every timeboxing turn is refused,
    including continuing a session that stands. Correct fail-closed behaviour;
    written into docs/setup/development.md so it does not read as a total outage
    to whoever is on call.
  • The eval aborts on transport flakiness in roughly half of runs
    (asyncio.gather without return_exceptions). Not CI-gate-ready.

Compounding

docs/superpowers/notes/2026-09-referent-interface-strain.md records nine
places the interface strained to fit timeboxing, and six items judged not
interface strain. It is the input to the marshal's provider and to #160, and it
is the honest answer to the map's own test: does the second provider get
cheaper because the first exists?

Verification

3056 passed, 2 skipped, 1 xfailed; -m "not slow" → 3444 passed.
tests/unit/test_slack_timeboxing_routing.py is byte-identical to main
(blob 3db9b8a) — it encodes the resolver-order contract from #310 and was
never touched.

Every refusal has a test that fails when the refusal is removed, verified by
independent disable-and-restore runs rather than by the implementers' reports.

🤖 Generated with Claude Code

hugocool and others added 20 commits September 8, 2026 19:48
…cidents (#345)

A message with no structural owner gets one judgement over the user's standing
things, and delivery goes to the referent's own surface. The trigger: on
2026-09-05 13:51 "can you replan today so the gym is before dinner?" opened a
fresh five-stage session for a day committed at 01:41, and the request text was
discarded on the way in. `standing_for` had known the day was committed all
morning; the door never asked it.

The design is measured rather than argued. Frozen fixtures built from real
`timeboxing_session_states` rows, 8 draws per case on the flash pin at
`reasoning: minimal`, ~0.4s per call:

- one call over all candidates beats per-candidate scoring, which collapses to
  0/8 on most cases because a candidate judged alone has no contrast;
- one prompt distinction carries most of the quality, 74/112 -> 95/112;
- how the never-used fact is *carried* is inside the noise floor (~2/112), but
  dropping that row costs 8/8 -> 2/8 on "cancel that session", picking the
  wrong day;
- the plan gist (block titles WITH times) is the largest single effect,
  28/56 -> 53/56 on the two-parallel-sessions incident;
- and it does not invent matches: four probes naming blocks in neither plan
  answer `none` 8/8 with the gist, against 0/8-6/8 without. A content-free
  descriptor is not the cautious option -- it fails both ways.

Two design points are enforcement rather than prose. The resolver returns a
referent and never an action, so resolve-then-act is a return type; and the
catalog is a callable seam taking an owner and an explicit as-of, so #352's
door and the eval are consumers alongside the routing rung.

Boundary agreed directly with #352's owner (admonish-1-56 [6988fa]), whose
blind review corrected two fixture labels and supplied the false-positive probe
that inverted the risk it was meant to confirm.

The spike rides along as the primary source; it carries its fixtures inline and
reproduces without the ledger, which matters because the ledger moves under a
run.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…gn (#345)

TDD throughout, one test cycle per task, each ending in a committable
deliverable. The order is descriptor, provider seam, resolver, guard, provider,
query, rung, eval, strain log -- so the model call is testable with a stub by
task three and the rung, which is the only task touching the 5,085-line
handlers.py, arrives last with everything under it already proven.

Three things the plan pins down rather than leaving to taste:

- the eval's fixtures are inline and dated, with the docstring saying WHY, since
  reading the live ledger measures its drift and the store keeps no history;
- the thresholds come from the measured runs (>=85% of draws, and the four
  false-positive probes unanimous, because a wrong answer there becomes a
  duplicate session at a door that creates);
- task 4 is an AST guard that block titles are read by a judge and never
  compared by code, and task 8 step 3 breaks the prompt on purpose to prove the
  eval is not vacuous.

Task 9 is the compounding deliverable Hugo asked for by name: a strain log
written during the build, recording what the interface had to grow to fit
timeboxing and which of it is general, as the input to the task marshal's own
provider.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cks and times

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… by the host

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…erent and never an action

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tion checks

- Add check for both ast.Import and ast.ImportFrom for slack imports
- Add test for forbidden builtin functions (sorted, any, all, max, min, sum, set)
  called on gist to prevent content reordering or filtering

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nces

Fixes regression where nested gist usage like min(len(self.gist), GIST_LIMIT)
was incorrectly flagged. The guard now only fails on direct calls like
sorted(self.gist) or set(self.gist), allowing structural operations that
don't inspect content meaning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The session store writes updated_at as naive UTC. Tagging it with as_of.tzinfo
was wrong when as_of was not in UTC, silently computing wrong elapsed times
and biasing router judgements. Use UTC consistently with the store's convention.

Add test that catches this when as_of is non-UTC (Europe/Amsterdam).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ch plan's gist

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… split

A summary containing a comma is CSV-quoted by render_plan's _escape, and
line.split(",") broke a quoted field into two pieces, shifting every
column after it -- the end time came back as the start time and the
summary carried a stray quote. csv.reader respects that quoting, so the
gist shown to the referent judge no longer silently misdescribes the plan.
A genuinely unparseable row still fails closed to an empty gist.

Adds two tests that fail against the old split(",") parser and pass
against csv.reader, plus a committed-day-beyond-horizon exclusion test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…minting a second one (#345)

The 2026-09-05 13:51 incident: "can you replan today so the gym is before
dinner?" typed top-level in #plan-sessions opened a fresh five-stage session
for a day that had been committed at 01:41. No structural resolver owned that
message -- it was top-level, so there was no thread to own -- and the channel
default sent it straight through the door that creates.

The referent rung goes last among the resolvers and ahead of every door that
can create. It asks one judgement -- which standing thing is this about --
and nothing more: what to do about the answer belongs to the surface that
owns the state. Resolved redirects the turn and points at it from where the
user typed; ambiguous asks and opens nothing; none, and every way the
judgement can decline, falls through to exactly today's behaviour. There is
no pattern fallback.

The catalog is drawn before any session is opened, so it can never offer the
row this very message would mint. That ordering is the guarantee and it is
tested directly, not inferred from timestamps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ive thread (#345)

Four findings from review, all of them the same shape: a place where the rung
could still end in a second session for a day that already had one.

A partial catalog is no longer read as "nothing stands". `build_catalog` says
so in `complete`, and this rung is the consumer that field was built for: in
front of a door that creates, a `none` drawn from a catalog a provider failed
to fill now asks instead of creating. Outside such a door it still falls
through, because asking there is noise. A resolver that *raises* is a
different fact and still falls through unchanged.

A positive resolution the route cannot deliver -- a session key naming a whole
DM, an agent focus refuses -- used to be dropped to a `logger.info` and then
create anyway. That is the incident with the machinery built to catch it
reporting success. It is now metered and said out loud, and it never continues
into the creating branch.

The session store is asked whatever `agent_type` already says. It used to be
asked only when `agent_type` was not already `timeboxing_agent`, which in the
planning channel is never -- the channel default says so before anyone has
claimed anything. On the app_mention path, where focus is never auto-recovered,
that left a live session's own thread with no structural claim, and after a
restart a judgement could send "move the gym earlier" to a different day.
Structural ownership is a fact and beats a judgement (#310).

The rung also moves to after the acknowledgement. A model round trip before
the first frame is silence a user cannot tell from a dropped message, and only
the app_mention path had an ack to hide it. It still runs ahead of every door
that can create, which is the ordering that matters. It also makes the origin's
first message the route's own by construction rather than by racing the
background memory task, so the harness needs no stub for that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…handoff door obeys it (#345)

The rung's own predicate for "is there a door that creates ahead of this
message" was `agent_type == "timeboxing_agent" and not thread_ts`. It was a
third reading of a condition two other sites already derived their own way,
and it missed the door a plain "can you replan today so the gym is before
dinner?" actually goes through: typed anywhere but the planning channel, the
turn goes to the receptionist, nothing about it reads as timeboxing, and the
receptionist hands it off into a session surface built over a day the catalog
could not see.

`_turn_can_mint_a_session` replaces all of it with one reading, anchored on
`open_session_surface` -- the thing that mints -- and on the fact that any
agent which is not the timeboxing one can hand the turn to it. It over-answers
on purpose, because nobody can know before the turn whether the model will
hand off, and the two mistakes are not symmetric: asking when nothing would
have been created costs a retry, creating over a day that already stands is
the incident.

The door answers back. The check sits inside the sole caller of
`open_session_surface`, so a third door cannot be added without inheriting it,
and a mint the rung was told could not happen is logged and metered rather
than inherited in silence.

Also: the branch that reads `catalog_complete` now says why the other two
outcomes need not, and a resolution that cannot be delivered names where the
plan lives instead of only saying it cannot be reached.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…versation (#345)

The rung refused up front whenever a door that could create lay ahead. That
was every message reaching the receptionist, because the receptionist can hand
the turn off -- so during a store outage someone asking what the weather is
was told the planner could not check their calendar. A rare failure with the
blast radius of every conversation.

The direction was right and the place was wrong. "Would this turn mint a
session" is not a yes or no: on `timeboxing_agent` with no thread to continue
it is certain, and there the turn has nothing else to do, so refusing up front
is right. Everywhere else it depends on a judgement nobody has made yet --
whether the model hands off -- so the turn now answers, carrying the fact that
the catalog was short, and whatever door turns out to create refuses at that
moment instead.

Both paths say the same words. Which internal door the turn took is not
something a user should have to model, and a hard refusal would have surfaced
as "something went wrong", naming a bug rather than a store that could not be
read.

The two doors `_begin_timeboxing_session_surface` never sees -- the legacy
fallback and the in-thread handoff -- ask `isinstance(msg, StartTimeboxing)`
of the message the builder actually produced. That reads the decision rather
than re-deriving it, so it cannot drift, and it closes the gap round 2 left.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sage type (#345)

Creation is decided by whether the session key is already known. Three rounds
asked the event shape or the message type instead, and all three were wrong the
same way: `StartTimeboxing` is not the only creating message.
`TimeboxingUserReply` mints one through `on_user_reply`'s
`_ensure_uncommitted_session`, and on the kernel backend the route sends neither
message and mints at `load_or_create`. So `MINT_NO` -- "a thread to continue, or
a DM whose key names a session that already exists" -- was a claim about the
store made without asking the store, and a DM top-level message and a
first-touch thread reply both wrote a row under it.

`_session_mint_prospect` and its tri-state are gone, and so are both
`isinstance(msg, StartTimeboxing)` guards. In their place, one question asked at
every door that hands a turn to timeboxing: is the catalog short, and does the
key this turn will use name nothing yet? It fails closed -- an unreadable store
answers the same as an empty one, because that is the condition that made the
catalog short in the first place.

Four doors ask it now, up from two-and-a-half: the session surface builder
(which mints the key it would ask about, so it refuses outright), the route's
own turn on both backends, the in-thread handoff fallback, and the redirect
branch, which nothing had guarded.

The harness was the reason none of this showed. It stubbed the kernel turn and
the runtime delivery with functions that wrote nothing, so the two doors that
mint without `open_session_surface` were invisible and both guards could be
replaced with `False` for a green suite. Both fakes now write the row the real
thing writes, and the new tests assert on rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… n=8

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rits

Records every place the referent-catalog interface strained to fit
timeboxing across Tasks 1-8, verified against the code as shipped at
77024eb rather than against the design doc's wording. Nine entries
judged general (locator addressing, gist's forced flatten-to-string
round trip, accepts duplicating the session state machine, day's
overloaded None, what catalog_complete required of the consumer
across Task 7's four fix rounds, the resolver's blind spot on
non-text creating doors, the shared observability key, the AST
guard's package-scoped blind spot, and as_of as a filtering horizon
rather than a point-in-time read) plus a named list of what was
judged Slack-routing or eval-fixture noise rather than interface
strain, so the boundary is stated rather than left implicit.

Also raises #443 against #345 per CLAUDE.md's
docs-ticket rule: docs/architecture/ needs the provider contract,
and src/fateforger/referents/AGENTS.md does not exist yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…stops minting (#345)

The final wave before this branch is offered for merge. Eight findings, one
pass.

Two blockers in the rung. The line naming which referent was chosen was written
into the origin message and then overwritten within the same turn by
`_origin_link_to_thread`'s `chat_update` -- so it survived in a DM and was
erased in #plan-sessions, the incident's own channel. Since the rung delivers
rather than waiting for a press, that line is the whole remaining protection
against a confidently wrong referent; it is now carried into the final message.
And `_resolve_referent` returned `None` on every failure, so a model outage over
a catalog that plainly contained the standing day fell through to the minting
door -- the incident, with a test asserting it. It now separates "declined to
ask" (no resolver, no store: today's behaviour exactly) from "asked and failed
over a non-empty catalog", which is treated as a partial catalog and refuses.

The resolver eval named the wrong instrument. Task 8 measured that the rate does
not discriminate a known prompt regression (87-90% with the load-bearing
sentence deleted); the probe-unanimity gate is what catches it, and it appeared
only as an inline comment. The docstring now says so and the rate assertion
reads as the collapse floor it is. The eval was not run.

`_plan_gist` re-parsed the rendered table with the rows sitting beside it in the
same payload -- the one thing `schedule_render.py` rules out and both
`candidate_display_text` and `required_blocks.slugs_on_candidate` already
follow. It reads `payload["rows"]` now; the table stays as a documented pre-rows
fallback, reading its columns by name out of `tmbx.core.render.COLUMNS` instead
of at 3/4/5, and the fixtures move to the real 11-column header.

The rung's `none` could not undo the rung's own redirect: a DM's origin key is
the stable `{channel}:dm`, so the redirect outlived the judgement that set it
and silently carried the next message to the old session. It is cleared on
`NoReferent`, and the design's out-of-scope line now describes what the code
does.

Also: an aware `updated_at` keeps its offset instead of being overwritten, the
referents package points at the interface-strain note, and the operator section
of docs/setup/development.md says why a session-store outage refuses every
timeboxing turn including the ones that should continue.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DECISION: standing things are offered options — a referent catalog in front of agent selection

2 participants