From 2b15bbae2ece0471ae7ed32131012becd3de1301 Mon Sep 17 00:00:00 2001 From: Nigel Jones Date: Tue, 11 Aug 2026 17:33:27 +0100 Subject: [PATCH 1/6] docs: migrate unique content from docs/dev/ before deletion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three pieces of useful content from docs/dev/ notes that have no equivalent in the published docs or docstrings: - mellea/core/backend.py: replace stale docs/dev/ reference in generate_from_context and _generate_from_context docstrings with the actual rationale — action is passed separately from ctx so shared context is referentially equal across calls, avoiding deep-copies in rejection sampling and parallel requirement checks. - docs/docs/advanced/lora-and-alora-adapters.md: add 'How automatic routing works' section documenting the three exceptions to aLoRA routing (flag, LLMaJRequirement subtype, adapter exception) and the ALoraRequirement escape hatch. Previously only the flag was mentioned with no explanation of when or why it applies. - docs/docs/concepts/plugins.mdx: add note after component_post_error explaining why component_pre_create/component_post_create are not implemented (Component is a Protocol, not an ABC) and pointing to component_pre_execute as the alternative. These changes are preparation for removing docs/dev/ entirely in a follow-up PR (ref #1482, #1483). Assisted-by: IBM Bob Signed-off-by: Nigel Jones --- docs/docs/advanced/lora-and-alora-adapters.md | 19 +++++++++++++++++++ docs/docs/concepts/plugins.mdx | 7 +++++++ mellea/core/backend.py | 14 ++++++++++---- 3 files changed, 36 insertions(+), 4 deletions(-) diff --git a/docs/docs/advanced/lora-and-alora-adapters.md b/docs/docs/advanced/lora-and-alora-adapters.md index da5594530..e1fb05248 100644 --- a/docs/docs/advanced/lora-and-alora-adapters.md +++ b/docs/docs/advanced/lora-and-alora-adapters.md @@ -149,6 +149,25 @@ When `backend.add_adapter()` is called, Mellea automatically routes requirement validation through the adapter for any `req()` calls on that session. The adapter runs at the `check_requirement` prompt position — fast, with minimal context overhead. +## How automatic routing works + +When an adapter is loaded via `backend.add_adapter()`, Mellea automatically routes +`req()` validation calls through it rather than falling back to LLM-as-a-judge. The +rule is: use the most specific available method. In practice this means the aLoRA +adapter is preferred whenever one is loaded, with three exceptions: + +1. `backend.default_to_constraint_checking_alora` is set to `False` — the adapter + is loaded but routing is suppressed for the entire backend instance. +2. The requirement uses the `LLMaJRequirement` subtype explicitly — the caller is + asking for LLM-as-a-judge regardless of what adapters are loaded. +3. The adapter throws an exception — Mellea falls back to LLM-as-a-judge + automatically. + +If you want to force the adapter path even when using `generate_from_context` +directly (bypassing the normal `validate()` call), use `ALoraRequirement` from +`mellea.stdlib.requirements` — routing through the adapter is then guaranteed +regardless of `default_to_constraint_checking_alora`. + ## Disable adapter validation To run without adapter validation (for benchmarking or debugging): diff --git a/docs/docs/concepts/plugins.mdx b/docs/docs/concepts/plugins.mdx index 85c6bd882..3bf9e7ae8 100644 --- a/docs/docs/concepts/plugins.mdx +++ b/docs/docs/concepts/plugins.mdx @@ -510,6 +510,13 @@ async def log_latency(payload, ctx): - Error logging and alerting - Failure analysis +:::note +**Creation hooks** (`component_pre_create` / `component_post_create`) are not +implemented. `Component` is a `Protocol`, not an abstract base class, so there is no +single interception point that covers all component implementations. Use +`component_pre_execute` for pre-execution policy enforcement instead. +::: + --- ### Generation pipeline diff --git a/mellea/core/backend.py b/mellea/core/backend.py index adc911496..c06e3cd0a 100644 --- a/mellea/core/backend.py +++ b/mellea/core/backend.py @@ -75,8 +75,13 @@ async def generate_from_context( """Generates a model output from a context. May not mutate the context. This must be called from a running event loop as it creates a task to run the generation request. Args: - action: The last item of the context should be passed in as an `action` instead of as part of the `ctx`. See `docs/dev/generate_signature_decisions.md`. - ctx: The rest of the context. + action: The component to generate from. Passed separately from `ctx` rather + than appended to it so that shared context is referentially equal across + calls — no deep copy is needed when multiple requests (e.g. rejection + sampling iterations, parallel requirement checks) run over the same + context. This also lets the backend see the exact generation target + without having to extract it from the context tail. + ctx: The rest of the context, excluding `action`. format: A response format to used for structured outputs / constrained decoding. model_options: Any model options to upsert into the defaults for this call. tool_calls: If `True`, then tool calls are extracts from the `action` `Component`. Assumption: if tool_calls is enabled, then the action `Component` has a TemplateRepresentation @@ -143,8 +148,9 @@ async def _generate_from_context( """Backend implementers should override this method to generate the actual response. Args: - action: The last item of the context should be passed in as an `action` instead of as part of the `ctx`. See `docs/dev/generate_signature_decisions.md`. - ctx: The rest of the context. + action: The component to generate from. See `generate_from_context` for the + rationale behind the action/context split. + ctx: The rest of the context, excluding `action`. format: A response format to used for structured outputs / constrained decoding. model_options: Any model options to upsert into the defaults for this call. tool_calls: If `True`, then tool calls are extracts from the `action` `Component`. Assumption: if tool_calls is enabled, then the action `Component` has a TemplateRepresentation From cec7b268210b8d8d02932f7e4a25b1412bcd32cd Mon Sep 17 00:00:00 2001 From: Nigel Jones Date: Tue, 11 Aug 2026 18:42:17 +0100 Subject: [PATCH 2/6] docs: close gaps left by the docs/dev/ migration prep MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ec304499 migrated three pieces of confirmed-current content out of docs/dev/ ahead of its planned deletion (#1482, #1483), but missed others and left dangling references that the deletion would break: - mellea/core/backend.py, mellea/plugins/hooks/component.py: add two more pieces of still-current rationale that had no equivalent in code — the open architectural risk in the action/ctx split once span-based backends land, and why component_post_success/_error are separate hooks rather than one success/failure union. - mellea/backends/huggingface.py, mellea/telemetry/metrics.py, mellea/stdlib/sampling/sampling_algos/budget_forcing_alg.py: repoint or drop three in-code comments that literally said "See docs/dev/....md". One pointed at a filename that never existed (generate_signature_decisions.md vs the real generate_ctx_signature.md); one cited a doc for content the doc never actually contained. Both predate this change but would have become silently broken once docs/dev/ is gone. - docs/examples/*/README.md: drop seven "Related Documentation" bullets linking to docs/dev/ files that are about to disappear. All seven point at files the prior stale-marking audit (82704ae2) had already flagged, so there's no live replacement to link to instead. Assisted-by: Claude Code Signed-off-by: Nigel Jones --- docs/examples/agents/README.md | 1 - docs/examples/context/README.md | 1 - docs/examples/generative_stubs/README.md | 3 +-- docs/examples/instruct_validate_repair/README.md | 3 +-- docs/examples/intrinsics/README.md | 1 - docs/examples/melp/README.md | 3 +-- docs/examples/mify/README.md | 1 - mellea/backends/huggingface.py | 4 +++- mellea/core/backend.py | 6 +++++- mellea/plugins/hooks/component.py | 8 ++++++++ .../stdlib/sampling/sampling_algos/budget_forcing_alg.py | 3 ++- mellea/telemetry/metrics.py | 3 +-- 12 files changed, 22 insertions(+), 15 deletions(-) diff --git a/docs/examples/agents/README.md b/docs/examples/agents/README.md index 869540991..6fb7e43ef 100644 --- a/docs/examples/agents/README.md +++ b/docs/examples/agents/README.md @@ -36,5 +36,4 @@ An alternative implementation of the ReACT pattern using Mellea's instruct-valid ## Related Documentation -- See `docs/dev/tool_calling.md` for more on tool integration - See `mellea/stdlib/requirements/tool_reqs.py` for tool requirements \ No newline at end of file diff --git a/docs/examples/context/README.md b/docs/examples/context/README.md index e7b8b3752..2cb44619f 100644 --- a/docs/examples/context/README.md +++ b/docs/examples/context/README.md @@ -95,4 +95,3 @@ ctx = WindowCompactor(size=0).compact(ctx) # drop body, kee - See `mellea/stdlib/context/` for context and compactor implementations - See `mellea/stdlib/sampling/` for sampling strategies - See `mellea/stdlib/frameworks/react.py` for the ReACT loop -- See `docs/dev/spans.md` for context architecture details diff --git a/docs/examples/generative_stubs/README.md b/docs/examples/generative_stubs/README.md index ae72b8483..bf9d2947b 100644 --- a/docs/examples/generative_stubs/README.md +++ b/docs/examples/generative_stubs/README.md @@ -57,5 +57,4 @@ with start_session() as m: ## Related Documentation -- See `mellea/stdlib/components/genstub.py` for implementation -- See `docs/dev/mellea_library.md` for design philosophy \ No newline at end of file +- See `mellea/stdlib/components/genstub.py` for implementation \ No newline at end of file diff --git a/docs/examples/instruct_validate_repair/README.md b/docs/examples/instruct_validate_repair/README.md index 013654f19..49de71465 100644 --- a/docs/examples/instruct_validate_repair/README.md +++ b/docs/examples/instruct_validate_repair/README.md @@ -169,5 +169,4 @@ result = m.instruct( ## Related Documentation - See `mellea/stdlib/requirements/` for requirement types -- See `mellea/stdlib/sampling/` for sampling strategies -- See `docs/dev/mellea_library.md` for design philosophy \ No newline at end of file +- See `mellea/stdlib/sampling/` for sampling strategies \ No newline at end of file diff --git a/docs/examples/intrinsics/README.md b/docs/examples/intrinsics/README.md index dddf197f7..07c0961e7 100644 --- a/docs/examples/intrinsics/README.md +++ b/docs/examples/intrinsics/README.md @@ -164,5 +164,4 @@ Full example showing multiple adapter functions working together in a RAG pipeli - See `mellea/stdlib/components/intrinsic/` for adapter function implementations - See `mellea/backends/adapters/` for adapter system -- See `docs/dev/intrinsics_and_adapters.md` for architecture details - See `docs/docs/examples/granite-switch/README.md` for more about granite-switch \ No newline at end of file diff --git a/docs/examples/melp/README.md b/docs/examples/melp/README.md index 7610b7994..14fe0b2f8 100644 --- a/docs/examples/melp/README.md +++ b/docs/examples/melp/README.md @@ -54,5 +54,4 @@ actual_result = force(composed) ## Related Documentation -- See `mellea/stdlib/functional.py` for functional programming primitives -- See `docs/dev/mellea_library.md` for design philosophy \ No newline at end of file +- See `mellea/stdlib/functional.py` for functional programming primitives \ No newline at end of file diff --git a/docs/examples/mify/README.md b/docs/examples/mify/README.md index 0c3f6b06b..1e346ae69 100644 --- a/docs/examples/mify/README.md +++ b/docs/examples/mify/README.md @@ -93,5 +93,4 @@ Objects decorated with `@mify` implement the `MifiedProtocol`, which provides: ## Related Documentation - See `mellea/stdlib/components/mify.py` for implementation -- See `docs/dev/mify.md` for design details - See `mellea/templates/` for template system \ No newline at end of file diff --git a/mellea/backends/huggingface.py b/mellea/backends/huggingface.py index b44af1814..f6a31c138 100644 --- a/mellea/backends/huggingface.py +++ b/mellea/backends/huggingface.py @@ -499,7 +499,9 @@ async def _generate_from_context( # Requirements can be automatically rerouted to a requirement adapter. if isinstance(action, Requirement): - # See docs/dev/requirement_aLoRA_rerouting.md + # See "How automatic routing works" in + # docs/docs/advanced/lora-and-alora-adapters.md for the three + # exceptions to this rule. reroute_to_alora = self.default_to_constraint_checking_alora adapter_name = "requirement-check" diff --git a/mellea/core/backend.py b/mellea/core/backend.py index c06e3cd0a..7cbc2383b 100644 --- a/mellea/core/backend.py +++ b/mellea/core/backend.py @@ -80,7 +80,11 @@ async def generate_from_context( calls — no deep copy is needed when multiple requests (e.g. rejection sampling iterations, parallel requirement checks) run over the same context. This also lets the backend see the exact generation target - without having to extract it from the context tail. + without having to extract it from the context tail. This head/tail split + is specific to the current linear-context stdlib patterns; it does not + generalize to a poset of possible generation points, which span-based + backends will need to express, so the signature may need to change again + once that work lands. ctx: The rest of the context, excluding `action`. format: A response format to used for structured outputs / constrained decoding. model_options: Any model options to upsert into the defaults for this call. diff --git a/mellea/plugins/hooks/component.py b/mellea/plugins/hooks/component.py index b49b41d91..8373b5928 100644 --- a/mellea/plugins/hooks/component.py +++ b/mellea/plugins/hooks/component.py @@ -39,6 +39,14 @@ class ComponentPreExecutePayload(MelleaBasePayload): class ComponentPostSuccessPayload(MelleaBasePayload): """Payload for `component_post_success` — after successful component execution. + Split from `component_post_error` rather than one `component_post` hook + with a success/failure union: the two carry disjoint fields (`result` and + `sampling_results` here vs. `error` and `stack_trace` on the error + payload), and plugins commonly care about only one outcome (a metrics + collector doesn't subscribe to errors; an audit logger may only want + them). A single hook would force nullable fields on both sides for no + registration benefit. + Attributes: action_id: UUID correlating pre/post hooks for a single component execution. component_type: Class name of the executed component. diff --git a/mellea/stdlib/sampling/sampling_algos/budget_forcing_alg.py b/mellea/stdlib/sampling/sampling_algos/budget_forcing_alg.py index e7e34bc29..83eec143d 100644 --- a/mellea/stdlib/sampling/sampling_algos/budget_forcing_alg.py +++ b/mellea/stdlib/sampling/sampling_algos/budget_forcing_alg.py @@ -49,7 +49,8 @@ async def think_budget_forcing( Args: backend: OllamaModelBackend instance to use for generation. action: The last item of the context, passed as an `action` instead of as part - of the `ctx`. See `docs/dev/generate_signature_decisions.md`. + of the `ctx`. See `Backend.generate_from_context` for the rationale + behind the action/context split. ctx: The current conversation context. format: Optional Pydantic model for constrained decoding of the response. tool_calls: If `True`, tool calling is enabled. diff --git a/mellea/telemetry/metrics.py b/mellea/telemetry/metrics.py index f23772b69..f0a3910c8 100644 --- a/mellea/telemetry/metrics.py +++ b/mellea/telemetry/metrics.py @@ -1004,8 +1004,7 @@ def _get_adapter_function_invocations_counter() -> Any: # in the codebase, pre-existing, already-shipped `Intrinsic*` symbols # (the `Intrinsic` component, `call_intrinsic`, etc.) still use the old # name and are renamed in a later, coordinated phase of Epic #929 (#1136) - # rather than here. See docs/dev/adapter_observability.md for the full - # rationale. (Applies to all three metrics below.) + # rather than here. (Applies to all three metrics below.) _adapter_function_invocations_counter = create_counter( "mellea.adapter_function.invocations", description="Total number of adapter function invocations", From b2b922ff40f1032012acd107446f9fd24df07a3e Mon Sep 17 00:00:00 2001 From: Nigel Jones Date: Tue, 11 Aug 2026 18:54:48 +0100 Subject: [PATCH 3/6] docs: delete docs/dev/ now that its current content is migrated MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ref #1482. A maintainer questioned keeping design notes outside the published docs at all — this replaces the mark-as-stale approach (#1483) with removing the directory entirely, now that the two prior commits have folded its still-current content into code docstrings, the published lora-and-alora-adapters/plugins docs, and (for the bare-label-vs-mellea.*-attribute convention) the mellea-telemetry skill. The other six files (constrained_decoding.md, intrinsics_and_adapters.md, mellea_library.md, mify.md, spans.md, tool_calling.md) were already confirmed stale, unfinished, or unverified against current code by the #1483 audit, so nothing further needed extracting from them before deletion. Assisted-by: Claude Code Signed-off-by: Nigel Jones --- docs/dev/adapter_observability.md | 135 ----------------------- docs/dev/constrained_decoding.md | 23 ---- docs/dev/generate_ctx_signature.md | 16 --- docs/dev/hook_system.md | 139 ------------------------ docs/dev/intrinsics_and_adapters.md | 38 ------- docs/dev/mellea_library.md | 15 --- docs/dev/mify.md | 73 ------------- docs/dev/requirement_aLoRA_rerouting.md | 90 --------------- docs/dev/spans.md | 20 ---- docs/dev/tool_calling.md | 73 ------------- 10 files changed, 622 deletions(-) delete mode 100644 docs/dev/adapter_observability.md delete mode 100644 docs/dev/constrained_decoding.md delete mode 100644 docs/dev/generate_ctx_signature.md delete mode 100644 docs/dev/hook_system.md delete mode 100644 docs/dev/intrinsics_and_adapters.md delete mode 100644 docs/dev/mellea_library.md delete mode 100644 docs/dev/mify.md delete mode 100644 docs/dev/requirement_aLoRA_rerouting.md delete mode 100644 docs/dev/spans.md delete mode 100644 docs/dev/tool_calling.md diff --git a/docs/dev/adapter_observability.md b/docs/dev/adapter_observability.md deleted file mode 100644 index 973b49de0..000000000 --- a/docs/dev/adapter_observability.md +++ /dev/null @@ -1,135 +0,0 @@ -# Adapter function lifecycle, options, and observability - -Epic #929 Phase 2, issue #1140. Covers three things landed together in the -same PR: the narrowed `AdapterMixin` verb contract, the shared -`resolve_model_options` helper, and the `AdapterFunctionMetricsPlugin` skeleton. - -## AdapterMixin verb contract - -`AdapterMixin` (`mellea/backends/adapters/adapter.py`) exposes **seven** -verbs, not the four stated in #1140's acceptance criteria. That's a direct -conflict with the issue text as written: Phase 1 (PR #1269) already added -`resolve_adapter()`, which depends on `base_model_name` and `add_adapter` -staying on the mixin, so trimming to four verbs isn't possible without -breaking Phase 1. The count below reflects what actually ships. - -### Universal (every backend implements these) - -- `base_model_name` — the underlying model's identifier. Read directly by - `resolve_adapter()` to construct new adapters lazily. -- `add_adapter(adapter)` — registers an adapter with the backend. - `resolve_adapter()` calls this internally the first time an adapter name - is resolved. -- `list_adapters()` — returns every adapter the backend *knows about*, - whether or not it's currently active. Both `LocalHFBackend` and - `OpenAIBackend` now share this "registered/known" contract: - `LocalHFBackend.list_adapters()` reads `self._added_adapters` (previously - it read `self._loaded_adapters`, which only included adapters that had - been explicitly loaded — that mismatch with `OpenAIBackend`'s semantics is - fixed as part of this issue). - -### Reality-specific (each backend overrides only its own) - -Each of the following raises `NotImplementedError` on the mixin by default; -a backend overrides only the verb matching its own adapter reality. - -- `load_peft_adapter(name)` / `unload_peft_adapter(name)` — LocalFile/PEFT - reality (`LocalHFBackend`). Loads or unloads LoRA/aLoRA weights from disk. - Renamed from the previous `load_adapter`/`unload_adapter`. -- `render_controls(name, active: bool)` — Embedded/Granite Switch reality - (`OpenAIBackend`). Weights are already baked into the served model, so - there's nothing to load or unload; this verb exists for future - control-token rendering. `active=True`/`False` map to the intended - `activate()`/`deactivate()` calls once #1142 wires EmbeddedBinding. -- `set_request_adapter(name)` — ServerMediated reality. No backend - implements this yet; the verb name is defined for when that reality is - built. - -`resolve_adapter()` and `adapter_scope()` are unchanged Phase 1 scaffolding -and out of scope for this issue — their real wiring into -`WeightsBinding.activate()`/`deactivate()` belongs to #1141/#1142. - -## resolve_model_options - -`mellea/backends/_options.py` centralizes the model-options merge logic that -`LocalHFBackend._simplify_and_merge` and `OpenAIBackend._simplify_and_merge` -each used to duplicate. Precedence, lowest to highest: - -```text -backend_defaults < helper_defaults < call_options -``` - -`remap` translates backend/caller-specific option names to `ModelOption` -keys before merging; `helper_defaults` is assumed to already be in -`ModelOption` key form. `call_intrinsic` (`mellea/stdlib/components/intrinsic/_util.py`) -also routes through this helper for its `TEMPERATURE: 0.0` default, so -caller-supplied `model_options` can't be silently clobbered by a hardcoded -default — the same class of bug PR #972 fixed elsewhere. - -## AdapterFunctionMetricsPlugin (skeleton) - -`mellea/telemetry/metrics_plugins.py` adds `AdapterFunctionMetricsPlugin`, hooking -`adapter_function_invocation_complete` and `adapter_function_phase_complete` -(`mellea/plugins/hooks/adapter_function.py`). Three metrics: - -- `mellea.adapter_function.invocations` (counter) — labels: `name`, `revision`, - `binding_type`, `adapter_type`, `outcome` (`success` | `schema_error` | - `error`). -- `mellea.adapter_function.phase_duration` (histogram, unit `s`) — - labels: `name`, `phase` (`prepare` | `activate` | `generate` | `parse` | - `deactivate`). -- `mellea.adapter_function.parse_failures` (counter) — labels: `name`, `revision`. - Incremented automatically whenever an invocation's `outcome` is - `schema_error` (i.e. an `AdapterSchemaMismatchError`), acting as a - schema-drift detector. - -No production code fires these hooks yet — this is a skeleton, unit-tested -against synthetic payloads only (`test/telemetry/test_metrics_plugins.py`). -Real wiring from `prepare`/`activate`/`generate`/`parse`/`deactivate` is -expected to go in with #1141 (LocalFileBinding) and #1142 (EmbeddedBinding). - -## Span tree (structure) - -Span *emission* ships with the Bindings (#1141/#1142) — no span code lands in -this PR. What this issue fixes is the *shape*, so the traces align with the -metrics and follow Mellea's existing tracing conventions rather than a bespoke -scheme. Spans are opened through the `start_*_span` helper family in -`mellea/telemetry/tracing.py` (mirroring `start_backend_span` / -`start_action_span`): the span is named by its operation with `gen_ai.*` set -where the semantic conventions apply, and Mellea-specific fields are attached -under the `mellea.*` prefix — the same convention as `mellea.action_type`, -`mellea.num_actions`, etc. - -An invocation opens one parent span with a child span per lifecycle phase: - -- **Parent** (the invocation) — carries `mellea.adapter_function.name`, - `mellea.adapter_function.revision`, `mellea.adapter_function.binding_type`, - `mellea.adapter_function.adapter_type`, and - `mellea.adapter_function.outcome`, mirroring the - `mellea.adapter_function.invocations` counter. -- **Children** (one per phase: `prepare`, `activate`, `generate`, `parse`, - `deactivate`) — each carries `mellea.adapter_function.phase` and - corresponds one-to-one with a `mellea.adapter_function.phase_duration` - histogram sample of the same phase. - -Note the deliberate split, consistent with the rest of Mellea: **metric labels -are bare** (`name`, `phase`, `revision`, …) while **span attributes are -`mellea.*`-prefixed** — same values, different surface, each following its -signal type's existing convention. - -## Content capture (`MELLEA_TRACES_CONTENT`) - -Span *metadata* — names, revisions, phase durations, outcomes — is always safe -to record. Adapter *input and output content* — prompts, retrieved documents, -generated text — is gated behind the **existing** `MELLEA_TRACES_CONTENT` -environment variable: the same content-capture gate Mellea's other spans -already use (it also honours `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`), -**off by default**, so traces never capture PII or proprietary content unless -explicitly opted in. When unset or falsey, the phase spans carry metadata only; -when set truthy, they additionally attach the adapter's input/output content. -The adapter-function spans **reuse this gate rather than introducing a new one**; -content attributes are attached when the Bindings (#1141/#1142) emit spans. - -(#1140's acceptance criteria named this `MELLEA_TRACE_CONTENT`; the real, -already-implemented variable is `MELLEA_TRACES_CONTENT` — see -`mellea/telemetry/tracing.py`.) diff --git a/docs/dev/constrained_decoding.md b/docs/dev/constrained_decoding.md deleted file mode 100644 index 4318443ef..000000000 --- a/docs/dev/constrained_decoding.md +++ /dev/null @@ -1,23 +0,0 @@ -# Constrained Decoding - -## How do constraints get defined? - -Should we be thinking bigger than pydantic? Should it be possible to pass arbitrary grammars? If so, what's the abstract interface for those? Should this be factored out into llm-io? - -## How do constraints get passed around? - -The `m` framework currently uses the `format` argument to pydantic schemas, **outside of model args**. Should we be using `@@@format@@@` within ModelArgs instead? Hendrik describes the behavior of model args like this (paraghased by Nathan): - -> If a keyword had meaning across multiple types of backends, and if it means the same thing in all of those backends but has different names, then we use the `@@@`-style args so that the user can pass these args across all backends in the same way. Otherwise, the arguments in model_args are passed along verbatim. - -This argues for `@@@format@@@` as opposed to a dedicated `format` option in the method signature. Or, in the alternative, for an entire re-think of ModelArgs. - -## Integration with grammar-targeted LLMs - -Some LLMs target generation in a particular grammar. Examples include: - * ALoRAs that target very simple grammars - * code generatorrs that target particular PLs - * models (or model modes) tuned to generate JSON - * models (or model modes) tuned to generate YAML or particular fragments of YAML (such as k8s configs) - -Should we be doing constrained decoding in these cases, or should we treat deviation from the grammar as an exception? Probably the answer is "it depends". Masataro had a nice idea of **taking the sum of logits of grammatically feasible completions** and ensuring that this sum is above some threshold. How would supporting this change the interface described in the "How do constraints get defined?" section? \ No newline at end of file diff --git a/docs/dev/generate_ctx_signature.md b/docs/dev/generate_ctx_signature.md deleted file mode 100644 index a5e58f064..000000000 --- a/docs/dev/generate_ctx_signature.md +++ /dev/null @@ -1,16 +0,0 @@ -# Splitting the `head` and `tail` of the Context on generate calls - -We have decided to split the context into an "action" and "the rest of the context"; i.e., instead of `generate : ctx, ... -> output`, we use `generate: action, ctx, ... -> output`. - - This "car/cdr" separation of the final element from the rest is done because there are many situations where many different requests are made over the same context. Examples include multiple requirement checking, rejection sampling, and so on. - -Advantages of this approach: - * shared context is referentially equal, which makes memory management extremely simple. - * Certain types of code -- especially requirement checking -- are much easier to write. Because the Context does not have to be deep-copied. - -Disadvantages of this approach: - * This solution is extremely specific to a few examples/patterns from stdlib. When we have `span`-based backends, there could be many different points in the span from which generation could continue. The solutino to that problem will sort of rhyme -- separating the generation target from th rest of the context.t However, the current signature is NOT a good solution. So it's possible we will have to change how this works in the fture. - * Not parsimonious with how context is normally used, and perhaps confusing, particularly in the most-common situation whwere the context is "just" a normal chat history. - * It is not yet clear what meaning this will have when contexts cannot be linearized. In particular: what if there's a poset and multiple generation opportunities within that poset? How do we "place the cursor"? Does this design choice make it harder to "place the cursor"? - * Contexts are not in fact immutable, so we have to be extremely careful about when a context gets modified, and may even need to introduce semaphores. - diff --git a/docs/dev/hook_system.md b/docs/dev/hook_system.md deleted file mode 100644 index 35bdae623..000000000 --- a/docs/dev/hook_system.md +++ /dev/null @@ -1,139 +0,0 @@ -# Mellea Plugin Hook System — Internal Design Notes - -> **User-facing documentation:** [Plugins & Hooks](../../docs/docs/concepts/plugins.mdx) covers usage, registration, execution modes, hook types reference, and patterns. This file retains only internal design rationale and decisions for contributors. - ---- - -## Design principles - -1. **Consistent interface**: All hooks follow the same async pattern with payload and context parameters -2. **Composable**: Multiple plugins can register for the same hook, executing in priority order -3. **Fail-safe**: Hook failures can be handled gracefully without breaking core execution -4. **Minimal intrusion**: Plugins are opt-in; default Mellea behavior remains unchanged without plugins. Plugins work identically whether invoked through a session (`m.instruct(...)`) or via the functional API (`instruct(backend, context, ...)`) -5. **Architecturally aligned**: Hook categories reflect Mellea's true abstraction boundaries — Session lifecycle, Component lifecycle, and the (Backend, Context) generation pipeline -6. **Code-first**: Plugins are defined and composed in Python. The `@hook` decorator and `Plugin` base class are the primary registration mechanisms; YAML configuration is a secondary option for deployment-time overrides -7. **Functions-first**: The simplest plugin is a plain async function decorated with `@hook`. Class-based plugins (via the `Plugin` base class) exist for stateful, multi-hook scenarios but are not required - ---- - -## Concurrency model - -Hooks use Python's `async`/`await` cooperative multitasking. Because Python's event loop only switches execution at `await` points, hook code won't be interrupted mid-logic. This means: - -- **Sequential when awaited**: Calling `await hook(...)` keeps control flow deterministic — the hook completes before the caller continues. -- **Race conditions only at `await` points**: Shared state is safe to read and write between `await` calls within a single hook. Races only arise if multiple hooks modify the same shared state and are dispatched concurrently. -- **No preemptive interruption**: Unlike threads, a hook handler runs uninterrupted until it yields control via `await`. - ---- - -## Hook invocation responsibilities - -Hooks are called from Mellea's base classes (`Component.aact()`, `Backend.generate()`, `SamplingStrategy.run()`, etc.). This means hook invocation is a framework-level concern, and authors of new backends, sampling strategies, or components do not need to manually insert hook calls. - -The caller (the base class method) is responsible for both invoking the hook and processing the result. Processing means checking the result for one of three possible outcomes: - -1. **Continue with original payload** — `PluginResult(continue_processing=True)` with no `modified_payload`. The caller proceeds unchanged. -2. **Continue with modified payload** — `PluginResult(continue_processing=True, modified_payload=...)`. The plugin manager applies the hook's payload policy, accepting only changes to writable fields and discarding unauthorized modifications. The caller uses the policy-filtered payload in place of the original. -3. **Block execution** — `PluginResult(continue_processing=False, violation=...)`. The caller raises or returns early with structured error information. - -Hooks cannot redirect control flow, jump to arbitrary code, or alter the calling method's logic beyond these outcomes. This is enforced by the `PluginResult` type. - ---- - -## Payload design principles - -1. **Strongly typed** — Each hook has a dedicated payload dataclass (not a generic dict). This enables IDE autocompletion, static analysis, and clear documentation of what each hook receives. -2. **Sufficient (maximize-at-boundary)** — Each payload includes everything available at that point in time. Post-hooks include the pre-hook fields plus results. This avoids forcing plugins to maintain their own state across pre/post pairs. -3. **Frozen (immutable)** — Payloads are frozen Pydantic models (`model_config = ConfigDict(frozen=True)`). Plugins cannot mutate payload attributes in place. To propose changes, plugins must call `payload.model_copy(update={...})` and return the copy via `PluginResult.modified_payload`. This ensures every modification is explicit and flows through the policy system. -4. **Policy-controlled** — Each hook type declares a `HookPayloadPolicy` specifying which fields are writable. The plugin manager applies the policy after each plugin returns, accepting only changes to writable fields and silently discarding unauthorized modifications. This separates "what the plugin can observe" from "what the plugin can change" — and enforces it at the framework level. -5. **Serializable** — Payloads should be serializable for external (MCP-based) plugins that run out-of-process. All payload fields use types that can round-trip through JSON or similar formats. -6. **Versioned** — Payload schemas carry a `payload_version` so plugins can detect incompatible changes at registration time rather than at runtime. -7. **Isolation** — Each plugin receives a copy-on-write (CoW) snapshot of the payload. Mutable containers (dicts, lists) are wrapped so mutations in one plugin do not affect others. Plugins should not cache payloads beyond the hook invocation — payload fields reference live framework objects (`Context`, `Component`, `MelleaSession`) whose lifecycle is managed by the framework. - ---- - -## GlobalContext design (ambient metadata) - -The `GlobalContext` passed to hooks carries lightweight, cross-cutting ambient metadata that is useful to every hook regardless of type. Hook-specific data (context, session, action, etc.) belongs on the **typed payload**, not on the global context. - -### What goes in GlobalContext - -```python -# GlobalContext.state — same for all hook types -backend_name: str # Derived from backend.model_id (when backend is passed) -``` - -The `backend_name` is a lightweight string extracted from `backend.model_id`. The full `backend` and `session` objects are **not** stored in GlobalContext — this avoids giving plugins unchecked mutable access to core framework objects. - -### Design rationale - -Previously, `context`, `session`, and `backend` were passed both on payloads and in `GlobalContext.state`, creating duplication. The same mutable object accessible via two paths was a footgun — plugins could be confused about which to read/modify. The refactored design: - -1. **Payloads** are the primary API surface — typed, documented, policy-controlled -2. **GlobalContext** holds only truly ambient metadata (`backend_name`) that doesn't belong on any specific payload -3. No mutable framework objects (`Backend`, `MelleaSession`, `Context`) are stored in GlobalContext - ---- - -## Design decision: separate success/error hooks - -`component_post_success` and `component_post_error` are separate hooks rather than a single `component_post` with a sum type over success/failure. The reasons are: - -1. **Registration granularity** — Plugins subscribe to only what they need. An audit logger may only care about errors; a metrics collector may only care about successes. -2. **Distinct payload shapes** — Success payloads carry `result`, `generate_log`, and `sampling_results`; error payloads carry `exception`, `error_type`, and `stack_trace`. A sum type would force nullable fields or tagged unions, adding complexity for every consumer. -3. **Different execution modes** — Error hooks may be fire-and-forget (for alerting); success hooks may be blocking (for output transformation). Separate hooks allow per-hook execution timing configuration. - ---- - -## Design decision: component_pre_create / component_post_create deferral - -`component_pre_create` and `component_post_create` are not implemented. `Component` is currently a `Protocol`, not an abstract base class. This means Mellea has no ownership over component initialization: there are no guarantees about when or how subclass `__init__` methods run, and there is no single interception point that covers all `Component` implementations. - -Placing hook calls inside `Instruction.__init__` and `Message.__init__` works for those specific classes, but it is fragile (any user-defined `Component` subclass is invisible to the hooks) and architecturally wrong (the hook system should not need to be threaded manually into every `__init__`). - -If `Component` were refactored to an abstract base class, Mellea could wrap `__init__` at the ABC level and fire these hooks generically for all subclasses. Until then, use `component_pre_execute` for pre-execution policy enforcement. - ---- - -## Unimplemented hooks - -The following hooks are designed but not yet implemented. They are included in the design for completeness and may be implemented as demand arises. - -| Hook Point | Category | Notes | -| --- | --- | --- | -| `component_pre_create` | Component Lifecycle | Blocked on Component-as-ABC refactoring (see above) | -| `component_post_create` | Component Lifecycle | Blocked on Component-as-ABC refactoring (see above) | -| `generation_stream_chunk` | Generation Pipeline | Per-chunk interception during streaming | -| `adapter_pre_load` | Backend Adapter Ops | Before `backend.load_peft_adapter()` | -| `adapter_post_load` | Backend Adapter Ops | After adapter loaded | -| `adapter_pre_unload` | Backend Adapter Ops | Before `backend.unload_peft_adapter()` | -| `adapter_post_unload` | Backend Adapter Ops | After adapter unloaded | -| `context_update` | Context Operations | When context changes (append/reset) | -| `context_prune` | Context Operations | When context is trimmed for token budget | -| `error_occurred` | Error Handling | Cross-cutting hook for unrecoverable errors | - ---- - -## Scoping implementation - -A single `PluginManager` instance manages all plugins. Plugins are tagged with an optional `session_id`. At dispatch time, the manager filters: global plugins (no session tag) always run; session-tagged plugins run only when the dispatch context matches their session ID. - -With-block scopes use the same `session_id` tagging mechanism. Each `with` block gets a unique UUID scope ID; the plugin manager filters plugins by scope ID at dispatch time and deregisters them by scope ID on exit. - ---- - -## YAML configuration (secondary) - -For deployment-time configuration, plugins can be loaded from YAML. This is useful for enabling/disabling plugins or changing priorities without code changes. The `disabled` mode (`PluginMode.DISABLED`) is available in YAML configuration for deployment-time control but is not exposed in Mellea's public `PluginMode` enum. - ---- - -## Custom hook types - -The plugin framework supports custom hook types for domain-specific extension points beyond the built-in lifecycle hooks. This is particularly relevant for agentic patterns (ReAct, tool-use loops, etc.) where the execution flow is application-defined. Custom hooks use the same `@hook` decorator and follow the same calling convention, payload chaining, and result semantics. As agentic patterns stabilize in Mellea, frequently-used custom hooks may be promoted to built-in hooks. - ---- - -## Functional API support - -The functional API (`instruct(backend, context, ...)`) does not require a session. Hooks still fire at the same execution points. If global plugins are registered, they execute. If no plugins are registered, hooks are no-ops with zero overhead. Session-scoped plugins do not apply because there is no session. diff --git a/docs/dev/intrinsics_and_adapters.md b/docs/dev/intrinsics_and_adapters.md deleted file mode 100644 index 3d1375921..000000000 --- a/docs/dev/intrinsics_and_adapters.md +++ /dev/null @@ -1,38 +0,0 @@ -# Intrinsics and Adapters -Note: Mellea currently only supports IntrinsicAdapters and Intrinsics. - -## Basics -In Mellea, intrinsics are a type of Component that signals one or more of the following to a backend: -- a special adapter must be used for generation -- the input/output for generation must be transformed in a particular way -- the model options must be modified in a particular way - -These changes only happen when the intrinsic is the "action" of the request. Intrinsics should usually not be used as an item in the context of generation (in fact, by default, Intrinsics have no string representation). - -These changes are specified by the Adapter that corresponds to a given Intrinsic. Matching happens based on the adapter name and type. - -## Parts of an Intrinsic -Intrinsics specify: -- an adapter name (ie requirement-check) -- types of adapters suitable to be used (ie alora) -- any kwargs necessary (ie a requirement like "make sure the last user message is...") - -## Parts of an Adapter -Adapters specify: -- compatible backends -- adapter type -- functions for getting a path to load them - -## Using Intrinsics -Mellea Intrinsics currently use the routines under `mellea.formatters.granite` for loading adapters and formatting input/outputs. This means Mellea only allows intrinsics/adapters that follow this pattern. - -## Needed Future Work -### Custom Adapters / Intrinsics -Mellea should support custom intrinsic / adapter implementations. To do this: -- make backend `_generate_from_intrinsic` functions generic and utilize only common adapter functions -- adapters must specify a transformation function that encapsulates the input/output modifications necessary for their generation requests - -### Concurrency Checks -Some backends (currently only LocalHFBackend) that allow adapters to be loaded, cannot independently utilize these adapters without impacting other generation requests. - -These backends should support a generation lock that ensures requests are only performed when the correct set of adapters (or no adapters) are active. diff --git a/docs/dev/mellea_library.md b/docs/dev/mellea_library.md deleted file mode 100644 index 3baf14898..000000000 --- a/docs/dev/mellea_library.md +++ /dev/null @@ -1,15 +0,0 @@ -# Mellea should be as close to a library as possible - -We should make it possible to use mellea as a library (as opposed to a framework). - -In the context of LLM applications, the library vs framework distinction really boils down to how you treat the backend. - -If a piece of software insists on having an exclusive handle on the backend, then that piece of software does not compose with any other piece of software that also insists on an exclusive handle. They both want to be privileged with respect to the backend, so they cannot "play well" together. The `outlines` library is a good example of software that could've been a library but instead acts like a framework. Even `granite-io` takes on a framework-like role when it decides to actually call the backend, as opposed to operating over strings (or perhaps chat histories). - -Writing LLM libraries is kind of difficult. There is a very strong instinct to try to grab control of the backend. Mellea is no exception. In the "intro path", mellea definitely behaves like a framework. We hide the actual backend objects (`PretrainedModel`, `openai.Client`, etc.) from the user. - -But we should try to make it easy for certain parts of mellea to be used as a library. There are many ways in which we could allow mellea to compose with other libraries: - -1. We could have a `m.start_session_with_shared_backend(client:openai.Client)` and similarly for local ollama models and transformers models. Everything would work mostly the same after that, except we would have to make much weaker assumptions about the state of the backend (e.g., cache and LoRAs). -2. We could strive to keep the `Formatter` logic completely separate from Backend-specific code, and the legacy model behavior should treat each Component like a standalone user message. This way people could use `mellea` components without using the `mellea` backend and context management code. -3. We could strive to keep the `Cache` strategies agnostic to the rest of the code base, and figure out what their interface should be with respect to various backend sdks (and transformers in particular) diff --git a/docs/dev/mify.md b/docs/dev/mify.md deleted file mode 100644 index ab3af6e94..000000000 --- a/docs/dev/mify.md +++ /dev/null @@ -1,73 +0,0 @@ -# mify - -In classical programming, object-orientation provides a way to couple data and functionality. -Classes have fields and methods. Fields store data and methods operate over that data. - -The mellea library allows you to interface with objects in the same way, but with the added benefit that an LLM can perform operations for you. - -```python -import mellea - -m = mellea.start_session() - - -class Circle: - """A circle is defined by its center and a radius.""" - center_x: float - center_y: float - radius: float - - -c = Circle(1, 0, 1) - -mify(c) - -# .query is used to compute things. -circumference: float = m.query(c, "compute the circumference of the circle", - format=float) - -# .transform is used to create a new class of the same type but mutated. -flipped_circle = m.transform(c, "Mirror the circle across the y axis.") -``` - -Let's consider a slightly more complicated example. - -```python -class Customer: - customer_id: int - name: str - age: int - email_addr: str - employer: str - meeting_notes: List[str] - - def __init__(customer_id: int): - ... - - def send_email(subject: str, body: str): - ... - - def get_meeting_notes() -> List[str]: - ... -``` - -... - -```python -ctx = mellea.SingleShotContext(backend=WatsonX("ibm/granite4")) - -customer = Customer(customer_id=42) -mify(c) - -meetings_summary = m.query(c, "Summarize the last three interactions with this customer.") - -email_body = ctx.instruct("Based upon the summary of notes from recent meetings, write an email body encouraging the customer to purchase three cases of self-sealing stembolts", grounding_context={"meetings_summary": meetings_summary}) - -email_subject = ctx.instruct("Write a subject for this sales email.", grounding_context={"email_body": email_body}) - -customer.execute("send an email.", email_body, email_subject) -``` - -For more examples and information, see -- [Mify Examples](../examples/mify.py) -- [Mify Implementation](../../mellea/stdlib/mify.py) diff --git a/docs/dev/requirement_aLoRA_rerouting.md b/docs/dev/requirement_aLoRA_rerouting.md deleted file mode 100644 index 163493445..000000000 --- a/docs/dev/requirement_aLoRA_rerouting.md +++ /dev/null @@ -1,90 +0,0 @@ -# Rerouting Requirement Actions in `Backend.generate_*` calls - -Backend will often re-route a `generate` call where `action : Requirement` to an ALora. This document explains how and why that happens. - -## The Requirement Rerouting Rule - -## The Simple Rule - -The simplest version of the Requirement Rerouting Rule is: - -> The most specific constraint checking method will be used when validating generic `Requirement`s. - -The actual rule is slightly more complicated. - -## The Actual Rule - -If a `Requirement` is validated using a backend that could either use a `requirement-check` aLoRA or perform an LLMaJ prompt on the underlying model, then the aLoRA is used for validation, even if the `backend.generate_from_context` method is called instead of the `backend._generate_from_intrinsic` method. - -There are three exceptions to this rule: -1. `Backend.default_to_constraint_checking_alora` is set to `False` (this parameter defaults to `True`). -2. The `Requirement` has a more specific subtype that indicates a more specific intent (`LLMaJRequirement`). -3. The `ALoRA` requirement checker throws an exception. - -There is an exception (or disambiguation) to the first exception: If the user provides an `ALoRARequirement`, then the `backend.generate_from_context` call is rerouted to the constraint checking LoRA, regardless of the value of `default_to_constraint_checking_alora`. - -## Decision Rationale - -### Background and Problem Statement - -The `stdlib` has a `Requirement` class whose `validate` behavior is an LLMaJ call. - -Suppose that the user creates a backend and then adds a generic constraint checking aLoRA: - -```python -from mellea import start_session -from mellea.core import Requirement -from mellea.backends.adapters import IntrinsicAdapter - -m = start_session( - "huggingface.LocalHFBackend:ibm-granite/granite-4.0-micro") - -# By default, the AloraRequirement uses a IntrinsicAdapter with "requirement-check". -m.backend.add_adapter(IntrinsicAdapter("ibm-granite/rag-intrinsics-lib", "requirement-check", base_model_name="granite-4.0-micro")) - -m.instruct( - "Corporate wants you to find the difference between these two strings:\n\naaa\naba") -assert m.validate(Requirement( - description="The answer should mention that one of the strings has the letter b while the other doesn't.")) -``` - -Both the underlying model and the aLoRA adapter know how to validate this requirement, so which should be used? - -## Alternatives to the Proposed Rule - -1. Avoid the problem by forcing the user to be more explicit. -2. Respect control flow in the backends/alora mixins, and have the MelleaSession or the user explicitly implement the appropriate control flow. -3. Have the `Requirement.validate` implementation specify whatever control flow is desired for that particular requirement. - -### Advantages - -1. Reduced cognitive load. To first approximation, there is a simple rule that produces unsurprising results. The exceptions are rare and require explicit intervention from the user. If these exceptions are used, the user almost certainly knows exactly what they are doing. -2. Control is retained. If the user wants to specify the precise semantics of their validate call, then they can use the mpore specific `LLMaJRequirement` and `ALoraRequirement` classes. -3. The backend is the one that needs to make the choice about whether to handle KV cache. - - -### Disadvantages - -All backends that implement the aLoRA mixin need to implement this semantics. - - * This might be a blessing in disguise. It's actually not clear that ALora context construction can be done WLOG outside of the specific backend. - * That code is written rarely in any case. - * Depending on the truth of the first bullet point's conjecture, we can mitigate by implementing this routing in `m.validate` so that even if a backend contributor gets this wrong the proper behavior is still usually observed by most users. - -## Phase 1 change (Epic #929, issue #1136) - -Backends now use a **capability-based lookup** to find the `requirement-check` -adapter, replacing the old `isinstance` + `AdapterType` check: - -```python -# Before (Phase 0) -adapter = get_adapter_for_intrinsic("requirement-check", [AdapterType.ALORA], self._added_adapters) - -# After (Phase 1) -adapter = self._find_adapter("requirement-check", ("alora",)) -``` - -The logical rule (three exceptions above) is unchanged. The change is purely -in how the matching adapter is located: capability name and adapter type are -now read from `adapter.identity` (the new `Identity` dataclass introduced in -Phase 0, issue #1134) rather than derived from the adapter's class hierarchy. diff --git a/docs/dev/spans.md b/docs/dev/spans.md deleted file mode 100644 index 28c19d3b9..000000000 --- a/docs/dev/spans.md +++ /dev/null @@ -1,20 +0,0 @@ -# Design Document for Spans - -## Span Contexts - -We will introduce a SpanContext which will behave kind of like a heap but with transformer-running-on-GPU memory primitives instead of malloc/realloc/free. The public interface to a SpanContext will roughly correspond to the sort of stuff you can do in Span algebras, if you've seen some of that work. - -## Mapping STDLIB to Spans - -There are two broad philosophies to choose from for Spans. - -### The Span Representation Approach - -All Components and CBlocks get a __span_repr__ which maps the all things to a Span representation. The Component owner is responsible for saying how something gets represented as a Span, and is also responsible for defining caching boundaries (via a cache_boundary tag). - -### The Span Formatter Approach - -There is a Formatter which maps Components and CBlocks to Spans, as a pure function. Similar to how the TemplateFormatter works today. - -We need to document which approach we choose and discuss why it was chosen. - diff --git a/docs/dev/tool_calling.md b/docs/dev/tool_calling.md deleted file mode 100644 index fff491d94..000000000 --- a/docs/dev/tool_calling.md +++ /dev/null @@ -1,73 +0,0 @@ -# Tool Calling - -## Problem Statement - -Context management and execution of tool calls are inextricably linked, because most -models expect the output of a tool call to be added to the context at the -moment when the too lcall happens. This means that the `Session` must own the -code that actual performs a tool call. - -This is annoying because *what to do with a tool call* -- or even *how to -implement a tool call* -- is going to vary from application to application. - -We are then faced with two options: - -1. Provide some sort of object protocol for handling tool calls, whereby the - client responsible for tool calling is also responsible for executing a - callback on the session which appropriately modifies the session's context - in light of the tool response; or, -2. Come up with a small number of ways in which a tool may be called, and - expose those in the session. Anyone who wants to do something more complex - must then extend the Session class and implement their own too lcalling - logic. - -## Proposals - - -### Tool Calling Protocol Option - -Basically (2). - -Certain things such as `transform` have a default semantics in the -`MelleaSession` base class. - -For anyone who wants to do free-form tool calling, -there is a `MelleaSessionToolProtocol` mixin which must be inherited from and -implemented. - -### Nothing Fancy Option - -Pass back the `ModelOutputThunk` with tool calls, and do nothing else. - -Note that we already have a `ctx.insert` function, si instead of a mixin with -a protocol, the user is just supposed to know what they are supposed to do and -then use `m.ctx.insert` to implement the relevant logic. - -This is what's done with openai sdk in the status quo anyways. - -### Compromise? - -Can this be implemented such that if you don't specify a tool calling protocol -implementation then the behavior is equivalent to the Nothing Fancy Option? -Probably so. - - -## Final Proposal - -The ModelOutputThunk has a `tools` field where parsed tool calls are surfaced -to the user. This already exists and probably does not need additional -modification. - -1. For certain special tool calling protocols, the Session handles things - automatically for the user. E.g., `m.transform` and `m.query`. We need to - specify the precise semantics for what happens when a user provides tools - in the model_options when using `m.transform` -- probably, you flow through - into the next two cases. -2. If the `Session` has a `SessionToolCallingProtocol` implemented, then the - `def tool_call_result(...)` on that protocol must be called by the user - after a tool is executed. When that method is called, the context is - updated appropriately. We can also provide a `def call_tool(tool)` method - for convenience, which does both the tool call and the context management - for the user. -3. Otherwise, nothing happens. The user is responsible for updating their - context as needed. From 91ebacdc3e1988e5f252ef9b1eff35b6f4e4f945 Mon Sep 17 00:00:00 2001 From: Nigel Jones Date: Tue, 11 Aug 2026 19:21:57 +0100 Subject: [PATCH 4/6] docs(telemetry): document the session.py tracing exception at its import site #1464 item 2: the exception was already documented at each call site (the "Called directly, not via hook" comments), but not where a reader skimming imports for "how do I emit a span" would look first. Assisted-by: Claude Code Signed-off-by: Nigel Jones --- mellea/stdlib/session.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mellea/stdlib/session.py b/mellea/stdlib/session.py index bf3571f96..081234454 100644 --- a/mellea/stdlib/session.py +++ b/mellea/stdlib/session.py @@ -54,6 +54,12 @@ from ..plugins.types import HookType from ..stdlib import functional as mfuncs from ..telemetry.context import with_context + +# session.py is the one sanctioned exception to "library code fires hooks; +# plugins open spans" (see tracing_plugins.py): OTel Token attach/detach is +# task-affine, and each hook fires in a separate _run_async_in_thread Task, +# so this span pair can't be delegated to a plugin. Do not copy this import +# elsewhere under mellea/stdlib/ or mellea/backends/ — use a hook instead. from ..telemetry.tracing import ( finish_session_span, finish_session_startup_span, From 53ff3e12d044703a5913f5f4e01c55d0bb8ac631 Mon Sep 17 00:00:00 2001 From: Nigel Jones Date: Tue, 11 Aug 2026 21:23:23 +0100 Subject: [PATCH 5/6] test(telemetry): enforce the backends-never-import-tracing invariant MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #1464 item 4. Nothing currently violates this — mellea/backends/ has zero direct telemetry.tracing imports today — but nothing was enforcing it, which is exactly how #1454 happened: docs/dev's stale guidance led an implementer to add direct start_*_span calls in mellea/backends/, caught only by review. Resolves relative imports via AST rather than string-matching so it doesn't false-positive on tracing_plugins.py (a different, legitimately imported module) and doesn't miss a violation written with different whitespace or aliasing. Assisted-by: Claude Code Signed-off-by: Nigel Jones --- .../telemetry/test_tracing_import_boundary.py | 100 ++++++++++++++++++ 1 file changed, 100 insertions(+) create mode 100644 test/telemetry/test_tracing_import_boundary.py diff --git a/test/telemetry/test_tracing_import_boundary.py b/test/telemetry/test_tracing_import_boundary.py new file mode 100644 index 000000000..95673422f --- /dev/null +++ b/test/telemetry/test_tracing_import_boundary.py @@ -0,0 +1,100 @@ +# Copyright IBM Corp. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +"""Enforce the "library code fires hooks; plugins open spans" import boundary. + +See #1464: `mellea/backends/` (and the rest of `mellea/stdlib/`) must never +import `mellea.telemetry.tracing` directly, because a direct import is a +direct span-opening call — the thing that has to happen from a +`tracing_plugins.py` plugin instead so telemetry stays optional and +removable. `mellea/stdlib/session.py` is the one sanctioned exception, +documented at its import site: OTel `Token` attach/detach is task-affine and +can't be delegated to a plugin. +""" + +import ast +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[2] +SANCTIONED_EXCEPTION = "mellea/stdlib/session.py" + + +def _package_for(path: Path) -> str: + """Return the dotted package containing `path`, for relative-import resolution.""" + rel = path.relative_to(REPO_ROOT) + parts = list(rel.parts[:-1]) + if rel.stem != "__init__": + pass # module's package is its containing directory either way + return ".".join(parts) + + +def _resolve_relative_import(package: str, level: int, module: str | None) -> str: + """Mirror importlib's `_resolve_name` for `from . import x`-style imports.""" + bits = package.rsplit(".", level - 1) + base = bits[0] + return f"{base}.{module}" if module else base + + +def _imports_telemetry_tracing(path: Path) -> bool: + """Return True if `path` imports the `mellea.telemetry.tracing` module.""" + tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) + package = _package_for(path) + for node in ast.walk(tree): + if isinstance(node, ast.Import): + if any(alias.name == "mellea.telemetry.tracing" for alias in node.names): + return True + elif isinstance(node, ast.ImportFrom): + if node.level == 0: + resolved = node.module or "" + if resolved == "mellea.telemetry.tracing": + return True + else: + resolved = _resolve_relative_import(package, node.level, node.module) + if resolved == "mellea.telemetry.tracing": + return True + # `from ..telemetry import tracing` imports the submodule by name. + if resolved == "mellea.telemetry" and any( + alias.name == "tracing" for alias in node.names + ): + return True + return False + + +def _source_files(*subdirs: str) -> list[Path]: + files: list[Path] = [] + for subdir in subdirs: + files.extend((REPO_ROOT / subdir).rglob("*.py")) + return files + + +def test_backends_never_import_telemetry_tracing(): + """`mellea/backends/` must fire hooks, not open spans directly.""" + offenders = [ + str(f.relative_to(REPO_ROOT)) + for f in _source_files("mellea/backends") + if _imports_telemetry_tracing(f) + ] + assert offenders == [], ( + "These backend modules import mellea.telemetry.tracing directly, " + "which means they open spans instead of firing a hook for a " + "tracing_plugins.py plugin to open one. Fire the matching hook " + "instead (see mellea/telemetry/tracing_plugins.py for the " + f"pattern): {offenders}" + ) + + +def test_stdlib_tracing_import_is_only_the_sanctioned_exception(): + """`mellea/stdlib/session.py` is the one place allowed to import tracing directly.""" + offenders = [ + str(f.relative_to(REPO_ROOT)) + for f in _source_files("mellea/stdlib", "mellea/backends") + if _imports_telemetry_tracing(f) + ] + assert offenders == [SANCTIONED_EXCEPTION], ( + "Exactly one module (mellea/stdlib/session.py) is allowed to import " + "mellea.telemetry.tracing directly, because OTel Token attach/detach " + "there is task-affine and can't be delegated to a hook-driven " + f"plugin. Found: {offenders}. If this is a new legitimate exception, " + "document it at the import site the way session.py does, and update " + "this test's SANCTIONED_EXCEPTION." + ) From 918b4a2bff6211863a1cb573059d345508a5814f Mon Sep 17 00:00:00 2001 From: Nigel Jones Date: Tue, 11 Aug 2026 21:27:12 +0100 Subject: [PATCH 6/6] docs: explain how spans are produced in tracing.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #1464 item 1. tracing.md documented which spans exist and what attributes they carry, but never mentioned that library code fires hooks and mellea/telemetry/tracing_plugins.py opens the spans — the same gap that let docs/dev/adapter_observability.md's stale direct-span guidance go unnoticed until it caused a real revert (PR #1454). Also names test_tracing_import_boundary.py as the CI enforcement for this rule. Assisted-by: Claude Code Signed-off-by: Nigel Jones --- docs/docs/observability/tracing.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/docs/docs/observability/tracing.md b/docs/docs/observability/tracing.md index 0272173c3..7a3f5b451 100644 --- a/docs/docs/observability/tracing.md +++ b/docs/docs/observability/tracing.md @@ -97,6 +97,35 @@ from mellea.telemetry import is_tracing_enabled print(f"Tracing enabled: {is_tracing_enabled()}") ``` +## How spans are produced + +Library code never opens a span itself. `mellea/backends/*` and +`mellea/stdlib/*` call `invoke_hook(HookType.X, payload)` and move on; a +plugin in `mellea/telemetry/tracing_plugins.py` subscribes to that hook and +opens or closes the span. Nothing under `mellea/backends/` or +`mellea/stdlib/` imports `mellea.telemetry.tracing` — that module's +span-opening functions are called from the tracing plugins alone. + +This indirection is what makes tracing genuinely optional and removable: with +no plugins registered, firing a hook is a no-op, so nothing changes for +someone using Mellea without the `[telemetry]` extra. It is also what lets a +single plugin assemble one span tree from work spread across several +objects, threads, or call stacks — the `chat` span nested under `sampling` in +the hierarchy below, for instance, is opened by the tracing plugin that +subscribes to the `generation_pre_call`/`generation_post_call` hook pair +fired from `Backend.generate_from_context`, not by the backend itself. + +`mellea/stdlib/session.py` is the one sanctioned exception: it imports +`mellea.telemetry.tracing` directly for the `session`/`start_session` span +pair, because OTel `Token` attach/detach is task-affine and each hook fires +in a separate task, so it can't be delegated to a plugin. That exception is +documented at its import site — treat it as the exception, not a template +for a new span. + +> **Note:** `test/telemetry/test_tracing_import_boundary.py` enforces this in +> CI — it fails if anything under `mellea/backends/` imports +> `mellea.telemetry.tracing` directly. + ## What spans Mellea emits Mellea has two trace scopes.