-
Notifications
You must be signed in to change notification settings - Fork 149
feat: add prefix to component tool names #1432
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
akihikokuroda
wants to merge
10
commits into
generative-computing:main
Choose a base branch
from
akihikokuroda:issue95-1
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 7 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
612de1a
add prefix to component tool name
akihikokuroda d4930a3
add prefix to component tool name
akihikokuroda b3c98a0
add prefix to component tool name
akihikokuroda a2b3411
compoment id based prefixing
akihikokuroda 83dd5c1
add duplicate tool name example
akihikokuroda 243d475
updated component tool usage
akihikokuroda fe8fe84
fix merge error
akihikokuroda bb0bc47
review comments
akihikokuroda e2f580b
fix ollama schema handling
akihikokuroda 0e9469c
fix lint error
akihikokuroda File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| # Component Examples | ||
|
|
||
| This directory contains examples demonstrating Mellea's component system, particularly focusing on component ID-based tool prefixing for collision-free tool composition. | ||
|
|
||
| ## Files | ||
|
|
||
| ### `duplicate_tool_names.py` | ||
| **Main example** - Demonstrates how Mellea handles multiple components with identical tool names. | ||
|
|
||
| **What it shows:** | ||
| - Two components (`DatabaseComponent` and `SearchComponent`) both define a `query` tool | ||
| - Automatic tool prefixing using component IDs (`component_{ID}.query`) | ||
| - LLM receiving and using both prefixed tools | ||
| - Proper tool execution via Mellea's pipeline (enables telemetry) | ||
| - Component ID extraction from prefixed tool names | ||
|
|
||
| **Run it:** | ||
| ```bash | ||
| uv run python docs/examples/components/duplicate_tool_names.py | ||
| uv run pytest docs/examples/components/duplicate_tool_names.py -v | ||
| ``` | ||
|
|
||
| **View telemetry metrics:** | ||
| ```bash | ||
| export MELLEA_METRICS_ENABLED=true | ||
| export MELLEA_METRICS_CONSOLE=true | ||
| uv run python docs/examples/components/duplicate_tool_names.py | ||
| ``` | ||
|
|
||
| **Key outputs:** | ||
| - Tools extracted with ID-based prefixes: `component_1adeba40.query`, `component_1c611a00.query` | ||
| - LLM successfully calls both tools in a single prompt | ||
| - Tool calls executed via `_call_tools()` to enable telemetry recording | ||
| - Both tools' component IDs visible in `mellea.tool.calls` metrics | ||
|
|
||
| --- | ||
|
|
||
| ### `duplicate_tool_names_experiments.py` | ||
| **Advanced experiments** - Explores various scenarios and edge cases with component tool prefixing. | ||
|
|
||
| **Experiments:** | ||
| 1. **Three Components with Same Tool Name** - Verify scaling beyond 2 components | ||
| 2. **Tool Name Mapping Inspection** - Examine the extracted tool objects and structure | ||
| 3. **Prefixing Stability** - Show that same instances get same IDs, new instances get different IDs | ||
| 4. **Selective Tool Access** - Filter tools before passing to LLM | ||
| 5. **Tool Deduplication** - Behavior when same component added multiple times | ||
| 6. **LLM with Filtered Tools** - Call LLM with a subset of available tools | ||
|
|
||
| **Run it:** | ||
| ```bash | ||
| uv run python docs/examples/components/duplicate_tool_names_experiments.py | ||
| uv run pytest docs/examples/components/duplicate_tool_names_experiments.py -v | ||
| ``` | ||
|
|
||
| **Key findings:** | ||
| - Component IDs are stable for the same instance (multi-turn ready) | ||
| - New component instances get new IDs (expected behavior) | ||
| - Duplicate component instances are gracefully handled with warnings | ||
| - Tool filtering works by subsetting the tools dict | ||
| - LLM respects prompt guidance even when given multiple tools | ||
|
|
||
| --- | ||
|
|
||
| ### `pattern2_context_and_tools.py` | ||
| **Pattern 2 demonstration** - Shows how to combine components in context with automatic tool extraction for tool calling. | ||
|
|
||
| **What it shows:** | ||
| - Pattern 2 approach: Components in session context with automatic tool extraction | ||
| - How to add components to the session context | ||
| - Backend automatically extracts tools via `add_tools_from_context_actions()` when `tool_calls=True` | ||
| - Proper tool execution via Mellea's pipeline (enables telemetry) | ||
| - Multi-turn stability with component ID-based prefixing | ||
| - Components with templates render in the conversation | ||
|
|
||
| **Run it:** | ||
| ```bash | ||
| uv run python docs/examples/components/pattern2_context_and_tools.py | ||
| uv run pytest docs/examples/components/pattern2_context_and_tools.py -v | ||
| ``` | ||
|
|
||
| **View telemetry metrics:** | ||
| ```bash | ||
| export MELLEA_METRICS_ENABLED=true | ||
| export MELLEA_METRICS_CONSOLE=true | ||
| uv run python docs/examples/components/pattern2_context_and_tools.py | ||
| ``` | ||
|
|
||
| **Key concepts:** | ||
| - Pattern 1: Extract tools only (simple tool calling) | ||
| - Pattern 2: Components in context with auto-extraction (implicit tool passing) | ||
| - Both patterns use component ID-based prefixing | ||
| - NO explicit `ModelOption.TOOLS` needed - backend auto-extracts from context | ||
| - Components must have valid templates for rendering | ||
| - Each tool call recorded in `mellea.tool.calls` metric with component_id | ||
|
|
||
| --- | ||
|
|
||
| ### `telemetry_tool_calling_demo.py` | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here —
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. removed |
||
| **Telemetry demonstration** - Shows how to enable and view tool calling metrics. | ||
|
|
||
| **What it shows:** | ||
| - How to enable telemetry with environment variables | ||
| - Setting up console exporter for metric viewing | ||
| - Tool calls executed via `_call_tools()` to trigger telemetry hooks | ||
| - JSON format of OpenTelemetry metrics | ||
| - Component ID extraction in `mellea.tool.calls` metric | ||
| - Multi-exporter setup (Console, OTLP, Prometheus) | ||
|
|
||
| **Run it (with telemetry):** | ||
| ```bash | ||
| export MELLEA_METRICS_ENABLED=true | ||
| export MELLEA_METRICS_CONSOLE=true | ||
| uv run python docs/examples/components/telemetry_tool_calling_demo.py | ||
| ``` | ||
|
|
||
| **Run it (without telemetry):** | ||
| ```bash | ||
| uv run python docs/examples/components/telemetry_tool_calling_demo.py | ||
| ``` | ||
|
|
||
| **Key outputs:** | ||
| - Tool calls listed with their component IDs | ||
| - OpenTelemetry JSON metrics showing `mellea.tool.calls` counter | ||
| - Each tool invocation tracked with: | ||
| - `tool`: Full tool name (e.g., `component_203e1b50.query`) | ||
| - `status`: `"success"` or `"failure"` | ||
| - `component_id`: Extracted from tool name (e.g., `203e1b50`) | ||
|
|
||
| --- | ||
|
|
||
| ## Concepts | ||
|
|
||
| ### Component ID-Based Prefixing | ||
|
|
||
| When multiple components define tools with the same name, Mellea prevents collisions by prefixing each tool name with its component ID: | ||
|
|
||
| ``` | ||
| Original: query, query | ||
| Prefixed: component_1adeba40.query, component_1c611a00.query | ||
| ``` | ||
|
|
||
| **How it works:** | ||
| 1. Each component instance gets a unique ID: `hex(id(object))[-8:]` | ||
| 2. Tools from each component are extracted and prefixed | ||
| 3. Prefixed names are collision-free | ||
| 4. Same component instances always produce same IDs (stable for multi-turn) | ||
|
|
||
| --- | ||
|
|
||
| ## Key Takeaways | ||
|
|
||
| 1. **Composability**: Multiple components can safely define tools with identical names | ||
| 2. **Determinism**: Component IDs are stable within a session for the same instance | ||
| 3. **Flexibility**: You can control which tools reach the LLM via filtering | ||
| 4. **Scalability**: Works smoothly with 2, 3, or more components | ||
| 5. **Observability**: Prefixed names and component IDs enable tracing and debugging | ||
|
|
||
| --- | ||
|
|
||
| ## Related Source Files | ||
|
|
||
| - `mellea/backends/tools.py` - `add_tools_from_context_actions()` implementation | ||
| - `mellea/core/base.py` - `TemplateRepresentation` with component metadata | ||
| - `mellea/stdlib/functional.py` - `_call_tools()` implementation (executes tools via pipeline) | ||
| - `mellea/telemetry/metrics_plugins.py` - `ToolMetricsPlugin` (records tool metrics) | ||
| - `mellea/telemetry/metrics.py` - `record_tool_call()` function (telemetry recording) | ||
| - Tests: `test/backends/test_tool_helpers.py` - Unit tests for tool prefixing | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This section documents
duplicate_tool_names_experiments.pywith full run instructions and "key findings," but the file doesn't exist in this PR (onlyduplicate_tool_names.pyandpattern2_context_and_tools.pydo). Please add the file or drop this section.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed.