Repository navigation
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
OpenTelemetry observability currently exports traces but does not export GenAI metrics for Prompt Drivers. Add an optional SDK
meter_providertoOpenTelemetryObservabilityDriver, falling back to OpenTelemetry's global meter provider, and record metrics aroundPromptDriver.run()while preserving existing tracing behavior.The implementation records operation duration, input/output token usage counters, and per-operation input/output token histograms using the current GenAI metric conventions and inference token conventions. Metrics include provider, operation, and requested model; failures include the exception class as
error.type. Aggregate token counters use theunknownmodality. Prompt and response content are not included in metric attributes.Duration covers one logical Driver run, including retries. Token usage is recorded only from its final successful response; streaming uses the existing final cumulative usage aggregation. Unknown token counts are omitted, and reported zero counts are retained. Failed retry usage is unavailable and is not estimated. The supplied provider is flushed, but not shut down, when the context exits. Streaming time-to-first-chunk and per-chunk latency are outside this change.
Include a console metrics example, configuration/lifecycle documentation, and regression coverage for success, streaming, missing/zero usage, errors, retries, provider subclasses, global providers, context reuse, and calls outside Prompt Drivers.
Validation on Python 3.13.7 with the repository's frozen core/dev/OpenTelemetry dependencies:
python -m pytest tests/unit/drivers/observability tests/unit/drivers/prompt/test_base_prompt_driver.py tests/unit/observability tests/unit/common/test_observable.py tests/unit/structures tests/unit/tasks/test_prompt_task.py -q: 233 passed.ruff checkandruff format --check: passed across the repository.pyrighton the changed Driver and new documentation example: 0 errors.mdformat --check .github/ docs/: passed after normalizing Windows checkout line endings for the check; unrelated files retain their original checkout contents.typosandgit diff --cached --check: passed.The complete unit/integration suite and live provider/exporter integrations were not run. Tests use in-memory metrics and mock Prompt Drivers; no model API calls are needed.
Issue ticket number and link
Addresses #1790: #1790
📚 Documentation preview 📚: https://griptape--2368.org.readthedocs.build//2368/