Skip to content

fix: make the bandit reward's ARR term live and warn on missing qlib metrics - #1479

Open
michaelraziel-bit wants to merge 4 commits into
microsoft:mainfrom
michaelraziel-bit:fix/bandit-arr-metric-key
Open

fix: make the bandit reward's ARR term live and warn on missing qlib metrics#1479
michaelraziel-bit wants to merge 4 commits into
microsoft:mainfrom
michaelraziel-bit:fix/bandit-arr-metric-key

Conversation

@michaelraziel-bit

@michaelraziel-bit michaelraziel-bit commented Sep 8, 2026

Copy link
Copy Markdown

Fixes #1451

What was wrong

extract_metrics_from_experiment in rdagent/scenarios/qlib/proposal/bandit.py looked up the annualized return with a trailing space in the key ("...annualized_return "). Qlib never logs that key, .get silently returned the 0.0 default, and the ARR feature was always zero. The eighth feature (arr / -mdd) is derived from it, so it was always zero too. Between them that is 0.45 of the reward weight, so every default rdagent fin_quant run has been steering the factor/model bandit on IC, ICIR, Rank IC, Rank ICIR, IR and drawdown only.

slot feature weight before after
4 ARR 0.25 always 0 live
7 ARR / |MDD| 0.20 always 0 live

Nothing noticed because every lookup defaulted to 0.0 inside a bare except Exception, so a mistyped key, a missing key and a genuinely zero metric were indistinguishable.

What this PR does

Four commits, each droppable on its own:

  1. fix: remove the trailing space. This is the whole bug fix.
  2. refactor: move the Qlib metric key strings into rdagent/scenarios/qlib/metrics.py and import them from both bandit.py and feedback.py (IMPORTANT_METRICS is now built from the constants), so the two places cannot disagree on spelling again. Replace the silent defaults with a helper that logs a warning naming the missing key and the keys that were present, then still returns the default so a partial Qlib result degrades instead of crashing the loop. Narrow the outer except Exception to the case it was guarding (experiment.result is None after a failed run). Change the max_drawdown default from 1.0 to 0.0: Qlib reports drawdown as a number <= 0, so 1.0 would have put -1.0 in the feature vector and flipped the sign of the ratio whenever the key was missing.
  3. refactor: rename the sharpe field to calmar. The formula (arr / -mdd) is unchanged; the name now says what it computes. Metrics is only constructed inside bandit.py, so nothing else is touched. This commit is optional. If you would rather keep the name, or replace the feature with a true Sharpe from Qlib's information_ratio or mean / std (the current feature is a monotone function of ARR, which is already slot 4, and carries no volatility information), say so and I will adjust.
  4. test: test/qlib/test_bandit_metrics.py feeds the extractor a Series with the exact key names Qlib logs and checks that every field is read, that a missing key produces a warning naming it, that a missing drawdown stays neutral, and that changing ARR moves EnvController.reward. The first test fails on the old key. All tests are marked offline so they run in CI.

Reward vector semantics are otherwise unchanged: same eight slots, same weights, same -mdd sign convention. log/ui/app.py still has its own copy of the key list; left alone here to keep the diff focused, happy to switch it to the constants in a follow-up.

Checks

  • pytest test/qlib -q: 12 passed
  • black --check -l 120 . and isort --check . as in the Makefile: clean

馃 Generated with Claude Code

https://claude.ai/code/session_011Rf4jRK6VVz9qZQ9fcDErw


馃摎 Documentation preview 馃摎: https://RDAgent--1479.org.readthedocs.build/en/1479/

michaelraziel-bit and others added 4 commits September 8, 2026 18:12
The key "1day.excess_return_with_cost.annualized_return " (with a trailing
space) never matches anything Qlib logs, so result.get() always returned
the 0.0 default. That left the 0.25-weight ARR term of the bandit reward
permanently dead, and the derived return/drawdown feature with it.

Refs microsoft#1451

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Rf4jRK6VVz9qZQ9fcDErw
Collect the Qlib metric key strings in rdagent/scenarios/qlib/metrics.py
and import them from both bandit.py and feedback.py, so the two places
can no longer disagree on spelling.

In extract_metrics_from_experiment, replace the silent
result.get(key, 0.0) defaults with a helper that logs a warning naming
the missing key and the keys that were present, then returns the
default. The loop still degrades rather than dies on a partial Qlib
result, but a mistyped or missing key is now visible in the log. The
outer except Exception is narrowed to the one case it was protecting
against (experiment.result is None after a failed run), so programming
errors are no longer swallowed.

The default for max_drawdown becomes 0.0 instead of 1.0. Qlib reports
drawdown as a number <= 0, so a default of 1.0 would have put -1.0 in
the feature vector and flipped the sign of the return/drawdown ratio
whenever the key was missing.

Refs microsoft#1451

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Rf4jRK6VVz9qZQ9fcDErw
The eighth bandit feature is annualized return divided by the absolute
max drawdown. That is a Calmar-type ratio, not a Sharpe ratio: it has no
volatility term and is a monotone function of the ARR feature already in
slot 4. The formula is unchanged; only the field name now says what it
computes. Metrics is only constructed inside bandit.py, so no caller is
affected.

Refs microsoft#1451

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Rf4jRK6VVz9qZQ9fcDErw
Feed extract_metrics_from_experiment a Series carrying the exact key
names Qlib logs, and check that every field is read, that a missing key
produces a warning naming it, that a missing drawdown stays neutral, and
that changing ARR actually moves EnvController.reward. The first test
would have failed on the trailing-space key.

Refs microsoft#1451

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Rf4jRK6VVz9qZQ9fcDErw
@michaelraziel-bit

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bandit scheduler reward: ARR term never fires (trailing space in metric key), and sharpe is derived from it

1 participant