Skip to content

fix(attempt): fall back to original outputs when reverse translation is missing - #2081

Open
alireza-aminzadeh wants to merge 1 commit into
NVIDIA:mainfrom
alireza-aminzadeh:fix/issue-2078-outputs-for-fallback
Open

fix(attempt): fall back to original outputs when reverse translation is missing#2081
alireza-aminzadeh wants to merge 1 commit into
NVIDIA:mainfrom
alireza-aminzadeh:fix/issue-2078-outputs-for-fallback

Conversation

@alireza-aminzadeh

Copy link
Copy Markdown

outputs_for(lang) returned reverse_translation_outputs whenever the requested language did not match the prompt language, with no fallback. When reverse translation was never populated (default empty dict, empty list, or None placeholders only), detectors that call outputs_for(self.lang_spec) received nothing and reported no hits. prompt_for() already falls back to the original prompt in the same situation, so the attempt looked complete while its outputs were silently empty.

Fix

If a different language is requested and reverse translation is unpopulated, return the original outputs and log a warning. Populated reverse translations, including partial None alignment slots, are still returned as-is.

This is complementary to #2041 (list default / type contract). That PR keeps returning an empty list for an unpopulated translated language; this change is the remaining half described in #2078.

Not a duplicate

I searched open PRs for 2078 and for an outputs_for / reverse-translation fallback. #2041 is the type-default fix only. #2067 is the mitigation.Prefixes start-anchor change and is unrelated.

Verification

  • python -m pytest tests/test_attempt.py -k outputs_for -- 3 passed
  • python -m pytest tests/test_attempt.py -- 24 passed; the pre-existing test_attempt_sticky_params failure is a local missing lorem dependency, not this change
  • Existing populated reverse-translation path still returns reverse_translation_outputs
  • Unpopulated / empty / None-only / legacy {} cases fall back to original outputs
  • Partial reverse translation keeps output alignment

Fixes #2078

…is missing

Detectors that call outputs_for() for a language other than the prompt
language received an empty reverse_translation_outputs value and silently
scored no hits. Mirror prompt_for() by returning the original outputs and
logging a warning when reverse translation is unpopulated.

Signed-off-by: Alireza Aminzadeh <6909432+alireza-aminzadeh@users.noreply.github.com>
@feiiiiii5

Copy link
Copy Markdown
Contributor

As the issue reporter: this matches option (a) from #2078. The populated-guard (any(output is not None ...)) covers all three unpopulated shapes — missing key, empty list, and None-only placeholders — and the warning makes the fallback visible in the log instead of silently handing detectors an empty output list while the attempt still looks complete. Mirroring prompt_for() restores exactly the invariant the issue asked for. Thanks for picking this up and for the test coverage — the populated/unpopulated split in the new tests pins the behaviour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

outputs_for() lacks the prompt_for() fallback: detectors silently see empty outputs when reverse translation is unpopulated

2 participants