Skip to content

fix(validation): correct 'and' to 'or' in argument validation across multiple metrics - #3440

Open
AbdullahRasheed45 wants to merge 1 commit into
Lightning-AI:masterfrom
AbdullahRasheed45:fix/arg-validation-and-to-or
Open

fix(validation): correct 'and' to 'or' in argument validation across multiple metrics#3440
AbdullahRasheed45 wants to merge 1 commit into
Lightning-AI:masterfrom
AbdullahRasheed45:fix/arg-validation-and-to-or

Conversation

@AbdullahRasheed45

Copy link
Copy Markdown

Summary

Several argument validation guards across multiple metrics used the wrong logical operator ( instead of ). As a result, invalid arguments were silently accepted and caused cryptic runtime errors deep inside the metric computation rather than a clear ValueError at initialisation time.

This pattern was first identified in stat_scores.py (issues #3405, resolved in PRs #3406/#3416) and in retrieval metrics (issue #3378). This PR fixes the identical bug in all remaining affected files.

The Bug

The guards follow the pattern:

# Wrong: only fails when BOTH conditions are True
if not isinstance(x, SomeType) and x_is_invalid:
    raise ValueError(...)

For an invalid-value-but-correct-type input (e.g., min_sensitivity=2.0):

  • not isinstance(2.0, float)False
  • 2.0 is outside [0, 1]True
  • False and TrueFalse — no error raised!

Fix: replace with so validation fails when either the type or the value is wrong.

Files Changed

File Parameter Invalid values that bypassed validation
functional/classification/specificity_sensitivity.py (3 sites) min_sensitivity out-of-range floats, e.g. 2.0, -0.5
functional/classification/recall_fixed_precision.py (3 sites) min_precision out-of-range floats, e.g. 2.0, -0.1
functional/classification/sensitivity_specificity.py (3 sites) min_specificity out-of-range floats, e.g. 2.0, -0.5
regression/spearman.py num_outputs 0, negative integers
regression/pearson.py num_outputs 0, negative integers
regression/log_cosh.py num_outputs 0, negative integers
classification/group_fairness.py (2 sites) num_groups 0, 1, negative integers
image/psnrb.py block_size 0, negative integers

Minimal Reproduction

from torchmetrics.functional.classification import binary_specificity_at_sensitivity

# Before fix: no error raised for out-of-range float
binary_specificity_at_sensitivity(preds, target, min_sensitivity=2.0)

# After fix: raises ValueError immediately
# ValueError: Expected argument `min_sensitivity` to be an float in the [0,1] range, but got 2.0
from torchmetrics.regression import SpearmanCorrCoef

# Before fix: no error for zero num_outputs (int)
SpearmanCorrCoef(num_outputs=0)

# After fix: raises ValueError immediately
# ValueError: Expected argument `num_outputs` to be an int larger than 0, but got 0

…multiple metrics

Several validation guards used the wrong logical operator ('and' instead
of 'or'), so invalid arguments were silently accepted and caused cryptic
runtime errors rather than a clear ValueError at initialisation time.

Affected metrics and the invalid inputs that bypassed validation:

- BinarySpecificityAtSensitivity / MulticlassSpecificityAtSensitivity
  / MultilabelSpecificityAtSensitivity
  (specificity_sensitivity.py): min_sensitivity as an out-of-range float,
  e.g. 2.0, was accepted without error.

- BinaryRecallAtFixedPrecision / MulticlassRecallAtFixedPrecision
  / MultilabelRecallAtFixedPrecision
  (recall_fixed_precision.py): same bug for min_precision.

- BinarySensitivityAtSpecificity / MulticlassSensitivityAtSpecificity
  / MultilabelSensitivityAtSpecificity
  (sensitivity_specificity.py): same bug for min_specificity.

- SpearmanCorrCoef / PearsonCorrCoef / LogCoshError (spearman.py,
  pearson.py, log_cosh.py): num_outputs=0 or a negative integer was
  accepted without error.

- BinaryFairness / MulticlassFairness (group_fairness.py): num_groups=1
  or a negative integer was accepted without error.

- PeakSignalNoiseRatioWithBlockedEffect (psnrb.py): block_size=0 or a
  negative integer was accepted without error.

In every case the fix is the same: replace 'and' with 'or' so that an
invalid argument raises ValueError when either type-check or value-check
fails, not only when both fail simultaneously.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant