Skip to content

Fix NMI/AMI returning 0.0 instead of 1.0 for two single-cluster labelings - #3470

Open
AlejandroCoronadoN wants to merge 2 commits into
Lightning-AI:masterfrom
AlejandroCoronadoN:fix-nmi-ami-single-cluster
Open

Fix NMI/AMI returning 0.0 instead of 1.0 for two single-cluster labelings#3470
AlejandroCoronadoN wants to merge 2 commits into
Lightning-AI:masterfrom
AlejandroCoronadoN:fix-nmi-ami-single-cluster

Conversation

@AlejandroCoronadoN

Copy link
Copy Markdown

Fixes #3469.

normalized_mutual_info_score and adjusted_mutual_info_score return 0.0 when both preds and target are a single cluster (two zero-entropy labelings). scikit-learn, the documented reference for these metrics, returns 1.0 for this limit case because the two labelings trivially agree.

Repro:

import torch, numpy as np
import sklearn.metrics as skm
import torchmetrics.functional.clustering as tmc

a = np.array([0, 0, 0, 0]); ta = torch.tensor(a)
print(float(tmc.normalized_mutual_info_score(ta, ta)), skm.normalized_mutual_info_score(a, a))  # 0.0 vs 1.0
print(float(tmc.adjusted_mutual_info_score(ta, ta)), skm.adjusted_mutual_info_score(a, a))        # 0.0 vs 1.0

Fix: return 1.0 early when both labelings have exactly one unique value, matching sklearn. For adjusted_mutual_info_score I also added the check_cluster_labels(preds, target) call that the other clustering metrics already use, so the early return does not bypass input validation (float inputs still raise, verified). No non-degenerate result changes.

Tests: added a both_single_cluster case parametrized over every average_method to both functional test files, using sklearn as the reference oracle. ruff is clean.

@mergify
mergify Bot requested a review from a team August 20, 2026 12:45
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 33%. Comparing base (d184220) to head (cdb6add).
⚠️ Report is 11 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #3470    +/-   ##
=======================================
- Coverage      37%     33%    -4%     
=======================================
  Files         349     349            
  Lines       19901   19914    +13     
=======================================
- Hits         7264    6524   -740     
- Misses      12637   13390   +753     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify mergify Bot added the ready label Aug 20, 2026
@AlejandroCoronadoN

Copy link
Copy Markdown
Author

Thanks for the review, @justusschock. This has drifted a bit behind master; happy to rebase it if that helps it land, and glad to make any other changes you need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NMI/AMI return 0.0 instead of 1.0 for two single-cluster labelings

3 participants