Skip to content

Support scalar-like analytics values#4659

Closed
YuanTingHsieh wants to merge 1 commit into
NVIDIA:mainfrom
YuanTingHsieh:codex/analytics-numpy-scalars
Closed

Support scalar-like analytics values#4659
YuanTingHsieh wants to merge 1 commit into
NVIDIA:mainfrom
YuanTingHsieh:codex/analytics-numpy-scalars

Conversation

@YuanTingHsieh

@YuanTingHsieh YuanTingHsieh commented May 21, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • Normalize scalar-like analytics values when constructing AnalyticsData.
  • Accept Python int/float values as before, plus scalar-like objects with .item() such as numpy.float32 and 0-D numpy arrays.
  • Normalize numeric values inside SCALARS and METRICS dictionaries as well.
  • Add unit coverage for generic scalar-like objects and numpy scalar/0-D array values.

Why

Client tracking writers currently reject values such as numpy.float32 for single scalar metrics because analytics validation only accepts built-in Python float/int. This can happen with code like:

running_loss += cost.cpu().detach().numpy() / images.size()[0]

Centralizing the normalization in AnalyticsData keeps SummaryWriter, MLflowWriter, WandBWriter, and lower-level analytics logging paths consistent without adding a numpy dependency to core API code.

@YuanTingHsieh YuanTingHsieh force-pushed the codex/analytics-numpy-scalars branch from 74ee977 to c3b18c0 Compare May 21, 2026 19:13
@YuanTingHsieh YuanTingHsieh force-pushed the codex/analytics-numpy-scalars branch from c3b18c0 to 977ddc7 Compare May 21, 2026 19:17
@YuanTingHsieh YuanTingHsieh changed the title [codex] Support scalar-like analytics values Support scalar-like analytics values May 21, 2026
@YuanTingHsieh YuanTingHsieh marked this pull request as ready for review May 21, 2026 19:29
Copilot AI review requested due to automatic review settings May 21, 2026 19:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves analytics logging ergonomics by normalizing “scalar-like” numeric values (e.g., objects with .item() such as NumPy scalars and 0-D arrays) when constructing AnalyticsData, so downstream writers receive built-in Python int/float consistently.

Changes:

  • Normalize scalar-like values for SCALAR/METRIC analytics types during AnalyticsData validation.
  • Normalize numeric values inside SCALARS/METRICS dict payloads.
  • Add unit tests covering generic scalar-like objects and NumPy scalar / 0-D array inputs.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
nvflare/apis/analytix.py Adds numeric scalar normalization logic during validation and applies it to scalar and dict-based analytics payloads.
tests/unit_test/apis/analytix_test.py Adds test coverage for scalar-like normalization, including NumPy scalars and 0-D arrays (skipped if NumPy is unavailable).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread nvflare/apis/analytix.py
@greptile-apps

greptile-apps Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR centralizes normalization of scalar-like analytics values inside AnalyticsData, accepting not just Python float/int but also any object exposing a scalar .item() method (e.g. numpy.float32, 0-D numpy arrays) for SCALAR, METRIC, SCALARS, and METRICS data types.

  • A new _normalize_numeric_scalar static method uses duck-typing (.item() + .shape == ()) to extract a Python float/int from scalar-like objects without importing numpy.
  • _validate_data_types now returns the (possibly normalized) value, which the constructor assigns to self.value, keeping all writers consistent.
  • Unit tests cover a generic ScalarLike stub, numpy scalars, and 0-D numpy arrays for both single-scalar and dict-of-scalars cases.

Confidence Score: 4/5

Safe to merge; the normalization logic is sound and the existing test suite is largely preserved.

The core normalization path is correct and well-tested for the primary use cases. The main gap is that non-normalizable values inside a SCALARS/METRICS dict are silently kept rather than rejected, so a mistyped dict entry would slip past _validate_data_types and only fail later inside a writer. Additionally, the new item() path means numpy booleans are now accepted as scalars where they were previously rejected — a minor but undocumented behavioral expansion.

Pay closest attention to the elif data_type in [AnalyticsDataType.METRICS, AnalyticsDataType.SCALARS] normalization loop in nvflare/apis/analytix.py (lines 197-202) and the corresponding test coverage gap for AnalyticsDataType.METRICS in the test file.

Important Files Changed

Filename Overview
nvflare/apis/analytix.py Adds _normalize_numeric_scalar static method and rewires _validate_data_types to normalize scalar-like values (numpy scalars, 0-D arrays) for SCALAR/METRIC/SCALARS/METRICS types; returns the (possibly normalized) value instead of void.
tests/unit_test/apis/analytix_test.py Adds three new test cases: generic scalar-like normalization, dict value normalization for SCALARS, and numpy scalar/0-D array normalization; METRICS dict normalization is not covered.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[AnalyticsData.__init__] --> B[_validate_data_types]
    B --> C{data_type?}
    C -->|SCALAR or METRIC| D[_normalize_numeric_scalar]
    D --> E{isinstance float or int?}
    E -->|Yes| F[return True plus value]
    E -->|No| G{has callable item?}
    G -->|No| H[return False plus value]
    G -->|Yes| I{shape attribute exists?}
    I -->|Yes and shape not empty tuple| H
    I -->|No or shape is empty tuple| J[call item]
    J --> K{result is float or int?}
    K -->|Yes| L[return True plus scalar]
    K -->|No| H
    F --> M[value normalized stored in self.value]
    L --> M
    H --> N[raise TypeError for SCALAR or METRIC]
    C -->|METRICS or PARAMETERS or SCALARS| O{isinstance dict?}
    O -->|No| P[raise TypeError]
    O -->|Yes| Q{METRICS or SCALARS?}
    Q -->|Yes| R[loop normalize each dict entry]
    R --> S{entry normalizable?}
    S -->|Yes| T[replace with Python float or int]
    S -->|No| U[keep original value silently]
    T --> V[value = normalized_dict]
    U --> V
    Q -->|No - PARAMETERS| W[value unchanged]
    C -->|TEXT| X{isinstance str?}
    X -->|No| Y[raise TypeError]
    C -->|TAGS| Z{isinstance dict?}
    Z -->|No| AA[raise TypeError]
    M --> BB[return value to init]
    V --> BB
    W --> BB
Loading

Reviews (1): Last reviewed commit: "Support scalar-like analytics values" | Re-trigger Greptile

Comment thread nvflare/apis/analytix.py
Comment thread tests/unit_test/apis/analytix_test.py
Comment thread nvflare/apis/analytix.py

Copy link
Copy Markdown
Collaborator Author

Superseded by replacement PR #4683, which was recreated from latest main after this PR glitched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants