Skip to content

[PyOV] Convert integer and bool arrays to a bf16 model input by value - #39183

Open
Somay-kousis wants to merge 4 commits into
openvinotoolkit:masterfrom
Somay-kousis:fix/bf16-input-from-int-array
Open

Somay-kousis wants to merge 4 commits into
openvinotoolkit:masterfrom
Somay-kousis:fix/bf16-input-from-int-array

Conversation

@Somay-kousis

@Somay-kousis Somay-kousis commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Details:

  • What a user sees, with x = np.array([1, 2, 3, 0]) fed to a bf16 model input:

    array fed to a bf16 input master, share_inputs=True master, share_inputs=False this PR (both)
    int8/32/64, uint8/32/64, bool, e.g. [1, 2, 3, 0] ValueError (broadcast) or wrong values [0.0078125, 2., 32., 0.], no error converted by value: [1., 2., 3., 0.]
    float32 / float64 converted ([PyOV] Fix float32/float64 inputs to a bf16 model input #38951) converted ([PyOV] Fix float32/float64 inputs to a bf16 model input #38951) unchanged
    2-byte: float16, int16, uint16 (raw bf16 bits) raw bits (byte-swapped if big-endian) float16: raw bits; int16/uint16: written by value (wrong) raw bits, any byte order
    other dtypes (str, bytes, void, complex, object) unchanged
  • Cause: Tensor.data of a bf16 tensor is a float16 view of the bits, and only float32/float64 arrays went through _to_bf16_view. update_tensor assigned ints into the float16 view by value (float16 bits then read as bf16), and value_to_tensor did value.view(float16).

  • Fix in data_dispatcher.py: _is_bf16_convertible (bool, int, uint and float arrays that are not 2 bytes wide) picks _to_bf16_view on both paths. float16/int16/uint16 arrays (_is_bf16_bits) are taken as raw bits on both paths through _bf16_bits_view, which also normalizes the byte order; before, update_tensor assigned int16/uint16 by value, so bf16 data stored as uint16 broke with share_inputs=False. Docstrings updated (the generated .pyi stub is left as is).

  • Rounding: ints go through float32 in _to_bf16_view, which is also what ops.constant(x, dtype=Type.bf16) does, so the bits match it, including int64/uint64 values above 2**32.

  • Not changed: 0-d arrays, numpy scalars and Python ints/lists into a bf16 input still raise (no wrong values).

  • New tests in tests/test_runtime/test_sync_infer_request.py:

    • test_infer_bf16_input_from_int_array: int8, uint8, int32, int64 and bool, list and dict inputs, share_inputs on and off.
    • test_infer_bf16_input_from_int_array_rounds_like_constant: int32, int64 and uint64 values that need rounding get the same bits as ops.constant(data, dtype=Type.bf16).
    • test_infer_bf16_input_from_16bit_array_keeps_raw_bits: float16, int16 and uint16 bit patterns (1.0, -2.0, 0.0, inf), contiguous and strided, list and dict inputs, share_inputs on and off.
    • test_infer_bf16_input_from_16bit_array_keeps_raw_bits_any_byte_order: the same bits as >u2, >i2, >f2.

Tickets:

AI Assistance:

  • AI assistance used: yes
  • AI tools helped find the bug and write the fix and the tests; I reviewed every line, and an independent review pass asked for the int64/uint64 rounding cases. Validation on macOS arm64 with the 2026.5.0 nightly wheel, using this branch's data_dispatcher.py:
    • All new test cases pass. With master's data_dispatcher.py: the 26 int/bool cases fail; of the 24 float16/int16/uint16 raw-bits cases, the 8 int16/uint16 ones with share_inputs=False fail (the rest already pass on master); 5 of the 6 big-endian cases fail.
    • test_sync_infer_request.py, test_async_infer_request.py, test_compiled_model.py, tests/test_utils/test_data_dispatch.py, test_tensor_string.py: 845 passed, 7 skipped.
    • 36,000 random int and bool values (9 dtypes, min/max, negatives) through infer with share_inputs on and off: 0 bit differences from ops.constant(x, dtype=Type.bf16).
    • flake8 (max-line-length 120) on data_dispatcher.py: clean.

Signed-off-by: Somay-kousis <somaykaush@gmail.com>
@Somay-kousis
Somay-kousis requested a review from a team as a code owner October 24, 2026 23:46
@github-actions github-actions Bot added the category: Python API OpenVINO Python bindings label Oct 24, 2026
@sys-openvino-ci sys-openvino-ci added the ExternalPR External contributor label Oct 24, 2026
@almilosz
almilosz requested a balanced review from Copilot October 26, 2026 08:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

1 open issue
What changed in this PR

Fixes incorrect bf16 input handling in Python bindings by ensuring integer/boolean NumPy arrays are converted to bf16 by value (matching ops.constant(..., dtype=bf16) behavior) rather than being reinterpreted as raw bf16 bits.

Changes:

  • Add _is_bf16_convertible() and use it to route int/bool (and non-float16 floats) through _to_bf16_view() for bf16 inputs.
  • Update bf16 conversion docstrings / typing stub to reflect the broader set of convertible dtypes.
  • Add regression tests for bf16 inputs from int/bool arrays (including share_inputs=True) and for rounding parity with ops.constant.
File Description
src/​bindings/​python/​tests/​test_runtime/​test_sync_infer_request.py Adds regression + rounding-parity tests for bf16 inputs from int/bool arrays.
src/​bindings/​python/​src/​openvino/​utils/​data_helpers/​data_dispatcher.pyi Updates stub docs/signatures to include _is_bf16_convertible and revised bf16 conversion description.
src/​bindings/​python/​src/​openvino/​utils/​data_helpers/​data_dispatcher.py Implements _is_bf16_convertible and applies it in both tensor creation and tensor update paths for bf16.

Review effort: Lite

Comment thread src/bindings/python/src/openvino/utils/data_helpers/data_dispatcher.pyi Outdated
Signed-off-by: Somay-kousis <somaykaush@gmail.com>
@p-wysocki

Copy link
Copy Markdown
Contributor

With this change, uint16/int16 arrays passed to a bf16 input are converted by value. On master with share_inputs=True (the default for compiled(x)), they are passed through as raw bf16 bits. For example, compiled(np.array([0x3F80], dtype=np.uint16)) returns 1.0 on master and with this PR the returned value is most likely different.

Could you keep raw-bits behavior for all 2-byte dtypes? This should also fix share_inputs=False, which currently gives garbage for uint16 bits. Please add int16/uint16 bit patterns to the tests, with both share_inputs values.

Signed-off-by: Somay-kousis <somaykaush@gmail.com>
@Somay-kousis

Copy link
Copy Markdown
Contributor Author

Thanks, you're right. Done in 937d2ec: float16, int16 and uint16 arrays are taken as raw bf16 bits on both paths now, and only the other bool/int/float arrays are converted by value. share_inputs=False with uint16/int16 bits now gives the right values too (master wrote them by value).

While at it, the raw-bits view also normalizes byte order, so >u2/>i2/>f2 give the same bits as native arrays (master byte-swapped them with share_inputs=True). 2-byte str/void/structured dtypes are left out and behave as on master.

New tests: test_infer_bf16_input_from_16bit_array_keeps_raw_bits (float16/int16/uint16 bit patterns for 1.0, -2.0, 0.0, inf; contiguous and strided; list and dict; both share_inputs) and ..._any_byte_order. I also compared every numeric dtype, plus str, bytes, void, complex, object and 0-d inputs, against master on both paths: only the cases above change.

@p-wysocki

Copy link
Copy Markdown
Contributor

build_jenkins

@p-wysocki p-wysocki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, a bit more test coverage and in my opinion we can proceed with merge.

assert np.array_equal(result[0], data.astype(np.float32))


@pytest.mark.parametrize("numpy_dtype", [np.float16, np.int16, np.uint16])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[random line] Please add a 2D / non-contiguous int case. The int tests only cover contiguous 1D data, and share_inputs=False has no to_c_style copy in front of the conversion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 8307de9: test_infer_bf16_input_from_2d_int_array, int32/int64/uint8 arrays of shape (4, 3) that are C-contiguous, transposed (Fortran order) and strided (every other column), with both share_inputs values. All 18 cases pass; with master's data_dispatcher.py all 18 fail.

Signed-off-by: Somay-kousis <somaykaush@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: Python API OpenVINO Python bindings ExternalPR External contributor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: [PyOV] Integer arrays to a bf16 model input get wrong values

4 participants