Repository navigation
[PyOV] Convert integer and bool arrays to a bf16 model input by value - #39183
Somay-kousis wants to merge 4 commits into
Conversation
Signed-off-by: Somay-kousis <somaykaush@gmail.com>
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
1 open issue
What changed in this PR
Fixes incorrect bf16 input handling in Python bindings by ensuring integer/boolean NumPy arrays are converted to bf16 by value (matching ops.constant(..., dtype=bf16) behavior) rather than being reinterpreted as raw bf16 bits.
Changes:
- Add
_is_bf16_convertible()and use it to route int/bool (and non-float16 floats) through_to_bf16_view()for bf16 inputs. - Update bf16 conversion docstrings / typing stub to reflect the broader set of convertible dtypes.
- Add regression tests for bf16 inputs from int/bool arrays (including
share_inputs=True) and for rounding parity withops.constant.
| File | Description |
|---|---|
| src/bindings/python/tests/test_runtime/test_sync_infer_request.py | Adds regression + rounding-parity tests for bf16 inputs from int/bool arrays. |
| src/bindings/python/src/openvino/utils/data_helpers/data_dispatcher.pyi | Updates stub docs/signatures to include _is_bf16_convertible and revised bf16 conversion description. |
| src/bindings/python/src/openvino/utils/data_helpers/data_dispatcher.py | Implements _is_bf16_convertible and applies it in both tensor creation and tensor update paths for bf16. |
Review effort: Lite
Signed-off-by: Somay-kousis <somaykaush@gmail.com>
|
With this change, uint16/int16 arrays passed to a bf16 input are converted by value. On master with share_inputs=True (the default for compiled(x)), they are passed through as raw bf16 bits. For example, Could you keep raw-bits behavior for all 2-byte dtypes? This should also fix |
Signed-off-by: Somay-kousis <somaykaush@gmail.com>
|
Thanks, you're right. Done in 937d2ec: float16, int16 and uint16 arrays are taken as raw bf16 bits on both paths now, and only the other bool/int/float arrays are converted by value. While at it, the raw-bits view also normalizes byte order, so New tests: |
|
build_jenkins |
p-wysocki
left a comment
There was a problem hiding this comment.
Overall LGTM, a bit more test coverage and in my opinion we can proceed with merge.
| assert np.array_equal(result[0], data.astype(np.float32)) | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("numpy_dtype", [np.float16, np.int16, np.uint16]) |
There was a problem hiding this comment.
[random line] Please add a 2D / non-contiguous int case. The int tests only cover contiguous 1D data, and share_inputs=False has no to_c_style copy in front of the conversion.
There was a problem hiding this comment.
Added in 8307de9: test_infer_bf16_input_from_2d_int_array, int32/int64/uint8 arrays of shape (4, 3) that are C-contiguous, transposed (Fortran order) and strided (every other column), with both share_inputs values. All 18 cases pass; with master's data_dispatcher.py all 18 fail.
Signed-off-by: Somay-kousis <somaykaush@gmail.com>

Details:
What a user sees, with
x = np.array([1, 2, 3, 0])fed to a bf16 model input:[1, 2, 3, 0]ValueError(broadcast) or wrong values[0.0078125, 2., 32., 0.], no error[1., 2., 3., 0.]Cause:
Tensor.dataof a bf16 tensor is a float16 view of the bits, and only float32/float64 arrays went through_to_bf16_view.update_tensorassigned ints into the float16 view by value (float16 bits then read as bf16), andvalue_to_tensordidvalue.view(float16).Fix in
data_dispatcher.py:_is_bf16_convertible(bool, int, uint and float arrays that are not 2 bytes wide) picks_to_bf16_viewon both paths. float16/int16/uint16 arrays (_is_bf16_bits) are taken as raw bits on both paths through_bf16_bits_view, which also normalizes the byte order; before,update_tensorassigned int16/uint16 by value, so bf16 data stored as uint16 broke withshare_inputs=False. Docstrings updated (the generated.pyistub is left as is).Rounding: ints go through float32 in
_to_bf16_view, which is also whatops.constant(x, dtype=Type.bf16)does, so the bits match it, including int64/uint64 values above 2**32.Not changed: 0-d arrays, numpy scalars and Python ints/lists into a bf16 input still raise (no wrong values).
New tests in
tests/test_runtime/test_sync_infer_request.py:test_infer_bf16_input_from_int_array: int8, uint8, int32, int64 and bool, list and dict inputs, share_inputs on and off.test_infer_bf16_input_from_int_array_rounds_like_constant: int32, int64 and uint64 values that need rounding get the same bits asops.constant(data, dtype=Type.bf16).test_infer_bf16_input_from_16bit_array_keeps_raw_bits: float16, int16 and uint16 bit patterns (1.0, -2.0, 0.0, inf), contiguous and strided, list and dict inputs, share_inputs on and off.test_infer_bf16_input_from_16bit_array_keeps_raw_bits_any_byte_order: the same bits as>u2,>i2,>f2.Tickets:
AI Assistance:
data_dispatcher.py:data_dispatcher.py: the 26 int/bool cases fail; of the 24 float16/int16/uint16 raw-bits cases, the 8 int16/uint16 ones with share_inputs=False fail (the rest already pass on master); 5 of the 6 big-endian cases fail.test_sync_infer_request.py,test_async_infer_request.py,test_compiled_model.py,tests/test_utils/test_data_dispatch.py,test_tensor_string.py: 845 passed, 7 skipped.inferwith share_inputs on and off: 0 bit differences fromops.constant(x, dtype=Type.bf16).data_dispatcher.py: clean.