[blas][rocblas] Support int8 inputs with float output in gemm_batch

[zjin-lcf]

Dependency

This is stacked on #761 and contains its commits. Only the last commit,
[blas][rocblas] Support int8 inputs with float output in gemm_batch, is new here; the
test-side tolerance work and the cancellation regression belong to that PR. Please review
#761 first, after which I will rebase this onto develop so the diff reduces to the rocBLAS
backend and one added regression.

Summary

rocBLAS reaches int8 inputs only with an int32 output and compute type, which takes int32
alpha and beta, whereas oneMath specifies a float output and float scalars for this
combination. This accumulates the products exactly in an int32 workspace and applies the
float scalars in a scaling kernel afterwards, covering the buffer strided, USM strided and
USM grouped entry points in both layouts.

• k is bounded so the int32 accumulator cannot wrap. Every int8 magnitude is at most 128,
  so 128 * 128 bounds a product and k above INT32_MAX / (128 * 128) reports
  unimplemented.
• Sizes whose workspace or kernel range would overflow are rejected before anything is
  allocated, using checked multiplication and addition rather than a check on the product.
• An ldc below the row count is rejected. rocBLAS rejects such a call itself, but only
  from its host task, which does not hold back the separately submitted scaling kernel, and
  the kernel would fold one column of C onto the next.
• Grouped scaling stays a single kernel submission regardless of group or batch count, by
  locating the group that owns each entry with a binary search over per-group metadata.
• The scaling kernels launch over a flat range, since HIP cannot map every large
  multi-dimensional range onto its grid. The added Int8Int8SinglePrecisionLargePrimeRange
  regression holds a large prime in n to cover that.

Verification

Built with a DPC++ build against ROCm 7.2.4 and run on an AMD Instinct MI300A (gfx942).

The GEMM batch suite passes in both dispatch modes and both layouts:

$ ./bin/test_main_blas_ct --gtest_filter="*GemmBatch*-*Int8*:*GemmBatchUsmTests.Complex*"
[  PASSED  ] 32 tests.
$ ./bin/test_main_blas_rt --gtest_filter="*GemmBatch*-*Int8*:*GemmBatchUsmTests.Complex*"
[  PASSED  ] 32 tests.

GemmBatchUsmTests.Complex* is excluded because it is flaky on this machine independently of
this change: it hangs under compile-time dispatch and dies with hipErrorIllegalAddress under
runtime dispatch on an unmodified develop checkout as well.

The int8 cases are excluded because they cannot execute in this environment. Any SYCL kernel
launched from a backend shared library trips an assertion in the DPC++ build I have:

Assertion `It != m_DeviceKernelInfoMap.end()' failed.
  at sycl/source/detail/program_manager/program_manager.cpp:1594

This is not specific to this change. The untouched Gerc row-major test fails identically,
since rocblas_level2.cpp also launches a parallel_for from the backend library. I would
appreciate a run of *Int8* in the batch suite on a machine with a released toolchain.

To cover the paths the unit tests cannot reach here, I exercised the backend directly from
standalone programs linked against libonemath_blas_rocblas.so, which do run on this device:

strided alpha and beta set   PASS
strided beta zero            PASS
strided alpha zero           PASS
strided both zero            PASS
grouped, one kernel total    PASS

negative strided size        PASS
strided ldc below m          PASS
workspace overflow           PASS
grouped ldc below m          PASS
oversized grouped range      PASS

These check the strided and grouped fallback against an exact integer reference for every
combination of zero and nonzero alpha and beta in both layouts, that grouped scaling submits
one kernel, and that the rejections above report invalid_argument.

Made with Cursor