Skip to content

[releease/2.12] Support for gfx1250#3322

Closed
rraminen wants to merge 5 commits into
ROCm:release/2.12from
rraminen:cherry-pick-gfx1250-rel2.12
Closed

[releease/2.12] Support for gfx1250#3322
rraminen wants to merge 5 commits into
ROCm:release/2.12from
rraminen:cherry-pick-gfx1250-rel2.12

Bug fix - Remove drop_seed, drop_offset as there are not used in the …

3ba5494
Select commit
Loading
Failed to load commit list.
ROCm Repo Management API / Jenkins failed Jun 17, 2026 in 39m 26s

Initialize/Build PyTorch: error in 'archiveArtifacts' step

Initialize / Build PyTorch / Shell Script

Error in sh step, with arguments #!/usr/bin/bash set -o pipefail ./build_pytorch.sh 2>&1 | tee build_pytorch.log .

script returned exit code 1
Build log
Build log truncated.

[2026-06-17T17:12:10.962Z] -- MKL libraries: /opt/conda/envs/py_3.12/lib/libmkl_intel_lp64.a;/opt/conda/envs/py_3.12/lib/libmkl_gnu_thread.a;/opt/conda/envs/py_3.12/lib/libmkl_core.a;-fopenmp;/usr/lib/x86_64-linux-gnu/libpthread.a;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/x86_64-linux-gnu/libdl.a
[2026-06-17T17:12:10.962Z] -- MKL include directory: /opt/conda/envs/py_3.12/include
[2026-06-17T17:12:10.962Z] -- MKL OpenMP type: GNU
[2026-06-17T17:12:10.962Z] -- MKL OpenMP library: -fopenmp
[2026-06-17T17:12:10.962Z] CMake Deprecation Warning at third_party/pthreadpool/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
[2026-06-17T17:12:10.962Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:10.962Z]   CMake.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:10.962Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:10.962Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] CMake Deprecation Warning at third_party/FXdiv/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
[2026-06-17T17:12:10.962Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:10.962Z]   CMake.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:10.962Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:10.962Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] -- The ASM compiler identification is GNU
[2026-06-17T17:12:10.962Z] -- Found assembler: /opt/cache/bin/cc
[2026-06-17T17:12:10.962Z] CMake Deprecation Warning at third_party/psimd/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
[2026-06-17T17:12:10.962Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:10.962Z]   CMake.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:10.962Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:10.962Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] CMake Deprecation Warning at third_party/FP16/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
[2026-06-17T17:12:10.962Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:10.962Z]   CMake.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:10.962Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:10.962Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] -- Brace yourself, we are building NNPACK
[2026-06-17T17:12:10.962Z] CMake Deprecation Warning at third_party/NNPACK/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
[2026-06-17T17:12:10.962Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:10.962Z]   CMake.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:10.962Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:10.962Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] -- Performing Test NNPACK_ARCH_IS_X86_32
[2026-06-17T17:12:10.962Z] -- Performing Test NNPACK_ARCH_IS_X86_32 - Failed
[2026-06-17T17:12:10.962Z] CMake Warning (dev) at third_party/NNPACK/CMakeLists.txt:110 (FIND_PACKAGE):
[2026-06-17T17:12:10.962Z]   Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
[2026-06-17T17:12:10.962Z]   are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
[2026-06-17T17:12:10.962Z]   the cmake_policy command to set the policy and suppress this warning.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] This warning is for project developers.  Use -Wno-dev to suppress it.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] -- Found PythonInterp: /opt/conda/envs/py_3.12/bin/python (found version "3.12.13")
[2026-06-17T17:12:10.962Z] -- NNPACK backend is x86-64
[2026-06-17T17:12:10.962Z] -- Downloading six (Python package) to /var/lib/jenkins/pytorch/build/confu-srcs/six (define PYTHON_SIX_SOURCE_DIR to avoid it)
[2026-06-17T17:12:10.962Z] CMake Deprecation Warning at CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
[2026-06-17T17:12:10.962Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:10.962Z]   CMake.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:10.962Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:10.962Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] CMake Warning (dev) at /opt/conda/envs/py_3.12/lib/python3.12/site-packages/cmake/data/share/cmake-3.31/Modules/ExternalProject/shared_internal_commands.cmake:1276 (message):
[2026-06-17T17:12:10.962Z]   The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
[2026-06-17T17:12:10.962Z]   not set.  The policy's OLD behavior will be used.  When using a URL
[2026-06-17T17:12:10.962Z]   download, the timestamps of extracted files should preferably be that of
[2026-06-17T17:12:10.962Z]   the time of extraction, otherwise code that depends on the extracted
[2026-06-17T17:12:10.962Z]   contents might not be rebuilt if the URL changes.  The OLD behavior
[2026-06-17T17:12:10.962Z]   preserves the timestamps from the archive instead, but this is usually not
[2026-06-17T17:12:10.962Z]   what you want.  Update your project to the NEW behavior or specify the
[2026-06-17T17:12:10.962Z]   DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
[2026-06-17T17:12:10.962Z]   robustness issue.
[2026-06-17T17:12:10.962Z] Call Stack (most recent call first):
[2026-06-17T17:12:10.962Z]   /opt/conda/envs/py_3.12/lib/python3.12/site-packages/cmake/data/share/cmake-3.31/Modules/ExternalProject.cmake:3041 (_ep_add_download_command)
[2026-06-17T17:12:10.962Z]   CMakeLists.txt:6 (ExternalProject_Add)
[2026-06-17T17:12:10.962Z] This warning is for project developers.  Use -Wno-dev to suppress it.
[2026-06-17T17:12:10.962Z] 
[2026-06-17T17:12:10.962Z] -- Configuring done (0.0s)
[2026-06-17T17:12:10.962Z] -- Generating done (0.0s)
[2026-06-17T17:12:10.962Z] -- Build files have been written to: /var/lib/jenkins/pytorch/build/confu-deps/six-download
[2026-06-17T17:12:10.962Z] [1/9] Creating directories for 'six'
[2026-06-17T17:12:12.831Z] [2/9] Performing download step (download, verify and extract) for 'six'
[2026-06-17T17:12:12.831Z] -- Downloading...
[2026-06-17T17:12:12.831Z]    dst='/var/lib/jenkins/pytorch/build/confu-deps/six-download/six-prefix/src/six-1.11.0.tar.gz'
[2026-06-17T17:12:12.831Z]    timeout='none'
[2026-06-17T17:12:12.831Z]    inactivity timeout='none'
[2026-06-17T17:12:12.831Z] -- Using src='https://pypi.python.org/packages/16/d8/bc6316cf98419719bd59c91742194c111b6f2e85abac88e496adefaf7afe/six-1.11.0.tar.gz'
[2026-06-17T17:12:12.831Z] -- [download 100% complete]
[2026-06-17T17:12:12.831Z] -- [download 23% complete]
[2026-06-17T17:12:12.831Z] -- [download 32% complete]
[2026-06-17T17:12:12.831Z] -- [download 37% complete]
[2026-06-17T17:12:12.831Z] -- [download 42% complete]
[2026-06-17T17:12:12.831Z] -- [download 46% complete]
[2026-06-17T17:12:12.831Z] -- [download 97% complete]
[2026-06-17T17:12:12.831Z] -- [download 100% complete]
[2026-06-17T17:12:12.831Z] -- verifying file...
[2026-06-17T17:12:12.831Z]        file='/var/lib/jenkins/pytorch/build/confu-deps/six-download/six-prefix/src/six-1.11.0.tar.gz'
[2026-06-17T17:12:12.831Z] -- Downloading... done
[2026-06-17T17:12:12.831Z] -- extracting...
[2026-06-17T17:12:12.831Z]      src='/var/lib/jenkins/pytorch/build/confu-deps/six-download/six-prefix/src/six-1.11.0.tar.gz'
[2026-06-17T17:12:12.831Z]      dst='/var/lib/jenkins/pytorch/build/confu-srcs/six'
[2026-06-17T17:12:12.831Z] -- extracting... [tar xfz]
[2026-06-17T17:12:12.831Z] -- extracting... [analysis]
[2026-06-17T17:12:12.831Z] -- extracting... [rename]
[2026-06-17T17:12:12.831Z] -- extracting... [clean up]
[2026-06-17T17:12:12.831Z] -- extracting... done
[2026-06-17T17:12:12.831Z] [3/9] No update step for 'six'
[2026-06-17T17:12:12.831Z] [4/9] No patch step for 'six'
[2026-06-17T17:12:12.831Z] [5/9] No configure step for 'six'
[2026-06-17T17:12:12.831Z] [6/9] No build step for 'six'
[2026-06-17T17:12:12.831Z] [7/9] No install step for 'six'
[2026-06-17T17:12:12.831Z] [8/9] No test step for 'six'
[2026-06-17T17:12:12.831Z] [9/9] Completed 'six'
[2026-06-17T17:12:12.831Z] -- Building for XNNPACK_TARGET_PROCESSOR: x86_64
[2026-06-17T17:12:12.831Z] -- Generating microkernels.cmake
[2026-06-17T17:12:12.831Z] Duplicate microkernel definition: src/qs8-qc4w-packw/gen/qs8-qc4w-packw-x8c8-gemm-goi-avxvnni.c and src/qs8-qc4w-packw/gen/qs8-qc4w-packw-x8c8-gemm-goi-scalar.c
[2026-06-17T17:12:12.832Z] Duplicate microkernel definition: src/qs8-qc4w-packw/gen/qs8-qc4w-packw-x8c8-gemm-goi-avxvnni.c and src/qs8-qc4w-packw/gen/qs8-qc4w-packw-x8c8-gemm-goi-avx256vnni.c (1th function)
[2026-06-17T17:12:12.832Z] No microkernel found in src/reference/unary-elementwise.cc
[2026-06-17T17:12:12.832Z] No microkernel found in src/reference/packing.cc
[2026-06-17T17:12:12.832Z] No microkernel found in src/reference/binary-elementwise.cc
[2026-06-17T17:12:13.276Z] -- LLVM FileCheck Found: /opt/rocm/llvm/bin/FileCheck
[2026-06-17T17:12:13.276Z] -- Found Git: /usr/bin/git (found version "2.54.0")
[2026-06-17T17:12:13.276Z] -- Google Benchmark version: v1.9.3, normalized to 1.9.3
[2026-06-17T17:12:13.276Z] -- Looking for shm_open in rt
[2026-06-17T17:12:13.276Z] -- Looking for shm_open in rt - found
[2026-06-17T17:12:13.276Z] -- Performing Test HAVE_CXX_FLAG_WALL
[2026-06-17T17:12:13.809Z] -- Performing Test HAVE_CXX_FLAG_WALL - Success
[2026-06-17T17:12:13.809Z] -- Performing Test HAVE_CXX_FLAG_WEXTRA
[2026-06-17T17:12:14.198Z] -- Performing Test HAVE_CXX_FLAG_WEXTRA - Success
[2026-06-17T17:12:14.198Z] -- Performing Test HAVE_CXX_FLAG_WSHADOW
[2026-06-17T17:12:14.606Z] -- Performing Test HAVE_CXX_FLAG_WSHADOW - Success
[2026-06-17T17:12:14.606Z] -- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL
[2026-06-17T17:12:15.023Z] -- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL - Success
[2026-06-17T17:12:15.023Z] -- Performing Test HAVE_CXX_FLAG_WOLD_STYLE_CAST
[2026-06-17T17:12:15.451Z] -- Performing Test HAVE_CXX_FLAG_WOLD_STYLE_CAST - Success
[2026-06-17T17:12:15.451Z] -- Performing Test HAVE_CXX_FLAG_WCONVERSION
[2026-06-17T17:12:15.874Z] -- Performing Test HAVE_CXX_FLAG_WCONVERSION - Success
[2026-06-17T17:12:15.874Z] -- Performing Test HAVE_CXX_FLAG_WERROR
[2026-06-17T17:12:16.618Z] -- Performing Test HAVE_CXX_FLAG_WERROR - Success
[2026-06-17T17:12:16.618Z] -- Performing Test HAVE_CXX_FLAG_WSUGGEST_OVERRIDE
[2026-06-17T17:12:16.618Z] -- Performing Test HAVE_CXX_FLAG_WSUGGEST_OVERRIDE - Success
[2026-06-17T17:12:16.618Z] -- Performing Test HAVE_CXX_FLAG_PEDANTIC
[2026-06-17T17:12:17.558Z] -- Performing Test HAVE_CXX_FLAG_PEDANTIC - Success
[2026-06-17T17:12:17.558Z] -- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS
[2026-06-17T17:12:17.976Z] -- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS - Success
[2026-06-17T17:12:17.976Z] -- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32
[2026-06-17T17:12:18.386Z] -- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 - Failed
[2026-06-17T17:12:18.386Z] -- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING
[2026-06-17T17:12:20.278Z] -- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING - Success
[2026-06-17T17:12:20.278Z] -- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS
[2026-06-17T17:12:21.439Z] -- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS - Success
[2026-06-17T17:12:21.439Z] -- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED
[2026-06-17T17:12:22.429Z] -- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED - Success
[2026-06-17T17:12:22.429Z] -- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING
[2026-06-17T17:12:23.950Z] -- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING - Success
[2026-06-17T17:12:23.950Z] -- Performing Test HAVE_CXX_FLAG_WD654
[2026-06-17T17:12:24.779Z] -- Performing Test HAVE_CXX_FLAG_WD654 - Failed
[2026-06-17T17:12:24.779Z] -- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY
[2026-06-17T17:12:25.213Z] -- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY - Failed
[2026-06-17T17:12:25.213Z] -- Performing Test HAVE_CXX_FLAG_COVERAGE
[2026-06-17T17:12:26.893Z] -- Performing Test HAVE_CXX_FLAG_COVERAGE - Success
[2026-06-17T17:12:26.893Z] -- Compiling and running to test HAVE_STD_REGEX
[2026-06-17T17:12:29.151Z] -- Performing Test HAVE_STD_REGEX -- success
[2026-06-17T17:12:29.151Z] -- Compiling and running to test HAVE_GNU_POSIX_REGEX
[2026-06-17T17:12:29.991Z] -- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
[2026-06-17T17:12:29.991Z] -- Compiling and running to test HAVE_POSIX_REGEX
[2026-06-17T17:12:30.820Z] -- Performing Test HAVE_POSIX_REGEX -- success
[2026-06-17T17:12:30.820Z] -- Compiling and running to test HAVE_STEADY_CLOCK
[2026-06-17T17:12:32.510Z] -- Performing Test HAVE_STEADY_CLOCK -- success
[2026-06-17T17:12:32.510Z] -- Compiling and running to test HAVE_PTHREAD_AFFINITY
[2026-06-17T17:12:33.391Z] -- Performing Test HAVE_PTHREAD_AFFINITY -- success
[2026-06-17T17:12:33.900Z] -- Found Numa: /usr/include
[2026-06-17T17:12:33.900Z] -- Found Numa  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so)
[2026-06-17T17:12:33.900Z] CMake Deprecation Warning at third_party/ittapi/CMakeLists.txt:7 (cmake_minimum_required):
[2026-06-17T17:12:33.900Z]   Compatibility with CMake < 3.10 will be removed from a future version of
[2026-06-17T17:12:33.900Z]   CMake.
[2026-06-17T17:12:33.900Z] 
[2026-06-17T17:12:33.900Z]   Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
[2026-06-17T17:12:33.900Z]   to tell CMake that the project requires at least <min> but has been updated
[2026-06-17T17:12:33.900Z]   to work with policies introduced by <max> or earlier.
[2026-06-17T17:12:33.900Z] 
[2026-06-17T17:12:33.900Z] 
[2026-06-17T17:12:34.282Z] -- Using third party subdirectory Eigen.
[2026-06-17T17:12:34.282Z] -- Found Python: /opt/conda/envs/py_3.12/bin/python (found version "3.12.13") found components: Interpreter Development.Module NumPy
[2026-06-17T17:12:34.282Z] -- Using third_party/pybind11.
[2026-06-17T17:12:34.282Z] -- pybind11 include dirs: /var/lib/jenkins/pytorch/cmake/../third_party/pybind11/include
[2026-06-17T17:12:35.120Z] -- Checking for module 'mpi-c'
[2026-06-17T17:12:35.120Z] --   Package 'mpi-c', required by 'virtual:world', not found
[2026-06-17T17:12:35.506Z] -- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS) 
[2026-06-17T17:12:36.823Z] -- Checking for module 'mpi-cxx'
[2026-06-17T17:12:36.823Z] --   Package 'mpi-cxx', required by 'virtual:world', not found
[2026-06-17T17:12:36.823Z] -- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS) 
[2026-06-17T17:12:36.823Z] -- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND) 
[2026-06-17T17:12:36.823Z] CMake Warning at cmake/Dependencies.cmake:904 (message):
[2026-06-17T17:12:36.823Z]   Not compiling with MPI.  Suppress this warning with -DUSE_MPI=OFF
[2026-06-17T17:12:36.823Z] Call Stack (most recent call first):
[2026-06-17T17:12:36.823Z]   CMakeLists.txt:947 (include)
[2026-06-17T17:12:36.823Z] 
[2026-06-17T17:12:36.823Z] 
[2026-06-17T17:12:36.823Z] -- Found OpenMP_C: -fopenmp
[2026-06-17T17:12:36.823Z] -- Found OpenMP_CXX: -fopenmp
[2026-06-17T17:12:36.823Z] -- Found OpenMP: TRUE
[2026-06-17T17:12:36.823Z] -- Adding OpenMP CXX_FLAGS: -fopenmp
[2026-06-17T17:12:36.823Z] -- Will link against OpenMP libraries: /opt/conda/envs/py_3.12/lib/libgomp.so
[2026-06-17T17:12:36.823Z] -- Found nvtx3: /var/lib/jenkins/pytorch/third_party/NVTX/c/include
[2026-06-17T17:12:36.823Z] Building PyTorch for GPU arch: gfx90a;gfx908;gfx942
[2026-06-17T17:12:36.823Z] -- Found HIP: /opt/rocm (found suitable version "7.2.53211-97f5574fe2", minimum required is "1.0")
[2026-06-17T17:12:36.823Z] HIP VERSION: 7.2.53211-97f5574fe2
[2026-06-17T17:12:37.677Z] -- The HIP compiler identification is Clang 22.0.0
[2026-06-17T17:12:37.677Z] -- Detecting HIP compiler ABI info
[2026-06-17T17:12:38.879Z] -- Detecting HIP compiler ABI info - done
[2026-06-17T17:12:38.879Z] -- Check for working HIP compiler: /opt/rocm/lib/llvm/bin/clang++ - skipped
[2026-06-17T17:12:38.879Z] -- Detecting HIP compile features
[2026-06-17T17:12:38.879Z] -- Detecting HIP compile features - done
[2026-06-17T17:12:38.879Z] hip VERSION: 7.2.53211
[2026-06-17T17:12:38.879Z] HIP version: 7.2.53211
[2026-06-17T17:12:38.879Z] -- Reading ROCM version from: /opt/rocm/include/rocm-core/rocm_version.h
[2026-06-17T17:12:38.879Z] -- Content: 
[2026-06-17T17:12:38.879Z] 
[2026-06-17T17:12:38.879Z] ***** ROCm version from rocm_version.h ****
[2026-06-17T17:12:38.879Z] 
[2026-06-17T17:12:38.879Z] ROCM_VERSION_DEV: 7.2.4
[2026-06-17T17:12:38.879Z] ROCM_VERSION_DEV_MAJOR: 7
[2026-06-17T17:12:38.879Z] ROCM_VERSION_DEV_MINOR: 2
[2026-06-17T17:12:38.879Z] ROCM_VERSION_DEV_PATCH: 4
[2026-06-17T17:12:38.879Z] ROCM_VERSION_DEV_INT:   70204
[2026-06-17T17:12:38.879Z] HIP_VERSION_MAJOR: 7
[2026-06-17T17:12:38.879Z] HIP_VERSION_MINOR: 2
[2026-06-17T17:12:38.879Z] TORCH_HIP_VERSION: 702
[2026-06-17T17:12:38.879Z] 
[2026-06-17T17:12:38.879Z] ***** Library versions from cmake find_package *****
[2026-06-17T17:12:38.879Z] 
[2026-06-17T17:12:38.879Z] amd_comgr VERSION: 3.0.0
[2026-06-17T17:12:38.879Z] rocrand VERSION: 4.2.0
[2026-06-17T17:12:38.879Z] hiprand VERSION: 3.1.0
[2026-06-17T17:12:38.879Z] rocblas VERSION: 5.2.0
[2026-06-17T17:12:38.879Z] hipblas VERSION: 3.2.0
[2026-06-17T17:12:38.879Z] miopen VERSION: 3.5.1
[2026-06-17T17:12:38.879Z] hipfft VERSION: 1.0.22
[2026-06-17T17:12:38.879Z] hipsparse VERSION: 4.2.0
[2026-06-17T17:12:38.879Z] rocprim VERSION: 4.2.0
[2026-06-17T17:12:38.879Z] hipcub VERSION: 4.2.0
[2026-06-17T17:12:38.879Z] rocthrust VERSION: 4.2.0
[2026-06-17T17:12:38.879Z] hipsolver VERSION: 3.2.0
[2026-06-17T17:12:38.879Z] rocsolver VERSION: 3.32.0
[2026-06-17T17:12:38.879Z] rocshmem VERSION: 3.3.0
[2026-06-17T17:12:38.879Z] hiprtc VERSION: 7.2.53211
[2026-06-17T17:12:38.879Z] hipblaslt VERSION: 1.2.2
[2026-06-17T17:12:38.879Z] rccl VERSION: 2.27.7
[2026-06-17T17:12:38.879Z] hsa-runtime64 VERSION: 1.18.70204
[2026-06-17T17:12:38.879Z] rocm_smi VERSION: 7.8.0.0-rocm-rel-7.2-93-97f5574fe2
[2026-06-17T17:12:38.879Z] hipsparselt VERSION: 0.2.6
[2026-06-17T17:12:39.822Z] hipblaslt is using scale pointer outer vec
[2026-06-17T17:12:39.822Z] INFOCompiling with HIP for AMD.
[2026-06-17T17:12:39.822Z] INFOForcing USE_SYSTEM_NCCL to ON since it's required by using RCCL
[2026-06-17T17:12:39.822Z] TORCH_HIP_VERSION=702 is added as a compiler defines
[2026-06-17T17:12:39.822Z] -- Disabling Kernel Assert for ROCm
[2026-06-17T17:12:39.822Z] -- RCCL Found!
[2026-06-17T17:12:39.822Z] CMake Warning at cmake/Dependencies.cmake:1152 (message):
[2026-06-17T17:12:39.822Z]   Not using XPU, so disabling USE_XCCL.  Suppress this warning with
[2026-06-17T17:12:39.822Z]   -DUSE_XCCL=OFF.
[2026-06-17T17:12:39.822Z] Call Stack (most recent call first):
[2026-06-17T17:12:39.822Z]   CMakeLists.txt:947 (include)
[2026-06-17T17:12:39.822Z] 
[2026-06-17T17:12:39.822Z] 
[2026-06-17T17:12:39.822Z] -- Performing Test UV_LINT_W4
[2026-06-17T17:12:39.822Z] -- Performing Test UV_LINT_W4 - Failed
[2026-06-17T17:12:39.822Z] -- Performing Test UV_LINT_NO_UNUSED_PARAMETER_MSVC
[2026-06-17T17:12:39.822Z] -- Performing Test UV_LINT_NO_UNUSED_PARAMETER_MSVC - Failed
[2026-06-17T17:12:39.822Z] -- Performing Test UV_LINT_NO_CONDITIONAL_CONSTANT_MSVC
[2026-06-17T17:12:40.242Z] -- Performing Test UV_LINT_NO_CONDITIONAL_CONSTANT_MSVC - Failed
[2026-06-17T17:12:40.242Z] -- Performing Test UV_LINT_NO_NONSTANDARD_MSVC
[2026-06-17T17:12:40.969Z] -- Performing Test UV_LINT_NO_NONSTANDARD_MSVC - Failed
[2026-06-17T17:12:40.969Z] -- Performing Test UV_LINT_NO_NONSTANDARD_EMPTY_TU_MSVC
[2026-06-17T17:12:42.345Z] -- Performing Test UV_LINT_NO_NONSTANDARD_EMPTY_TU_MSVC - Failed
[2026-06-17T17:12:42.345Z] -- Performing Test UV_LINT_NO_NONSTANDARD_FILE_SCOPE_MSVC
[2026-06-17T17:12:43.674Z] -- Performing Test UV_LINT_NO_NONSTANDARD_FILE_SCOPE_MSVC - Failed
[2026-06-17T17:12:43.674Z] -- Performing Test UV_LINT_NO_NONSTANDARD_NONSTATIC_DLIMPORT_MSVC
[2026-06-17T17:12:44.557Z] -- Performing Test UV_LINT_NO_NONSTANDARD_NONSTATIC_DLIMPORT_MSVC - Failed
[2026-06-17T17:12:44.557Z] -- Performing Test UV_LINT_NO_HIDES_LOCAL
[2026-06-17T17:12:44.965Z] -- Performing Test UV_LINT_NO_HIDES_LOCAL - Failed
[2026-06-17T17:12:44.965Z] -- Performing Test UV_LINT_NO_HIDES_PARAM
[2026-06-17T17:12:46.565Z] -- Performing Test UV_LINT_NO_HIDES_PARAM - Failed
[2026-06-17T17:12:46.565Z] -- Performing Test UV_LINT_NO_HIDES_GLOBAL
[2026-06-17T17:12:48.174Z] -- Performing Test UV_LINT_NO_HIDES_GLOBAL - Failed
[2026-06-17T17:12:48.174Z] -- Performing Test UV_LINT_NO_CONDITIONAL_ASSIGNMENT_MSVC
[2026-06-17T17:12:49.663Z] -- Performing Test UV_LINT_NO_CONDITIONAL_ASSIGNMENT_MSVC - Failed
[2026-06-17T17:12:49.663Z] -- Performing Test UV_LINT_NO_UNSAFE_MSVC
[2026-06-17T17:12:50.640Z] -- Performing Test UV_LINT_NO_UNSAFE_MSVC - Failed
[2026-06-17T17:12:50.640Z] -- Performing Test UV_LINT_WALL
[2026-06-17T17:12:51.534Z] -- Performing Test UV_LINT_WALL - Success
[2026-06-17T17:12:51.534Z] -- Performing Test UV_LINT_NO_UNUSED_PARAMETER
[2026-06-17T17:12:52.172Z] -- Performing Test UV_LINT_NO_UNUSED_PARAMETER - Success
[2026-06-17T17:12:52.172Z] -- Performing Test UV_LINT_STRICT_PROTOTYPES
[2026-06-17T17:12:52.596Z] -- Performing Test UV_LINT_STRICT_PROTOTYPES - Success
[2026-06-17T17:12:52.596Z] -- Performing Test UV_LINT_EXTRA
[2026-06-17T17:12:53.149Z] -- Performing Test UV_LINT_EXTRA - Success
[2026-06-17T17:12:53.149Z] -- Performing Test UV_LINT_UTF8_MSVC
[2026-06-17T17:12:53.980Z] -- Performing Test UV_LINT_UTF8_MSVC - Failed
[2026-06-17T17:12:53.980Z] -- Performing Test UV_F_STRICT_ALIASING
[2026-06-17T17:12:55.136Z] -- Performing Test UV_F_STRICT_ALIASING - Success
[2026-06-17T17:12:55.136Z] -- summary of build options:
[2026-06-17T17:12:55.136Z]     Install prefix:  /var/lib/jenkins/pytorch/torch
[2026-06-17T17:12:55.136Z]     Target system:   Linux
[2026-06-17T17:12:55.136Z]     Compiler:
[2026-06-17T17:12:55.136Z]       C compiler:    /opt/cache/bin/cc (GNU)
[2026-06-17T17:12:55.136Z]       CFLAGS:         
[2026-06-17T17:12:55.136Z] 
[2026-06-17T17:12:55.136Z] -- Found uv: 1.51.0 (found version "1.51.0") 
[2026-06-17T17:12:55.136Z] -- Performing Test HAS_WNO_MISSING_TEMPLATE_ARG_LIST_AFTER_TEMPLATE_KW
[2026-06-17T17:12:55.555Z] -- Performing Test HAS_WNO_MISSING_TEMPLATE_ARG_LIST_AFTER_TEMPLATE_KW - Failed
[2026-06-17T17:12:55.555Z] CMake Warning at cmake/Dependencies.cmake:1208 (message):
[2026-06-17T17:12:55.555Z]   TensorPipe doesn't yet support ROCm
[2026-06-17T17:12:55.555Z] Call Stack (most recent call first):
[2026-06-17T17:12:55.555Z]   CMakeLists.txt:947 (include)
[2026-06-17T17:12:55.555Z] 
[2026-06-17T17:12:55.555Z] 
[2026-06-17T17:12:55.555Z] -- Gloo build as SHARED library
[2026-06-17T17:12:55.555Z] -- Found HIP: /opt/rocm (found suitable version "7.2.53211", minimum required is "1.0")
[2026-06-17T17:12:55.555Z] HIP library name: amdhip64
[2026-06-17T17:12:55.555Z] -- Found Python3: /opt/conda/envs/py_3.12/bin/python3.12 (found version "3.12.13") found components: Interpreter
[2026-06-17T17:12:56.105Z] Successfully preprocessed all matching files.
[2026-06-17T17:12:56.986Z] -- GLOO_USE_TORCH_DTYPES : 1 /var/lib/jenkins/pytorch;/var/lib/jenkins/pytorch/build
[2026-06-17T17:12:56.986Z] -- Found Python3: /opt/conda/envs/py_3.12/bin/python (found version "3.12.13") found components: Interpreter
[2026-06-17T17:12:56.986Z] -- ONNX_PROTOC_EXECUTABLE: $<TARGET_FILE:protobuf::protoc>
[2026-06-17T17:12:56.986Z] -- Protobuf_VERSION: Protobuf_VERSION_NOTFOUND
[2026-06-17T17:12:56.986Z] Generated: /var/lib/jenkins/pytorch/build/third_party/onnx/onnx/onnx_onnx_torch-ml.proto
[2026-06-17T17:12:56.986Z] Generated: /var/lib/jenkins/pytorch/build/third_party/onnx/onnx/onnx-operators_onnx_torch-ml.proto
[2026-06-17T17:12:56.986Z] Generated: /var/lib/jenkins/pytorch/build/third_party/onnx/onnx/onnx-data_onnx_torch.proto
[2026-06-17T17:12:56.986Z] -- 
[2026-06-17T17:12:56.986Z] -- ******** Summary ********
[2026-06-17T17:12:56.986Z] --   CMake version                     : 3.31.6
[2026-06-17T17:12:56.986Z] --   CMake command                     : /opt/conda/envs/py_3.12/lib/python3.12/site-packages/cmake/data/bin/cmake
[2026-06-17T17:12:56.986Z] --   System                            : Linux
[2026-06-17T17:12:56.986Z] --   C++ compiler                      : /opt/cache/bin/c++
[2026-06-17T17:12:56.986Z] --   C++ compiler version              : 13.3.0
[2026-06-17T17:12:56.986Z] --   CXX flags                         :  -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -Wnon-virtual-dtor
[2026-06-17T17:12:56.986Z] --   Build type                        : Release
[2026-06-17T17:12:56.986Z] --   Compile definitions               : ROCM_VERSION=70204;TORCH_HIP_VERSION=702;USE_LAYERNORM_FAST_RECIPROCAL;ONNX_ML=1;ONNXIFI_ENABLE_EXT=1
[2026-06-17T17:12:56.986Z] --   CMAKE_PREFIX_PATH                 : /opt/conda/envs/py_3.12/lib/python3.12/site-packages;/opt/conda/envs/py_3.12;/opt/rocm
[2026-06-17T17:12:56.986Z] --   CMAKE_INSTALL_PREFIX              : /var/lib/jenkins/pytorch/torch
[2026-06-17T17:12:56.986Z] --   CMAKE_MODULE_PATH                 : /opt/rocm/lib/cmake/hip;/opt/rocm/lib/x86_64-linux-gnu/cmake/hip;/var/lib/jenkins/pytorch/cmake/Modules;/var/lib/jenkins/pytorch/cmake/public/../Modules_CUDA_fix
[2026-06-17T17:12:56.986Z] -- 
[2026-06-17T17:12:56.986Z] --   ONNX version                      : 1.18.0
[2026-06-17T17:12:56.986Z] --   ONNX NAMESPACE                    : onnx_torch
[2026-06-17T17:12:56.986Z] --   ONNX_USE_LITE_PROTO               : OFF
[2026-06-17T17:12:56.986Z] --   USE_PROTOBUF_SHARED_LIBS          : OFF
[2026-06-17T17:12:56.986Z] --   ONNX_DISABLE_EXCEPTIONS           : OFF
[2026-06-17T17:12:56.986Z] --   ONNX_DISABLE_STATIC_REGISTRATION  : OFF
[2026-06-17T17:12:56.986Z] --   ONNX_WERROR                       : OFF
[2026-06-17T17:12:56.986Z] --   ONNX_BUILD_TESTS                  : OFF
[2026-06-17T17:12:56.986Z] --   BUILD_SHARED_LIBS                 : OFF
[2026-06-17T17:12:56.986Z] -- 
[2026-06-17T17:12:56.986Z] --   Protobuf compiler                 : $<TARGET_FILE:protobuf::protoc>
[2026-06-17T17:12:56.986Z] --   Protobuf includes                 : 
[2026-06-17T17:12:56.986Z] --   Protobuf libraries                : 
[2026-06-17T17:12:56.986Z] --   ONNX_BUILD_PYTHON                 : OFF
[2026-06-17T17:12:56.986Z] -- Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor
[2026-06-17T17:12:56.986Z] -- Adding -DNDEBUG to compile flags
[2026-06-17T17:12:56.986Z] -- Checking prototype magma_get_sgeqrf_nb for MAGMA_V2
[2026-06-17T17:12:57.841Z] -- Checking prototype magma_get_sgeqrf_nb for MAGMA_V2 - False
[2026-06-17T17:12:57.841Z] -- Compiling with MAGMA support
[2026-06-17T17:12:57.841Z] -- MAGMA INCLUDE DIRECTORIES: /opt/rocm/magma/include
[2026-06-17T17:12:57.841Z] -- MAGMA LIBRARIES: /opt/rocm/magma/lib/libmagma.so
[2026-06-17T17:12:57.841Z] -- MAGMA V2 check: 0
[2026-06-17T17:12:57.841Z] -- No SVE support on this machine.
[2026-06-17T17:12:57.841Z] -- Looking for sbgemm_
[2026-06-17T17:12:59.491Z] -- Looking for sbgemm_ - not found
[2026-06-17T17:12:59.491Z] -- Looking for shgemm_
[2026-06-17T17:13:00.313Z] -- Looking for shgemm_ - not found
[2026-06-17T17:13:00.313Z] -- Found a library with LAPACK API (mkl).
[2026-06-17T17:13:00.313Z] disabling CUDA because NOT USE_CUDA is set
[2026-06-17T17:13:00.313Z] -- Will build oneDNN Graph
[2026-06-17T17:13:00.313Z] -- Will build oneDNN UKERNEL
[2026-06-17T17:13:00.313Z] -- MKLDNN_CPU_RUNTIME = OMP
[2026-06-17T17:13:00.313Z] -- DNNL_TARGET_ARCH: X64
[2026-06-17T17:13:00.313Z] -- DNNL_LIBRARY_NAME: dnnl
[2026-06-17T17:13:00.313Z] CMake Warning (dev) at /opt/conda/envs/py_3.12/lib/python3.12/site-packages/cmake/data/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:441 (message):
[2026-06-17T17:13:00.313Z]   The package name passed to `find_package_handle_standard_args` (OpenMP_C)
[2026-06-17T17:13:00.313Z]   does not match the name of the calling package (OpenMP).  This can lead to
[2026-06-17T17:13:00.313Z]   problems in calling code that expects `find_package` result variables
[2026-06-17T17:13:00.313Z]   (e.g., `_FOUND`) to follow a certain pattern.
[2026-06-17T17:13:00.313Z] Call Stack (most recent call first):
[2026-06-17T17:13:00.313Z]   cmake/Modules/FindOpenMP.cmake:635 (find_package_handle_standard_args)
[2026-06-17T17:13:00.313Z]   third_party/ideep/mkl-dnn/cmake/OpenMP.cmake:41 (find_package)
[2026-06-17T17:13:00.313Z]   third_party/ideep/mkl-dnn/CMakeLists.txt:94 (include)
[2026-06-17T17:13:00.313Z] This warning is for project developers.  Use -Wno-dev to suppress it.
[2026-06-17T17:13:00.313Z] 
[2026-06-17T17:13:00.313Z] -- Found OpenMP_C: -fopenmp
[2026-06-17T17:13:00.313Z] CMake Warning (dev) at /opt/conda/envs/py_3.12/lib/python3.12/site-packages/cmake/data/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:441 (message):
[2026-06-17T17:13:00.314Z]   The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
[2026-06-17T17:13:00.314Z]   does not match the name of the calling package (OpenMP).  This can lead to
[2026-06-17T17:13:00.314Z]   problems in calling code that expects `find_package` result variables
[2026-06-17T17:13:00.314Z]   (e.g., `_FOUND`) to follow a certain pattern.
[2026-06-17T17:13:00.314Z] Call Stack (most recent call first):
[2026-06-17T17:13:00.314Z]   cmake/Modules/FindOpenMP.cmake:635 (find_package_handle_standard_args)
[2026-06-17T17:13:00.314Z]   third_party/ideep/mkl-dnn/cmake/OpenMP.cmake:41 (find_package)
[2026-06-17T17:13:00.314Z]   third_party/ideep/mkl-dnn/CMakeLists.txt:94 (include)
[2026-06-17T17:13:00.314Z] This warning is for project developers.  Use -Wno-dev to suppress it.
[2026-06-17T17:13:00.314Z] 
[2026-06-17T17:13:00.314Z] -- Found OpenMP_CXX: -fopenmp
[2026-06-17T17:13:00.314Z] -- Enabled testing coverage: CI
[2026-06-17T17:13:00.314Z] -- Enabled workload: TRAINING
[2026-06-17T17:13:00.314Z] -- Enabled primitives: ALL
[2026-06-17T17:13:00.314Z] -- Enabled primitive CPU ISA: ALL
[2026-06-17T17:13:00.314Z] -- Enabled primitive GPU ISA: ALL
[2026-06-17T17:13:00.314Z] -- Enabled GeMM kernels ISA: ALL
[2026-06-17T17:13:00.314Z] -- Primitive cache is enabled
[2026-06-17T17:13:00.314Z] -- Experimental functionality for ukernels is enabled
[2026-06-17T17:13:00.314Z] -- Graph component is enabled
[2026-06-17T17:13:00.314Z] -- Found MKL-DNN: TRUE
[2026-06-17T17:13:00.314Z] -- Looking for clock_gettime in rt
[2026-06-17T17:13:01.293Z] -- Looking for clock_gettime in rt - found
[2026-06-17T17:13:01.293Z] -- Looking for mmap
[2026-06-17T17:13:01.845Z] -- Looking for mmap - found
[2026-06-17T17:13:01.845Z] -- Looking for shm_open
[2026-06-17T17:13:02.244Z] -- Looking for shm_open - found
[2026-06-17T17:13:02.244Z] -- Looking for shm_unlink
[2026-06-17T17:13:02.244Z] -- Looking for shm_unlink - found
[2026-06-17T17:13:02.244Z] -- Looking for malloc_usable_size
[2026-06-17T17:13:03.423Z] -- Looking for malloc_usable_size - found
[2026-06-17T17:13:03.423Z] -- Looking for posix_fallocate
[2026-06-17T17:13:04.313Z] -- Looking for posix_fallocate - found
[2026-06-17T17:13:04.313Z] -- <FindZVECTOR>
[2026-06-17T17:13:04.313Z] -- Performing Test COMPILE_OUT_ZVECTOR
[2026-06-17T17:13:04.737Z] -- Performing Test COMPILE_OUT_ZVECTOR - Failed
[2026-06-17T17:13:04.737Z] -- ZVECTOR flags were NOT set.
[2026-06-17T17:13:04.737Z] -- </FindZVECTOR>
[2026-06-17T17:13:04.737Z] -- {fmt} version: 12.1.0
[2026-06-17T17:13:04.737Z] -- Build type: Release
[2026-06-17T17:13:04.737Z] -- Using Kineto with Roctracer support
[2026-06-17T17:13:04.737Z] -- Configuring Kineto dependency:
[2026-06-17T17:13:04.737Z] --   KINETO_SOURCE_DIR = /var/lib/jenkins/pytorch/third_party/kineto/libkineto
[2026-06-17T17:13:04.737Z] --   KINETO_BUILD_TESTS = OFF
[2026-06-17T17:13:04.737Z] --   KINETO_LIBRARY_TYPE = static
[2026-06-17T17:13:04.737Z] --  ROCM_SOURCE_DIR = /opt/rocm
[2026-06-17T17:13:04.737Z] -- Could not find nvcc, please set CUDAToolkit_ROOT.
[2026-06-17T17:13:04.737Z] -- Reading ROCM version from: /opt/rocm/include/rocm-core/rocm_version.h
[2026-06-17T17:13:04.737Z] -- ROCM major: 7
[2026-06-17T17:13:04.737Z] -- ROCM minor: 2
[2026-06-17T17:13:04.737Z] -- ROCM patch: 4
[2026-06-17T17:13:04.737Z] -- Building with: rocprofiler-sdk
[2026-06-17T17:13:04.737Z] --  Building with rocprofiler-sdk
[2026-06-17T17:13:04.737Z] --  ROCTRACER_INCLUDE_DIR = /opt/rocm/include/roctracer
[2026-06-17T17:13:04.737Z] --  DYNOLOG_INCLUDE_DIR = /var/lib/jenkins/pytorch/third_party/kineto/libkineto/third_party/dynolog/
[2026-06-17T17:13:04.737Z] --  IPCFABRIC_INCLUDE_DIR = /var/lib/jenkins/pytorch/third_party/kineto/libkineto/third_party/dynolog//dynolog/src/ipcfabric/
[2026-06-17T17:13:04.737Z] -- Configured Kineto
[2026-06-17T17:13:04.737Z] CMake Warning at CMakeLists.txt:963 (message):
[2026-06-17T17:13:04.737Z]   Setting USE_FBGEMM_GENAI for gfx942/gfx950 to ON by default, doing ROCM
[2026-06-17T17:13:04.737Z]   build
[2026-06-17T17:13:04.737Z] 
[2026-06-17T17:13:04.737Z] 
[2026-06-17T17:13:04.737Z] CMake Error at CMakeLists.txt:972 (cmake_dependent_option):
[2026-06-17T17:13:04.737Z]   cmake_dependent_option Macro invoked with incorrect arguments for macro
[2026-06-17T17:13:04.737Z]   named: CMAKE_DEPENDENT_OPTION
[2026-06-17T17:13:04.737Z] 
[2026-06-17T17:13:04.737Z] 
[2026-06-17T17:13:04.737Z] -- Using AOTriton Runtime from pre-compiled binary https://github.com/ROCm/aotriton/releases/download/0.11.2b/aotriton-0.11.2b-manylinux_2_28_x86_64-rocm7.2-shared.tar.gz.    Set env variables AOTRITON_INSTALL_FROM_SOURCE=1 to build from source.
[2026-06-17T17:13:04.737Z] -- Download AOTriton pre-compiled GPU images from https://github.com/ROCm/aotriton/releases/download/0.11.2b/aotriton-0.11.2b-images-amd-gfx90a.tar.gz.
[2026-06-17T17:13:04.737Z] -- Download AOTriton pre-compiled GPU images from https://github.com/ROCm/aotriton/releases/download/0.11.2b/aotriton-0.11.2b-images-amd-gfx942.tar.gz.
[2026-06-17T17:13:04.737Z] -- GCC 13.3.0: Adding gcc and gcc_s libs to link line
[2026-06-17T17:13:04.737Z] -- Performing Test HAS_WERROR_RETURN_TYPE
[2026-06-17T17:13:05.149Z] -- Performing Test HAS_WERROR_RETURN_TYPE - Success
[2026-06-17T17:13:05.149Z] -- Performing Test HAS_WERROR_NON_VIRTUAL_DTOR
[2026-06-17T17:13:06.227Z] -- Performing Test HAS_WERROR_NON_VIRTUAL_DTOR - Success
[2026-06-17T17:13:06.227Z] -- Performing Test HAS_WERROR_BRACED_SCALAR_INIT
[2026-06-17T17:13:06.227Z] -- Performing Test HAS_WERROR_BRACED_SCALAR_INIT - Failed
[2026-06-17T17:13:06.227Z] -- Performing Test HAS_WERROR_RANGE_LOOP_CONSTRUCT
[2026-06-17T17:13:07.053Z] -- Performing Test HAS_WERROR_RANGE_LOOP_CONSTRUCT - Success
[2026-06-17T17:13:07.053Z] -- Performing Test HAS_WERROR_BOOL_OPERATION
[2026-06-17T17:13:08.868Z] -- Performing Test HAS_WERROR_BOOL_OPERATION - Success
[2026-06-17T17:13:08.868Z] -- Performing Test HAS_WNARROWING
[2026-06-17T17:13:09.698Z] -- Performing Test HAS_WNARROWING - Success
[2026-06-17T17:13:09.698Z] -- Performing Test HAS_WNO_MISSING_FIELD_INITIALIZERS
[2026-06-17T17:13:10.531Z] -- Performing Test HAS_WNO_MISSING_FIELD_INITIALIZERS - Success
[2026-06-17T17:13:10.531Z] -- Performing Test HAS_WNO_UNKNOWN_PRAGMAS
[2026-06-17T17:13:11.874Z] -- Performing Test HAS_WNO_UNKNOWN_PRAGMAS - Success
[2026-06-17T17:13:11.874Z] -- Performing Test HAS_WNO_UNUSED_PARAMETER
[2026-06-17T17:13:13.207Z] -- Performing Test HAS_WNO_UNUSED_PARAMETER - Success
[2026-06-17T17:13:13.207Z] -- Performing Test HAS_WNO_STRICT_OVERFLOW
[2026-06-17T17:13:13.871Z] -- Performing Test HAS_WNO_STRICT_OVERFLOW - Success
[2026-06-17T17:13:13.871Z] -- Performing Test HAS_WNO_STRICT_ALIASING
[2026-06-17T17:13:14.374Z] -- Performing Test HAS_WNO_STRICT_ALIASING - Success
[2026-06-17T17:13:14.374Z] -- Performing Test HAS_WNO_STRINGOP_OVERFLOW
[2026-06-17T17:13:15.175Z] -- Performing Test HAS_WNO_STRINGOP_OVERFLOW - Success
[2026-06-17T17:13:15.176Z] -- Performing Test HAS_WVLA_EXTENSION
[2026-06-17T17:13:15.176Z] -- Performing Test HAS_WVLA_EXTENSION - Failed
[2026-06-17T17:13:15.176Z] -- Performing Test HAS_WSUGGEST_OVERRIDE
[2026-06-17T17:13:16.030Z] -- Performing Test HAS_WSUGGEST_OVERRIDE - Success
[2026-06-17T17:13:16.030Z] -- Performing Test HAS_WNEWLINE_EOF
[2026-06-17T17:13:16.425Z] -- Performing Test HAS_WNEWLINE_EOF - Failed
[2026-06-17T17:13:16.425Z] -- Performing Test HAS_WINCONSISTENT_MISSING_OVERRIDE
[2026-06-17T17:13:16.800Z] -- Performing Test HAS_WINCONSISTENT_MISSING_OVERRIDE - Failed
[2026-06-17T17:13:16.800Z] -- Performing Test HAS_WINCONSISTENT_MISSING_DESTRUCTOR_OVERRIDE
[2026-06-17T17:13:17.408Z] -- Performing Test HAS_WINCONSISTENT_MISSING_DESTRUCTOR_OVERRIDE - Failed
[2026-06-17T17:13:17.408Z] -- Performing Test HAS_WNO_ERROR_OLD_STYLE_CAST
[2026-06-17T17:13:18.281Z] -- Performing Test HAS_WNO_ERROR_OLD_STYLE_CAST - Success
[2026-06-17T17:13:18.281Z] -- Performing Test HAS_WCONSTANT_CONVERSION
[2026-06-17T17:13:18.281Z] -- Performing Test HAS_WCONSTANT_CONVERSION - Failed
[2026-06-17T17:13:18.281Z] -- Performing Test HAS_WNO_ALIGNED_ALLOCATION_UNAVAILABLE
[2026-06-17T17:13:18.840Z] -- Performing Test HAS_WNO_ALIGNED_ALLOCATION_UNAVAILABLE - Failed
[2026-06-17T17:13:18.840Z] -- Performing Test HAS_QUNUSED_ARGUMENTS
[2026-06-17T17:13:19.236Z] -- Performing Test HAS_QUNUSED_ARGUMENTS - Failed
[2026-06-17T17:13:19.236Z] -- Performing Test HAS_FALIGNED_NEW
[2026-06-17T17:13:20.926Z] -- Performing Test HAS_FALIGNED_NEW - Success
[2026-06-17T17:13:20.926Z] -- Performing Test HAS_WNO_MAYBE_UNINITIALIZED
[2026-06-17T17:13:21.299Z] -- Performing Test HAS_WNO_MAYBE_UNINITIALIZED - Success
[2026-06-17T17:13:21.300Z] -- Performing Test HAS_FSTANDALONE_DEBUG
[2026-06-17T17:13:21.300Z] -- Performing Test HAS_FSTANDALONE_DEBUG - Failed
[2026-06-17T17:13:21.300Z] -- Performing Test HAS_FNO_MATH_ERRNO
[2026-06-17T17:13:23.151Z] -- Performing Test HAS_FNO_MATH_ERRNO - Success
[2026-06-17T17:13:23.151Z] -- Performing Test HAS_FNO_TRAPPING_MATH
[2026-06-17T17:13:23.579Z] -- Performing Test HAS_FNO_TRAPPING_MATH - Success
[2026-06-17T17:13:23.579Z] -- Performing Test HAS_WERROR_FORMAT
[2026-06-17T17:13:25.223Z] -- Performing Test HAS_WERROR_FORMAT - Success
[2026-06-17T17:13:25.223Z] -- Performing Test HAS_WNO_DANGLING_REFERENCE
[2026-06-17T17:13:25.652Z] -- Performing Test HAS_WNO_DANGLING_REFERENCE - Success
[2026-06-17T17:13:25.652Z] -- Performing Test HAS_WNO_ERROR_DANGLING_REFERENCE
[2026-06-17T17:13:26.080Z] -- Performing Test HAS_WNO_ERROR_DANGLING_REFERENCE - Success
[2026-06-17T17:13:26.080Z] -- NUMA paths:
[2026-06-17T17:13:26.080Z] -- /usr/include
[2026-06-17T17:13:26.080Z] -- /usr/lib/x86_64-linux-gnu/libnuma.so
[2026-06-17T17:13:26.080Z] -- Looking for backtrace
[2026-06-17T17:13:27.239Z] -- Looking for backtrace - found
[2026-06-17T17:13:27.239Z] -- backtrace facility detected in default set of libraries
[2026-06-17T17:13:27.239Z] -- Found Backtrace: /usr/include
[2026-06-17T17:13:29.855Z] -- headers outputs: 
[2026-06-17T17:13:30.342Z] -- sources outputs: 
[2026-06-17T17:13:31.241Z] -- declarations_yaml outputs: 
[2026-06-17T17:13:31.241Z] -- Performing Test COMPILER_SUPPORTS_NO_AVX256_SPLIT
[2026-06-17T17:13:32.043Z] -- Performing Test COMPILER_SUPPORTS_NO_AVX256_SPLIT - Success
[2026-06-17T17:13:32.043Z] -- Using ATen parallel backend: OMP
[2026-06-17T17:13:32.043Z] Building PyTorch for GPU arch: gfx90a;gfx908;gfx942
[2026-06-17T17:13:32.043Z] HIP VERSION: 7.2.53211
[2026-06-17T17:13:32.043Z] -- Reading ROCM version from: /opt/rocm/include/rocm-core/rocm_version.h
[2026-06-17T17:13:32.043Z] -- Content: //Copyright �� Advanced Micro Devices, Inc., or its affiliates.
[2026-06-17T17:13:32.043Z] //SPDX-License-Identifier: MIT
[2026-06-17T17:13:32.043Z] 
[2026-06-17T17:13:32.043Z] #ifndef _ROCM_VERSION_H_
[2026-06-17T17:13:32.043Z] #define _ROCM_VERSION_H_
[2026-06-17T17:13:32.043Z] 
[2026-06-17T17:13:32.043Z] 
[2026-06-17T17:13:32.043Z] #ifdef __cplusplus
[2026-06-17T17:13:32.043Z] extern "C" {
[2026-06-17T17:13:32.043Z] #endif  // __cplusplus
[2026-06-17T17:13:32.043Z] 
[2026-06-17T17:13:32.043Z] 
[2026-06-17T17:13:32.043Z] #if defined(_WIN32) || defined(__CYGWIN__)
[2026-06-17T17:13:32.043Z] #define LIB_API_PUBLIC __declspec(dllexport)
[2026-06-17T17:13:32.043Z] #define ATTRIBUTE_NON_NULL
[2026-06-17T17:13:32.043Z] #else
[2026-06-17T17:13:32.043Z] #define LIB_API_PUBLIC __attribute__((visibility("default")))
[2026-06-17T17:13:32.043Z] #define ATTRIBUTE_NON_NULL __attribute__((nonnull))
[2026-06-17T17:13:32.043Z] #endif
[2026-06-17T17:13:32.043Z] 
[2026-06-17T17:13:32.043Z] #define ROCM_VERSION_MAJOR   7
[2026-06-17T17:13:32.044Z] #define ROCM_VERSION_MINOR   2
[2026-06-17T17:13:32.044Z] #define ROCM_VERSION_PATCH   4
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] #define ROCM_BUILD_INFO	     "7.2.4.0-93-97f5574fe2"
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] typedef enum {
[2026-06-17T17:13:32.044Z] 	VerSuccess=0,
[2026-06-17T17:13:32.044Z] 	VerIncorrectParameters,
[2026-06-17T17:13:32.044Z] 	VerMemoryAllocationFailed,
[2026-06-17T17:13:32.044Z] 	VerValuesNotDefined,
[2026-06-17T17:13:32.044Z] 	VerErrorMAX		//This should always be last value in the enumerations
[2026-06-17T17:13:32.044Z] } VerErrors;
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] //  API for getting the version
[2026-06-17T17:13:32.044Z] //  Return val :  VerErros : API execution status.  The parameters are valid only when the execution status is SUCCESS==0
[2026-06-17T17:13:32.044Z] LIB_API_PUBLIC VerErrors getROCmVersion(unsigned int* Major, unsigned int* Minor, unsigned int* Patch) ATTRIBUTE_NON_NULL ;
[2026-06-17T17:13:32.044Z] //  Usage :
[2026-06-17T17:13:32.044Z] //  int mj=0,mn=0,p=0,ret=0;
[2026-06-17T17:13:32.044Z] //  ret=getROCMVersion(&mj,&mn,&p);
[2026-06-17T17:13:32.044Z] //  if(ret !=VerSuccess )  // error occured
[2026-06-17T17:13:32.044Z] //
[2026-06-17T17:13:32.044Z] //  check for the values and
[2026-06-17T17:13:32.044Z] //
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] //API for building build info on console
[2026-06-17T17:13:32.044Z] LIB_API_PUBLIC VerErrors printBuildInfo();
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] #ifdef __cplusplus
[2026-06-17T17:13:32.044Z] }  // end extern "C" block
[2026-06-17T17:13:32.044Z] #endif
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] #endif  //_ROCM_VERSION_H_  header guard
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] hip VERSION: 7.2.53211
[2026-06-17T17:13:32.044Z] HIP version: 7.2.53211
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] ***** ROCm version from rocm_version.h ****
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] ROCM_VERSION_DEV: 7.2.4
[2026-06-17T17:13:32.044Z] ROCM_VERSION_DEV_MAJOR: 7
[2026-06-17T17:13:32.044Z] ROCM_VERSION_DEV_MINOR: 2
[2026-06-17T17:13:32.044Z] ROCM_VERSION_DEV_PATCH: 4
[2026-06-17T17:13:32.044Z] ROCM_VERSION_DEV_INT:   70204
[2026-06-17T17:13:32.044Z] HIP_VERSION_MAJOR: 7
[2026-06-17T17:13:32.044Z] HIP_VERSION_MINOR: 2
[2026-06-17T17:13:32.044Z] TORCH_HIP_VERSION: 702
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] ***** Library versions from cmake find_package *****
[2026-06-17T17:13:32.044Z] 
[2026-06-17T17:13:32.044Z] amd_comgr VERSION: 3.0.0
[2026-06-17T17:13:32.044Z] rocrand VERSION: 4.2.0
[2026-06-17T17:13:32.044Z] hiprand VERSION: 3.1.0
[2026-06-17T17:13:32.044Z] rocblas VERSION: 5.2.0
[2026-06-17T17:13:32.044Z] hipblas VERSION: 3.2.0
[2026-06-17T17:13:32.044Z] miopen VERSION: 3.5.1
[2026-06-17T17:13:32.044Z] hipfft VERSION: 1.0.22
[2026-06-17T17:13:32.044Z] hipsparse VERSION: 4.2.0
[2026-06-17T17:13:32.044Z] rocprim VERSION: 4.2.0
[2026-06-17T17:13:32.044Z] hipcub VERSION: 4.2.0
[2026-06-17T17:13:32.044Z] rocthrust VERSION: 4.2.0
[2026-06-17T17:13:32.044Z] hipsolver VERSION: 3.2.0
[2026-06-17T17:13:32.044Z] rocsolver VERSION: 3.32.0
[2026-06-17T17:13:32.044Z] rocshmem VERSION: 3.3.0
[2026-06-17T17:13:32.044Z] hiprtc VERSION: 7.2.53211
[2026-06-17T17:13:32.044Z] hipblaslt VERSION: 1.2.2
[2026-06-17T17:13:32.044Z] rccl VERSION: 2.27.7
[2026-06-17T17:13:32.044Z] hsa-runtime64 VERSION: 1.18.70204
[2026-06-17T17:13:32.044Z] rocm_smi VERSION: 7.8.0.0-rocm-rel-7.2-93-97f5574fe2
[2026-06-17T17:13:32.044Z] hipsparselt VERSION: 0.2.6
[2026-06-17T17:13:34.646Z] hipblaslt is using scale pointer outer vec
[2026-06-17T17:13:34.646Z] ROCm is enabled.
[2026-06-17T17:13:34.646Z] -- Found OpenSSL: /opt/conda/envs/py_3.12/lib/libcrypto.so (found version "3.6.3")
[2026-06-17T17:13:34.646Z] -- Check size of long double
[2026-06-17T17:13:36.050Z] -- Check size of long double - done
[2026-06-17T17:13:36.050Z] -- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE
[2026-06-17T17:13:36.457Z] -- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE - Success
[2026-06-17T17:13:36.457Z] -- Performing Test COMPILER_SUPPORTS_FLOAT128
[2026-06-17T17:13:37.258Z] -- Performing Test COMPILER_SUPPORTS_FLOAT128 - Success
[2026-06-17T17:13:37.258Z] -- Performing Test COMPILER_SUPPORTS_QUADMATH
[2026-06-17T17:13:38.089Z] -- Performing Test COMPILER_SUPPORTS_QUADMATH - Success
[2026-06-17T17:13:38.089Z] -- Performing Test COMPILER_SUPPORTS_SSE2
[2026-06-17T17:13:39.453Z] -- Performing Test COMPILER_SUPPORTS_SSE2 - Success
[2026-06-17T17:13:39.453Z] -- Performing Test COMPILER_SUPPORTS_SSE4
[2026-06-17T17:13:42.312Z] -- Performing Test COMPILER_SUPPORTS_SSE4 - Success
[2026-06-17T17:13:42.312Z] -- Performing Test COMPILER_SUPPORTS_AVX
[2026-06-17T17:13:45.887Z] -- Performing Test COMPILER_SUPPORTS_AVX - Success
[2026-06-17T17:13:45.887Z] -- Performing Test COMPILER_SUPPORTS_FMA4
[2026-06-17T17:13:48.457Z] -- Performing Test COMPILER_SUPPORTS_FMA4 - Success
[2026-06-17T17:13:48.457Z] -- Performing Test COMPILER_SUPPORTS_AVX2
[2026-06-17T17:13:51.679Z] -- Performing Test COMPILER_SUPPORTS_AVX2 - Success
[2026-06-17T17:13:51.679Z] -- Performing Test COMPILER_SUPPORTS_AVX512F
[2026-06-17T17:13:54.304Z] -- Performing Test COMPILER_SUPPORTS_AVX512F - Success
[2026-06-17T17:13:54.304Z] -- Support for OpenMP disabled by CMake option
[2026-06-17T17:13:54.304Z] -- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES
[2026-06-17T17:13:55.407Z] -- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES - Success
[2026-06-17T17:13:55.407Z] -- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH
[2026-06-17T17:13:56.916Z] -- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH - Success
[2026-06-17T17:13:56.916Z] -- Performing Test COMPILER_SUPPORTS_SYS_GETRANDOM
[2026-06-17T17:13:58.211Z] -- Performing Test COMPILER_SUPPORTS_SYS_GETRANDOM - Success
[2026-06-17T17:13:58.211Z] -- Configuring build for SLEEF-v3.8.0
[2026-06-17T17:13:58.211Z]    Target system: Linux-6.8.0-124-generic
[2026-06-17T17:13:58.211Z]    Target processor: x86_64
[2026-06-17T17:13:58.211Z]    Host system: Linux-6.8.0-124-generic
[2026-06-17T17:13:58.211Z]    Host processor: x86_64
[2026-06-17T17:13:58.211Z]    Detected C compiler: GNU @ /opt/cache/bin/cc
[2026-06-17T17:13:58.211Z]    CMake: 3.31.6
[2026-06-17T17:13:58.211Z]    Make program: /opt/conda/envs/py_3.12/bin/ninja
[2026-06-17T17:13:58.211Z] -- Using option `-Wall -Wno-unused-function -Wno-attributes -Wno-unused-result -Wno-psabi -ffp-contract=off -fno-math-errno -fno-trapping-math` to compile libsleef
[2026-06-17T17:13:58.211Z] -- Building shared libs : OFF
[2026-06-17T17:13:58.211Z] -- Building static test bins: OFF
[2026-06-17T17:13:58.211Z] -- MPFR : LIB_MPFR-NOTFOUND
[2026-06-17T17:13:58.211Z] -- GMP : LIBGMP-NOTFOUND
[2026-06-17T17:13:58.211Z] -- RT : /usr/lib/x86_64-linux-gnu/librt.a
[2026-06-17T17:13:58.211Z] -- FFTW3 : LIBFFTW3-NOTFOUND
[2026-06-17T17:13:58.211Z] -- OPENSSL : 3.6.3
[2026-06-17T17:13:58.211Z] -- SDE : SDE_COMMAND-NOTFOUND
[2026-06-17T17:13:58.211Z] -- COMPILER_SUPPORTS_OPENMP : 
[2026-06-17T17:13:58.211Z] AT_INSTALL_INCLUDE_DIR include/ATen/core
[2026-06-17T17:13:58.211Z] core header install: /var/lib/jenkins/pytorch/build/aten/src/ATen/core/TensorBody.h
[2026-06-17T17:13:58.211Z] core header install: /var/lib/jenkins/pytorch/build/aten/src/ATen/core/aten_interned_strings.h
[2026-06-17T17:14:22.196Z] -- Performing Test HAS_WNO_UNUSED_VARIABLE
[2026-06-17T17:14:22.789Z] -- Performing Test HAS_WNO_UNUSED_VARIABLE - Success
[2026-06-17T17:14:22.789Z] -- Performing Test HAS_WNO_UNUSED_BUT_SET_VARIABLE
[2026-06-17T17:14:23.622Z] -- Performing Test HAS_WNO_UNUSED_BUT_SET_VARIABLE - Success
[2026-06-17T17:14:23.622Z] -- Performing Test HAS_WNO_DANGLING_POINTER
[2026-06-17T17:14:24.464Z] -- Performing Test HAS_WNO_DANGLING_POINTER - Success
[2026-06-17T17:14:24.464Z] -- Performing Test HAS_WNO_MISSING_BRACES
[2026-06-17T17:14:24.948Z] -- Performing Test HAS_WNO_MISSING_BRACES - Success
[2026-06-17T17:14:24.948Z] -- Performing Test HAS_WNO_UNUSED_BUT_SET_PARAMETER
[2026-06-17T17:14:27.032Z] -- Performing Test HAS_WNO_UNUSED_BUT_SET_PARAMETER - Success
[2026-06-17T17:14:27.032Z] -- Performing Test HAS_WNO_ERROR_NONNULL
[2026-06-17T17:14:28.340Z] -- Performing Test HAS_WNO_ERROR_NONNULL - Success
[2026-06-17T17:14:28.340Z] -- Performing Test HAS_WNO_ERROR_ARRAY_BOUNDS
[2026-06-17T17:14:28.713Z] -- Performing Test HAS_WNO_ERROR_ARRAY_BOUNDS - Success
[2026-06-17T17:14:28.713Z] INFOcaffe2 ROCM_SOURCE_DIR = /opt/rocm
[2026-06-17T17:14:28.713Z] INFOUSE_NVSHMEM=ON USE_ROCM=ON rocshmem_FOUND=1
[2026-06-17T17:14:28.713Z] -- rocSHMEM found, building with rocSHMEM support for arches: gfx90a:xnack-;gfx90a:xnack+;gfx942
[2026-06-17T17:14:28.713Z] -- ROCSHMEM_INCLUDE_DIR: '/opt/rocm-7.2.4/include'
[2026-06-17T17:14:30.022Z] CMake Warning at CMakeLists.txt:1355 (message):
[2026-06-17T17:14:30.022Z]   Generated cmake files are only fully tested if one builds with system glog,
[2026-06-17T17:14:30.022Z]   gflags, and protobuf.  Other settings may generate files that are not well
[2026-06-17T17:14:30.022Z]   tested.
[2026-06-17T17:14:30.022Z] 
[2026-06-17T17:14:30.022Z] 
[2026-06-17T17:14:30.022Z] -- 
[2026-06-17T17:14:30.022Z] -- ******** Summary ********
[2026-06-17T17:14:30.022Z] -- General:
[2026-06-17T17:14:30.022Z] --   CMake version         : 3.31.6
[2026-06-17T17:14:30.022Z] --   CMake command         : /opt/conda/envs/py_3.12/lib/python3.12/site-packages/cmake/data/bin/cmake
[2026-06-17T17:14:30.022Z] --   System                : Linux
[2026-06-17T17:14:30.022Z] --   C++ compiler          : /opt/cache/bin/c++
[2026-06-17T17:14:30.022Z] --   C++ compiler id       : GNU
[2026-06-17T17:14:30.022Z] --   C++ compiler version  : 13.3.0
[2026-06-17T17:14:30.022Z] --   Using ccache if found : ON
[2026-06-17T17:14:30.022Z] --   Found ccache          : CCACHE_PROGRAM-NOTFOUND
[2026-06-17T17:14:30.022Z] --   CXX flags             :  -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOXPUPTI=ON -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -DC10_NODEPRECATED -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -faligned-new -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-dangling-reference -Wno-error=dangling-reference -Wno-stringop-overflow
[2026-06-17T17:14:30.022Z] --   Shared LD flags       :  -Wl,--no-as-needed -rdynamic
[2026-06-17T17:14:30.022Z] --   Static LD flags       : 
[2026-06-17T17:14:30.022Z] --   Module LD flags       : 
[2026-06-17T17:14:30.022Z] --   Build type            : Release
[2026-06-17T17:14:30.022Z] --   Compile definitions   : ROCM_VERSION=70204;TORCH_HIP_VERSION=702;USE_LAYERNORM_FAST_RECIPROCAL;ONNX_ML=1;ONNXIFI_ENABLE_EXT=1;ONNX_NAMESPACE=onnx_torch;IDEEP_USE_MKL;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1;HAVE_POSIX_FALLOCATE=1;USE_EXTERNAL_MZCRC;MINIZ_DISABLE_ZIP_READER_CRC32_CHECKS
[2026-06-17T17:14:30.022Z] --   CMAKE_PREFIX_PATH     : /opt/conda/envs/py_3.12/lib/python3.12/site-packages;/opt/conda/envs/py_3.12;/opt/rocm
[2026-06-17T17:14:30.022Z] --   CMAKE_INSTALL_PREFIX  : /var/lib/jenkins/pytorch/torch
[2026-06-17T17:14:30.022Z] --   USE_GOLD_LINKER       : OFF
[2026-06-17T17:14:30.022Z] -- 
[2026-06-17T17:14:30.022Z] --   TORCH_VERSION         : 2.12.0
[2026-06-17T17:14:30.022Z] --   BUILD_STATIC_RUNTIME_BENCHMARK: ON
[2026-06-17T17:14:30.022Z] --   BUILD_BINARY          : OFF
[2026-06-17T17:14:30.022Z] --   BUILD_CUSTOM_PROTOBUF : ON
[2026-06-17T17:14:30.022Z] --     Link local protobuf : ON
[2026-06-17T17:14:30.022Z] --   BUILD_PYTHON          : True
[2026-06-17T17:14:30.022Z] --     Python version      : 3.12.13
[2026-06-17T17:14:30.022Z] --     Python executable   : /opt/conda/envs/py_3.12/bin/python
[2026-06-17T17:14:30.022Z] --     Python library      : 
[2026-06-17T17:14:30.022Z] --     Python includes     : /opt/conda/envs/py_3.12/include/python3.12
[2026-06-17T17:14:30.022Z] --     Python site-package : /opt/conda/envs/py_3.12/lib/python3.12/site-packages
[2026-06-17T17:14:30.022Z] --   BUILD_SHARED_LIBS     : ON
[2026-06-17T17:14:30.022Z] --   CAFFE2_USE_MSVC_STATIC_RUNTIME     : OFF
[2026-06-17T17:14:30.022Z] --   BUILD_TEST            : True
[2026-06-17T17:14:30.022Z] --   BUILD_JNI             : OFF
[2026-06-17T17:14:30.022Z] --   BUILD_MOBILE_AUTOGRAD : OFF
[2026-06-17T17:14:30.022Z] --   BUILD_LITE_INTERPRETER: OFF
[2026-06-17T17:14:30.022Z] --   INTERN_BUILD_MOBILE   : 
[2026-06-17T17:14:30.022Z] --   TRACING_BASED         : OFF
[2026-06-17T17:14:30.022Z] --   USE_BLAS              : 1
[2026-06-17T17:14:30.022Z] --     BLAS                : mkl
[2026-06-17T17:14:30.022Z] --     BLAS_HAS_SBGEMM     : 
[2026-06-17T17:14:30.022Z] --     BLAS_HAS_SHGEMM     : 
[2026-06-17T17:14:30.022Z] --   USE_LAPACK            : 1
[2026-06-17T17:14:30.022Z] --     LAPACK              : mkl
[2026-06-17T17:14:30.022Z] --   USE_ASAN              : OFF
[2026-06-17T17:14:30.022Z] --   USE_LSAN              : OFF
[2026-06-17T17:14:30.022Z] --   USE_TSAN              : OFF
[2026-06-17T17:14:30.022Z] --   USE_CPP_CODE_COVERAGE : OFF
[2026-06-17T17:14:30.022Z] --   USE_CUDA              : OFF
[2026-06-17T17:14:30.023Z] --   USE_XPU               : OFF
[2026-06-17T17:14:30.023Z] --   USE_ROCM              : ON
[2026-06-17T17:14:30.023Z] --     ROCM_VERSION                  : 7.2.4
[2026-06-17T17:14:30.023Z] --     USE_FLASH_ATTENTION           : ON
[2026-06-17T17:14:30.023Z] --     USE_MEM_EFF_ATTENTION         : ON
[2026-06-17T17:14:30.023Z] --     USE_ROCM_CK_SDPA              : OFF
[2026-06-17T17:14:30.023Z] --     USE_ROCM_CK_GEMM              : ON
[2026-06-17T17:14:30.023Z] --     USE_LAYERNORM_FAST_RECIPROCAL : ON
[2026-06-17T17:14:30.023Z] --   BUILD_NVFUSER         : 
[2026-06-17T17:14:30.023Z] --   USE_EIGEN_FOR_BLAS    : 
[2026-06-17T17:14:30.023Z] --   USE_EIGEN_FOR_SPARSE  : OFF
[2026-06-17T17:14:30.023Z] --   USE_FBGEMM            : OFF
[2026-06-17T17:14:30.023Z] --   USE_MSLK              : 
[2026-06-17T17:14:30.023Z] --   USE_KINETO            : ON
[2026-06-17T17:14:30.023Z] --   USE_GFLAGS            : OFF
[2026-06-17T17:14:30.023Z] --   USE_GLOG              : OFF
[2026-06-17T17:14:30.023Z] --   USE_LITE_PROTO        : OFF
[2026-06-17T17:14:30.023Z] --   USE_PYTORCH_METAL     : OFF
[2026-06-17T17:14:30.023Z] --   USE_PYTORCH_METAL_EXPORT     : OFF
[2026-06-17T17:14:30.023Z] --   USE_MPS               : OFF
[2026-06-17T17:14:30.023Z] --   CAN_COMPILE_METAL     : 
[2026-06-17T17:14:30.023Z] --   CAN_COMPILE_METAL_40  : 
[2026-06-17T17:14:30.023Z] --   USE_MKL               : ON
[2026-06-17T17:14:30.023Z] --     USE_STATIC_MKL      : OFF
[2026-06-17T17:14:30.023Z] --   USE_MKLDNN            : ON
[2026-06-17T17:14:30.023Z] --   USE_MKLDNN_ACL        : OFF
[2026-06-17T17:14:30.023Z] --   USE_MKLDNN_CBLAS      : OFF
[2026-06-17T17:14:30.023Z] --   USE_PRIORITIZED_TEXT_FOR_LD : OFF
[2026-06-17T17:14:30.023Z] --   USE_UCC               : OFF
[2026-06-17T17:14:30.023Z] --   USE_ITT               : ON
[2026-06-17T17:14:30.023Z] --   USE_XCCL              : OFF
[2026-06-17T17:14:30.023Z] --   USE_NCCL              : ON
[2026-06-17T17:14:30.023Z] --     USE_SYSTEM_NCCL     : ON
[2026-06-17T17:14:30.023Z] --   Found ROCSHMEM         : /opt/rocm-7.2.4/include
[2026-06-17T17:14:30.023Z] --   USE_NNPACK            : ON
[2026-06-17T17:14:30.023Z] --   USE_NUMPY             : ON
[2026-06-17T17:14:30.023Z] --   USE_OBSERVERS         : ON
[2026-06-17T17:14:30.023Z] --   USE_OPENCL            : OFF
[2026-06-17T17:14:30.023Z] --   USE_OPENMP            : ON
[2026-06-17T17:14:30.023Z] --   USE_MIMALLOC          : OFF
[2026-06-17T17:14:30.023Z] --   USE_VULKAN            : OFF
[2026-06-17T17:14:30.023Z] --   USE_PROF              : OFF
[2026-06-17T17:14:30.023Z] --   USE_PYTORCH_QNNPACK   : ON
[2026-06-17T17:14:30.023Z] --   USE_XNNPACK           : ON
[2026-06-17T17:14:30.023Z] --   USE_DISTRIBUTED       : ON
[2026-06-17T17:14:30.023Z] --     USE_MPI               : OFF
[2026-06-17T17:14:30.023Z] --     USE_GLOO              : ON
[2026-06-17T17:14:30.023Z] --     USE_GLOO_WITH_OPENSSL : OFF
[2026-06-17T17:14:30.023Z] --     USE_GLOO_IBVERBS      : OFF
[2026-06-17T17:14:30.023Z] --     USE_TENSORPIPE        : ON
[2026-06-17T17:14:30.023Z] --   Public Dependencies  : caffe2::mkl
[2026-06-17T17:14:30.023Z] --   Private Dependencies : Threads::Threads;pthreadpool;cpuinfo;pytorch_qnnpack;nnpack;XNNPACK;microkernels-prod;ittnotify;fp16;caffe2::openmp;tensorpipe;nlohmann;moodycamel;gloo;rt;fmt::fmt-header-only;kineto;gcc_s;gcc;dl
[2026-06-17T17:14:30.023Z] --   Public CUDA Deps.    : 
[2026-06-17T17:14:30.023Z] --   Private CUDA Deps.   : __caffe2_nccl
[2026-06-17T17:14:30.023Z] --   USE_COREML_DELEGATE     : OFF
[2026-06-17T17:14:30.023Z] --   BUILD_LAZY_TS_BACKEND   : ON
[2026-06-17T17:14:30.023Z] --   USE_ROCM_KERNEL_ASSERT : OFF
[2026-06-17T17:14:30.023Z] -- Configuring incomplete, errors occurred!
[2026-06-17T17:14:30.433Z] -- Checkout nccl release tag: v2.29.7-1
[2026-06-17T17:14:30.791Z] 
[2026-06-17T17:14:30.791Z] ERROR Backend subprocess exited when trying to invoke build_wheel
[2026-06-17T17:14:30.791Z] + sccache_epilogue
[2026-06-17T17:14:30.791Z] + echo '::group::Sccache Compilation Log'
[2026-06-17T17:14:30.791Z] + echo '=================== sccache compilation log ==================='
[2026-06-17T17:14:30.791Z] + python /var/lib/jenkins/pytorch/.ci/pytorch/print_sccache_log.py /root/sccache_error.log
[2026-06-17T17:14:30.791Z] + echo '=========== If your build fails, please take a look at the log above for possible reasons ==========='
[2026-06-17T17:14:30.791Z] + sccache --show-stats
[2026-06-17T17:14:30.791Z] + sccache --stop-server
[2026-06-17T17:14:30.791Z] + echo ::endgroup::
[2026-06-17T17:14:30.791Z] ::group::Sccache Compilation Log
[2026-06-17T17:14:30.791Z] =================== sccache compilation log ===================
[2026-06-17T17:14:30.791Z] =========== If your build fails, please take a look at the log above for possible reasons ===========
[2026-06-17T17:14:30.791Z] Compile requests                    272
[2026-06-17T17:14:30.792Z] Compile requests executed           133
[2026-06-17T17:14:30.792Z] Cache hits                            1
[2026-06-17T17:14:30.792Z] Cache hits (C/C++)                    1
[2026-06-17T17:14:30.792Z] Cache misses                        102
[2026-06-17T17:14:30.792Z] Cache misses (C/C++)                101
[2026-06-17T17:14:30.792Z] Cache misses (HIP)                    1
[2026-06-17T17:14:30.792Z] Cache hits rate                    0.97 %
[2026-06-17T17:14:30.792Z] Cache hits rate (C/C++)            0.98 %
[2026-06-17T17:14:30.792Z] Cache hits rate (HIP)              0.00 %
[2026-06-17T17:14:30.792Z] Cache timeouts                        0
[2026-06-17T17:14:30.792Z] Cache read errors                     0
[2026-06-17T17:14:30.792Z] Forced recaches                       0
[2026-06-17T17:14:30.792Z] Cache write errors                    0
[2026-06-17T17:14:30.792Z] Cache errors                         22
[2026-06-17T17:14:30.792Z] Cache errors (C/C++)                 22
[2026-06-17T17:14:30.792Z] Compilations                        102
[2026-06-17T17:14:30.792Z] Compilation failures                  8
[2026-06-17T17:14:30.792Z] Non-cacheable compilations            0
[2026-06-17T17:14:30.792Z] Non-cacheable calls                  13
[2026-06-17T17:14:30.792Z] Non-compilation calls               126
[2026-06-17T17:14:30.792Z] Unsupported compiler calls            0
[2026-06-17T17:14:30.792Z] Average cache write               0.011 s
[2026-06-17T17:14:30.792Z] Average compiler                  0.484 s
[2026-06-17T17:14:30.792Z] Average cache read hit            0.000 s
[2026-06-17T17:14:30.792Z] Failed distributed compilations       0
[2026-06-17T17:14:30.792Z] 
[2026-06-17T17:14:30.792Z] Non-cacheable reasons:
[2026-06-17T17:14:30.792Z] multiple input files                 13
[2026-06-17T17:14:30.792Z] 
[2026-06-17T17:14:30.792Z] Cache location                  Local disk: "/root/.cache/sccache"
[2026-06-17T17:14:30.792Z] Use direct/preprocessor mode?   no
[2026-06-17T17:14:30.792Z] Version (client)                0.13.0
[2026-06-17T17:14:30.792Z] Cache size                          239 KiB
[2026-06-17T17:14:30.792Z] Max cache size                       10 GiB
[2026-06-17T17:14:30.792Z] Stopping sccache server...
[2026-06-17T17:14:30.792Z] Compile requests                    272
[2026-06-17T17:14:30.792Z] Compile requests executed           133
[2026-06-17T17:14:30.792Z] Cache hits                            1
[2026-06-17T17:14:30.792Z] Cache hits (C/C++)                    1
[2026-06-17T17:14:30.792Z] Cache misses                        102
[2026-06-17T17:14:30.792Z] Cache misses (C/C++)                101
[2026-06-17T17:14:30.792Z] Cache misses (HIP)                    1
[2026-06-17T17:14:30.792Z] Cache hits rate                    0.97 %
[2026-06-17T17:14:30.792Z] Cache hits rate (C/C++)            0.98 %
[2026-06-17T17:14:30.792Z] Cache hits rate (HIP)              0.00 %
[2026-06-17T17:14:30.792Z] Cache timeouts                        0
[2026-06-17T17:14:30.792Z] Cache read errors                     0
[2026-06-17T17:14:30.792Z] Forced recaches                       0
[2026-06-17T17:14:30.792Z] Cache write errors                    0
[2026-06-17T17:14:30.792Z] Cache errors                         22
[2026-06-17T17:14:30.792Z] Cache errors (C/C++)                 22
[2026-06-17T17:14:30.792Z] Compilations                        102
[2026-06-17T17:14:30.792Z] Compilation failures                  8
[2026-06-17T17:14:30.792Z] Non-cacheable compilations            0
[2026-06-17T17:14:30.792Z] Non-cacheable calls                  13
[2026-06-17T17:14:30.792Z] Non-compilation calls               126
[2026-06-17T17:14:30.792Z] Unsupported compiler calls            0
[2026-06-17T17:14:30.792Z] Average cache write               0.011 s
[2026-06-17T17:14:30.792Z] Average compiler                  0.484 s
[2026-06-17T17:14:30.792Z] Average cache read hit            0.000 s
[2026-06-17T17:14:30.792Z] Failed distributed compilations       0
[2026-06-17T17:14:30.792Z] 
[2026-06-17T17:14:30.792Z] Non-cacheable reasons:
[2026-06-17T17:14:30.792Z] multiple input files                 13
[2026-06-17T17:14:30.792Z] 
[2026-06-17T17:14:30.792Z] Cache location                  Local disk: "/root/.cache/sccache"
[2026-06-17T17:14:30.792Z] Use direct/preprocessor mode?   no
[2026-06-17T17:14:30.792Z] Version (client)                0.13.0
[2026-06-17T17:14:30.792Z] Cache size                          239 KiB
[2026-06-17T17:14:30.792Z] Max cache size                       10 GiB
[2026-06-17T17:14:30.792Z] ::endgroup::

Output truncated.

Details

  • Kill older PR Builds (3.4 sec)
  • Initialize (38 min)
    • Download CI scripts (38 sec)
    • Checkout Pytorch (4 min 3 sec)
    • Check base Docker image existence (17 sec)
    • Pull Docker Image (14 min)
    • Build PyTorch (17 min)
      Error: script returned exit code 1 - logs
      Error: PyTorch build failed hudson.AbortException: script returned exit code 1 - logs
      Error: No artifacts found that match the file pattern ".whl, test_artifacts.zip". Configuration error?* - logs
  • Tests (37 sec)
    • Test PyTorch (8 ms)
      • Test PyTorch (17 sec)
    • Test Distributed (7 ms)
      • Test Distributed (17 sec)
    • Test Inductor (7 ms)
      • Test Inductor (17 sec)
    • Test PyTorch Slow (10 ms)
      • Test PyTorch Slow (17 sec)
    • Microbenchmark (33 sec)
      • Microbenchmark (17 sec)
  • Post Build (3.7 sec)
  • Declarative: Post Actions (7.3 sec)