Skip to content

Reduce latency cycles for _mm_movemask_epi8 on Armv7-A #769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Cuda-Chen wants to merge 6 commits into
base: master
Choose a base branch
from
+426 −142
Open

Reduce latency cycles for _mm_movemask_epi8 on Armv7-A #769

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
dd51d88
perf: Reduce latency cycles for '_mm_movemask_epi8' on Armv7-A
Cuda-Chen Jul 31, 2026
54feef7
[DO NOT MERGE] Benchmark A32
Cuda-Chen Aug 1, 2026
c14eecf
[DO NOT MERGE] Add benchmark with original impl
Cuda-Chen Aug 1, 2026
9cfd894
[DO NOT MERGE] Apply suggestion of benchmarking
Cuda-Chen Aug 1, 2026
cf8a5e5
[DO NOT MERGE] Apply suggestions
Cuda-Chen Aug 1, 2026
21ab6ca
[DO NOT MERGE] Apply benchmark suggestion
Cuda-Chen Aug 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/benchmark-a32.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Benchmark A32

on: [push, pull_request, workflow_dispatch]

jobs:
benchmark-a32:
name: Benchmark A32 (Armv7-A) on A64 Native

@cubic-dev-ai cubic-dev-ai Bot Aug 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This benchmark runs A32 instructions on an AArch64 core in compatibility mode, so the resulting cycle counts reflect Ampere Altra's ARMv7 execution rather than real ARMv7-A silicon. The latency numbers here are a sanity check and cannot validate the microarchitecture-specific latency-reduction claim in the PR description; the on-hardware ARMv7-A benchmarking mentioned there remains necessary before relying on the ~30→21 cycle figures.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/benchmark-a32.yml, line 7:

<comment>This benchmark runs A32 instructions on an AArch64 core in compatibility mode, so the resulting cycle counts reflect Ampere Altra's ARMv7 execution rather than real ARMv7-A silicon. The latency numbers here are a sanity check and cannot validate the microarchitecture-specific latency-reduction claim in the PR description; the on-hardware ARMv7-A benchmarking mentioned there remains necessary before relying on the ~30→21 cycle figures.</comment>

<file context>
@@ -0,0 +1,29 @@
+
+jobs:
+  benchmark-a32:
+    name: Benchmark A32 (Armv7-A) on A64 Native
+    if: contains(toJSON(github.event.head_commit.message), 'Merge pull request ') == false
+    timeout-minutes: 30
</file context>

if: contains(toJSON(github.event.head_commit.message), 'Merge pull request ') == false
timeout-minutes: 30
runs-on: ubuntu-24.04-arm

@cubic-dev-ai cubic-dev-ai Bot Aug 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Reported latency will describe the A64 host's A32 execution path, not Armv7-A, so it cannot validate the stated Armv7-A cycle reduction. Run this performance benchmark on an Armv7-A runner/device, or label results as A64 A32-compat measurements.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/benchmark-a32.yml, line 10:

<comment>Reported latency will describe the A64 host's A32 execution path, not Armv7-A, so it cannot validate the stated Armv7-A cycle reduction. Run this performance benchmark on an Armv7-A runner/device, or label results as A64 A32-compat measurements.</comment>

<file context>
@@ -0,0 +1,25 @@
+    name: Benchmark A32 (Armv7-A) on A64 Native
+    if: contains(toJSON(github.event.head_commit.message), 'Merge pull request ') == false
+    timeout-minutes: 30
+    runs-on: ubuntu-24.04-arm
+    steps:
+      - name: Checkout code
</file context>

@cubic-dev-ai cubic-dev-ai Bot Aug 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The goal of this PR is a ~9-cycle latency reduction validated by microbenchmark, but the numbers come from a shared, frequency-scaling arm64 VM where nanosecond-level latency/dependent-chain measurements are noisy and hard to reproduce. The results are indicative only and could easily hide or falsely confirm the expected gain. Consider documenting the expected variance, running more repetitions, or validating on pinned/dedicated ARMv7 hardware before treating the gain as confirmed.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/benchmark-a32.yml, line 10:

<comment>The goal of this PR is a ~9-cycle latency reduction validated by microbenchmark, but the numbers come from a shared, frequency-scaling arm64 VM where nanosecond-level latency/dependent-chain measurements are noisy and hard to reproduce. The results are indicative only and could easily hide or falsely confirm the expected gain. Consider documenting the expected variance, running more repetitions, or validating on pinned/dedicated ARMv7 hardware before treating the gain as confirmed.</comment>

<file context>
@@ -0,0 +1,29 @@
+    name: Benchmark A32 (Armv7-A) on A64 Native
+    if: contains(toJSON(github.event.head_commit.message), 'Merge pull request ') == false
+    timeout-minutes: 30
+    runs-on: ubuntu-24.04-arm
+    steps:
+      - name: Checkout code
</file context>

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Download dependencies
run: |
sudo dpkg --add-architecture armhf
sudo apt-get update -q -y
sudo apt-get install -q -y libc6:armhf crossbuild-essential-armhf g++-arm-linux-gnueabihf

- name: Build and Run movemask benchmark natively in A32 mode
run: |
echo "=== Optimized Implementation (VPADD) ==="
make clean > /dev/null
make bench-movemask CROSS_COMPILE=arm-linux-gnueabihf- EXEC_WRAPPER=

@cubic-dev-ai cubic-dev-ai Bot Aug 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The steps run a cross-compiled ARMv7 armhf binary natively on the AArch64 ubuntu-24.04-arm runner by clearing EXEC_WRAPPER, which relies on the host kernel supporting 32-bit A32 execution (CONFIG_COMPAT). If the runner kernel lacks this, the bench-movemask steps fail with an exec-format error; this is not guaranteed on GitHub-hosted arm64 runners. Consider verifying on the actual runner or keeping the qemu-arm fallback for robustness.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/benchmark-a32.yml, line 25:

<comment>The steps run a cross-compiled ARMv7 armhf binary natively on the AArch64 `ubuntu-24.04-arm` runner by clearing `EXEC_WRAPPER`, which relies on the host kernel supporting 32-bit A32 execution (CONFIG_COMPAT). If the runner kernel lacks this, the `bench-movemask` steps fail with an exec-format error; this is not guaranteed on GitHub-hosted arm64 runners. Consider verifying on the actual runner or keeping the `qemu-arm` fallback for robustness.</comment>

<file context>
@@ -0,0 +1,29 @@
+        run: |
+          echo "=== Optimized Implementation (VPADD) ==="
+          make clean > /dev/null
+          make bench-movemask CROSS_COMPILE=arm-linux-gnueabihf- EXEC_WRAPPER=
+          
+          echo "=== Original Implementation (VSRA Baseline) ==="
</file context>

@cubic-dev-ai cubic-dev-ai Bot Aug 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The job builds an ARMv7 (armhf) binary and runs it natively via EXEC_WRAPPER= on an AArch64 runner. Native AArch32 execution only works if the runner CPU implements the AArch32 state at EL0 (e.g. Cortex-A / Ampere Altra) and the kernel has CONFIG_COMPAT; Neoverse V-class and similar cores drop AArch32, in which case the binary fails with 'Exec format error'/SIGILL and the benchmark never runs. The repo already uses qemu-arm for ARMv7 in main.yml for exactly this reason. Consider verifying the runner's AArch32 support or falling back to $(EXEC_WRAPPER) (QEMU) as insurance so the job reports numbers rather than dying.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/benchmark-a32.yml, line 25:

<comment>The job builds an ARMv7 (armhf) binary and runs it natively via EXEC_WRAPPER= on an AArch64 runner. Native AArch32 execution only works if the runner CPU implements the AArch32 state at EL0 (e.g. Cortex-A / Ampere Altra) and the kernel has CONFIG_COMPAT; Neoverse V-class and similar cores drop AArch32, in which case the binary fails with 'Exec format error'/SIGILL and the benchmark never runs. The repo already uses qemu-arm for ARMv7 in main.yml for exactly this reason. Consider verifying the runner's AArch32 support or falling back to $(EXEC_WRAPPER) (QEMU) as insurance so the job reports numbers rather than dying.</comment>

<file context>
@@ -0,0 +1,29 @@
+        run: |
+          echo "=== Optimized Implementation (VPADD) ==="
+          make clean > /dev/null
+          make bench-movemask CROSS_COMPILE=arm-linux-gnueabihf- EXEC_WRAPPER=
+          
+          echo "=== Original Implementation (VSRA Baseline) ==="
</file context>


echo "=== Original Implementation (VSRA Baseline) ==="
make clean > /dev/null
make bench-movemask CROSS_COMPILE=arm-linux-gnueabihf- EXEC_WRAPPER= CXXFLAGS="-DBENCH_ORIGINAL_MOVEMASK_EPI8"
16 changes: 15 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -361,14 +361,28 @@ fuzz-clean:
$(RM) -r $(FUZZ_CORPUS)
$(RM) crash-* leak-* timeout-* oom-*

.PHONY: clean check check-main check-ieee754 check-nan check-aes check-ubsan check-asan check-strict-aliasing check-uninit check-macros check-differential generate-golden coverage-report indent ieee754 nan aes fuzz fuzz-verbose fuzz-clean
# Movemask benchmark
BENCH_MOVEMASK_SRC = tests/bench_movemask.cpp
BENCH_MOVEMASK_EXEC = tests/bench_movemask

$(BENCH_MOVEMASK_EXEC): $(BENCH_MOVEMASK_SRC) sse2neon.h
$(CXX) -O3 $(ARCH_CFLAGS) $(CXXFLAGS) -I. -std=gnu++14 $(LDFLAGS) -o $@ $<

bench-movemask: $(BENCH_MOVEMASK_EXEC)
ifeq ($(processor),$(filter $(processor),aarch64 arm64 arm armv7l))
$(CC) $(ARCH_CFLAGS) -c sse2neon.h
endif
$(EXEC_WRAPPER) $^

.PHONY: clean check check-main check-ieee754 check-nan check-aes check-ubsan check-asan check-strict-aliasing check-uninit check-macros check-differential generate-golden coverage-report indent ieee754 nan aes fuzz fuzz-verbose fuzz-clean bench-movemask
clean:
$(RM) $(OBJS) $(EXEC) $(deps) sse2neon.h.gch
$(RM) $(IEEE754_OBJS) $(IEEE754_EXEC) $(ieee754_deps)
$(RM) $(NAN_OBJS) $(NAN_EXEC) $(nan_deps)
$(RM) $(AES_OBJS) $(AES_EXEC) $(aes_deps)
$(RM) $(DIFFERENTIAL_OBJS) $(DIFFERENTIAL_EXEC) $(differential_deps)
$(RM) $(FUZZ_EXEC)
$(RM) $(BENCH_MOVEMASK_EXEC)

-include $(deps)
-include $(ieee754_deps)
Expand Down
Loading
Loading