cmd/compile: legacy SSE encodings emitted in functions using simd/archsimd intrinsics cause AVX-SSE transition penalties

Go version

go version go1.26.5 linux/amd64 (also reproduced cross-compiling from darwin/arm64)

Output of `go env` in your module/workspace

GOARCH=amd64
GOOS=linux
GOEXPERIMENT=simd
GOAMD64=v1 (also reproduces with v3/v4)

What did you do?

Compiled functions using simd/archsimd vector operations:

package repro

import "simd/archsimd"

func ShiftImm(x archsimd.Uint32x8) archsimd.Uint32x8 {
	return x.ShiftAllRight(27)
}

What did you see happen?

The scalar shift count is materialized into an XMM register with a legacy-encoded (non-VEX) MOVQ in the middle of VEX code:

MOVL $0x1b, AX
MOVQ AX, X1        // 66 48 0f 6e c8   <- legacy SSE encoding, no VEX prefix
VPSRLD X1, Y0, Y0  // c5 fd d2 c1      <- VEX

Executing a legacy SSE instruction while the upper halves of the YMM/ZMM registers are dirty triggers an AVX-SSE state transition penalty on Intel CPUs (Intel 64 and IA-32 Architectures Optimization Reference Manual, "Mixing AVX Code with SSE Code"). Since surrounding archsimd code generally leaves the uppers dirty, every call through such a function pays the penalty.

Measured impact (Intel Xeon Platinum 8581C, Emerald Rapids): a parquet bloom filter block insert using ShiftAllRight(27) between 256-bit operations ran at 132ns/op instead of 2ns/op — about 65x slower — with ~96% of profile samples on the two instructions following the legacy MOVQ. Replacing the scalar-count shift with a per-lane ShiftRight fed by a constant vector loaded from memory restored the expected performance.

A second manifestation is the compiler's stack zeroing idiom, which clears zeroed stack memory with legacy-encoded MOVUPS stores of the X15 zero register even inside functions full of EVEX instructions:

func ReduceStore(x archsimd.Uint64x8) uint64 {
	var t [8]uint64
	acc := archsimd.LoadUint64x8Slice(t[:])
	acc = acc.Add(x)
	acc.StoreSlice(t[:])
	var n uint64
	for _, v := range t {
		n += v
	}
	return n
}

compiles to:

MOVUPS X15, (CX)       // 44 0f 11 39     <- legacy SSE zeroing stores
MOVUPS X15, 0x10(CX)   // 44 0f 11 79 10
MOVUPS X15, 0x20(CX)
MOVUPS X15, 0x30(CX)
VPADDQ (SP), Z0, Z0    // 62 f1 fd 48 d4  <- EVEX
VMOVDQU64 Z0, (SP)     // 62 f1 fe 48 7f

(the vector load/store intrinsics themselves are correctly EVEX-encoded; only the zeroing is legacy). In the code where we found this — a vector accumulator reduced through a stack array once per call — the transition penalty cost ~155ns per call, and in CPU profiles the time is attributed to the first EVEX instruction of the next call, which makes the root cause hard to find. Rewriting the reduction with GetHi/GetLo/GetElem (register-only, no stack array to zero) removed it.

Both cases were found while porting hand-written assembly kernels in parquet-go to archsimd (parquet-go/parquet-go#584; analysis notes in the repository under docs/archsimd-port.md).

What did you expect to see?

Functions containing VEX/EVEX instructions should use VEX encodings for all compiler-generated XMM/YMM traffic (scalar-to-vector moves, spills, stack copies, zeroing), or the compiler should manage the transition with VZEROUPPER at the boundaries. GCC and Clang emit vmovq/vmovups inside AVX-enabled functions for this reason.

For the first repro, the ideal lowering uses the immediate form of the shift, which needs no scalar materialization at all:

VPSRLD $0x1b, Y0, Y0   // c5 fd 72 d0 1b — immediate form, single instruction

and if the register-count form is kept, the move should at minimum be VEX-encoded:

MOVL $0x1b, AX
VMOVQ AX, X1           // c4 e1 f9 6e c8 — VEX encoding of the same move
VPSRLD X1, Y0, Y0      // c5 fd d2 c1

For the second manifestation, the zeroing stores should use the VEX encoding of the same instruction:

VMOVUPS X15, (CX)       // c5 78 11 39 — VEX encoding of the same store
VMOVUPS X15, 0x10(CX)
VMOVUPS X15, 0x20(CX)
VMOVUPS X15, 0x30(CX)

Adopting VEX encodings would also allow wider zeroing stores: 32 bytes per VMOVDQU from a YMM register, or 64 bytes per VMOVDQU64 from a ZMM register, halving or quartering the number of stores in functions that already use those register widths (provided the zero register's upper bits are known zero, e.g. by materializing it with a VEX VPXOR, which zeroes the full width).

Since the fix could take several forms (per-instruction lowering fixes, promoting legacy encodings to VEX at the assembler level when a function contains AVX instructions, or VZEROUPPER insertion), filing this as an issue rather than a CL to discuss the preferred approach.

docs: issue 1 filed as golang/go#80835

One nit, unlike what the short advice mentions mixing VEX EVEX and SSE is fine, but you need all of your VEX and EVEX instructions to use XMM or smaller.

I still agree this needs to be fixed.

cc @golang/compiler

Related Issues

simd/archsimd: AVX instructions do not make use of memory operands #78159 (closed)
simd/archsimd: would intrinsics eliminate Go/assembly boundary overhead for SIMD prefilter? #77647 (closed)
cmd/compile: GOEXPERIMENT=simd allocates 512-bit vector values from 15 of the 32 zmm registers #80828
cmd/compile: archsimd MulAdd cannot accumulate in place or fold a broadcast memory operand (no VFMADD231PS, no .BCST), though the assembler encodes both #80829
cmd/asm: confusing names, encodings for VEX instructions #14068 (closed)
cmd/compile: suboptimal zeroing of multiple registers on amd64 #76449 (closed)
simd/archsimd: BroadcastFloat32x16 is emulated, and every inlined non-intrinsic wrapper costs a 1-byte anchor NOP in the loop body #80830
cmd/compile: simd operations not marked as hoistable #79984
cmd/compile: archsimd VPSRLW missing constant-folding rewrite rules + suboptimal NotEqual codegen #78138
cmd/objdump: disassembly of AVX instructions generated by archsimd produces nonsense. #78065 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

perf(simd): fix legacy XORPS and serial chains in optimistic float bo…

Looks like we need special handling in ssa->prog lowering, I will take a look!

The current workaround is that you call archsimd.ClearAVXUpperBits when you're done with simd operations. Long term, I'd hope the compiler choose the AVX instructions instead of SSE when it can see AVX is supported, e.g. when a CPU feature check present, or GOAMD64=v3 or above is set, or following an AVX instruction (e.g. a simd operation).

I also hope the compiler can insert VZEROUPPER automatically, e.g. when it is done with simd operations. The exact location to insert may be a bit tricky.

fix(xxhash): per lane shift counts in the AVX2 kernels

perf: AVX2 tier for xxhash MultiSum64Uint32 (#598)

Change https://go.dev/cl/825186 mentions this issue: cmd/compile: give critical-edge split blocks the CPU features of their edge

cmd/compile: don't let no-code SIMD values imply CPU features

cmd/compile: give critical-edge split blocks the CPU features of thei…

Do the day's pass four tiles at a time where the build allows

cmd/compile: use VEX encodings for lowered Zero and Move when AVX is …

cmd/compile: use VEX encodings for spills, moves, and copies when AVX…

docs: the upstream watch read Gerrit only, and a question sat five we…

Is automatic VZEROUPPER insertion tracked here? The linked CLs only change instruction encodings, I'm worried it gets lost when those land.

Even when every instruction in a simd function is VEX/EVEX, the caller gets control back with dirty upper halves. With go1.27.1/gotip (linux/amd64, GOEXPERIMENT=simd):

//go:build goexperiment.simd

package main

import "simd/archsimd"

//go:noinline
func AddSlices(dst, a, b []float32) {
	for len(dst) >= 8 && len(a) >= 8 && len(b) >= 8 {
		archsimd.LoadFloat32x8(a).Add(archsimd.LoadFloat32x8(b)).Store(dst)
		dst, a, b = dst[8:], a[8:], b[8:]
	}
}

func main() {}

compiles to a VADDPS on Y registers and a RET. There is no VZEROUPPER anywhere in the package, at the default GOAMD64 or at GOAMD64=v3. Any legacy-SSE code that runs afterwards (memmove, the stack zeroing from this issue) pays the transition on Intel. I only have AMD hardware, so I haven't measured it.

For comparison, clang 23 with -O2 -march=skylake compiles the equivalent AVX2 intrinsics loop to vzeroupper followed by ret, and when the loop is inlined into a caller that then calls memcpy, it places the vzeroupper right before the tail call.

erigontech/erigon#24410 hit this in a hex encoder and works around it with ClearAVXUpperBits after the loop.

Should this be tracked here or in a separate issue? Happy to file one.

Change https://go.dev/cl/847806 mentions this issue: encoding/hex: vectorize Encode and Decode on amd64 and arm64

docs: retract the #144 status change — those CLs do not touch keel's …

@mauri870

@mauri870 — on "should this be tracked here or in a separate issue": I think separate, and I have
a measurement that bears on your worry about it getting lost when those CLs land.

Your case is a boundary problem; the one I reported here is an interior one. My report was an
AVX-512 GEMM microkernel where the compiler's 128-bit spill idiom is legacy-SSE encoded inside
an all-EVEX loop — 36 non-VEX movups, e.g. movups %xmm2,0x210(%rsp), interleaved among 88
vfmadd213ps, 177 vmovdqu64, 44 vmovss and 44 vbroadcastss. A VZEROUPPER at the function
boundary cannot reach that: the transitions, if they are being paid, are paid mid-loop inside one
basic block, and the dirty uppers are ZMM-wide rather than YMM-wide. So "insert VZEROUPPER
before returns" would be a complete fix for your repro and would not touch mine.

And the two CLs linked here do not touch mine either — measured, not assumed. I built two
toolchains that differ only by CL 825185 and CL 825186. They are consecutive (0b2fd4aa9c08 is
exactly one commit ahead of fbea197d3279, and that commit is the second CL), so I pinned the
base to the later one and reverted both from it. Same source tree, same host (i9-9960X,
Skylake-X, bare metal), -trimpath, and an identical VERSION in both arms so the only
difference is the reverts.

	both CLs	both reverted
`compile` binary sha256	`2f5c402e…`	`5a2eecfb…` (toolchains differ)
disassembly sha256, filename header stripped	`428d3575…`	`428d3575…` (identical)
non-VEX `movups`/`movaps` in the kernel	36	36
non-VEX `movups`/`movaps`, whole test binary	6416	6416

396,471 lines of disassembly, byte-identical; the revert deleted 136 lines across 4 files and the
compilers hash differently, so it applied — it just doesn't reach this code. So the interior
case survives those CLs landing, which is the thing you were worried about, one shape over. On
that basis #80835 looks like at least two sub-cases to me, and the one that stays open after these
CLs is not obviously the one they were filed against.

Scope, so this isn't weighted more than it deserves. The 36 are all in a 6×32 tile my library
benchmarks but does not ship — it spills by design and is held out of dispatch. The two shipped
AVX-512 kernels (2×32, 4×32) carry zero non-VEX moves. So this is a real instance of the
encoding, in a shape no user's code path reaches.

I also have no timing, and I'd rather say so than imply otherwise. My figures are static
instruction counts. I have Intel AVX-512 hardware but no counter-based measurement of AVX-SSE
transitions, so I have attributed nothing to them — and the A/B above is precisely why I still
can't: with no encoding delta there is no counterfactual to time. Two things in my own data also
argue against assuming the penalty dominates: the 65× in the original report is Emerald Rapids,
while my Skylake-X host pays about 1.9× on the same code, which a transition-penalty story has
to explain and so far doesn't. One other data point, from a separate sweep: GOAMD64 v1 through v4
leave these encodings unchanged (with the control that the same sweep does move the package
listing at v3), so for the interior case GOAMD64 is not the knob either.

Happy to file the boundary issue if you'd rather not, though it's your repro and your finding, so
I'd suggest you keep it. If it helps, the write-up with the listings is docs/spill-report.md
section 11.3–11.5 in github.com/scttfrdmn/keel, and the A/B above is under
archive/go80835-encoding-ab/ including the section that states which parts are unmeasured.

docs: the golang/go#80835 reply, sent and archived as posted

Navigation Menu

Repository navigation

cmd/compile: legacy SSE encodings emitted in functions using simd/archsimd intrinsics cause AVX-SSE transition penalties #80835

Go version

Output of `go env` in your module/workspace

What did you do?

What did you see happen?

What did you expect to see?

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

cmd/compile: legacy SSE encodings emitted in functions using simd/archsimd intrinsics cause AVX-SSE transition penalties #80835

Description

Go version

Output of go env in your module/workspace

What did you do?

What did you see happen?

What did you expect to see?

Activity

Jorropo commented on Aug 12, 2026

Jorropo commented on Aug 12, 2026

gabyhelp commented on Aug 12, 2026

JunyangShao commented on Aug 13, 2026

cherrymui commented on Aug 13, 2026

cherrymui commented on Aug 13, 2026

14 remaining items

gopherbot commented on Aug 31, 2026

mauri870 commented on Oct 8, 2026

gopherbot commented on Oct 9, 2026

scttfrdmn commented on Oct 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Output of `go env` in your module/workspace