feat(ds4): add indexed layer-major prefill

[davide221]

Summary

• add typed exact, dense, and sparse DeepSeek V4 prefill modes; exact
  remains the default
• execute opt-in prefill layer-major with batched causal compressor updates and
  device-resident HC/FFN state
• add the DS4 learned-indexer QAT/score/mask kernels and D=512 compact sparse
  attention
• add gfx1151 ROCmFPX batched MMQ, shared activation quantization, and grouped
  output projection without materialized permutations

The branch is rebased directly on current main (a8465cc, including #496).
The feature uses the typed --ds4-prefill option and adds no prefill tuning
environment variables.

Integration and safety

• optimized prefill requires one local HIP backend with all experts resident;
  unsupported split, remote, hybrid, CUDA, and non-DS4 configurations fail
  during initialization instead of silently selecting a wrong graph
• DS4 flash attention has an explicit backend contract; CPU and unsupported GPU
  launches reject it rather than falling through to generic attention
• grouped-source projection is a distinct GGML operation, so unsupported
  backends cannot silently consume its physical layout as an ordinary matmul
• HC pre/post/output operators reject unsupported higher-dimensional planes
  instead of silently dropping them
• inverse-RoPE fallback output uses dedicated LDS scratch, avoiding a race with
  concurrent softmax-score readers
• graph allocators, model-owned host mirrors, and cached tensor pointers are
  released before model teardown/reload
• the rebase preserves feat(ds4): add DSpark speculative decoding (+27% on Strix Halo) #496's DSpark fused-verification path for q=2-4; longer
  prefill batches use the layer-major path
• exact prefill remains tokenwise and unchanged

Performance

DeepSeek-V4-Flash-ROCMFP2-STRIX.gguf, gfx1151 at 2.9 GHz, ROCm 7.2.4,
--chunk 2048 --max-ctx 8192 --ds4-prefill sparse, all experts resident, no
draft, and prefix/prefill/disk caches disabled.

Three consecutive 2K -> 4K -> 8K cycles ran in one server process:

Cycle	2,139 tokens	4,039 tokens	7,649 tokens
Cold	192.5 tok/s	224.4 tok/s	246.2 tok/s
Warm 1	297.5 tok/s	275.4 tok/s	252.8 tok/s
Warm 2	297.3 tok/s	276.2 tok/s	251.0 tok/s

RSS was identical after both warm cycles (101,715,404 KiB), confirming a stable
allocator high-water mark across mixed prompt shapes. Earlier clean runs on the
less memory-pressured lucebox2 reached 327-328 tok/s at 2K, 313-317 tok/s at
4K, and 281-289 tok/s at 8K.

This supports approximately 300 tok/s for warm 2K prefill, not 300+ tok/s at
every context length. A separate ROCmFP3 long-context run sustained 248-256
tok/s around 8K and 221.9 tok/s at 23.6K.

Correctness boundary

• Candidate exact mode and current-main exact mode produced byte-identical
  deterministic output in two runs each: 44 tokens, SHA-256
  d0dce91aeef336d620595aec276c28cb7deeafb58628686f5dadb5d519f79e55.

• On a real ROCmFP3 artifact, deterministic scored smoke tests passed equally:

  Mode	GSM8K smoke	HumanEval smoke
  exact tokenwise	10/10	3/3
  dense, chunk 2048	10/10	3/3
  sparse, chunk 2048	10/10	3/3

• dense changes floating-point reduction topology and sparse additionally
  prunes cache entries. They are explicitly opt-in/approximate and may change
  generated tokens; the scored smoke gate establishes task-level parity on this
  small corpus, not byte-identical logits or broad benchmark equivalence.

Validation

• clean ROCm 7.2/gfx1151 Release build on rebased head ef3f53a
• full CTest suite on lucebox4: 20/20 passed, including ROCmFPX, DS4 HC,
  indexed-attention, fused-verification, and server tests
• NVIDIA sm_86 engine build (HIP-only DS4 kernels excluded from CUDA)
• GPU regressions for configured sparse keep cap > live history, inverse-RoPE
  fallback scratch isolation, and row-strided HC-post input
• CPU reference tests for grouped output projection and indexer QAT/score/mask
• exact candidate-vs-main raw-token comparison
• three-cycle mixed-context throughput and memory-stability gate
• git diff --check; no conflict markers

Review follow-up

Commit 8dc4602 resolves the valid Cubic findings: inverse-RoPE scratch race,
three HC shape-contract gaps, and the grouped-source backend contract. Commit
410acd8 gates the HIP-only attention regressions on CUDA builds. Reported
telemetry fields were already aggregated. Later long chunks intentionally
rebuild when kv_start changes graph topology; a position-only rebind would be
incorrect and remains a separate optimization. Commit ef3f53a resolves the
#496 rebase integration without changing the optimized prefill contract.

[cubic-dev-ai]

All reported issues were addressed across 35 files

Not reviewed (too large): server/src/deepseek4/deepseek4_graph.cpp (~2,141 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[davide221]

Final review completed at a682401.

• GitHub reports the PR MERGEABLE / CLEAN; all final-head checks are green and no review threads remain unresolved.
• Lucebox4: final HIP rebuild succeeded, CTest passed 20/20, and deepseek4_unit passed 100 consecutive runs. The tested source matched the pushed head byte-for-byte.
• Correctness boundary: the default --ds4-prefill exact path matched current main byte-for-byte for the 44-token comparison (SHA-256 d0dce91aeef336d620595aec276c28cb7deeafb58628686f5dadb5d519f79e55). dense and sparse are explicit opt-in approximate modes; the roughly 300 tok/s measurements are from sparse, not the byte-identical path.
• The core DS4 graph implementation was manually audited because Cubic skipped that large file. Valid review findings were fixed; stale or unsafe suggestions were not applied.

No known merge blocker remains. A final full-model ROCmFPX throughput rerun was not possible on Lucebox4 because its target model artifact is incomplete; the recorded sustained throughput comes from the earlier performance head, while subsequent commits are correctness hardening and dead-code removal.

[davide221]