feat(ds4): add DSpark speculative decoding (+27% on Strix Halo)

[davide221]

Summary

• bridge DeepSeek4/MTP DSpark auxiliary heads into the draft GGUF contract
• add the DeepSeek4 DSpark loader, proposal graph, target feature capture, batched verification, rollback, and streaming/cancellation integration
• add confidence-adaptive q=2/3/4 verification while retaining fixed-width controls
• preserve the feat(ds4): add fused HIP decode for ROCmFPX #503 fused ROCmFPX target path and leave standard autoregressive generation unchanged unless DFLASH_DS4_SPEC=1

Rebase and hardening

This branch is rebuilt directly on current main after #502 and #503 merged. The rebase removes the old stacked-branch conflict and includes lifecycle/error-path hardening found during review:

• current backend and generation-result APIs
• target/drafter compatibility, tensor-shape, tensor-contract, and file-bounds validation
• graph, snapshot, backend, and drafter cleanup
• repeated-request snapshot safety
• streaming callbacks, cancellation, and failure propagation
• verifier hooks that correctly bypass cached single-token graphs when captures/logits are required
• fused verification that preserves capture ordering, ratio-4 boundaries, hash-table bounds, and the configured routed-expert top-k
• rejection rollback that restores overwritten SWA history after the raw KV ring wraps

Strix Halo validation

Validated on a 128 GiB Strix Halo Radeon 8060S with ROCm 7.2.4, monolithic fused decode, --ds4-expert-top-k 4, platform profile performance, and GPU performance level high.

• full HIP build: pass
• CTest: 20/20 pass
• synthetic malformed-shape, unknown-tensor, and truncated-GGUF loader regressions: pass
• real 11 GiB DSpark drafter load + GPU forward: pass
• wrapped-ring partial-rejection and full-restore regression: pass
• targeted post-wrap restore + normal replay: response text exactly matched a direct normal-verifier run
• final confidence-adaptive production run after the rollback fix: 10/10 correct (5 GSM + 5 Math), 2,437 output tokens, 31.30 tok/s weighted
• matched autoregressive control: 10/10 correct, 25.31 tok/s weighted

Weighted throughput is total completion tokens divided by summed decode time. The measured DSpark uplift over the matched AR control is about 24%.

The 30+ tok/s result is configuration-specific: DSpark, fused decode, --ds4-expert-top-k 4, and the stated performance/high clock profiles. The expert-top-k flag is a separate approximate inference policy, not part of DSpark itself. Batched and token-at-a-time graph shapes can also differ around near-tied logits; quality was scored directly rather than claiming bit-for-bit AR equivalence. The DS4 documentation now calls out that DFLASH_DS4_FUSED_VERIFY=1 is the opt-in throughput profile and is not a byte-identity mode.

Follow-up

Remote DSpark execution and shared transport remain isolated in #505.

[davide221]

Validation update (2026-07-09): the converter/loader bridge in this PR was validated on lucebox2 with the converted DSpark drafter artifact. In the follow-on DeepSeek4 runtime worktree, DSpark loads successfully, prefill feature capture and generation are wired, the batched-verify causality bug is fixed, and causal batched output matched exact sequential verification in paired probes. Adaptive EWMA width and fused batched verify are working; fused verify reached approximately 20 tok/s (peak 20.1 tok/s), versus roughly 13–15 tok/s for the earlier dynamic causal path. Exact sequential verification measured 1.78–1.91 accepted candidates per 4-candidate step. These runtime changes are follow-on work in the hub-494 experiment worktree and are not claimed as already included in PR 496.

[davide221]

Updated DSpark adaptive width in 1fc4f00.

• Fused Markov generation now returns confidence scores in the existing token-id synchronization (no extra host round trip).
• A compatible confidence head automatically selects q=2/q=3/q=4 from cumulative prefix confidence.
• Artifacts without a compatible confidence head fall back to the existing EWMA controller.
• No new environment variables or rejected confidence+EWMA hybrid policy were added.
• Expert k=3 is not included.

Validation on lucebox2 / gfx1151 with the exact committed source:

• HIP dflash_server build: pass.
• GSM 5/5 + MATH 5/5: 10/10.
• Server-side decode: 1,485 tokens / 50.769 s = 29.25 tok/s weighted.
• Fixed-q=4 reference: 29.49 tok/s weighted.

The confidence policy therefore retains essentially all high-acceptance throughput while adapting verification width on harder prompts.

[cubic-dev-ai]

All reported issues were addressed across 14 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 15 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

2 issues found across 15 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/deepseek4/deepseek4_dspark.h">

<violation number="1" location="server/src/deepseek4/deepseek4_dspark.h:132">
P3: `GenerateRequest` forward declaration is unused and suggests a dependency this API does not have; remove it to keep the DSpark interface minimal.</violation>
</file>

<file name="server/src/deepseek4/deepseek4_dspark_spec.cpp">

<violation number="1" location="server/src/deepseek4/deepseek4_dspark_spec.cpp:462">
P3: After the feature window fills, every committed token copies the entire DSpark context window, adding avoidable multi-megabyte host-memory traffic to the decode hot path. Keep feature columns in a circular buffer (materializing contiguous context only when required) so speculative throughput does not degrade after 128 positions.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>