docs(skills): add GPU performance analysis and inference optimization skills - #6129
Draft
shanmugamr1992 wants to merge 1 commit into
Draft
docs(skills): add GPU performance analysis and inference optimization skills#6129shanmugamr1992 wants to merge 1 commit into
shanmugamr1992 wants to merge 1 commit into
Conversation
… skills Add two skills covering the profile-to-optimization workflow for Megatron-Core inference performance work: - nsight-system-analysis: analyze nsys traces to find the per-iteration gap between two implementations or break down a single profile. Covers iteration anchoring, GPU busy/idle split, per-category kernel breakdown, module slicing between GEMM anchors, exposed communication, and single-forward-pass kernel composition, with scripts for each step. - optimize-inference-siddharth: what to change, in what order, and what never to break when an architecture is slower than vLLM. Covers CUDA-graph scope and bucket coverage, the inference_optimized MoE stack, Triton production hygiene, per-step host overhead, decision gates that quantify a lever's ceiling before any code is written, competitor-trace differential analysis, and the A/B protocol needed to make sub-1% wins falsifiable. The skill is self-maintaining: it documents how to extend, correct, and prune itself as campaigns learn more. Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two
skills/guides covering the profile-to-optimization workflow for Megatron-Core inference performance work. Documentation only — no changes to any library or training code.nsight-system-analysis— analyze nsys traces to find the per-iteration gap between two implementations, or break down a single profile. Covers iteration anchoring, GPU busy/idle split (union-of-intervals, not per-stream sums), per-category kernel breakdown, module slicing between GEMM anchors, exposed communication, and single-forward-pass kernel composition. Ships scripts for each step plus a report template and SQL recipes.optimize-inference-siddharth— what to change, in what order, and what never to break when an architecture is slower than vLLM. Encodes the patterns behind Siddharth Singh's (@sidsingh-nvidia) 2026 inference work: CUDA-graph scope and bucket coverage, theinference_optimizedMoE stack (NVLS AllGather-V, fused grouped GEMM, shared-expert overlap, pad-row masking), Mamba/SSM scratch sizing, Triton production hygiene, and per-step host overhead — as eleven hard rules with the failure each one prevents.It also adds methodology validated on a Qwen3-30B-A3B EP4 campaign that took mcore from 66% to 81% of the vLLM DP4+EP baseline on 4xGB200:
The second skill is self-maintaining:
references/updating-this-skill.mddocuments the triggers, routing, quality bar, deletion policy, and revision log, so campaigns extend and prune it rather than letting it go stale.Test plan
Documentation only; no code paths are touched and nothing is imported by the library or tests.
SKILL.mdfrontmatter blocks parse as YAML, withnamematching the directoryscripts/*.pybyte-compile;taxonomy_template.ymlparseslicense: Apache-2.0and SPDX headers consistent with the repo, matching the convention of the existingskills/