Repository navigation
feat: add a performance dashboard tying latency benchmarks (G1–G4) to a live chart #53
Description
H3. feat: add a performance dashboard tying latency benchmarks (G1–G4) to a live chart
Labels: visualization, performance
Priority: Low
Proposed Solution: Extend pipeline_dashboard.html to plot the CI-tracked benchmark history from B2/G4 rather than a static number.
Acceptance Criteria: [ ] dashboard reads from committed benchmark artifacts
Dependencies: B2, G4
Estimated Difficulty: M
Estimated Time: 1 day
16. Milestones / Project Board
| Milestone | Scope | Key Issues |
|---|---|---|
| M1 — Correctness Hardening | Fix the real bugs before anything else | A1, A2, A4, A6, A7, A8 |
| M2 — Validation | Prove the numbers are right, not just present | A3, A5, D2 |
| M3 — CI/Test Breadth | Cover CUDA, ML, closed-loop, fuzz, sanitizers | B1–B7 |
| M4 — Architecture & Hygiene Cleanup | Build system, repo bloat, formatting | C1–C6, F1–F6 |
| M5 — Performance Proof | Real end-to-end numbers, SIMD, CUDA comparison | G1–G4 |
| M6 — Submission Polish | Docs, visualizations, ML reproducibility, assumptions stated | D1, D3, D4, E1–E5, H1–H3 |
17. Development Roadmap
**Immediate (this week): ** A1, A2, A4, A6 — these are the correctness bugs a judge or careful reviewer would find fastest, and they're each ≤2 days.
**Next week: ** A3, A5, A8, B3, B5, B6 — validation tests and sanitizer/fuzz coverage, so the "it works" claim has evidence behind it.
**Next month: ** B1, B2, B4, D1, D2, C1, C2, F1–F4 — CI breadth, ML reproducibility, build consolidation, GitHub hygiene.
**Pre-submission: ** G1–G4 (real end-to-end performance numbers, since "10 ms real-time" is a headline PS9 criterion and needs to survive scrutiny), E5 (dataset assumptions stated), C6 (assumptions documented).
**Post-submission / research publication track: ** D3, D4, E3, H1–H3 — the kind of polish that matters more for an SN Computer Science-style paper than for the hackathon deadline itself.
18. Final Scorecard (0–10)
| Category | Score | Rationale |
|---|---|---|
| Scientific Correctness | 7 | Core math is standard and appears implemented correctly on inspection; unproven by dedicated golden-value/end-to-end validation tests |
| Software Engineering | 6 | Real modular C library with a working build, but struct-duplication landmine (A2) and 9 redundant build scripts (C1) |
| Documentation | 8 | Unusually thorough for a hackathon repo — 11 docs files, dedicated compliance mapping doc, paper draft |
| Mathematics | 7 | Zernike/Fried/Kolmogorov/coupling models are textbook-appropriate; missing convergence/condition reporting (A8) |
| AI/ML | 6 | Mature pipeline breadth (MLP/CNN/LSTM/ablation/noise-robustness), but headline metrics not reproducible from a fresh clone (D1/B4) |
| Performance | 6 | OpenMP + CUDA present; latency claims not independently, end-to-end verified (G1) |
| Real-Time Capability | 6 | Architecturally sound for <10 ms, but the actual end-to-end number (vs. hot-path-only microbenchmark) is unverified |
| Visualization | 8 | Strong breadth of static and interactive visualizations |
| Research Depth | 7 | Correct concepts, missing explicit literature citation trail (E3) |
| Innovation | 6 | Predictive-AO LSTM and coupling-based DM model are reasonable, not groundbreaking, choices — appropriate for the problem |
| Reproducibility | 5 | Weakest category — ML artifacts and some benchmark numbers cannot currently be regenerated from git clone alone |
| Maintainability | 6 | Good in-code documentation and naming; undermined by struct duplication (A2) and build-script sprawl (C1) |
Composite: 6.8 / 10
19. Risk Assessment
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| ISRO evaluators ask to reproduce the ML/latency numbers and can't from the public repo | High | High | D1, B4, G1 |
| A hidden test-set-vs-real-dataset geometry mismatch surfaces only at submission time | Medium | High | A7, E5 |
| Struct-duplication bug (A2) causes a crash under a future contributor's change, discovered late | Medium | High | A2, address before any further feature work on rippra_api.c |
| Reviewer discounts the submission for unlabeled simplifying assumptions (DM coupling model) looking like an oversight rather than a deliberate choice | Medium | Medium | C6, E5 |
| Undiscovered memory bug in the C core surfaces during a live demo | Low–Medium | High | B6 (sanitizers), B5 (fuzz) |
20. Final Verdict
🟢 Submit-capable with a focused hardening pass, not a rewrite. The prior stale audit's "do not submit" verdict no longer reflects the codebase. The core PS9 deliverables are implemented with reasonable algorithmic choices. The highest-leverage work between now and submission is not new features — it's (1) fixing the handful of real correctness bugs in Category A, (2) proving the numbers with validation tests and end-to-end benchmarks rather than asserting them in docs, and (3) making the ML claims reproducible from a clean clone. Everything in Categories C, F, and H is polish that helps a reviewer trust the project but won't change whether the core algorithm works.
Activity
purvanshjoshi commentedon Jul 1, 2026
[LEGIT - DECISION & IMPLEMENTATION PLAN]
This is a legitimate dashboard enhancement.
To implement this:
visualizations/pipeline_dashboard.htmlto load benchmark logs dynamically.purvanshjoshi commentedon Jul 4, 2026
This issue remains valid. No live performance dashboard plots CI-tracked benchmark history.
Suggested approach:
Shivamkumar31 commentedon Aug 1, 2026
i like to work on this
Shivamkumar31 commentedon Aug 8, 2026
Hi @sr-857, I'd like to work on this issue.
I understand the goal as connecting the existing G1–G4 performance benchmarks to a historical dashboard rather than displaying static latency values.
My planned approach is:
visualizations/pipeline_dashboard.htmlto consume that history and render time-series latency charts.I'll first inspect the existing benchmark/CI implementation to avoid duplicating functionality. If this approach matches the intended direction, I'd be happy to take this issue.
purvanshjoshi commentedon Aug 9, 2026
Thanks for the detailed plan, @Shivamkumar31 — the direction (history, not static values) is right. Before we talk assignment, a few technical points I'd like your take on, since I want to make sure the fit is right for this one:
The file referenced in the issue doesn't exist yet. The issue says "extend
pipeline_dashboard.html", but there is novisualizations/pipeline_dashboard.htmlin the repo today (only static PNGs/mockups). So this is effectively building a new dashboard, not extending one. How would you approach scaffolding it, and would you keep it as a self-contained static page (no build step) or generate it?Where does the history live? CI currently uploads benchmark outputs (
centroid.txt,e2e.txt,openmp.txt,simd.txt,scaling.txt) as a per-run artifact namedperformance-benchmarks(.github/workflows/ci.yml:139-143). GitHub Actions artifacts are per-run and ephemeral (default retention 90 days) — they are not a durable cross-run history. Your step 3 says "persist through the existing CI/artifact mechanism". If artifacts expire, how would the dashboard accumulate historical points across many runs? Would you commit a JSON history file to the repo, use a dedicated branch, or rely on artifact download APIs?There's already a parser/regression check.
tools/check_benchmark_regression.pyalready regex-parses those text files and compares againstbenchmarks/baseline.json(threshold +20%). Your step 2 proposes a JSON standard. Would your JSON schema replace that parser, or coexist with it? Where would the JSON be generated — inside the C benchmarks themselves, or a Python converter in CI?Noise on shared runners. The benchmarks run on
ubuntu-latest(2-vCPU shared runners). Raw per-run latency is noisy (the baseline file itself is annotated "measured on ubuntu-latest GH Actions runner"). A naive time-series chart will look like a scatter plot. How would you handle noise — rolling median, min-of-N runs, confidence bands — so the dashboard shows trends rather than runner jitter?G1–G4 scope. G1–G4 in the audit doc span centroiding, reconstruction, DM mapping, end-to-end hot-path, plus SIMD and OpenMP scaling. Which subset do you consider "canonical" for the primary chart, and how would you represent the OpenMP multi-thread series (
scaling.txt, 1/2/4/8 threads) in the same visualization?Not assigning yet — I'd like to see how you'd answer these, then we can proceed. If you'd rather discuss a specific point or propose adjustments, happy to hear it.