Add nvbench-compare-robust --plot-output/--plot-along-output options

docs/nvbench_compare_robust.md

@@ -64,14 +64,29 @@ nvbench-compare-robust --display explain reference.json compare.json
 nvbench-compare-legacy reference.json compare.json
 ```
 
-Plot the comparison summary, or plot timings along a positive numeric axis. Add
+Plot the comparison summary, or plot timings along a positive numeric axis. By
+default, plotting uses Matplotlib's interactive `plt.show()` behavior. Add
 `--dark` to the summary plot when it should use a dark theme:
 
 ```bash
 nvbench-compare-robust --plot --dark reference.json compare.json
 nvbench-compare-robust --plot-along "Elements{io}" reference.json compare.json
 ```
 
+Save plots to files when running in CI, remote shells, or scripted workflows:
+
+```bash
+nvbench-compare-robust --plot --plot-output compare.png reference.json compare.json
+nvbench-compare-robust \
+  --plot-along "Elements{io}" \
+  --plot-along-output "plots/{benchmark}-device{device}-{axis}.png" \
+  reference.json compare.json
+```
+
+When `--plot` and `--plot-along` are used together, choose one output mode for
+both plots: either omit both output options to show both plots interactively, or
+provide both `--plot-output` and `--plot-along-output` to save both plots.
+
 Generate Python code with bulk sample/frequency filenames for every displayed
 row:
 
@@ -246,6 +261,11 @@ file. The generated script contains a `bulk_rows` list. Each entry corresponds
 to one row that `nvbench-compare-robust` prints in its display tables after all
 benchmark, axis, device, and threshold filters are applied.
 
+This output is also useful when the built-in `--plot` or `--plot-along` views
+are too generic. The generated `bulk_rows` data and `load_bulk_data(row)` helper
+let users build custom Matplotlib, Seaborn, or notebook visualizations from the
+same paired comparison rows used by the tool.
+
 Use `stdout` instead of a file path to print the generated Python code:
 
 ```bash
@@ -298,6 +318,11 @@ Each `bulk_rows` entry includes:
 The generated script also defines `load_bulk_data(row)`, which reads the
 float32 sample and frequency files for a selected row.
 
+When directory inputs are used, `bulk_rows` contains rows from all matching JSON
+file pairs. Each row records its `reference_json` and `compare_json` path, so
+custom plotting code can group rows by source file, benchmark, device, axis, or
+reason code.
+
 Select the first displayed row:
 
 ```python
@@ -317,6 +342,17 @@ If `-b` and `-a` narrow the report to one comparison of interest, the desired
 entry is usually available positionally as `bulk_rows[0]`. If duplicate states
 remain after filtering, use `occurrence` to distinguish them.
 
+Plot the selected row with regular Python plotting tools:
+
+```python
+import matplotlib.pyplot as plt
+
+plt.hist(arrays["reference_samples"], alpha=0.5, label="reference")
+plt.hist(arrays["compare_samples"], alpha=0.5, label="compare")
+plt.legend()
+plt.show()
+```
+
 ## Time Estimates And Intervals
 
 `nvbench-compare-robust` first tries to build a robust timing input for both
@@ -646,3 +682,59 @@ fraction: use `--threshold-diff 5` for a 5% threshold.
 
 This option affects table output. It does not change summary counters or the
 data used by `--plot-along`.
+
+### `--plot-output PATH`
+
+Save the summary plot generated by `--plot` to `PATH` and do not call
+`plt.show()`. When this option is omitted, `--plot` keeps the default
+interactive behavior.
+
+Interactive and file-based plot output cannot be mixed in one invocation. If
+`--plot` and `--plot-along` are both requested, provide output paths for both
+plots or omit output paths for both plots.
+
+Directory comparisons can generate one summary plot per matched JSON file pair.
+When multiple summary plots resolve to the same output path, later plots are
+written to the first available sibling path using `-copy-N` before the file
+extension, such as `compare-copy-1.png`. The same disambiguation is used if the
+requested file already exists. The actual saved path and any disambiguation
+warning are written to stderr.
+
+### `--plot-along-output PATH_OR_TEMPLATE`
+
+Save plots generated by `--plot-along` to files and do not call `plt.show()`.
+Because `--plot-along` may produce one plot per benchmark/device pair, this
+option accepts filename templates with these fields:
+
+- `{benchmark}`: benchmark name
+- `{device}`: compare-device id
+- `{axis}`: selected plot axis name
+- `{pair}`: zero-based positional device-pair index
+
+Field values are sanitized before substitution so benchmark or axis names from
+input JSON cannot introduce path separators. Directory structure should be
+written literally in the template, as in `plots/{benchmark}.png`.
+
+For example:
+
+```bash
+nvbench-compare-robust \
+  --plot-along "Elements{io}" \
+  --plot-along-output "plots/{benchmark}-pair{pair}-{axis}.png" \
+  reference.json compare.json
+```
+
+A plain path without template fields is valid. If multiple plot-along figures
+resolve to the same path, later plots are written to the first available sibling
+path using `-copy-N` before the file extension, such as
+`plot-along-copy-1.png`. The same disambiguation is used if the requested file
+already exists. The actual saved path and any disambiguation warning are
+written to stderr.
+
+To avoid copy-suffixed filenames, narrow the comparison with `--benchmark`,
+`--axis`, `--reference-devices`, or `--compare-devices`, pass the JSON files of
+interest directly instead of a directory, or add more template fields such as
+`{benchmark}`, `{device}`, `{pair}`, and `{axis}`. For duplicate-heavy
+benchmarks, crowded legends, presentation-quality plots, or a different
+grouping scheme, use `--bulk-debug-python` to export the paired rows and build a
+custom visualization.