Align pdsh benchmarks and library defaults

[TomAugspurger]

Description

This updates the defaults in our pdsh benchmarks to match the defaults from the library. This will make it easier to understand what values are being used based just on the command run.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[pentschev]

LGTM. Thanks Tom.

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Chores
  • Updated CLI argument defaults: --max-io-threads now defaults to 4, and --native-parquet now defaults to False.
  • Corrected CLI help text for various command-line arguments to reflect accurate default values and configuration mappings.

Walkthrough

CLI argument defaults and help text are updated across benchmark and streaming configuration files to reflect actual runtime behavior. The --max-io-threads default increases from 2 to 4, --native-parquet defaults to False instead of True, and --num-py-executors help text corrects the documented default from 1 to 8.

Changes

CLI Configuration Updates

Layer / File(s)	Summary
CLI argument defaults and documentation python/cudf_polars/cudf_polars/experimental/benchmarks/utils_new_frontends.py, python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/options.py	Default values and help text updated for benchmark and streaming configuration arguments: --max-io-threads default increased to 4, --native-parquet default set to False, and --num-py-executors help text corrected to reflect actual default of 8.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Align pdsh benchmarks and library defaults' accurately and directly describes the main change in the pull request—updating benchmark default values to match library defaults across two configuration files.
Description check	✅ Passed	The description is directly related to the changeset and explains the intent of aligning pdsh benchmark defaults with library defaults, matching the purpose of the code modifications.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/options.py`:
- Around line 644-646: The help text for the --num-py-executors option
incorrectly names the env var as CUDF_POLARS__NUM_PY_EXECUTORS; update that help
string to the actual env var used at runtime,
CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS. Locate the option definition for
--num-py-executors in options.py (the field wired to
CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS) and replace the incorrect env var
reference in its help/description so the documentation matches the configured
environment variable.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e862b61a-eafd-41bc-98f1-cedbe61d240b

📥 Commits

Reviewing files that changed from the base of the PR and between 7d84936 and f7bbb0b.

📒 Files selected for processing (2)

• python/cudf_polars/cudf_polars/experimental/benchmarks/utils_new_frontends.py
• python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/options.py

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix --num-py-executors env var name in help text.

Line 645 currently references CUDF_POLARS__NUM_PY_EXECUTORS, but the field is wired to CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS (Line 313). The help text should match runtime behavior.

Suggested patch

         help=textwrap.dedent("""\
             Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
-            Env: CUDF_POLARS__NUM_PY_EXECUTORS.
+            Env: CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS.
             Built-in default: 8."""),

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
	Env: CUDF_POLARS__NUM_PY_EXECUTORS.
	Built-in default: 1."""),
	Built-in default: 8."""),
	Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
	Env: CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS.
	Built-in default: 8."""),

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/options.py`
around lines 644 - 646, The help text for the --num-py-executors option
incorrectly names the env var as CUDF_POLARS__NUM_PY_EXECUTORS; update that help
string to the actual env var used at runtime,
CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS. Locate the option definition for
--num-py-executors in options.py (the field wired to
CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS) and replace the incorrect env var
reference in its help/description so the documentation matches the configured
environment variable.