Skip to content

Align pdsh benchmarks and library defaults #22399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TomAugspurger wants to merge 3 commits into
base: main
Choose a base branch
from
+5 −5
Open

Align pdsh benchmarks and library defaults #22399

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6ec28a8
Align pdsh benchmarks and library defaults
TomAugspurger May 6, 2026
af6140a
Merge branch 'main' into tom/cudf-polars-cli-alignment
TomAugspurger May 7, 2026
f7bbb0b
Merge branch 'main' into tom/cudf-polars-cli-alignment
TomAugspurger May 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions python/cudf_polars/cudf_polars/experimental/benchmarks/utils_new_frontends.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1752,15 +1752,15 @@ def build_parser(num_queries: int = 22) -> argparse.ArgumentParser:
)
parser.add_argument(
"--max-io-threads",
default=2,
default=4,
type=int,
help="Maximum number of IO threads for rapidsmpf runtime.",
help="Sets cudf_polars.utils.config.StreamingExecutor.max_io_threads.",
)
parser.add_argument(
"--native-parquet",
action=argparse.BooleanOptionalAction,
default=True,
help="Use C++ read_parquet nodes.",
default=False,
help="Sets cudf_polars.utils.config.ParquetOptions.use_rapidsmpf_native.",
)
parser.add_argument(
"-o",
Expand Down
2 changes: 1 addition & 1 deletion python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/options.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -643,7 +643,7 @@ def _add_cli_args(parser: argparse.ArgumentParser) -> None:
help=textwrap.dedent("""\
Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
Env: CUDF_POLARS__NUM_PY_EXECUTORS.
Built-in default: 1."""),
Built-in default: 8."""),
Comment on lines 644 to +646
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix --num-py-executors env var name in help text.

Line 645 currently references CUDF_POLARS__NUM_PY_EXECUTORS, but the field is wired to CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS (Line 313). The help text should match runtime behavior.

Suggested patch 
         help=textwrap.dedent("""\
             Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
-            Env: CUDF_POLARS__NUM_PY_EXECUTORS.
+            Env: CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS.
             Built-in default: 8."""),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
Env: CUDF_POLARS__NUM_PY_EXECUTORS.
Built-in default: 1."""),
Built-in default: 8."""),
Max workers for the Python ThreadPoolExecutor inside RapidsMPF.
Env: CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS.
Built-in default: 8."""),
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/cudf_polars/cudf_polars/experimental/rapidsmpf/frontend/options.py`
around lines 644 - 646, The help text for the --num-py-executors option
incorrectly names the env var as CUDF_POLARS__NUM_PY_EXECUTORS; update that help
string to the actual env var used at runtime,
CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS. Locate the option definition for
--num-py-executors in options.py (the field wired to
CUDF_POLARS__EXECUTOR__NUM_PY_EXECUTORS) and replace the incorrect env var
reference in its help/description so the documentation matches the configured
environment variable.

)
g.add_argument(
"--raise-on-fail",
Expand Down
Loading