Skip to content

session: use resource group runtime state for paging#69633

Open
JmPotato wants to merge 1 commit into
pingcap:masterfrom
JmPotato:codex/rc-effective-hard-limit
Open

session: use resource group runtime state for paging#69633
JmPotato wants to merge 1 commit into
pingcap:masterfrom
JmPotato:codex/rc-effective-hard-limit

Conversation

@JmPotato

@JmPotato JmPotato commented Jul 3, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: ref tikv/pd#10612

Problem Summary:

PD resource control now exposes whether the runtime request-unit token bucket has a limited burst. TiDB's tidb_paging_size_bytes gate still depended only on the static resource-group metadata in InfoSchema, so it could miss limited-burst settings produced by service-limit override logic.

This PR intentionally points TiDB at temporary personal fork commits to validate the cross-repo integration before the upstream PD/client-go dependency pins are available.

What changed and how does it work?

  • Replaces github.com/tikv/pd/client with github.com/JmPotato/pd/client v0.0.0-20260703063321-7f8ee3eaf371.
  • Replaces github.com/tikv/client-go/v2 with github.com/JmPotato/client-go/v2 v2.0.0-20260703060738-b9da313d68a6 for mock compatibility with the updated PD client interface.
  • Makes GetDistSQLCtx prefer ResourceGroupsController.GetResourceGroupRuntimeState().HasLimitedBurst when deciding whether tidb_paging_size_bytes can be forwarded.
  • Falls back to the existing InfoSchema burst-limit check while the runtime state is not available yet.
  • Updates TiDB test mocks for the newer PD client watch response and keyspace/service-discovery interfaces.
  • Regenerates Bazel deps and BUILD files with make bazel_prepare.

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)
  • Integration test
  • No need to test

Manual test steps:

go test ./pkg/session -run TestDistSQLCtxPagingSizeBytesRequiresHardCappedResourceGroup -count=1
go test ./pkg/domain/infosync -run TestNonExistent -count=1
go test ./pkg/executor -run TestNonExistent -count=1
go test ./pkg/store/mockstore/unistore -run TestNonExistent -count=1
make bazel_prepare
bazel test --remote_cache=https://cache.hawkingrei.com/bazelcache --noremote_upload_local_results //pkg/session:session_test --test_filter=TestDistSQLCtxPagingSizeBytesRequiresHardCappedResourceGroup
bazel test --remote_cache=https://cache.hawkingrei.com/bazelcache --noremote_upload_local_results //pkg/domain/infosync:infosync_test //pkg/executor:executor_test //pkg/store/mockstore/unistore:unistore_test --test_filter=TestNonExistent
git diff --check

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Fix the tidb_paging_size_bytes gating so Resource Control limited-burst state reported by PD is recognized before falling back to static resource-group metadata.

Summary by CodeRabbit

  • Bug Fixes

    • Improved resource control paging behavior so paging size limits are applied more consistently.
    • Keyspace lookups now return a not-found error when an entry is missing, avoiding unexpected failures.
    • Updated mock watch/discovery behavior to better align with current runtime watch responses.
  • Chores

    • Refreshed multiple Go dependencies (including OpenTelemetry, gRPC, OAuth).
    • Updated pinned dependency metadata and removed an obsolete pinned package.

@ti-chi-bot ti-chi-bot Bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jul 3, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 3, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign bb7133 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 3, 2026
@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR updates Go dependency pins and replaces two tikv client paths with forked versions. It also switches resource-manager mocks to PD metastorage watch responses, refactors session paging checks, and adjusts mockstore PD keyspace lookup behavior.

Changes

Dependency pin updates

Layer / File(s) Summary
Pin updates and replace directives
DEPS.bzl, go.mod
Bumps multiple dependency pins, removes github.com/zeebo/errs, and adds replace directives for github.com/tikv/pd/client and github.com/tikv/client-go/v2 to forked pseudo-versions.

PD metastorage watch API migration

Layer / File(s) Summary
Resource manager mock watch channel migration
pkg/domain/infosync/BUILD.bazel, pkg/domain/infosync/resource_manager_client.go, pkg/executor/adapter_internal_test.go
Adds the PD metastorage dependency and changes mock watch emission/return types from raw event slices to *metastorage.WatchResponse.
Session resource group paging helper
pkg/session/session.go
Moves paging-size gating into resourceGroupAllowsPagingSizeBytes, which checks runtime resource-group state first and then infoschema fallback.
Mockstore PD keyspace and client conn updates
pkg/store/mockstore/unistore/pd.go
Adds a no-op RemoveClientConn and refactors keyspace lookup to return ENTRY_NOT_FOUND for missing IDs or names, plus LoadKeyspaceByID.

Estimated code review effort: 2 (Simple) | ~12 minutes

Possibly related PRs

  • pingcap/tidb#68091: Also changes pkg/session/session.go around PagingSizeBytes and resource-control gating.
  • pingcap/tidb#69320: Also updates overlapping dependency pins in DEPS.bzl and go.mod.
  • pingcap/tidb#69503: Also touches DEPS.bzl generation and dependency pin updates for Go modules.

Suggested labels: ok-to-test, approved, lgtm

Suggested reviewers: cfzjywxk, yudongusa, zyguan

Poem

I’m a bunny with a tiny grin,
New deps hop out, new deps hop in,
WatchResponses wiggle, bright and neat,
Paging bytes now tap their feet,
Thump-thump—this patch is quite the treat 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately highlights the main change: using resource-group runtime state for paging decisions.
Description check ✅ Passed The description covers the problem, issue reference, implementation details, tests, side effects, and release note, matching the template well.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jul 3, 2026
Signed-off-by: JmPotato <github@ipotato.me>
@JmPotato JmPotato force-pushed the codex/rc-effective-hard-limit branch from a71b708 to 23f168d Compare July 3, 2026 06:36

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/session/session.go (1)

3552-3563: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add unit test coverage for the new paging helper.

resourceGroupAllowsPagingSizeBytes is a pure function with three distinct branches (nil/empty guard, runtime-state path, infoschema fallback) implementing the exact behavior this PR is meant to fix. A focused unit test exercising each branch would guard against regressions in this hard-limit detection logic.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/session/session.go` around lines 3552 - 3563, Add focused unit tests for
resourceGroupAllowsPagingSizeBytes to cover all three branches: the nil/empty
guard, the ResourceGroupsController runtime-state path, and the InfoSchema
fallback. Use the function name resourceGroupAllowsPagingSizeBytes and the
Domain/ResourceGroupsController/GetResourceGroupRuntimeState/InfoSchema.ResourceGroupByName
paths to build table-driven cases that assert the expected boolean result for
each scenario.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/session/session.go`:
- Around line 3552-3563: Add focused unit tests for
resourceGroupAllowsPagingSizeBytes to cover all three branches: the nil/empty
guard, the ResourceGroupsController runtime-state path, and the InfoSchema
fallback. Use the function name resourceGroupAllowsPagingSizeBytes and the
Domain/ResourceGroupsController/GetResourceGroupRuntimeState/InfoSchema.ResourceGroupByName
paths to build table-driven cases that assert the expected boolean result for
each scenario.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 45c5e12d-58be-4f33-b7e6-c4df9a4a82d0

📥 Commits

Reviewing files that changed from the base of the PR and between 8be4bd0 and a71b708.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (7)
  • DEPS.bzl
  • go.mod
  • pkg/domain/infosync/BUILD.bazel
  • pkg/domain/infosync/resource_manager_client.go
  • pkg/executor/adapter_internal_test.go
  • pkg/session/session.go
  • pkg/store/mockstore/unistore/pd.go

@ti-chi-bot

ti-chi-bot Bot commented Jul 3, 2026

Copy link
Copy Markdown

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

📖 For more info, you can check the "Contribute Code" section in the development guide.


Notice: To remove the do-not-merge/needs-tests-checked label, please finished the tests then check the finished items in description.

For example:

Tests <!-- At least one of them must be included. -->

- [x] Unit test
- [ ] Integration test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No code

‼️ Must keep the HTML comments <!-- At least one of them must be included. -->

📖 For more info, you can check the "Contribute Code" section in the development guide.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/session/session.go (1)

3552-3563: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a comment explaining the hard-limit/paging rationale.

The logic here is non-obvious: paging is only allowed when the resource group has a limited (hard-capped) burst — state.HasLimitedBurst or GetBurstLimitAdjusted() >= 0 — and disabled when the group is unlimited. Without context, a reader could easily assume the opposite (that an unlimited group would allow larger paging). A short comment on why hard-capped burst is required to safely enable paging (and why runtime state is preferred over InfoSchema) would aid future maintainers.

📝 Proposed comment addition
+// resourceGroupAllowsPagingSizeBytes reports whether paging can be enabled for the given
+// resource group. Paging is only safe when the group has a hard-capped (limited) burst,
+// since an unlimited-burst group may not enforce the same throttling contract that paging
+// relies on. Runtime resource-group state is preferred; InfoSchema's static burst-limit
+// metadata is used as a fallback until runtime state is available (e.g. right after group
+// creation or a service-limit override).
 func resourceGroupAllowsPagingSizeBytes(dom *domain.Domain, resourceGroupName string) bool {

Based on coding guidelines: "Comments SHOULD explain non-obvious intent, constraints, invariants, concurrency guarantees, SQL/compatibility contracts, or important performance trade-offs."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/session/session.go` around lines 3552 - 3563, Add a short explanatory
comment in resourceGroupAllowsPagingSizeBytes clarifying that paging is only
enabled for resource groups with a hard-limited burst (state.HasLimitedBurst or
GetBurstLimitAdjusted() >= 0), not for unlimited groups. Mention that the
runtime state from ResourceGroupsController is preferred when available because
it reflects the current effective limit, with InfoSchema used only as a
fallback.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/session/session.go`:
- Around line 3552-3563: Add a short explanatory comment in
resourceGroupAllowsPagingSizeBytes clarifying that paging is only enabled for
resource groups with a hard-limited burst (state.HasLimitedBurst or
GetBurstLimitAdjusted() >= 0), not for unlimited groups. Mention that the
runtime state from ResourceGroupsController is preferred when available because
it reflects the current effective limit, with InfoSchema used only as a
fallback.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 5ea9a75c-7b3f-40a0-be55-9d52fa63001f

📥 Commits

Reviewing files that changed from the base of the PR and between a71b708 and 23f168d.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (7)
  • DEPS.bzl
  • go.mod
  • pkg/domain/infosync/BUILD.bazel
  • pkg/domain/infosync/resource_manager_client.go
  • pkg/executor/adapter_internal_test.go
  • pkg/session/session.go
  • pkg/store/mockstore/unistore/pd.go
🚧 Files skipped from review as they are similar to previous changes (6)
  • pkg/domain/infosync/BUILD.bazel
  • pkg/executor/adapter_internal_test.go
  • pkg/domain/infosync/resource_manager_client.go
  • go.mod
  • pkg/store/mockstore/unistore/pd.go
  • DEPS.bzl

@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 56.86275% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.6693%. Comparing base (b15a36a) to head (23f168d).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #69633        +/-   ##
================================================
- Coverage   76.3233%   75.6693%   -0.6541%     
================================================
  Files          2041       2078        +37     
  Lines        560726     582364     +21638     
================================================
+ Hits         427965     440671     +12706     
- Misses       131860     139566      +7706     
- Partials        901       2127      +1226     
Flag Coverage Δ
integration 45.6037% <56.8627%> (+5.8985%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4471% <ø> (ø)
parser ∅ <ø> (∅)
br 63.9515% <ø> (+1.2302%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ti-chi-bot

ti-chi-bot Bot commented Jul 3, 2026

Copy link
Copy Markdown

@JmPotato: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-build-next-gen 23f168d link true /test pull-build-next-gen
idc-jenkins-ci-tidb/unit-test 23f168d link true /test unit-test
idc-jenkins-ci-tidb/build 23f168d link true /test build
pull-unit-test-next-gen 23f168d link true /test pull-unit-test-next-gen
pull-integration-realcluster-test-next-gen 23f168d link true /test pull-integration-realcluster-test-next-gen

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant