Repository navigation
[feature](cloud) Support SHOW TRANSACTION WHERE LABEL LIKE in cloud mode - #70764
Draft
jaredmdobson wants to merge 5 commits into
Draft
jaredmdobson wants to merge 5 commits into
jaredmdobson wants to merge 5 commits into
Conversation
### Background SHOW TRANSACTION already supports label patterns in shared-nothing deployments. Cloud MetaService stores a label index that can point to several transaction attempts. ### What problem does this PR solve? Issue Number: TODO(author): the issue is opened with the PR Related PR: None PR Type: Feature Problem Summary: Cloud SHOW TRANSACTION WHERE LABEL LIKE currently returns an unsupported-mode error. A paginated MetaService scan gives the FE the transaction attempts needed to apply the existing label regex and render normal transaction rows. ### Details - Scope: Cloud-mode transaction lookup by label pattern. - Design: Scan encoded keys for the database and literal prefix, batch-read attempts, and resume by transaction ID when a page boundary splits a retried label. - Usage: SHOW TRANSACTION FROM my_db WHERE LABEL LIKE 'load_%'. - Default behavior: MetaService scans at most 10000 label keys per call and the FE returns up to 1000 matching rows sorted by descending transaction ID. - Compatibility: The new RPC reuses the existing KV format. An older MetaService produces an upgrade message when an upgraded FE attempts the new RPC. - Deployment modes: Cloud uses the RPC and the existing database LOAD check. Shared-nothing continues to use the local transaction manager. ### Testing Reviewed the SQL, FE, RPC, and KV call paths and compared protobuf definitions across supported branches. Checked changed C++ with clang-format 16 and reviewed new Java imports and line lengths. Added MetaService, FE, and cloud regression coverage for execution in CI. Doris was not built and the tests were not run locally. ### Release note Cloud deployments support SHOW TRANSACTION WHERE LABEL LIKE after MetaService is upgraded. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: Yes, document cloud support and the FE row cap in the SHOW TRANSACTION reference. - Pre-submission status: Partially verified: runtime tests and full FE Checkstyle were not run. ### AI Disclosure - Tool: Codex - Generated: Implementation, tests, and commit description - Human review: Pending Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d return newest matches Use fixed-size pages and enforce a statement-wide scan budget. Retain the newest matching transactions and bound each MetaService page by its row, read, and elapsed-time budgets. Cover resumable batches and regex filtering. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…TRANSACTION LABEL LIKE ### Background Issue Number: None ### Details Advance the cursor before honoring the page duration budget and resume partial pages after FDB read timeouts. Charge the FE budget for label and attempt reads, and enforce the statement deadline between RPCs. ### Testing clang-format 16, Java Checkstyle, and git diff --check passed. Doris builds and runtime tests were skipped. ### Release note Cloud SHOW TRANSACTION LABEL LIKE limits keys scanned per statement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HOW TRANSACTION LABEL LIKE Send the remaining key allowance to MetaService and keep complete final pages. Use fixed-width cursors and cover range-boundary resumption. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…SHOW TRANSACTION LABEL LIKE Drain fetched labels with no pending attempts at the read limit. Reject malformed cursors and negative allowances, and cover MemTxnKv pagination. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design summary
New MetaService RPC,
scan_txn_label. It range-scans the database'stxn_labelkeys under the literal prefix of the pattern. It then batch-reads thetxn_infoof every attempt recorded under each label. The scan stays inside the database's key range and never reaches another database or instance.Bounded per call. Each call is limited by:
max_scan_txn_label_keys, 2,000);txn_infokeys it reads (same 2,000 cap);max_scan_txn_label_limit, 1,000);scan_txn_label_max_duration_ms, 2,000 ms);The call stops when it hits any of these and returns an opaque
next_key.Pagination that always advances. A cursor points either at the next unread attempt inside a label (so a label with many retried attempts can span pages) or just past a finished label. Every OK page returns rows, moves the cursor forward, or ends the scan. A FoundationDB timeout or
TXN_TOO_OLDafter some progress returns the rows read so far and a cursor to the first unread item. The same error before any progress is returned as an error. The server rejects malformed cursors withINVALID_ARGUMENT.FE behavior.
CloudGlobalTransactionMgr.getDbTransInfoByLabelMatchderives the literal prefix from the regexShowTransactionCommandbuilds, and uses an empty prefix whenever pruning would be unsafe.cloud_show_txn_label_match_max_rows(1,000) matches by transaction ID, newest first.cloud_show_txn_label_match_max_scan_keys(100,000) keys and tocloud_show_txn_label_match_timeout_ms(30 s) orquery_timeout, whichever is lower. Both are checked between pages, along with connection kill.Compatibility. The proto changes are additive. An FE that calls an older MetaService gets
UNIMPLEMENTED(bRPC maps an unknown method to it), which surfaces as "Please upgrade MetaService …". Upgrade MetaService before FE.What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
In cloud mode,
SHOW TRANSACTION FROM db WHERE LABEL LIKE 'x%'fails with "Not supported in cloud mode". Shared-nothing serves it from the FE's in-memory transaction manager. In cloud mode transactions live in MetaService, and no RPC could list them by label. This PR adds a bounded, paged label scan to MetaService and uses it from the cloud transaction manager.Details
meta_service_txn.cpp(scan_txn_label),meta_service.h,meta_service_helper.h,keys.{h,cpp}(txn_label_key_prefix),bvars,config.h. Proto:cloud.proto(ScanTxnLabelRequest/Response).CloudGlobalTransactionMgr.java,MetaServiceClient.java,MetaServiceProxy.java,Config.java.SHOW TRANSACTION … WHERE LABEL LIKE.WHERE STATUS =stays unsupported in cloud mode.cloud_show_txn_label_match_max_scan_keysor narrow the pattern.KILL QUERYis noticed between pages, not during an in-flight RPC.ShowTransactionCommandapplies.Testing
UT Job on a Linux ARM64 build of the PR head:
CloudGlobalTransactionMgrTestmeta_service_test, all enabled tests exceptFdb/*, which needs a real FoundationDBScanTxnLabel,ScanTxnLabelFinishesAtExactBudgetandScanTxnLabelResumesAfterSecondRangeGetFailsdoris_cloudlinkNew coverage:
txn_info;TXN_TOO_OLDon the batch read and on the second range read, draining all 1,001 IDs in order with no duplicates.cloud_p0/test_cloud_show_txn_label_like.groovy. It runs two stream loads that share a label prefix and one that doesn't, and checks the result and its ordering. It was not run in a cloud regression environment for this PR.Release note
Cloud mode supports
SHOW TRANSACTION FROM db WHERE LABEL LIKE '...', returning up to the newest 1,000 matching transactions. Upgrade MetaService before FE.Check List (For Author)
AI Disclosure