Skip to content

[feature](cloud) Support SHOW TRANSACTION WHERE LABEL LIKE in cloud mode - #70764

Draft
jaredmdobson wants to merge 5 commits into
apache:masterfrom
jaredmdobson:feat-cloud-show-txn-label-like
Draft

jaredmdobson wants to merge 5 commits into
apache:masterfrom
jaredmdobson:feat-cloud-show-txn-label-like

Conversation

@jaredmdobson

Copy link
Copy Markdown

Design summary

  • New MetaService RPC, scan_txn_label. It range-scans the database's txn_label keys under the literal prefix of the pattern. It then batch-reads the txn_info of every attempt recorded under each label. The scan stays inside the database's key range and never reaches another database or instance.

  • Bounded per call. Each call is limited by:

    • the label keys it reads (max_scan_txn_label_keys, 2,000);
    • the txn_info keys it reads (same 2,000 cap);
    • the rows it returns (max_scan_txn_label_limit, 1,000);
    • a time budget (scan_txn_label_max_duration_ms, 2,000 ms);
    • the remaining key allowance the FE sends.

    The call stops when it hits any of these and returns an opaque next_key.

  • Pagination that always advances. A cursor points either at the next unread attempt inside a label (so a label with many retried attempts can span pages) or just past a finished label. Every OK page returns rows, moves the cursor forward, or ends the scan. A FoundationDB timeout or TXN_TOO_OLD after some progress returns the rows read so far and a cursor to the first unread item. The same error before any progress is returned as an error. The server rejects malformed cursors with INVALID_ARGUMENT.

  • FE behavior.

    • CloudGlobalTransactionMgr.getDbTransInfoByLabelMatch derives the literal prefix from the regex ShowTransactionCommand builds, and uses an empty prefix whenever pruning would be unsafe.
    • It pages through the scan, applies the regex, and keeps the newest cloud_show_txn_label_match_max_rows (1,000) matches by transaction ID, newest first.
    • The statement is limited to cloud_show_txn_label_match_max_scan_keys (100,000) keys and to cloud_show_txn_label_match_timeout_ms (30 s) or query_timeout, whichever is lower. Both are checked between pages, along with connection kill.
    • If the scan cannot finish within those limits, the statement fails and asks for a longer literal prefix. It never returns a silently partial list.
  • Compatibility. The proto changes are additive. An FE that calls an older MetaService gets UNIMPLEMENTED (bRPC maps an unknown method to it), which surfaces as "Please upgrade MetaService …". Upgrade MetaService before FE.

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

In cloud mode, SHOW TRANSACTION FROM db WHERE LABEL LIKE 'x%' fails with "Not supported in cloud mode". Shared-nothing serves it from the FE's in-memory transaction manager. In cloud mode transactions live in MetaService, and no RPC could list them by label. This PR adds a bounded, paged label scan to MetaService and uses it from the cloud transaction manager.

Details

  • Change map: about 440 production lines and 600 test lines.
    • MetaService: meta_service_txn.cpp (scan_txn_label), meta_service.h, meta_service_helper.h, keys.{h,cpp} (txn_label_key_prefix), bvars, config.h. Proto: cloud.proto (ScanTxnLabelRequest/Response).
    • FE: CloudGlobalTransactionMgr.java, MetaServiceClient.java, MetaServiceProxy.java, Config.java.
  • Scope: cloud mode, SHOW TRANSACTION … WHERE LABEL LIKE. WHERE STATUS = stays unsupported in cloud mode.
  • Default behavior: shared-nothing is unchanged. Cloud returns at most the newest 1,000 matches. Shared-nothing has no row cap.
  • Known limits:
    • If the statement key budget runs out exactly when a range read fills its limit, the statement can fail even though no keys are left. Raise cloud_show_txn_label_match_max_scan_keys or narrow the pattern.
    • Keys read by MetaService's internal retries are not charged across attempts.
    • KILL QUERY is noticed between pages, not during an in-flight RPC.
  • Privileges: unchanged. The existing database-level check in ShowTransactionCommand applies.

Testing

UT Job on a Linux ARM64 build of the PR head:

Step Result
FE CloudGlobalTransactionMgrTest 73 run, 0 failed
FE Checkstyle passed
Cloud meta_service_test, all enabled tests except Fdb/*, which needs a real FoundationDB 313 run, 0 failed, including ScanTxnLabel, ScanTxnLabelFinishesAtExactBudget and ScanTxnLabelResumesAfterSecondRangeGetFails
doris_cloud link passed
clang-format 16 on changed C++ passed

New coverage:

  • MetaService:
    • prefix ranges, including escaped bytes and adjacent database IDs;
    • labels with several attempts, and labels holding only a versionstamp;
    • recycled txn_info;
    • limit clamping and per-request key allowance;
    • an exact-budget finish;
    • cursors within a label and between labels, checked to always advance (128/130/8 pages);
    • malformed, truncated, overflowing and legacy cursors;
    • injected timeout and TXN_TOO_OLD on the batch read and on the second range read, draining all 1,001 IDs in order with no duplicates.
  • FE:
    • prefix derivation;
    • regex filter;
    • newest-N across pages;
    • fixed page size;
    • scan-key budget and timeout errors;
    • unchanged-cursor rejection;
    • the old-MetaService error.
  • Regression: cloud_p0/test_cloud_show_txn_label_like.groovy. It runs two stream loads that share a label prefix and one that doesn't, and checks the result and its ordering. It was not run in a cloud regression environment for this PR.

Release note

Cloud mode supports SHOW TRANSACTION FROM db WHERE LABEL LIKE '...', returning up to the newest 1,000 matching transactions. Upgrade MetaService before FE.

Check List (For Author)

  • Behavior changed: Yes (see Details)
  • Does this need documentation: Yes. The SHOW TRANSACTION reference should state cloud support and the row cap. A doris-website PR will follow.
  • Pre-submission status: Partially verified. The unit tests, checkstyle, link and format passed (see Testing). The cloud regression suite was not run.

AI Disclosure

  • Tool: Codex (implementation and tests), Claude Code (specs, reviews and verification).
  • Generated: Code, tests and this description.
  • Human review: Pending. The author reviews before this PR leaves draft.

jaredmdobson and others added 5 commits November 18, 2026 22:59
### Background

SHOW TRANSACTION already supports label patterns in shared-nothing deployments.
Cloud MetaService stores a label index that can point to several transaction attempts.

### What problem does this PR solve?

Issue Number: TODO(author): the issue is opened with the PR
Related PR: None
PR Type: Feature

Problem Summary:

Cloud SHOW TRANSACTION WHERE LABEL LIKE currently returns an unsupported-mode error.
A paginated MetaService scan gives the FE the transaction attempts needed to apply
the existing label regex and render normal transaction rows.

### Details

- Scope: Cloud-mode transaction lookup by label pattern.
- Design: Scan encoded keys for the database and literal prefix, batch-read attempts,
  and resume by transaction ID when a page boundary splits a retried label.
- Usage: SHOW TRANSACTION FROM my_db WHERE LABEL LIKE 'load_%'.
- Default behavior: MetaService scans at most 10000 label keys per call and the FE
  returns up to 1000 matching rows sorted by descending transaction ID.
- Compatibility: The new RPC reuses the existing KV format. An older MetaService
  produces an upgrade message when an upgraded FE attempts the new RPC.
- Deployment modes: Cloud uses the RPC and the existing database LOAD check.
  Shared-nothing continues to use the local transaction manager.

### Testing

Reviewed the SQL, FE, RPC, and KV call paths and compared protobuf definitions
across supported branches. Checked changed C++ with clang-format 16 and reviewed
new Java imports and line lengths. Added MetaService, FE, and cloud regression
coverage for execution in CI. Doris was not built and the tests were not run locally.

### Release note

Cloud deployments support SHOW TRANSACTION WHERE LABEL LIKE after MetaService is upgraded.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: Yes, document cloud support and the FE row cap in the SHOW TRANSACTION reference.
- Pre-submission status: Partially verified: runtime tests and full FE Checkstyle were not run.

### AI Disclosure

- Tool: Codex
- Generated: Implementation, tests, and commit description
- Human review: Pending

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d return newest matches

Use fixed-size pages and enforce a statement-wide scan budget. Retain the
newest matching transactions and bound each MetaService page by its row,
read, and elapsed-time budgets. Cover resumable batches and regex filtering.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…TRANSACTION LABEL LIKE

### Background

Issue Number: None

### Details

Advance the cursor before honoring the page duration budget and resume
partial pages after FDB read timeouts. Charge the FE budget for label
and attempt reads, and enforce the statement deadline between RPCs.

### Testing

clang-format 16, Java Checkstyle, and git diff --check passed.
Doris builds and runtime tests were skipped.

### Release note

Cloud SHOW TRANSACTION LABEL LIKE limits keys scanned per statement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HOW TRANSACTION LABEL LIKE

Send the remaining key allowance to MetaService and keep complete final
pages. Use fixed-width cursors and cover range-boundary resumption.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…SHOW TRANSACTION LABEL LIKE

Drain fetched labels with no pending attempts at the read limit. Reject
malformed cursors and negative allowances, and cover MemTxnKv pagination.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant