Skip to content

feat: add RDF bloom payload model - #8354

Closed
discord9 wants to merge 2 commits into
GreptimeTeam:mainfrom
discord9:omos/rdf-bloom-common-query
Closed

feat: add RDF bloom payload model#8354
discord9 wants to merge 2 commits into
GreptimeTeam:mainfrom
discord9:omos/rdf-bloom-common-query

Conversation

@discord9

@discord9 discord9 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

This is PR 1a of the RDF Bloom stack. Follow-up fork-only stack PRs:

Depends on greptime-proto payload PR: GreptimeTeam/greptime-proto#325

Part of original umbrella draft PR: #8342

What's changed and what's your intention?

This PR introduces only the typed RDF Bloom payload model and serialization plumbing:

  • adds DynFilterPayload::JoinHashBloom as a typed remote dynamic filter payload variant
  • adds slim JoinHashBloomPayload model/proto conversion/validation
  • updates initial remote dynamic filter snapshot serde and payload-size accounting
  • updates greptime-proto dependency to PR feat: timestamp column support i64 #325
  • keeps the proto surface slim: no JoinHashKind, no BloomHashAlgorithm, and no serialized distinct_hash_count; df_seed0..3 remain for receiver-side DataFusion hash reconstruction

Bloom probing and HashTableLookupExpr encoding are intentionally split into follow-up PR1b/PR1c to keep this review bounded.

Validation run locally:

  • cargo fmt --all -- --check
  • git diff --check
  • CARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0 cargo test -p common-query request --lib

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

@github-actions github-actions Bot added size/XXL docs-not-required This change does not impact docs. labels Jun 24, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request transitions the project's DataFusion dependency to a new fork and introduces a join-hash Bloom filter payload model (JoinHashBloomPayload) along with its corresponding probe expression (JoinHashBloomProbeExpr) to optimize remote dynamic filter updates. It also adds robust encoding, decoding, and fallback mechanisms, supported by extensive unit tests. The code review feedback suggests restoring the deleted documentation comments for DynFilterUpdate and QueryRequest to preserve code clarity, and simplifying the ceiling division math in num_bits_ceil_bytes using a more standard integer arithmetic pattern.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/common/query/src/request.rs
Comment thread src/common/query/src/request.rs
Comment thread src/common/query/src/request/join_hash_bloom.rs
Signed-off-by: discord9 <discord9@163.com>
@discord9
discord9 force-pushed the omos/rdf-bloom-common-query branch from 8aaf683 to 532e9f9 Compare June 30, 2026 04:01
Signed-off-by: discord9 <discord9@163.com>
@discord9
discord9 force-pushed the omos/rdf-bloom-common-query branch from 532e9f9 to c14119b Compare July 6, 2026 09:46
@discord9 discord9 closed this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs. size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant