feat: add RDF bloom payload model - #8354
Conversation
There was a problem hiding this comment.
Code Review
This pull request transitions the project's DataFusion dependency to a new fork and introduces a join-hash Bloom filter payload model (JoinHashBloomPayload) along with its corresponding probe expression (JoinHashBloomProbeExpr) to optimize remote dynamic filter updates. It also adds robust encoding, decoding, and fallback mechanisms, supported by extensive unit tests. The code review feedback suggests restoring the deleted documentation comments for DynFilterUpdate and QueryRequest to preserve code clarity, and simplifying the ceiling division math in num_bits_ceil_bytes using a more standard integer arithmetic pattern.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
7768ea3 to
7b323f3
Compare
c49d164 to
8aaf683
Compare
Signed-off-by: discord9 <discord9@163.com>
8aaf683 to
532e9f9
Compare
Signed-off-by: discord9 <discord9@163.com>
532e9f9 to
c14119b
Compare
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
This is PR 1a of the RDF Bloom stack. Follow-up fork-only stack PRs:
Depends on greptime-proto payload PR: GreptimeTeam/greptime-proto#325
Part of original umbrella draft PR: #8342
What's changed and what's your intention?
This PR introduces only the typed RDF Bloom payload model and serialization plumbing:
DynFilterPayload::JoinHashBloomas a typed remote dynamic filter payload variantJoinHashBloomPayloadmodel/proto conversion/validationJoinHashKind, noBloomHashAlgorithm, and no serializeddistinct_hash_count;df_seed0..3remain for receiver-side DataFusion hash reconstructionBloom probing and HashTableLookupExpr encoding are intentionally split into follow-up PR1b/PR1c to keep this review bounded.
Validation run locally:
cargo fmt --all -- --checkgit diff --checkCARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0 cargo test -p common-query request --libPR Checklist
Please convert it to a draft if some of the following conditions are not met.