Repository navigation
Conversation
…rapper when the filter is also merged apache#68684 Cherry-picked from apache#68684 (d357885). Resolve branch-4.1 conflicts by retaining the existing compile-check and runtime filter interfaces. Adapt the unit tests to the branch's expression type and omit assertions for the master-only bucket-prune cache; keep the wrapper ownership and filter-value checks. Tests: ./run-be-ut.sh -j 40 --run --filter='RuntimeFilterMergerTest.*:RuntimeFilterProducerTest.*:RuntimeFilterWrapperTest.*' (cherry picked from commit d357885)
Backport the ownership simplification and arrival-count fix from apache#70661 (8d5b213, 4f12834, ab348b3, 1e3335d). Always clone the first merger input and the Bloom filter used during an IN-to-Bloom transition, so callers do not need to prove exclusive ownership. Record an input arrival and mark the merger ready only after accumulation succeeds. Keep local and remote filter delivery unchanged. Adapt the failed-merge test to branch-4.1 with a deterministic Bloom-size mismatch, since its Bloom directory allocation bypasses the memory tracker. Tests: focused ASAN BE runtime-filter UT, broad RF-related ASAN BE UT, rfv2 SQL regression, C++ format checks, and FE checkstyle.
Member
Author
|
run buildall |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backport the runtime-filter ownership fixes from #68684 and #70661 to branch-4.1. A producer wrapper can be consumed directly by a local target while its filter is also merged for another target. Reusing that wrapper in the merger lets later merges change a filter that the direct consumer is already probing.
Validation
BUILD_TYPE=ASAN ./run-be-ut.sh --run -j 16 --filter='RuntimeFilterMergerTest.*:RuntimeFilterProducerTest.*:RuntimeFilterWrapperTest.*'— 44/44 passed.BUILD_TYPE=ASAN ./run-be-ut.sh --run -j 16 --filter='*RuntimeFilter*:*BloomFilter*:*MinMax*:*SyncSize*-TableReaderTest.CountStarFallbackKeepsLateRuntimeFilterCarrierValues'— 291/291 passed../build.sh --be -j 32— passed.(cd fe && mvn -o -pl fe-core -am -DskipTests validate)— passed.doris-local-regression run -d query_p0/runtimefilterV2 -s rfv2— 1/1 suite passed on a single-FE/single-BE cluster.clang-format --dry-run --Werror; repository format check andgit diff --checkpassed.Multi-BE RPC, mixed-version operation, and production Bloom-copy performance have not been validated locally.