You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
OnPair has no LikeKernel, so LIKE on an OnPair column decodes the values and runs the generic pattern. The compressor keeps whichever of FSST and OnPair is smaller per column, and on ClickBench that is OnPair for URL, Title and Referer (about 30 to 40 % smaller than FSST in the dict layout), so their LIKE queries never reach the FSST matcher.
Proposed solution
A LikeKernel for OnPair on top of the onpair crate's existing search (contains, starts_with, equals), the same way compare.rs already uses it, with the pattern subset of the FSST matcher. A local prototype took ClickBench Q20 from 55.0 to 44.0 ms. I'm working on it.
After that I'd like to follow up with a faster contains: OnPair's longest prefix match makes the token at a position depend only on the next 16 bytes, so a needle can be searched as a few anchor tokens in the code stream. A first probe is exact and about 1 ns per row for selective needles.
Additional context
Open question for the follow-up: would the faster contains fit better in the onpair crate or in vortex-onpair?
Activity
robert3005 commentedon Oct 25, 2026
@gargiulofrancesco have been working on OnPair string comparison algorithm. You can see initial results here https://www.youtube.com/watch?v=IfCMC-4PEzI