Skip to content

LIKE pushdown for OnPair #10758

Open
Open

Description

@mightsleep

Problem or motivation

OnPair has no LikeKernel, so LIKE on an OnPair column decodes the values and runs the generic pattern. The compressor keeps whichever of FSST and OnPair is smaller per column, and on ClickBench that is OnPair for URL, Title and Referer (about 30 to 40 % smaller than FSST in the dict layout), so their LIKE queries never reach the FSST matcher.

Proposed solution

A LikeKernel for OnPair on top of the onpair crate's existing search (contains, starts_with, equals), the same way compare.rs already uses it, with the pattern subset of the FSST matcher. A local prototype took ClickBench Q20 from 55.0 to 44.0 ms. I'm working on it.

After that I'd like to follow up with a faster contains: OnPair's longest prefix match makes the token at a position depend only on the next 16 bytes, so a needle can be searched as a few anchor tokens in the code stream. A first probe is exact and about 1 ns per row for selective needles.

Additional context

Open question for the follow-up: would the faster contains fit better in the onpair crate or in vortex-onpair?

Activity

  1. robert3005 commented on Oct 25, 2026

    @robert3005
    Contributor

    @gargiulofrancesco have been working on OnPair string comparison algorithm. You can see initial results here https://www.youtube.com/watch?v=IfCMC-4PEzI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureA feature request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions