Chopper: a small open model that cuts only the needed sentences out of retrieved text #11552
wiiiktor
started this conversation in
Show and tell
Replies: 1 comment 1 reply
|
Here’s a pair I’d try in Spanish: “Abrimos los lunes de 9 a 6. Los lunes feriados permanecemos cerrados.” Ask “¿Abren este lunes feriado?” and then “¿A qué hora abren un lunes normal?” Same two sentences, but dropping I’d be curious to see the actual spans Chopper keeps for each question. That would help tell whether a wrong answer came from the cut or from the model reading it. |
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi! Sharing a small open-source tool that may be useful here.
https://github.com/wiiiktor/chopper_demo
A new, ultra-fast method (50 ms on a consumer-grade GPU): a per-token decision [useful? yes/no] for pages of text. Created in November 2026 in Poland, EU. A tiny edge model offering 82% of the quality of Claude Opus 5.5.
70-second video: https://youtu.be/KMvmdXCEOJU
A retriever brings back 20 text snippets, but usually only one or two sentences in them answer the question. The chopper is not an LLM summarising them: a small encoder (0.3B parameters) reads the question and all 20 snippets together in one pass and decides for every token whether it is useful. No text generation, so it takes about 50 ms per question (~2,400 tokens), 36 ms with the 0.14B version.
What it keeps are exact quotes ("chops"), never rewritten, so every chop can be traced back to its source. If the answer is not there at all, it says so, so you can search again.
Test made with Qwen3-14B answering 400 questions, one retrieval each:
pip install chopper-ragworks with LangChain, LlamaIndex, Haystack or plain Python. Models (MIT): https://huggingface.co/wiiiktor/chopper_mmbertFeedback is very welcome, especially where it fails.
All reactions