Repository navigation

Multi vendor support: Fixes for download_docs & WARN for lack of OPENROUTER_API_KEY - #5

Open

rpoornac wants to merge 105 commits into

v-shobhit:masterfrom

hans-intel:multi-vendor-support

Commits on Dec 20, 2024

[Automated Commit] Format Codebase
mlcommons-bot
committed
9b59d2e

Commits on Sep 18, 2025

Merge branch 'mlcommons:master' into master
v-shobhit
authored
fe51c12
[Automated Commit] Format Codebase
github-actions[bot]
committed
1f2666c

Commits on Sep 19, 2025

Add download_pdf.py
v-shobhit
committed
ec84225

Commits on Sep 22, 2025

Add artefacts downloading scripts
v-shobhit
committed
a77eb4d
Fix downloading script
v-shobhit
committed
9f0db17
[Automated Commit] Format Codebase
github-actions[bot]
committed
b4dcd57
cleanup read_pdf.py
v-shobhit
committed
e0f28b2
add env setup files
v-shobhit
committed
d301bff
add single-shot retrieval
v-shobhit
committed
daa07f8
renmae setup file
v-shobhit
committed
23aaea6
Add README
v-shobhit
committed
a15bc6d
name fix
v-shobhit
committed
fb5f157
Add reranker
v-shobhit
committed
94b11d1
Add README.md
v-shobhit
committed
9275aaa

Commits on Sep 26, 2025

Save vector db
hans-intel
committed
c5c7213
Support Intel XPU for embedding and reranker model
hans-intel
committed
5142a06
Separate tok_k for retrieval and reranking
hans-intel
committed
9f5d8c5
save url mapping in passage for evaluation
hans-intel
committed
41cd5c4
Implement evaluation
hans-intel
committed
7b9390f

Commits on Sep 29, 2025

Commits on Sep 30, 2025

Support txt file to ingest: --passages renamed to --ingest
hans-intel
committed
beeb74f

Commits on Oct 1, 2025

Add stemmer to bm25
hans-intel
committed
1f70b6c
Add missing ingest function for vectordb
hans-intel
committed
94a46cb
Add metrics recall, precision, F1, MAP (Mean Average Precision)
hans-intel
committed
1140a8a

Commits on Oct 2, 2025

implment retrieval strategies (top_p, relative, elbow, ...)
hans-intel
committed
4fd651c
rename evaluate() to evaluate_retrieval()
hans-intel
committed
140add3

Commits on Oct 3, 2025

support html parsing (bs4)
hans-intel
committed
4a8deb1
code cleanup
hans-intel
committed
9710776
Fix vector db ingestion
hans-intel
committed
1e84b9c
fix r2r variation (torch seed)
hans-intel
committed
e72e2c7
improve parsing (remove wiki metadata, default sentence boundary)
hans-intel
committed
35f5bc2

Commits on Oct 6, 2025

Add perf monitoring feature (--benchmark)
hans-intel
committed
6c6d160

Commits on Oct 7, 2025

Add indexing trend measurement
hans-intel
committed
fba871e

Commits on Oct 8, 2025

Add vector indexing option
hans-intel
committed
9396387

Commits on Oct 10, 2025

fix read_docs measurement
hans-intel
committed
bb3737e
Merge branch 'e2e-rag-eval-adv' into e2e-rag
hans-intel
committed
f6b4396
Merge branch 'e2e-rag-r2r-fix' into e2e-rag
hans-intel
committed
a385e67
Merge branch 'e2e-rag-bsoup' into e2e-rag
hans-intel
committed
9b28f7b
Merge branch 'e2e-rag-latency-measurement' into e2e-rag
hans-intel
committed
b4b0e02
Merge branch 'e2e-rag-vector-indexing-option' into e2e-rag
hans-intel
committed
7aee971

Commits on Oct 14, 2025

Implment IVF nprobe option
hans-intel
committed
3c8a004

Commits on Oct 15, 2025

Update README
hans-intel
committed
5a9120c
Add feature save/load embeddings (--load-embeddings)
hans-intel
committed
ac61fc0

Commits on Oct 17, 2025

Embedding is done in multiple devices
hans-intel
committed
b39e8ce
Centralized parameter management
hans-intel
committed
bfcb570
Support top_p in vector db
hans-intel
committed
f5775df
Support parallelization in read_docs (--processes)
hans-intel
committed
5ded358
Change default max retrieval to 20 from 100
hans-intel
committed
1ae5f56
fix result display -- deduplicated urls
hans-intel
committed
f47c4f5
Fix correctly passing retrieval strategy params to filter
hans-intel
committed
a004b93

Commits on Oct 18, 2025

Commits on Oct 26, 2025

More detailed analyses for FRAMES prompts
hans-intel
committed
0fedf4f

Commits on Oct 27, 2025

Commits on Oct 29, 2025

improve query writer prompt (WIP)
hans-intel
committed
1a58fab
Support hpu devices
hans-intel
committed
a4171c8

Commits on Oct 30, 2025

Query writer prompt WIP
hans-intel
committed
9d9254f
Query writer prompt WIP
hans-intel
committed
34f5501
Query rewriter prompt WIP
hans-intel
committed
6c7b862

Commits on Nov 6, 2025

HPU reranking+embedding workaround (run on CPU)
hans-intel
committed
9b26053

Commits on Nov 9, 2025

HPU reranking+embedding workaround (run on CPU)
hans-intel
committed
d966e8b
HPU embedding fix
hans-intel
committed
936d5ab
fix read_docs to include .infobox
hans-intel
committed
2a00054
HPU reranking+embedding workaround (run on CPU)
hans-intel
committed
248e23f
HPU embedding fix
hans-intel
committed
0eedaa6
Support hpu devices
hans-intel
committed
ea50009
generate LLM answer single_shot_retrieval
hans-intel
committed
3994990
LLM answer for single shot retrieval and evaluation script
hans-intel
committed
eb57581
change default params to match FRAMES paper (k=5, n=5)
hans-intel
committed
b4da009
HPU reranking+embedding workaround (run on CPU)
hans-intel
committed
3c35831
copy url_mapping to output folder
hans-intel
committed
f2d4e55
Support hpu devices
hans-intel
committed
8bfd4ad
Saving result of multi-shot retrieval for evaluate.py
hans-intel
committed
ab9f902
Simplified query rewriter prompt as baseline
hans-intel
committed
1139d0d
improve scoring prompt
hans-intel
committed
068d9b3
Fix bm25 for single shot retrieval
hans-intel
committed
8b3b155

Commits on Nov 12, 2025

add scripts
hans-intel
committed
8d22e1f

Commits on Apr 29, 2026

multi hop accuracy improvement with oracle test into the same branch
mkankana
committed
1bfebe5

Commits on May 6, 2026

hybrid model usage 120B+20B. removed dead code
mkankana
committed
9a038e4

Commits on May 14, 2026

LLM calls to openrouter. logging all LLM calls and results
mkankana
committed
b9a73fc

Commits on May 17, 2026

added log sample result
mkankana
committed
bbc0e66
added log sample result
mkankana
committed
6670aa9

Commits on May 18, 2026

Add parallel multi-shot retrieval with per-query threading
hans-intel
committed
3cad55f
Add OUTPUT_DIR and TEMPERATURE as parameters
hans-intel
committed
3e8a0b4

Commits on May 19, 2026

Add retry with backoff for LLM calls and fix silent failures
hans-intel
committed
b6eacfa

Commits on May 21, 2026

Fix colbert to be used in late interaction way (query, doc embedding then MaxSim)
hans-intel
committed
ce33ce8

Commits on May 22, 2026

Commits on May 23, 2026

Decouple endpoint routing from OpenRouter
hans-intel
committed
ff80999
Update readme
hans-intel
committed
9527fdd

Commits on Jun 2, 2026