Skip to content

ci: let the test suite actually run — torch-free deps in the dev extra - #170

Merged
andylizf merged 1 commit into
StarTrail-org:mainfrom
dex0shubham:ci/run-torch-free-tests
Oct 8, 2026
Merged

andylizf merged 1 commit into
StarTrail-org:mainfrom
dex0shubham:ci/run-torch-free-tests

Conversation

@dex0shubham

Copy link
Copy Markdown
Contributor

Problem

The Tests job installs uv sync --extra dev, and dev was ["pytest>=8.0"].
Every test gated on fastapi, faiss or qdrant-client therefore skips in CI —
111 of 173 tests run.

That gap has already cost something concrete: the stale department-filter tests
fixed in #162 were broken by the Qdrant refactor in July and sat that way until
late September, because CI never imported the module. Anyone with the serve
extra installed saw five failures; CI saw a skip.

Why this is cheap

Testing the serve stack doesn't need torch. serve/src/pixelrag_serve/api.py
imports torch and transformers lazily, inside the functions that encode
queries (lines 350, 711, 756) — module import needs only numpy, fastapi, PIL and
pydantic, and core already carries numpy and pillow. The core dependency comment
already anticipates exactly this:

Kept in core — not the heavy embed/index extras — so pixelrag chunk,
pixelrag index, and the torch-free test suite import cleanly without pulling
torch/faiss.

So dev gains fastapi, faiss-cpu, pydantic, httpx (fastapi's
TestClient needs it) and pixelrag[qdrant]. They're listed individually
rather than as pixelrag[serve] precisely because that extra pulls torch,
torchvision and transformers, which no test touches.

No workflow change — the Tests job already runs --extra dev.

Verification

Using CI's own install command:

$ uv run --isolated --extra dev pytest tests/
before:  111 passed, 5 skipped
after:   173 passed, 1 skipped       (~42s)

The 62 newly-running tests include the full Qdrant backend-parity suite,
which has never run in CI — the VectorBackend contract tests whose entire
purpose is proving FAISS and Qdrant behave identically. The remaining skip is
the pdf extra, which needs poppler on the runner; left alone.

Cost: the Tests job additionally downloads faiss-cpu (~30 MB) and
qdrant-client/grpcio (~11 MB). No torch.

About the uv.lock diff

It's ~500 lines but inert, and worth checking rather than trusting:

  • same 123 packages before and after (grep -c '^name = ')
  • zero version changes (no version = line differs)
  • the change is that a newer uv drops marker = "sys_platform == 'darwin' or sys_platform == 'linux'" from dependency entries and reorders
    resolution-markers

Those markers are tautologies here: [tool.uv] sets
environments = ["sys_platform == 'linux'", "sys_platform == 'darwin'"], so
"darwin or linux" is always true within the declared resolution universe. Happy
to regenerate the lock with a pinned uv version if you'd rather keep the diff
minimal.

One risk worth naming

These 62 tests have never executed on Linux CI — they pass locally on macOS.
If any of them is platform-sensitive, this PR is where that surfaces, which is
the point. The CI run on this PR is the real verification; I'll fix anything it
turns up.

CI's Tests job installs `uv sync --extra dev`, which was pytest alone, so
every test gated on fastapi, faiss or qdrant-client silently skipped:
111 of 173 tests ran. The gap is what let StarTrail-org#162's stale department-filter
tests sit broken for two months — CI never imported them.

The serve stack doesn't actually need torch to be *tested*: api.py
imports torch and transformers lazily, inside the functions that encode
queries, and core already carries numpy and pillow (the core dependency
comment already anticipates "the torch-free test suite"). So the dev
extra gains fastapi, faiss-cpu, pydantic, httpx (TestClient) and
pixelrag[qdrant] — listed individually rather than as pixelrag[serve],
which would drag in torch + torchvision + transformers for nothing.

Verified with CI's own install command:

    uv run --isolated --extra dev pytest tests/
    before: 111 passed,   5 skipped
    after:  173 passed,   1 skipped

The 62 newly-running tests include the whole Qdrant backend-parity suite,
which has never run in CI. The one remaining skip is the pdf extra, which
needs poppler on the runner.

The uv.lock diff is large but inert: same 123 packages, no version
changes. `environments` restricts resolution to linux and darwin, so the
`sys_platform == 'darwin' or sys_platform == 'linux'` markers a newer uv
drops here are tautologies; the rest is marker reordering.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 2, 2026

Copy link
Copy Markdown

@dex0shubham is attempting to deploy a commit to the andylizf's projects Team on Vercel.

A member of the Team first needs to authorize it.

@dex0shubham

Copy link
Copy Markdown
Contributor Author

@andylizf gentle ping on this one. CI is green (the only red check is the Vercel preview, which needs a team member to authorize a deploy from a fork).

Short version: the Tests job installs --extra dev, which was pytest alone, so everything gated on fastapi/faiss/qdrant-client silently skipped — 111 of 173 tests actually ran. Adding those four deps plus pixelrag[qdrant] to dev takes CI to 173 passed, 1 skipped, including the Qdrant backend-parity suite that had never run there. No torch: api.py imports it lazily, so the tests don't need it. Job wall time is unchanged (74s → 73s).

One thing worth your call: the uv.lock diff is ~500 lines, but inert — same 123 packages, zero version changes. It's a newer uv dropping sys_platform == 'darwin' or sys_platform == 'linux' markers, which are tautologies given [tool.uv] environments. If you'd rather keep the lock diff minimal, tell me which uv version you pin to and I'll regenerate.

Happy to rebase or split this if it's easier to review.

@andylizf
andylizf merged commit b4fe763 into StarTrail-org:main Oct 8, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants