Skip to content

🐛 Fix: the temporal clone e2e follows the async copy it exercises - #203

Draft
medazizktata25 wants to merge 2 commits into
Guepard-Corp:mainfrom
medazizktata25:fix/temporal-clone-test-follows-async-copy
Draft

medazizktata25 wants to merge 2 commits into
Guepard-Corp:mainfrom
medazizktata25:fix/temporal-clone-test-follows-async-copy

Conversation

@medazizktata25

Copy link
Copy Markdown
Contributor

Related Issue

None filed.

What

e2e_clone_robustness::clone_range_and_temporal_hydration_elide fails on every run that can reach it:

temporal window hydrated (t0=0 t1=0)

The test is out of date, not the extension. Its temporal half was written for a synchronous fetch: it expects the query on a date window to raise rows_fetched. The commit after it (f7a4d83) changed that path, and the test was never updated:

  • an uncovered temporal window now federates to the source for an immediate answer;
  • it enqueues a background time copy of the window;
  • the worker's copies skip gfs.clone_stats (DEFER_BOOKKEEPING), so rows_fetched cannot move.

The data is right: the query returns exactly the source's rows. CI did not catch this because it runs none of this file's tests (they need the gfs-postgres image).

How

  1. test(clone): the temporal-window case follows the async copy it exercises. The temporal half now checks what the path does:
    • the first query federates once (federate_calls +1) and returns exactly the source's rows;
    • the queued copy lands: nothing left in gfs.copy_queue for the table, and the table is whole-cached or holds a cached range (waits up to 60 s);
    • the repeat query returns the source's rows without federating again.
  2. docs(clone): rows_fetched does not count the background worker's copies. A comment where clone_stats.rows_fetched is defined, saying what it counts: rows a backend copies for its own query, plus gfs.warm's, and not the worker's. A matching note in tpch_validate.sh, whose pulled() reads that column as rows pulled.

Review Guide

  1. crates/applications/cli/tests/e2e_clone_robustness.rs: the temporal half of clone_range_and_temporal_hydration_elide.
  2. crates/extensions/gfs/src/sql/schema.sql and tpch_validate.sh: comments only.

Testing

On Rust 1.93.1, macOS with Colima for Docker, and a gfs-postgres:16 image built from this tree.

  • e2e_clone_robustness: 5/5 pass, and they ran (not the early return for a missing image).
  • Calibration:
    • with an extension image whose worker never runs its typed copy jobs, the test fails with "the background copy of the window never landed";
    • with the image rebuilt from clean source, 5/5 pass.
  • cargo fmt --check clean.
  • Commit 2 changes only SQL -- comments and a shell comment, so the installed catalog is unchanged; the image was not rebuilt for it.
  • Not tested: Docker Desktop, podman, a Linux Docker host.

Documentation

  • The rows_fetched comment in schema.sql, and the note in tpch_validate.sh.

User Impact

None at runtime: one test and two comments.

Out of scope, found while checking this: the copy worker still shares locks with foreground queries, and a user query on a table being copied can fail with deadlock detected. I reproduced it; it will be reported separately.

  • gfs.take_token() takes gfs.budget FOR UPDATE before its unlimited early return, and the worker holds that for the whole copy.
  • The worker then updates the table's clone_source row, which a concurrent query holds from bump_access.

Checklist

  • Self-review completed
  • Format check passes (cargo fmt --check)
  • This PR can be safely reverted.

…ises

A query over an uncovered temporal window federates to the source and
enqueues a background copy of the window; the worker's copy skips the
clone_stats write by design, so rows_fetched never moves. The test still
expected the synchronous path it was written for and failed on every
run that could reach it ("temporal window hydrated (t0=0 t1=0)").

It now checks what the path does: the first query federates once and
matches the source, the queued copy lands (the table is whole-cached or
holds a cached range, with nothing left in gfs.copy_queue), and the
repeat query matches the source without federating again.

Checked against an extension image whose worker never copies: the test
fails with "the background copy of the window never landed".
The background worker's copies (temporal windows, whole-table and
predicate copies) skip clone_stats, so rows_fetched never includes them.
It counts what a backend copies for its own query, plus what gfs.warm
copies. Say so where the column is defined, and where tpch_validate.sh
reads it as the number of rows pulled.

This branch is waiting to be deployed

1 waiting deployment
pr-validation — 3cf42917 Waiting Oct 27, 2026 by medazizktata25 via build-and-test (ubuntu-latest) #298
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant