check: track a set of checked packs for partial checks#9909
Conversation
Replace the single last-pack-checked marker with a persisted set of checked pack ids, so a pack added between partial runs is verified regardless of how its content-sha256 name sorts.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9909 +/- ##
==========================================
+ Coverage 85.13% 85.34% +0.21%
==========================================
Files 93 93
Lines 15899 15932 +33
Branches 2428 2429 +1
==========================================
+ Hits 13535 13597 +62
+ Misses 1654 1631 -23
+ Partials 710 704 -6 ☔ View full report in Codecov by Harness. |
ThomasWaldmann
left a comment
There was a problem hiding this comment.
Also: does it handle Ctrl-C correctly?
Split the vertical import and trim an irrelevant line from the checked-packs comment.
b9214b6 to
0e68c7e
Compare
Yes, store.store() writes atomically and appends a sha256 that's verified on load, so an interrupt can't corrupt the set or mark a pack checked when it wasn't. The worst case you redo a last one minute of work, capped at 1 min now that checkpoints run every 60s. |
Also shrink the partial-check checkpoint window from 5 minutes to 1 minute so a Ctrl-C between checkpoints redoes at most a minute of verification work.
0e68c7e to
996331b
Compare
|
Docs not updated — data-structures.rst:34 still documents last-pack-checked ("key of last pack checked as text"). It should describe checked-packs instead. |
|
Checkpoint cost grows with repo size (note, not a blocker). The old checkpoint wrote a ~70-byte marker; the new one rewrites the whole table (41 bytes/pack) every 5 minutes — ~4 MB per checkpoint at 100k packs. Fine in practice, just no longer O(1). Guess we should reduce checkpoint frequency: every 30 mins. |
|
Re-reviewed the updated branch. All 62 The update resolves what I'd consider required: the docs are fixed (and better than asked — the old entry was misdocumented under Remaining minor points, none blocking:
Old marker cleanup. A repo that ran partial checks under older borg keeps its Schema guard on load. If a future borg changes Test gap: the skip itself is untested. The four tests prove the re-check cases, but none proves that an intact-recorded pack is skipped (the point of the mechanism), nor that a full check ignores the set. Both pass on your branch; feel free to take these: def _spy_hash(repository, monkeypatch):
hashed_keys = []
orig_hash = repository.store.hash
def spy(key):
hashed_keys.append(key)
return orig_hash(key)
monkeypatch.setattr(repository.store, "hash", spy)
return hashed_keys
def test_check_partial_skips_pack_recorded_intact(tmp_path, monkeypatch):
# a pack recorded intact this cycle is skipped on resume, not re-verified.
intact = fchunk(b"INTACT", chunk_id=H(1))
intact_id = sha256(intact).digest()
with Repository(str(tmp_path / "repo"), exclusive=True, create=True) as repository:
pack_key = "packs/" + bin_to_hex(intact_id)
repository.store_store(pack_key, intact)
tracker = PackTracker(repository.store)
tracker.record(intact_id, ok=True)
tracker.save()
hashed_keys = _spy_hash(repository, monkeypatch)
assert repository.check(repair=False, max_duration=3600) is True
assert pack_key not in hashed_keys # skipped, not re-verified
def test_check_full_ignores_recorded_set(tmp_path, monkeypatch):
# a FULL check re-verifies everything even if a partial cycle recorded the pack intact,
# and drops the set once the cycle is complete.
intact = fchunk(b"INTACT", chunk_id=H(1))
intact_id = sha256(intact).digest()
with Repository(str(tmp_path / "repo"), exclusive=True, create=True) as repository:
pack_key = "packs/" + bin_to_hex(intact_id)
repository.store_store(pack_key, intact)
tracker = PackTracker(repository.store)
tracker.record(intact_id, ok=True)
tracker.save()
hashed_keys = _spy_hash(repository, monkeypatch)
assert repository.check(repair=False) is True
assert pack_key in hashed_keys # full check verified it
after = PackTracker(repository.store)
after.load()
assert len(after) == 0 # set dropped after complete cycleTiny wording nits: the docs say "pack key -> timestamp, result" but it's the 32-byte binary pack id ("key" elsewhere in that doc means the store key like |
The repository check writes a marker holding the last pack id it verified, so a partial check (
--max-duration) resumes by skipping every pack up to that marker. This assumes the pack listing is sorted.Pack files are named by the sha256 of their content, so a pack added between two partial runs can sort anywhere. If its id sorts before the marker, the resumed check skips it and never verifies it.
This replaces the marker with a persisted set of checked pack ids, a
HashTableNTmapping pack id to(timestamp, result). A pack is skipped only when it appears in the set recorded as intact; a pack recorded as corrupt is re-verified rather than skipped. So a newly added pack gets checked whatever its name sorts to, and a pack that was corrupt in an earlier run is looked at again. The set is dropped once a cycle has checked every pack.The set is managed by a
PackTrackerclass. It persists viastore.store()(atomic write) with a sha256 appended over the serialized table; on load a blob whose sha256 does not match is discarded, so a rotted set re-checks everything instead of skipping an unverified pack. Ctrl-C is therefore safe: the on-disk set is never partially written and never records a pack that was not verified.The cycle is checkpointed every 30 minutes rather than every minute: the checkpoint now rewrites the whole table (~41 bytes/pack) instead of a fixed ~70-byte marker, so on large repos each save is no longer O(1). At 30-minute intervals that cost stays negligible while an interrupt still redoes at most 30 minutes of verification work.
The set is stored at
cache/checked-packs;docs/internals/data-structures.rstis updated to document it in place of the old marker.Closes #9897